Review Article
Austin J Public Health Epidemiol. 2021; 8(5): 1113.
Quality of Reporting Data for Covid-19 Cases in Sfax (Tunisia) in 2020-2021
Makhlouf M¹*, Ismahen B¹, Hayet A¹, Hedia K¹, Iadh M¹, Hejer L², Nissaf BA² and Jawhar M³
¹Department of Preventive Health, Tunisia
²ONMNE, Tunisia
³Regional Director of Health, Tunisia
*Corresponding author: Mohamed Makhlouf, Department of Preventive Health, Tunisia
Received: October 11, 2021; Accepted: November 24, 2021; Published: December 01, 2021
Abstract
Background: During the SARS-CoV-2 pandemic, a specific reporting system was set up in Tunisia to allow communication and timely follow-up of cases. These data are used among others to take public health measures to control this infection. However, effective decisions should be based on high quality data. Purpose: Our objective was to quantitatively evaluate the quality of the data of COVID-19 cases reporting in Sfax from March 2020 to June 2021. Methods: This is a cross-sectional, descriptive study that refers to the data collected in the reporting of covid-19 patients. Three quality attributes were studied, availability, promptness or punctuality and completeness. The source of the data was the Case Report Forms (CRFs) developed by the National Office of New and Emerging Diseases (ONMNE) collected mainly by telephone. A sample of 384 CRFs was selected by systematic random sampling. Findings: Only 29% of the MSDSs (15972) were available compared to the expected ones (38852). As for punctuality, 23% (n=87) were received at the surveillance cell within 48 hours of case confirmation. On the other hand, they reveal that the information recorded in the MSDS by the interveners was globally very insufficient and or illegible, especially the identification of the reporter and the contacts of the patients with scores of 53.9% and 43.9%. Overall, the quality of the data was judged “poor” in the majority of cases (72.13%). Conclusion: The present study revealed that our data reported during this pandemic are not timely and not satisfactory both quantitatively and qualitatively. We therefore recommend an evaluation of the entire communication system on a regular basis. Also, a mandatory training cycle for all primary care physicians and biologists in bio-statistics is necessary to better understand the usefulness of data collection, analysis and interpretation.
Keywords: COVID-19; Data quality; Evaluation; Tunisia (Sfax)
Introduction
Context and rationale
The World Health Organization (WHO) declared on January 30, 2020 COVID-19 is a public health emergency of international concern and on March 11 a global pandemic [1].
In Tunisia, the first case of covid-19 detected was on March 2, 2020 while in Sfax 15 days later. And since then, the epidemic has spread in waves throughout the Tunisian territory until today.
In order to allow the reporting and monitoring of COVID-19 cases in all health regions of the country, a specific reporting device has been set up using a manuscript developed by the National Observatory of New and Emerging Diseases (ONMNE). The objectives of this system are to monitor the number of cases and deaths related to COVID-19 in real time and to collect the information necessary to help manage episodes of clustered cases of COVID-19. This health information system allows us to make decisions about controlling this infection. This system will therefore generate data that will be collected by regional and national health surveillance teams.
However, effective public health decisions should be based on high quality data, i.e. data that are properly collected, analyzed, interpreted and timely. These same data are also needed to accurately assess the impact of different measures taken.
Regular assessment of data quality is therefore necessary to avoid the consequences of poor quality. In this regard, one wonders how satisfactory the data recorded in Sfax during this pandemic.
Presentation of Sfax region
Le the governorate of Sfax made up of 16 delegations is located in the south-east of Tunisia and has 1,017,235 inhabitants in 2020. Its economy is mainly based on agriculture (olive oil), fishing, phosphates and manufacturing industries. In terms of health, the region of Sfax is home to 2 university hospital centers, 4 regional hospitals and 16 health districts. A regional health surveillance system was officially created in 2014 within the Regional Health Directorate. Since the onset of the SARS-CoV-2 pandemic, a health watch unit has been created in each district. These units watch over the health events identified as priorities in terms of public health threat and those not defined a priori associated with a threat or suspected public health threat (Figure 1).
Figure 1: Health map of the region of Sfax.
Objectives
General objective: Evaluate the quality of the data for the reporting of COVID-19 cases in Sfax since the start of the pandemic (March 2020 - June 2021).
Specific objectives:
• Calculate the availability of manuscripts reporting positive cases.
• Calculate the promptness of the reporting forms.
• Estimate the completeness of these sheets.
Materials and Methods
The framework of the study
This report was developed within the framework of the accelerated training in intervention epidemiology (PHEP-BFE) organized by the National Observatory of New and Emerging Diseases (ONMNE).
Type of the study
It is a descriptive cross-sectional study that refers to the data collected for the reporting of covid-19 cases during the years 2020 and 2021.
Study period
June 14 to July 25, 2021.
Definitions
Confirmed positive case: Confirmation of infection is achieved by a positive RT-PCR test result (detection of viral RNA optimally in nasopharyngeal specimens) or by a positive antigenic test result (TDR-Ag).
Availability: The availability of an equipment or a system is a measure of performance. This quality indicator is obtained by dividing the time during which the equipment or system is operational by the total time during which it should have been operational.
Punctuality or Timeliness: It is determined whether the entities that submitted data did so before a given deadline.
Completeness: The completeness of the data is assessed by observing whether all entities that are supposed to report data actually do so.
Study population including
Source: COVID-19 positive case report forms according to the national definition of these cases developed by the NMOE.
Collection circuit of the Report Forms:
• The data on this form was collected most often through a remote telephone interview.
• The members of the local health watch unit in each district receive daily the list of confirmed positive cases by RT-PCR or by TDR-Ag. The completion of the reporting forms is done by telephone. Once completed, these forms will be sent to the preventive health department.
• For the laboratories: the forms are filled in before the results come out and once the result is positive it will be notified and the form will be sent to the preventive health department.
• Parameters studied by the report form: the dimensions studied in this form are the identification of the reporter, the identification of the patient, the clinical information, the exposure to the risks, the biological confirmation and the identification of the contacts.
Inclusion criteria: Any handwritten form of positive cases sent to the Preventive Health Department.
Non-inclusion criteria: Any suspected case file.
Sampling method
We opted for a systematic random sampling to select a representative sample of the report forms.
Sample size
It was calculated according to the Cochran formula with an expected frequency of 0.5, a confidence level of 95% and a precision of 5%. This resulted in a sample of 384 individuals. The sampling step was 33.
Data collection form: this is an observation grid (audit) composed of 6 dimensions
• The declarant with 6 items
• The patient with 10 items
• Clinical information with 20 items
• Risk exposure with 9 items
• Confirmation with 5 items
• Identification of contacts with 4 items
Collection techniques and tools
The audit form was completed by the author of this study.
Ethical and administrative considerations
Authorization to access the report forms was granted by the Regional Director of Health and the Director of Preventive Health in Sfax.
The plan of analysis of the results
Calculation of the availability: The availability of reporting forms (DFS) = Number of existing forms*100/ Total number of cases reported electronically.
Promptness/Punctuality Calculation: A score of 0 was assigned if the difference between the date of reporting and the date of confirmation exceeded 48 hours or one of the 2 dates was not reported and otherwise a score of 1.
Calculation of completeness: For each item, a score was assigned as follows: 1 point if the data is completely missing or illegible, 2 points if the data are incomplete or half-readable and 3 points if the data is complete and legible.
The sum of the points of the items allows obtaining a final score of the studied dimension.
To facilitate the interpretation of the results, we proceeded to the linear transformation of the scores of the six dimensions and the global score according to the following formula: final score = (Score obtained - Minimum possible score) * 100/(Maximum possible score - Minimum possible score). Thus, the new scores obtained varied from 0 to 100.
The quality of the overall data or of a dimension was judged
• “Poor” if the score obtained was <50%.
• “Average” if the score obtained is between 50 and 75%.
• “Satisfactory” if the score obtained is >= 75%.
The data collected was entered and analyzed by the Epi Info software version 7.2.4. Epi Info is a set of public domain programs developed by the Center for Disease Control and Prevention, USA (CDC) and running on the Microsoft Windows operating system. It is used by public health professionals for studying epidemics, managing databases for public health surveillance and other tasks and, in general, statistical applications and databases.
Once the data entry was completed, the obtained data were analyzed individually by items and then aggregated into dimensions.
The results were summarized in tables, graphs, means with standard deviations, and absolute and relative frequencies.
Results
Availability
As of June 30, 2021, only 15972 case reports (29%) are currently available at the Regional Health Directorate out of a total of 38852 registered COVID-19 cases (Figure 2).
Figure 2: Study of the availability of the FS COVID 19 in Sfax on June 30,
2021.
Promptness or Punctuality
Nearly one in four cases was reported on time (less than 48 hours) to the Regional Health Department during the SARS-COV2 pandemic (Figure 3).
Figure 3: Score attributed to the punctuality of reporting of COVID 19 cases
to the Regional Health Directorate of Sfax in 2020-21.
Completeness
Overall data analysis: According to the reporting or not of the data and their readability, the identification of the reporter and the identification of the contacts of the COVID-19 patients were the 2 dimensions with the lowest performance scores, 53.92 and 43.90 successively (Figure 4).
Figure 4: Global analysis of the 6 dimensions studied according to the
performance score obtained.
Overall, the quality of the COVID-19 case report data was judged “poor” in 72.13%. None of the forms received by the Regional Directorate of Sfax was “satisfactory” (Figure 5).
Figure 5: Overall quality of the data reported on the COVID-19 positive case
report form in the Sfax region during the period March 2020 - June 2021.
Analysis by dimension:
Declarant identification: Just over half of the CRFs (56.77%) were rated as “poor” and only 10.93% were satisfactory (Figure 6).
Figure 6: Overall quality of the “declarant Identification” dimension.
We note that globally all the items of this dimension did not reach a score of 50% and more particularly those concerning the “tampon” with 34.24%, the “workplace” with 25.78% and the “mail” with 14.71% (Figure 7).
Figure 7: Scores of the different items of the dimension “Declarant
identification”.
Patient identification: Almost 2/3 of the CRFs were satisfactory (59.11%) and few (4.16%) were rated as poor overall (Figure 8).
Figure 8: Overall quality of the “Patient Identification” dimension.
Except for the two items “Residence” and “Locality” which had scores below 50% (42.96 and 37.60% successively), all the others exceeded 67%, in particular “Name and surname” and “Date of birth” with 99.47 and 97.78% (Figure 9).
Figure 9: Scores of the different items of the dimension “Patient Identification”.
Clinical informations: The treatment of this dimension showed that almost 2/3 of the records had data qualified as “average” because they were reported in an incomplete or illegible way (Figure 10).
Figure 10: Overall quality of the “Clinical Informations” dimension.
The majority of items in this dimension had scores between 59% and 74%.
The “signs” and “asymptomatic” items had the highest scores (Figure 11).
Figure 11: Scores of the different items of the “Clinical Informations” dimension.
Exposure to risks: The analysis of this dimension showed that out of 5 cards, 2 had no information at all which was considered “poor” and the other 3 had incomplete or illegible information which was considered “average” (Figure 12).
Figure 12: Overall quality of the “Risk Exposure” dimension.
The score of the different items of this dimension was between 50 and 65%. However, the items with the lowest scores were “period of exposure” and “nature of exposure” with 55.2% and 54.29% (Figure 13).
Figure 13: Scores of the different items of the dimension “Exposure to risks”.
Confirmation of the diagnosis: In the majority of cases (70%), the quality of the data for this dimension was rated as “fair”. In addition, none of the records had “satisfactory” data (Figure 14).
Figure 14: Overall quality of the “Confirmation” dimension.
A score higher than 75% was revealed in 3 items, while in the 2 others, “Name of the laboratory” and “Type of sampling”, it reached successively 71.35% and 57.81% (Figure 15).
Figure 15: Scores of the different items of the dimension “Exposure to risks”.
Identification of contacts: For this dimension, in the majority of cases (84.89%), data were missing and therefore “poor” (Figure 16).
Figure 16: Overall quality of the “Contact Identification” dimension.
In this dimension, the score of 3 out of 4 items did not reach 40%, especially the name of the contact (Figure 17). In 64%, “close contact” was mentioned without any other information.
Figure 17: Scores of the different items of the dimension “Exposure to risks”.
Discussion
COVID-19 surveillance data are essential for detecting cases, monitoring the geographic spread and intensity of virus transmission, identifying trends by age, gender, vulnerable populations and settings, measuring health care impacts, adjusting and proportioning public health and social measures, and adapting the response to new developments, such as the introduction of vaccination and changing virus variants [1].
The evaluation of this system will provide information on its efficiency and effectiveness in monitoring communicable disease prevention and control activities. It aims to improve the quality of information and thus obtain a better service at the level of care. It is necessary to evaluate the relevance of the events selected, how the system identifies and reports these events, and how the system responds to health problems. Regular evaluation of surveillance systems is recommended [2-5].
In the present study, our objective was to quantitatively evaluate the quality of COVID-19 case reporting data in Sfax from March 2020 to June 2021.
The results thus obtained meet this objective well. Indeed, they revealed a quantitative as well as qualitative failure affecting the three quality attributes of the studied reporting data, namely the availability, the promptness or punctuality and the completeness of the COVID-19 reporting data via a manuscript.
Having access to reliable data in real time allows decision makers to act more quickly and effectively. Ensuring this availability of data requires a continuous and exhaustive data collection process, as well as data security and backup.
In Sfax, only 15972 report forms (29%) are currently available at the Directorate of Preventive Health out of a total of 38852 COVID-19 cases registered in Sfax. This shows that data collection in our region is not continuous and therefore much information will be lost.
As for the promptness or punctuality, only 26% of these forms were received in time (within 48 hours of the declaration of cases) at the Regional Directorate. This reveals that, in the majority of cases, there is a long delay between the need for information and the provision of the data. As a result, the value of our information system is decreasing.
Finally, as far as completeness is concerned, we found that the quality of the data reported on these forms was globally unsatisfactory, mainly concerning the “identification of the reporter” and the “tracing of contacts”.
However, this evaluation remains limited given the size of the sample and the fact that the data was collected through a remote interview. This type of telephone interview does not allow for the observation of the non-verbal communication of the interviewees and therefore reduces the quality of the information [6]. Similarly, it has been shown that the methodology used in the evaluation is strongly related to the results obtained.
Poor data quality can have tedious consequences on the public health decisions made.
Indeed, the quality of the dimension “identification of the reporter” judged globally “poor” could call into question the accuracy of the reported data.
Similarly, the quality of the “contact tracing” dimension was also rated as “poor” and would have a negative impact on the performance of the public health response. Indeed, contact tracing, along with rapid testing, isolation, and treatment of patients, is a key strategy for breaking the chain of transmission of SARS-CoV-2 and reducing COVID-19-related mortality. Tracing can also be used to pinpoint locations or activities that are the source of infection and thus allow for targeted public health and social measures [5].
The poor quality of the data found in the present study has been shown in other work. Indeed Claire Melamed has stated that data systems in Africa are fragile and inadequate to inform decisions, making it difficult to control the COVID-19 pandemic [7-12]. Similarly, Varun Vasudevan et al. (2020) found a wide disparity in the quality of COVID-19 data reporting across India [13-15].
Conclusion
Our evaluation of the epidemiological surveillance system has quantitatively shed light on the quality of the COVID-19 data communicated to the Regional Health Directorate of Sfax during the COVID-19 pandemic (2020-2021). It revealed that overall these data are neither timely nor satisfactory both quantitatively and qualitatively. These results would have a negative impact on the performance of the response to COVID-19.
Therefore, we suggest that improved analytical capacity is needed, given the many challenges and uncertainties associated with COVID-19. It is more urgent than ever to produce reliable data on key indicators on a regular basis. Early detection of problems, development of mitigation measures, and measurement of their effectiveness depend on it. Regular data collection needs to be improved, both at the HMIS level and at the health facility and household levels.
Similarly, we recommend that the entire paper and electronic communication system be evaluated on a regular basis and that analytical capacity be improved. Also, a mandatory training cycle for primary care physicians and biologists in biostatistics is needed to better understand the utility of data collection, analysis and interpretation.
Acknowledgement
• Mr. Le Directeur régional de la Santé de Sfax
• Mr. le Directeur de la santé Préventive de Sfax
• L’équipe de la cellule de veille sanitaire régionale
• les équipes de la veille sanitaire locale des circonscriptions
• Les biologistes privés de la région
• L’équipe de l’ONMNE
References
- COVID-19: Orientations pour la collecte des données des statistiques du travail. OMS. 2020.
- Comptez- moi! Collecte de données relatives aux droits de la personne: Site web de la CODB.
- Imen Ayouni, Jihen Maatoug, Wafa Dhouib, Nawel Zammit, Sihem Ben Fredj, Rim Ghammam, et al. Effective public health measures to mitigate the spread of COVID-19: a systematic review. BMC Public Health. 2021; 21: 1015.
- Zulfiqar A. Bhutta, Faisal Sultan, Aamer Ikram, Adil H. Haider, Assad Hafeez and Muhammad Islam. Balancing science and public policy in Pakistan’s COVID-19 response. World Health Organization (WHO) 2021. Open Access. Some rights reserved. This work is available under the CC BY-NC-SA 3.0 IGO license.
- Note d’information sur le dispositif de riposte au COVID-19. Global Fund. 2021.
- Module 3: Vérification des données et évaluation des systèmes. Organisation mondiale de la Santé. 2019.
- Evaluer les effets des différentes mesures de lutte contre le Covid-19, mission impossible. The Conversation. 2020.
- évaluations durant la pandémie de COVID-19. UNDP-IEO.
- Signalement de cas de COVID-19 dans les Etablissements sociaux et médico-sociaux. Application COVID-19 EHPAD/ESMS version 2. Guide pour les établissements. Santé Publique France. 2021.
- La veille et l’alerte sanitaire en France et dans le Nord Pas Calais. Institut de veille sanitaire.
- Esam Halboub, Mohammed Sultan Al-Ak’hali, Hesham M. Al-Mekhlafi and Mohammed Nasser Alhajj. Quality and readability of web-based Arabic health information on COVID-19: An infodemiological study. BMC Public Health. 2021; 21: 151.
- Une initiative pour combattre la COVID-19 avec les données. 20 Sci Dev Net. 2020.
- Jonathon D. Gass Jr, Anamika Misra, Mahendra Nath Singh Yadav, Fatima Sana, et al. Implementation and results of an integrated data quality assurance protocol in a randomized controlled trial in Uttar Pradesh, India. Trials. 2017; 18: 418.
- Ben Abdelaziz, N El Haddad, H Hannachi, S Nouira, S Mekki, et al. Qualité des supports de communication de crise lors de la pandémie de la COVID-19 au Grand Maghreb. 2021.
- Protocole pour l’évaluation des systèmes de surveillance épidémiologique. WHO/EMC/DIS/97.2.
Download PDF