Special Article - Dental Public Health
J Dent & Oral Disord. 2016; 2(7): 1037.
Inter and Intra-Examiner Reliability of Pulp Ulcer Fistula Abscess Index in the Representative Sample of Students in Saudi Arabia
Subait AA¹*, Ousaimi MA¹, Shohaib TA¹, Richi HN¹ and El-Metwally A²
¹Department of Public Health, King Saud bin Abdulaziz University for Health Sciences, Saudi Arabia
²Department of Public Health and Health Informatics, King Saud bin Abdulaziz University for Health Sciences, Saudi Arabia
*Corresponding author: Abeer Al Subait, Department of Public Health, College of Dentistry, King Saud bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
Received: May 04, 2016; Accepted: September 27, 2016; Published: October 03, 2016
Abstract
Introduction: No studies have been conducted in Saudi Arabia to assess the reliability of PUFA (Pulp Ulcer Fistula Abscess Index) which is indeed to analyze the functionality and reproducibility for giving a true and unbiased burden of untreated caries.
Methods: This cross-sectional study was conducted at Alnaeem Ibn Hammad Intermediate School from March 1st till 5th. One hundred and forty six students were examined for consequences of caries using PUFA/pufa (permanent/primary) indices. Kappa statistics was calculated for inter and intraexaminer reliability on the same time and for day 1 and 2 in both dentitions.
Results: For day 1 and 2 we found substantial to moderate agreement among two observer for different units of PUFA and pufa scale and for over all scale for primary and permanent dentition. For first observer we found substantial agreement for marking pulpal involvement, fistula, abscess and overall PUFA score. For the second observer we found almost perfect agreement for marking pulpal involvement and overall PUFA score with kappa statistics of 0.90 (CI: 0.82- 0.97 and 0.849 (CI: 0.75- 0.93) respectively. Intra-reliability for first observer had moderate agreement for marking pulpal involvement and overall pufa scores while for second observer we found substantial agreement for marking pulp and overall pufa score with kappa statistics 0.73 (CI: 0.50- 0.96) and 0.66 (CI: 0.41- 0.90) respectively.
Conclusion: Through this study we conclude that PUFA scale is reproducible and functional in determining burden of untreated caries in both primary and secondary dentition as results of over study indicate perfect to fair agreement of inter and intra examiner reliability. Further multicenter studies are recommended to support such findings.
Keywords: Pulp ulcer fistula abscess index; Inter-intra examiner reliability; Saudi Arabia
Introduction
Dental caries; as known; are caused by an imbalance between tooth re-mineralization and demineralization which results in destruction of tooth structure along with having pain and feeling discomfort [1]. In 2007, World Health Organization (WHO) emphasized on promoting new tools for a proper data collection to estimate a true burden of dental caries so that preventive strategies can be formulated [2]. Preventing and controlling dental caries highly depend upon a suitable examinations and correct diagnosis. Thus, it is really important to develop new tools or at least encourage using newly developed ones so a precise assessment of caries can be carried out which is considered very useful for a proper management of caries worldwide [3].
The DMF (Decayed Missing Filled) index which was formulated by Klein, Palmer and Knutson in 1938 has been used long time ago by researchers and clinicians for the assessment of dental caries [4]. The DMF index has many limitations in terms of the scores such as the number of teeth at risk, the assessment is confined on dentinal caries, not valid for elderly people, neither root caries nor progression of these caries can be determined and other few limitations [5,6]. Despite of these limitations, DMF index is still in used because of its validity, reliability and simplicity [3]. To overcome the limitations of DMF, various scales came into existence such as: SC (Significant Caries) index, ICDAS (International Caries Detection and Assessment System), (Specific Caries) index, CAST (Caries Assessment Spectrum and Treatment) index and PUFA (Pulp Ulcer Fistula Abscess) index [3]. The aim of this current article is to validate PUFA scale.
PUFA index was developed to outdo one of the drawbacks of DMF scale and unfortunately, no information is available on the clinical consequences of untreated dental caries, such as pulpal abscess which is considered a serious complication of dental caries [7]. PUFA records show multiple issues such as a visible pulp, oral mucosal ulcerations due to a root fragment or sharp cusp, a fistula and an abscess [3].
Many studies have been conducted in different parts of the world to assess the burden of untreated caries by using PUFA/pufa scores. Recently, a study was conducted in India reported that total 603 students of age 5-6 years old were examined for pufa with mean recorded 0.9 (SD: 1.93) and 38.6% have caries involving pulp. The ratio of untreated caries was 35% concluding that pufa index can be used as a tool to highlight the true burden of untreated dental caries [8]. World-wide different studies have been conducting to document the effectiveness of this scale in terms of assessment of untreated caries [7-12]. However none of the studies have studies the reliability of this tool. The aim of this article is to assess the reliability of PUFA/ pufa score in Saudi Arabia as this tool has shown promising outcomes in other parts of the world. To the best of our knowledge, no studies have been conducted in Saudi Arabia to assess the reliability of PUFA which is indeed an important step to analyze the functionality and reproducibility of such indexes so that can be used in future studies for giving a true and unbiased burden of untreated dental caries.
Materials and Methods
This cross-sectional study was conducted a tone of the National Guard schools; Alnaeem Ibn Hammad Intermediate School from March 1st until March 5th. A convenient sample of 146 intermediate students was examined for consequences of caries using the PUFA/ pufa indices. An approval was obtained from the Ministry of Education’s Office and ethical committee in Riyadh.
Customized Arabic informed consents with codes were given to the principle for distribution. After collecting the consent form an assigned teacher was given the lists of all classes and be requested to bring the students to the examination venue; one class at a time. The students were called by the teacher one after other in the same order written in the lists and any absent student was then highlighted right away.
The list was folded and handed back to the first examiner so that only the students’ codes were visible without knowing their names. Each student was called to be examined by the first examiner. After that, the student was guided to go directly to the adjacent room to be examined again by the second examiner which allows the student to return back to his class once the examination was completed. Absentees in the first and second days were examined in the third day that is March 3rd. All students were reexamined by the same both examiners two days after their first examination except for 10 students who were be absent on that day ending up with a total of 136 students.
Exclusion criteria
Students with special needs were excluded.
Data collection
Each tooth could be scored either as PUFA/pufa, or not scored at all. Then, the number of permanent teeth with pulpal involvement was counted to be recorded as P score, teeth with soft tissue ulceration were counted to be recorded as U score, teeth with a fistula were counted to be recorded as F score and teeth with an abscess were counted to be recorded as a score. The total summation of PUFA was recorded as PUFA score for each case. In the same way, pufa for primary teeth was recorded and the final score were ranged between 0-32 for PUFA index; including the third molars and 0-20 for pufa index.
Calibration
Two examiners were trained well by an experienced oral epidemiologist one week before starting the study to measure PUFA/ pufa indices. The session was lasting for 10 minutes looking at various photos of teeth with pulpal involvement, fistula, abscess, and soft tissue ulceration caused by sharp tooth edges.
Examination
All examinations were carried out by two independent examiners through visual examination using disposable mouth mirror and 2X2 gauze as well. The both examinations; performed by the first examiner and the second examiner were taken place in the library of the school and another room connected to the library respectively. A number of 146 students were examined on the first day by both examiners comparing with 136 students were examined two days later by the same examiners.
Statistical analysis
Data were entered twice using SPSS version 20.Inter and intraexaminer reliability of PUFA/pufa and PUFA/pufa scores were calculated using kappa statistics. Inter-examiner reliability was calculated twice; during the first examination and second examination respectively ended with calculated mean. The mean of all reliabilities was calculated to come up with an overall reliability of PUFA/pufa indices.
Results
Kappa statistics was calculated for inter and intra-examiner reliability of PUFA/pufa, and PUFA/pufa scores on the same day and for day 1 and 2 in permanent and primary dentations. For day 1, a substantial agreement was found between two observers of marking pulpal involvement in permanent dentition with kappa value of 0.75 (CI: 0.64-0.88). Total number of 104 students, were marked as free cavity from both observers. For fistula, a moderate agreement was found in permanent dentition among both observers with kappa statistics of 0.56 (CI: 0.07-1.00). Total number of 141 students, were marked as having no fistula in any tooth by both observers. For abscess, a slight agreement was found among both observers with kappa statistics of 0.19 (CI: 0.00-0.47). For overall PUFA score, a substantial agreement was found among both observers with kappa statistics of 0.70 (CI: 0.52-0.78) (Table 1).
First Observer
Second Observer
k and 95% CI
0
1
2
3
4
5
Inter observer kappa statistics of permanent dentition for both observers and for day 1
Pulp
0
1
2
3
4
104
2
0
0
0
5
20
1
0
0
1
3
5
0
0
0
1
1
2
1
-
-
0.754 (0.6471-0.8817)
Substantial agreement
Fistula
0
1
141
3
0
2
-
-
-
-
0.563 (0.0734-1)
Moderate agreement
Abscess
0
1
137
0
7
1
1
0
-
-
-
0.191 (0-0.4707)
Slight agreement
PUFA Score
0
1
2
3
5
97
2
0
0
0
10
20
1
0
0
1
3
7
0
0
0
1
0
0
2
0
0
0
2
0
0
0
0
0
1
0.700 (0.5285-0.7813)
Substantial agreement
Inter observer kappa statistics of primary dentition for both observers and for day 1
Pulp
0
1
2
3
127
4
1
0
2
8
1
0
0
0
0
1
-
0
0
1
1
-
0.613 (0.4376-0.8586)
Substantial agreement
Fistula
0
1
2
143
0
1
1
1
0
-
-
-
0.495 (0-1)
Moderate agreement
Abscess
0
1
143
0
2
1
-
-
-
-
0.495 (0-1)
Moderate agreement
pufa Score
0
1
2
3
4
126
4
1
0
0
1
8
0
0
0
0
2
1
1
0
0
0
0
1
0
0
0
0
0
1
0.7107 (0.5276-0.8938)
Substantial agreement
Table 1: Inter-observer kappa statistics of permanent and primary dentations for both observers and for day 1.
Further, kappa was calculated among both observers for day 1 in primary dentition and found a substantial agreement of marking the pulpal involvement with kappa statistics 0.61 (CI: 0.43-0.85) and a moderate agreement for fistula and abscess 0.49 (CI: 0.00-1.00). Total number of 143 students was marked as having neither fistula nor abscess in any tooth by both observers. For overall pufa score, a substantial agreement was found among both observers with kappa statistics of 0.71 (CI: 0.52-0.89) (Table 1).
Then, kappa for day 2 was calculated for both permanent and primary dentations and found in permanent dentition a substantial agreement among both observers of marking pulpal involvement and for overall PUFA score with kappa value 0.64 (CI: 0.46- 0.81) and 0.63 (CI:0.49- 0.76) respectively (Table 2). For primary dentition, it was found a moderate agreement for both observers on day 2 of marking pulpal involvement and overall pufa score with kappa statistics 0.59 (CI: 0.32- 0.86) and 0.58 (CI: 0.34- 0.83) respectively. For fistula, a substantial agreement was found of 0.66 (CI: 0.006-1.00) by both observers with no fistula in 134 students. For abscess, a fair agreement was found between both observers with kappa statistics of 0.32 (CI: 0.00 -0. 97) (Table 2).
First observer
Second observer
k and 95% CI
0
1
2
3
4
Inter observer kappa statistics of permanent dentition for both observers and for day 2
Pulp
0
1
2
3
4
97
1
0
0
0
7
14
3
0
0
0
4
7
0
1
0
1
0
1
0
-
0.6405 (0.4697- 0.8113)
Substantial agreement
Fistula
0
1
3
129
5
1
1
0
0
-
-
-
-0.012 (P value: 0.833)
Abscess
0
1
127
2
5
0
2
0
-
-
-0.020 (P value: 0.749)
PUFA
0
1
2
3
4
5
91
2
0
0
0
0
9
16
4
0
0
0
0
2
5
1
1
0
0
1
1
1
0
0
0
1
0
0
0
1
0.6303 (0.4926- 0.768)
Substantial agreement
Inter observer kappa statistics of primary dentition for both observers and for day 2
Pulp
0
1
2
3
4
123
4
0
0
0
1
5
1
0
0
0
0
0
1
1
-
-
0.5945 (0.3219- 0.8671)
Moderate agreement
Fistula
0
1
134
1
0
1
-
-
-
0.6634 (0.006-1)
Substantial agreement
Abscess
0
1
131
3
1
1
-
-
-
0.32 (0-0. 9765)
Fair agreement
Pufa
0
1
2
3
5
120
6
0
0
0
0
5
1
0
0
0
1
1
1
1
-
-
0.5874 (0.3412- 0.8336)
Moderate agreement
Table 2: Inter-observer kappa statistics of permanent and primary dentations for both observers and for day 2.
Moreover, intra-reliability for permanent and primary dentations was calculated for day 1 and 2. For the first observer, it was found a substantial agreement of marking pulpal involvement, fistula, abscess and overall PUFA score with kappa statistics of 0.734 (CI: 0.60- 0.86), 0.62 (CI: 0.25- 0.98), 0.66 (CI: 0.006-1.00) and 0.73 (CI: 0.58- 0.84) respectively for both days (Table 3). While for the second observer, it was found an almost perfect agreement of marking pulpal involvement and overall PUFA score with kappa statistics of 0.90 (CI: 0.82- 0.97 and 0.849 (CI: 0.75- 0.93) respectively (Table 3). The intrareliability in primary dentition for day 1 and 2 for the first and second observers was calculated and found for the first observer kappa statistics had a moderate agreement of marking pulpal involvement and overall pufa scores with values 0.50 (CI: 0.22- 0.80) and 0.51 (CI: 0.25- 0.76) respectively for both days and a perfect agreement of marking fistula with kappa statistics of 1 (CI: 1.00-1.00) for both days (Table 4). For the second observer, a substantial agreement of marking pulp and overall pufa score was found with kappa statistics of 0.73 (CI: 0.50- 0.96) and 0.66 (CI: 0.41- 0.90) respectively. Likewise the first observer, a perfect agreement of marking fistula was found with kappa statistics of 1 while for abscess a fair agreement was found with kappa statistics of 0.38 (CI: 0-1) for both days (Table 4).
Day 1
Day 2
k and 95% CI
0
1
2
3
4
5
Intra observer kappa statistics of permanent dentition for both days for observer 2
Pulp
0
1
2
3
4
99
5
0
0
0
3
16
1
0
0
2
1
6
0
1
0
0
0
1
0
0
0
0
1
0
-
0.734 (0.6025- 0.8661)
Substantial agreement
Fistula
0
1
129
1
2
3
-
0
1
-
-
0.622 (0.258- 0.987)
Substantial agreement
Abscess
0
1
134
0
1
1
-
-
-
-
0.663 (0.006-1)
Substantial agreement
PUFA
0
1
2
3
4
95
5
0
0
0
5
17
0
0
0
2
0
7
1
0
0
0
1
1
0
0
0
0
1
0
0
0
0
0
1
0.736 (0.5887- 0.8479)
Substantial agreement
Intra observer kappa statistics of permanent dentition for both days and for observer 2
Pulp
0
1
2
3
98
0
0
0
1
22
1
0
0
1
8
3
0
0
0
2
-
-
0.909 (0.821- 0.9782)
Almost perfect agreement
Fistula
0
1
133
2
1
0
-
-
-
-
-0.010 (P value: 0.902)
Abscess
0
1
2
125
4
0
1
4
0
1
0
1
-
-
-
0.606 (0.2978- 0.9142)
Moderate agreement
PUFA
0
1
2
3
4
5
91
2
0
0
0
0
1
25
3
0
0
0
0
0
8
0
1
0
0
0
0
2
1
0
0
1
0
0
0
1
-
0.849 (0.7592- 0.9392)
Almost perfect agreement
Table 3: Intra-observer kappa statistics of permanent dentition for both days and for observer 1 and 2.
Day 1
Day 2
k and 95% CI
0
1
2
3
4
Intra observer kappa statistics of primary dentition for both days and for observer 1
Pulp
0
1
2
3
119
5
0
0
5
4
0
0
0
0
1
0
0
0
0
1
0
0
1
0
0.507 (0.2208- 0.803)
Moderate agreement
Fistula
0
1
2
134
0
0
0
1
1
1 (1-1)
Perfect agreement
Abscess
0
1
131
1
4
0
-0.012 (P value: 0.861)
Pufa
0
1
2
3
4
114
5
0
0
0
7
5
0
0
0
1
0
1
0
0
0
0
0
1
0
0
0
0
0
1
0.510 (0.2571- 0.7633)
Moderate agreement
Intra observer kappa statistics of primary dentition for both days and for observer 2
Pulp
0
1
2
4
124
3
0
0
1
6
0
0
0
0
1
1
0.734 (0.5051- 0.9629)
Substantial agreement
Fistula
0
1
135
0
0
1
1.00 (1-1)
Perfect agreement
Abscess
0
1
132
2
1
1
0.389 (0-1)
Fair agreement
Pufa
0
1
2
5
122
4
0
0
2
4
0
0
0
0
3
1
0.661 (0.4172- 0.9058)
Substantial agreement
Table 4: Intra-observer kappa statistics of primary dentition for both days and for observer 1 and 2.
For both primary and secondary dentations, no sign of ulceration was found by the both observers in any individual thus kappa was not calculated for this component of score.
The prevalence of PUFA >0 for the first and second day was 26.1%, 29.6% and for primary dentition the prevalence of pufa>0 for the first and second day was 13.1% and 11% respectively and the mean scores for PUFA for the first and second day were 0.38 (SD: 0.38) and 0.42 (SD: 0.84) respectively and for primary dentition the mean scores for pufa for the first and second day were 0.19 (SD: 0.59) and 0.18 (SD: 0.60) respectively.
Discussion
A substantial decline in caries has been seen in many developed countries and one of the important reasons behind this decline is the increased number of usage of more advanced tools to distinguish between different stages of dental caries in order to diagnose the initial stages of caries as well. On the other hand, countries with low and average incomes are still facing a high burden of untreated caries. One of the major reasons is the paucity in diagnosis of early caries lesions which further leads to delay in a proper treatment and management at early stages [13-16]. In the last few decades, index PUFA/pufa has shown reliable outcomes with universally acceptable results for tooth decay and its severity. It is considered an easy and safe tool that can be used by non-dental professionals for quick and dependable scores [7].
In this study, Kappa statistics were calculated for inter and intraexaminer reliability of PUFA/pufa and PUFA/pufa scores for the same day and for day 1 and 2 in permanent and primary dentations. For day 1and 2, a substantial to moderate agreement was found among two observers for different units of PUFA/pufa scale and for overall scales for primary and permanent dentitions. Furthermore, the intra-reliability was calculated for permanent and primary dentitions for day 1 and 2. For the first observer, it was found as a substantial agreement of marking pulpal involvement, fistula, abscess and overall PUFA score for both days. For the second observer, an almost perfect agreement was found of marking pulpal involvement and overall PUFA score. Intra-reliability for primary dentition for day 1 and 2 for the first and second observers was calculated and found that for the first observer kappa statistics had a moderate agreement for marking pulpal involvement and overall pufa scores for both days. For this condob server, we found a substantial agreement of marking pulp and overall pufa score. However the wide CIs limits conclusion of the findings.
This study is considered the first study which has been conducted in Saudi Arabia to assess the reliability of such tool in this region. As Saudi Arabia is facing a huge burden of dental caries, advanced tools like PUFA/pufa are much needed to detect early and advance stages of caries. Thus, it was aimed to conduct this study in a single setting to check the feasibility of such tool in this context. A fairly positive result was found with a perfect to moderate agreement for individual items and for overall scores for both primary and permanent dentations. The agreement between the two examiners was also good enough for individual items and for overall score for both dentitions and for day 1 and 2 as well.
Inter and intra reliability assessments for day 1 and day 2 in both dentitions have highlighted the true functionality and reliability of this scale. However the wide CIs limits conclusion of the findings. Moreover, due to the paucity of studies whether locally or globally, making a comparison between these findings and the findings of other studies difficult. Thus, further studies are strongly recommended to support the findings of this study. Additionally, this study is considered as a first type of its nature, as it was conducted in single setting with a limited number of children. As a result of that, the findings are not generalizable for that purpose but still this study succeeded in providing preliminary results for future studies that will be conducted in a similar area.
Conclusion
Through this study we conclude that PUFA scale is reproducible and functional in determining burden of untreated caries in both primary and secondary dentition as results of over study indicate perfect to fair agreement of inter and intra examiner reliability. Further multicenter studies are recommended to support such findings.
References
- Petersen PE. The World Oral Health Report 2003: continuous improvement of oral health in the 21st century–the approach of the WHO Global Oral Health Programme. Community Dent Oral Epidemiol. 2003; 31: 3-24.
- World Health Organization. Oral Health: action plan for the promotion and integrated disease prevention. Sixtieth World Health Assembly. WHA60.17. Agenda item. 12.9.2007.
- Mehta A. Comprehensive review of caries assessment systems developed over the last decade. RSBO. 2012; 9: 316-321.
- Klein H, Palmer CE. Studies on dental caries: V. Familial resemblance in the caries experience of siblings. Public Health Reports. [1896-1970] 1938; 5: 1353-1364.
- Broadbent JM, Thomson WM. For debate: problems with the DMF index pertinent to dental caries data analysis. Community Dent Oral Epidemiol. 2005; 33: 400-409.
- Hiremath SS. Indices. In: Hiremath SS. Textbook of preventive and community Dentistry. 2 edn. India. Elsevier. 2011; 198-221.
- Monse B, Heinrich‐Weltzien R, Benzian H, Holmgren C, van Palenstein Helderman W. PUFA–An index of clinical consequences of untreated dental caries. Community Dent Oral Epidemiol. 2010; 38: 77-82.
- Mehta A, Bhalla S. Assessing consequences of untreated carious lesions using pufa index among 5-6 years old school children in an urban Indian population. Indian J Dent Res. 2014; 25: 150-153.
- Oziegbe EO, Esan TA. Prevalence and clinical consequences of untreated dental caries using PUFA index in suburban Nigerian school children. Eur Arch Paediatr Dent. 2013; 14: 227-231.
- Shanbhog R, Godhi BS, Nandlal B et al. Clinical consequences of untreated dental caries evaluated using PUFA index in orphanage children from India. J Int Oral Health: JIOH. 2013; 5: 1-9.
- Figueiredo MJ, de Amorim RG, Leal SC, Mulder J, Frencken JE. Prevalence and severity of clinical consequences of untreated dentine carious lesions in children from a deprived area of Brazil. Caries Res. 2011; 45: 435-442.
- Leal SC, Bronkhorst EM, Fan M et al. Untreated cavitated dentine lesions: impact on children’s quality of life. Caries Res. 2012; 46: 102-106.
- Nyvad B, Machiulskiene V, Bælum V. Reliability of a new caries diagnostic system differentiating between active and inactive caries lesions. Caries Res. 1999; 33: 252-260.
- Ismail AI, Sohn W, Tellez M, Amaya A, Sen A, Hasson H. The International Caries Detection and Assessment System [ICDAS]: an integrated system for measuring dental caries. Community Dent Oral Epidemiol. 2007; 35: 170-178.
- Whelton H. Overview of the impact of changing global patterns of dental caries experience on caries clinical trials. J Dent Res. 2004; 83: 29.
- McGinn T, Wyer PC, Newman TB, Keitz S, Leipzig R, Evidence-Based Medicine Teaching Tips Working Group. Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic). CMAJ. 2004; 171: 1369-1373.
Download PDF