Research Article
Austin J Emergency & Crit Care Med. 2015;2(1): 1009.
Pediatric Residents’ Critical Care Training and the Challenge of the New Accreditation Council for Graduate Medical Education Guidelines
Megan Barry1, Kristin Koehn2, Esma Birisci3, Craig Downs2 and Abdallah Dalabih2*
1Department of Pediatrics, Child Neurology, Vanderbilt University, USA
2Department of Child Health, University of Missouri, USA
3Department of Industrial and Manufacturing System Engineering, University of Missouri, USA
*Corresponding author: Abdallah Dalabih, Department of Child Health, University of Missouri, 400 N Keene Street, Columbia, MO 65201, USA
Received: December 02, 2014; Accepted: January 02, 2015 Published: January 05, 2015
Abstract
Objective: Our objective was to explore the difference in confidence levels reported in pediatric residency program directors’ evaluations compared to pediatric residents’ self-evaluations with regard to performance of eight critical care related skills. Our secondary objective was to determine if spending extra time in a critical care setting had an effect on the residents’ confidence level and perceived competency in those skills.
We elected to perform this survey prior to the adaptation of the new Accreditation Council for Graduate Medical Education (ACGME) recommendations in regard to the number of educational units (four week blocks) spent in a pediatric intensive care unit (PICU) setting.
Methods: We sent an online survey questioning resident exposure to critically ill children and the skills gained during residency to all United States pediatric program directors (PD). The elective anonymous survey targeted program directors and resident physicians in ACGME accredited pediatric training programs. Responses were collected over a two month period. We performed a Chi- Square analysis using IBM SPSS 22.0 and set a significance level of 0.05. The 95% confidence interval was calculated at 95%.
Results: A total of 547 PRs and 124 (49%) PDs responded. Of those, 352 residents had spent at least three months in a critical care setting. Overall, resident confidence in their competency in all categories was lower than the opinion of PDs. Residents required to provide critical care coverage outside of a structured rotation had a significantly higher level of confidence in five of the eight surveyed skills.
Conclusions: Although the PDs had higher confidence in their residents’ competency than the residents’ self evaluations, the survey showed that residents with more exposure to critical care settings had a higher level of confidence in their competency level than those with less exposure. The new ACGME regulations may decrease the critical care training time; we suggest that less time in those settings is unlikely to raise these competencies or the residents’ self-confidence.
Keywords: Pediatric Residency; Critical care; Education; ACGME
Abbreviations
ACGME: American Council of Graduate Medical Education; NICU: Neonatal Intensive Care Unit; PICU: Pediatric Intensive Care Unit; PR: Pediatric Resident; PD: Program Director
Introduction
In the United States, resident physicians are required to complete an Accreditation Council for Graduate Medical Education (ACGME) approved training program before independently practicing medicine. ACGME accredited programs adhere to a set of common training requirements outlining specific numbers of rotations in multiple areas, including medical subspecialties, inpatient medicine, intensive care, emergency medicine, and ambulatory care. The remainder of the required rotations can vary between programs based
on educational exposure and staffing of various services. This model leaves little room for personalized education and creates a “one size fits all” model, rather than focusing on areas of interest to the trainee.
The newly revised ACGME requirements, effective July 1, 2013, were designed to address the need for more individualized training. These new requirements give residents more emphasis on developing tracks of training within each program to better prepare them for the next step after residency. The new requirements decreased the minimum units of education in the neonatal intensive care unit (NICU) and the pediatric intensive care unit (PICU) to two units each [1]. This is an overall change from the previous requirement of five units total between the different intensive care units.
Pediatricians work hard to establish good rapport with families and to build relationships based upon trust. Due to this relationship, children with urgent medical needs often present to the pediatric office first [2]. Up to 70% of pre-hospital healthcare professionals practice in rural settings [3]. In rural America, pediatricians play an important role in providing care to a large population and may be the only provider with any pediatric experience, including pediatric critical care experience. Rural hospitals often lack support staff that are equipped to manage a critically ill pediatric patient. It is important for a pediatrician to possess the necessary procedural skills and confidence to stabilize an acutely ill patient prior to transfer to a tertiary care center. Many of the skills needed to stabilize an acutely ill child can be learned and practiced in the intensive care setting, which for many pediatricians occurs only during residency.
The goal of our study was to elicit the opinion of current pediatric residents (PR) and pediatric program directors (PD) regarding the level of confidence in procedural competency gained during residency, prior to adapting the new ACGME recommendations. Residents often gain the skills of recognizing critically ill patients and procedural skills during the time spent in the intensive care units. Therefore, our secondary objective was to determine if spending extra time in a critical care setting had an effect on the residents’ confidence level and perceived competency in those skills. We hypothesis that spending less time in a critical care setting will decrease the selfperceived confidence in recognizing and managing critically ill children.
Methods
We designed two online surveys that were approved by our institutional review board (IRB). The surveys consisted of fifteen questions each, including eight questions aimed at identifying the residents’ exposure to critically ill children and their perceived competency level (by the time of graduation) in those skills, from the perspectives of both the PRs and the PDs. The additional questions included the level of training, the number of months of NICU and PICU experience the respondent was to complete by the end of residency, and the number of months of intensive care training the resident had completed prior to answering the survey. We also asked if the residents had any exposure to critically ill children outside of a formal rotation in the critical care setting (e.g. night shift or call coverage). Confidence in competency level in eight skills and procedures was surveyed. To have a better understanding of the responders’ confidence in their ability to perform the skills, we did not give any examples of what determined competency in any skill or procedure. It was up to the respondent to judge his or her own confidence level in performing the surveyed skills.
The email addresses of all United States pediatric and medicinepediatric PDs were obtained from the American Medical Association’s FREIDA online. A link to the survey was included in an email to the PDs along with an explanation of the study. The PDs then made the decision whether or not to forward the survey to their residents. A second reminder email was sent to all program directors two weeks after the initial survey was delivered.
Our primary objective was to explore the difference in the confidence level perceived by pediatric residency program directors and by pediatric residents’ self-evaluation in performing eight critical care related skills. Our secondary objective was to determine if spending extra time in a critical care setting had an effect on the residents’ confidence level and perceived competency in those skills. The data was collected over a period of two months; only residents who spent three or more months in a critical care setting were included in the final analysis, as well as all of the PDs. For the statistical analysis we used a Chi- Square analysis run with IBM SPSS 22.0. A p-value of = 0.05 was considered statistically significant and a 95% confidence interval was also calculated for all of the data points.
Results
Of the 252 residency programs that were contacted a total of 125PDs (49.6%). A total of 547 PRs responded to the survey. It is unknown how many PRs received the survey from their program directors. The responses of residents with three or more months of critical care experience (n=352) at the time of the survey were used for the study. PDs thought residents were more competent than the residents’ self-reported competency in all categories surveyed (Table-1), and this difference was statistically significant.
Surveyed Skill
Program Director (N=125)
Pediatric Resident (N=352)
p-value
95% CI
Identifying critically ill patient
124 (99.2%)
334 (94.9%)
0.034
0.021 0.048
Initiating proper work up plan for critically ill children
122 (97.6%)
277 (78.7%)
< 0.001
0.159 0.208
Interpreting Chest X-Rays
115 (92%)
276 (78.4%)
0.001
0.104 0.169
Establishing vascular or intra-osseous Access
50 (40%)
56 (15.9%)
< 0.001
0.194 0.290
Recognizing the difference between respiratory distress and respiratory failure
118 (94.4%)
265 (75.3%)
< 0.001
0.157 0.218
Intubation
62 (49.6%)
< 0.001
0.135 0.238
Adjusting mechanical ventilation settings according to blood gases
92 (73.6%)
188 (53.4%)
< 0.001
0.162 0.257
Managing cardiopulmonary arrest
86 (68.8%)
87 (24.7%)
< 0.001
0.396 0.491
(Table-1) The self-perceived competency level of pediatric residents was lower than the program directors’ evaluation of the residents’ competency in critical care skills in all of the eight surveyed skill categories, and that was statistically significant. The resident group in this comparison had spent a minimum of three months in a critical care setting.
Table 1: A comparison between the self-perceived level of competency of residents compared to the program directors’ evaluation.
We further compared the confidence level of the same competencies between residents who provided critical care coverage outside of a structured critical care rotation with those who did not. Although both groups had spent at least three months in critical care setting, the first group had a statistically higher level of confidence in their own competency level in five of the eight surveyed categories (Table-2).
Surveyed Skill
Residents with extra critical care coverage (N=225)
Residents with no extra critical care coverage (N=127)
p-value
95 % CI
Identifying critically ill patient
217 (96.4%)
117 (92.1%)
0.077
-0.009 0.095
Initiating proper work up plan for critically ill children
181 (80.4%)
96 (75.6%)
0.286
-0.042 0.139
Interpreting Chest X-Rays
186 (82.7%)
90 (70.9%)
0.01
0.013 0.198
Establishing vascular or intra-osseous Access
42 (18.7%)
14 (11%)
0.06
0.002 0.151
Recognizing the difference between respiratory distress and respiratory failure
178 (79.1%)
88 (69.3%)
0.039
-0.002 0.190
Intubation
82 (36.4%)
27 (21.3%)
0.003
0.058 0.248
Adjusting mechanical ventilation settings according to blood gases
134 (59.6%)
54 (42.5%)
0.002
0.062 0.277
Managing cardiopulmonary arrest
67 (29.8%)
0.003
0.055 0.229
(Table-2) The self-perceived competency level of pediatric residents who have critical care coverage outside of structured critical care rotations in eight different skill categories. The resident group, who covered critical care units outside of structured rotations and spent a minimum of three months in a critical care setting, had a statistically significant higher self-perceived competency level in five of the eight surveyed categories than residents who did not have extra coverage and spent at least three months in a critical care setting.
Table 2: A comparison between the self-perceived level of competency between residents who had extra critical care coverage outside of a structured rotation, and those who did not.
When comparing residents who spent three or months of critical care rotations to those who had less than three months; residents who had longer critical care experience had a statistically significant higher level of confidence in all investigated eight categories (Table-3).
Surveyed Skill
Residents with less than 3 months ofcritical care experience
(N=197)Residents with 3 or more months of critical care experience
(N=352)p-value
95% Confidence Interval
Identifying critically ill patient
106 (53.8%)
334 (94.9%)
0.000
0.33% - 0.48%
Selection of the appropriate initial work up in a critically ill patient?
44 (22.3%)
277 (78.7%)
0.000
0.48%-0.63%
Reading chest x-rays?
82 (42.4%)
276 (78.4%)
0.000
0.28%-0.44%
Establishing access (IV, IO, central, etc) ?
13 (6.6%)
56 (15.9%)
0.002
0.038%-0.14%
Recognizing the difference between respiratory failure and respiratory distress?
61 (31%)
265 (75.3%)
0.000
0.36%-0.52%
Intubation Intubating patients?
12 (6.1%)
109 (30.9%)
0.000
0.19%-0.31%
Adjusting mechanical ventilator settings according to blood gases?
28 (14.4%)
188 (53.4%)
0.000
0.32%-0.46%
Managing cardiopulmonary arrest?
7 (3.6%)
87 (27.7%)
0.000
0.16%-0.26%
(Table-3) The self-perceived competency level of pediatric residents who have three or moremonths of critical care experience was higher in all eight different skill categories, and that difference was statistically significant.
Table 3: A comparison between the self-perceived level of competency between residents who had less thanthree months of critical care coverage, those who did.
When asked if they believed PRs were exposed to enough critically ill patients throughout residency, 93% of PDs and 86% of PRs answered affirmatively. The responding program directors’ program sizes are shown in Table-4.
Number of residents
number of responding program directors
Less than 5
4 (3.2%)
5-10
34 (27.2%)
11-15
41 (32.8%)
16-20
16 (12.8%)
21-25
10 (8%)
more than 26
20 (16%)
(Table-4) The responding residency program directors’ program sizes varied widely. The majority of responding program directors had between 11-15 residents per class.
Table 4: The size of the residency classes of the responding residency program directors.
Discussion
The ACGME aims to personalize residency training, making the PRs better prepared for general and subspecialty practice [1]. The skill set acquired through critical care training is essential for every practicing pediatrician, regardless of his or her desired practice setting. This survey demonstrates a gap between the PDs’ and PRs’ perception of the residents’ competency level in the surveyed categories. This may either suggest that PDs overestimate the competency level of their residents, or that PRs underestimate their competency level due to the expected lack of confidence during training. Multiple studies have shown that physicians have a limited ability to adequately self-assess, which may explain the gap in perceived competency level between the program directors and the residents [4,5]. Because self-assessment may not be accurate when compared to the PD observations, we compared the difference between two resident groups: a group that provided extra critical care coverage and a group that did not. The survey showed a statistically significant difference in self-assessed competency with increased exposure to critically ill patients.
When asked about their residents’ competency in managing cardiopulmonary arrest, only 69% of PDs thought the residents were competent in this skill, which is consistent with previous studies [6]. The survey exposed a deficiency in residents’ self confidence in their critical care competency level; only 25% of surveyed residents felt competent in managing a cardiac arrest.
Our study is limited in that the number of residents who received the survey is unknown. Due to privacy concerns and regulations, our IRB did not approve directly contacting the residents, thus we had to depend on the PDs’ willingness to forward the survey to their residents. The survey was also limited due to the subjectivity of the competency level, depending on the residents’ own confidence in their skills. We think that subjectivity is important, because it highlights the residents’ confidence that can only be built by exposure and constant practice.
Our survey was designed to examine and address opinions of overall confidence in residents’ competency in key procedural skills in the intensive care setting under the previous ACGME common program requirements. This study was not designed to provide solutions for those perceived competency deficiencies, but we suggest that less time in critical care (e.g. NICU, PICU) settings is unlikely to raise these competencies or the residents’ self-confidence, as it was obvious that the longer time spent in critical care settings was associated with higher level of self-confidence. Future studies should be conducted to examine possible solutions in order to improve competency in critical care skills needed for future practicing pediatricians. Future studies and surveys should be repeated to examine the effect of the ACGME guidelines (after being implemented for few years) on the residents’ self-perceived confidence.
References
- Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in pediatrics. 2012.
- Flores G, Weinstock DJ. The preparedness of pediatricians for emergencies in the office. What is broken, should we care, and how can we fix it? Arch Pediatr Adolesc Med. 1996; 150: 249-256.
- Moore B, Sapien R; Committee on Pediatric Emergency Medicine. The role of the pediatrician in rural emergency medical services for children. Pediatrics. 2012; 130: 978-982.
- Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA. 2006; 296: 1094-1102.
- Johnson D, Cujec B. Comparison of self, nurse, and physician assessment of residents rotating through an intensive care unit. Crit Care Med. 1998; 26: 1811-1816.
- Gaies MG, Landrigan CP, Hafler JP, Sandora TJ. Assessing procedural skills training in pediatric residency programs. Pediatrics. 2007; 120: 715-722.
Download PDF