Identifying Performance Problems Early In Residency

Keywords

Objective: The aim of this study is to compare oral health attitudes and behaviors between preclinical and clinical dental students.

Resident Education; Performance Problems; Remediation; Performance Evaluations

Editorial

The majority of residents will progress through residency, advancing from year to year without significant roadblocks or remediations. They will graduate and go on to practice as competent physicians. Unfortunately, a minority will be promoted and graduated despite persistent and potentially dangerous performance problems [1]. We suspect that this kind of event will be familiar to most, if not all, residency programs in most, if not all, specialties. It is our contention that many residency programs fail to identify these problems early and therefore are less successful in their remediation. In this perspective article, we describe strategies designed to specifically identify performance problems early in residency.

For our discussion, we define a resident with a performance problem as a resident noted by more than one faculty for negative reasons [2,3]. Performance problems can occur in medical knowledge, professionalism, clinical practice, or a combination of any of those. Such performance problems should trigger a feedback session, remediation, or a performance improvement plan. The resident should be made responsible for achieving timed milestones necessary for advancement and graduation. Such a plan should contain offers of resources to help the resident, along with increasingly authoritarian interventions when there is lack of acceptable progress [4]. It should be clear from the start that the resident is responsible for resolving his or her own performance issue. Residents are highly motivated to succeed and typically take responsibility for their performance. Performance issues are most often successfully addressed with a good performance improvement plan. Unfortunately, some performance problems are refractory and of sufficient severity to prevent promotion or graduation.

A significant number of performance issues are either not recognized early in residency, or are inadequately addressed [1]. A program director is then faced with the agonizing decision of what to do with an upper level resident with a refractory performance problem. Lingering doubts then arise about the resident's fitness for independent practice. There is often regret that the problem was not identified earlier, or if identified, not adequately addressed. There are a number of pressures to continue to promote and ultimately graduate a resident with refractory performance issues including the possibility of extra clinical duties distributed among remaining residents or faculty, fear of litigation, misguided mercy, and a sense of failure among faculty educators. In addition, unlike other work place situations, residents are relatively short-term employees and a program director will only need to deal with a performance issue for a finite period of time.

There can be emotional as well as practical costs associated with dismissing a resident from a program. Resident dismissals can affect class or program morale as the cohort "loses one of its own." Often times the remaining residents face a heavier clinical load to staff the gaps in coverage. However, there is also a cost of allowing a resident with significant refractory performance problems to graduate [2]. First and foremost there is a cost to patients - both those the residentcares for in residency and those cared for during the rest of his or her career. Time spent by faculty dealing with a refractory performance issue is much better spent in positive educational activities with the rest of the residency group. An inadequately addressed performance issue sends the message to other residents and faculty that certain substandard performance is, in fact, acceptable, thereby effectively normalizing deviance. Finally, a program director does no favor to a resident by allowing him or her to continue with unacceptable performance, resulting in eventual graduation without the necessary skill set for success.

The earlier those performance problems are identified, the earlier a performance improvement plan can be implemented and the earlier a severe refractory performance problem can be diagnosed. Early diagnosis of, and intervention taken toward, a refractory performance problem can prevent much of the cost of protracted indecision. What follows are strategies for the early detection of performance problems early in residency.

Early Evaluations

Most significant performance problems can be detected within the first six months of residency [1]. Strategies that we employ to diagnose performance problems early include:

An initial orientation month

Bringing all residents together for their first month, in their core specialty, with increased contact with the program director, associate program director and key faculty, increases the opportunity to diagnose any performance problems. In our Emergency Medicine residency, first year residents only rotate four months of their first year in the Emergency Department. Therefore, actual time with key faculty is inconsistent and spread throughout the year. During our orientation month model, the program director and key faculty now have the opportunity to observe and evaluate all the new residents in the clinical setting. In addition to the clinical activities, key faculty work collaboratively to evaluate residents in small groups, procedural skill labs and in simulation encounters.

Intern assessment exams utilizing standardized patients

Each resident is observed and evaluated by faculty in five separate patient scenarios with standardized patients at the beginning of residency. The residents are evaluated by observing faculty who provide immediate feedback. The standardized patients also provide feedback on the residents. These interactions provide unique insights into bedside manner, professionalism, and prior clinical experience.

Early verification of proficiencies for procedural skills

All new Emergency Medicine residents must complete verification of proficiencies (VOPs) in key procedures during their first month in the skills lab. These skills include intubation, chest tube placement, surgical airway, arterial line placement and ultrasound guided central venous line placement. These skill sessions are taught by key faculty who evaluate each resident and verify proficiency. Completion of verification of proficiency is necessary before the resident can perform the procedure under faculty supervision in the clinical setting. These faculty-intensive sessions during the first month increase the opportunity to diagnose any performance problem.

Front-loaded formal evaluations by program director

All residents have a formal evaluation with the program director after their first month. Performance in the first month is summarized and if there is a performance problem identified, a performance improvement plan is implemented with at least monthly evaluations until the performance problem is resolved. If no performance problem is identified at the one month evaluation, the next formal evaluation by the program director is at three months. If at the three month evaluation a performance problem is identified, a performance improvement plan is implemented with at least monthly evaluations until the performance problem is resolved. If no performance problems are identified at the three month evaluation, the next evaluation by the program director occurs in three months. If there is a performance problem identified, evaluations by the program director are then every 6 months. It is our opinion that valuable time may be wasted in diagnosing and addressing a performance issue by waiting for an initial semi-annual formal evaluation.

Direct Observation for chief complaints

Each resident is directly observed and evaluated by faculty as they evaluate three patients presenting with separate chief complaints. These experiences result in immediate formative feedback and summative comments for their files. These direct observations have the potential to illuminate several performance issues. Our recommendation is to complete several in the first six months of the intern year.

Multiple Evaluations and Multiple Evaluators

Reliable resident evaluation requires multiple evaluations [5]. Multiple evaluators are also preferable. End-of-shift evaluations are completed by faculty for our residents during their emergency department rotations. We obtain an average of fourteen evaluationsper month, for each resident, from five to ten unique evaluators. Evaluations rank the resident's performance during that shift using the six ACGME competencies on a one to eight numerical scale. In addition, there is a comment section and all faculty members are encouraged to document comments with recommendations for improvement. As faculty rate performance, the supervisor's entrustment decisions on the resident's clinical autonomy follow from his or her independently verifiable judgment of trainee competence [6]. Whenever faculty members mention a resident performance concern to the program director, they are requested to document their concern in the end of shift evaluations and discuss their concern with the resident. We stress to our faculty that they are evaluating potential future colleagues and their input is vital for the success of the program and our learners.

Addressing All Concerns

Any concerns about resident performance are shared with the resident, even if the concern is coming from a single source. We feel this transparency is important to let the resident know the perception of his or her performance. Any concern from two or more sources is taken very seriously and results in a separate meeting between the program director and resident. The faculty perception of a performance problem is shared with the resident and he or she is asked if he or she knows why the evaluators may have this perception. The response by the resident is very important. Does the resident have any insight into the performance problem? Is the resident quick to blame others or circumstances for any perception of a problem? Lack of any insight by the resident, and/or failure to take responsibility for a potential performance problem, are indicators the performance problem may be more resistant to resolution. The resident is asked what he or she can do to resolve the perception of a performance problem. This practical use of facilitative feedback involves the provision of comments and suggestions to facilitate recipients in their own revision [7]. A performance improvement plan is then crafted and implemented.

It could be argued that reacting to a concern, even from a single evaluator, may result in overreaction to non-existent problems. Many repeated such concerns may make a resident feel micromanaged or become less responsive to important feedback. Assuming there is a significant problem when none exists would be analogous to making a type I statistical error. However, we feel that missing a significant problem by not acting on a faculty concern is a more serious risk, analogous to a type II statistical error. In general, faculty do not share concerns about residents lightly and when they do, the concerns should be taken seriously. Actively addressing small problems early may prevent them from getting larger. In addition, we feel residents are owed the courtesy of knowing the perception of their performance.

Early Performance Improvement Plans

Performance plans are developed at the time of program director evaluation. To be effective, performance improvement plans must have a clear performance goal, measurement of improvement, time frame for improvement, and consequences of not improving. By learning about their own abilities through external feedback, learners are able to improve their self-monitoring [7]. One tendency is to confuse a clear performance goal with objectivity of measurement. A performance goal could be "no more concerns from faculty about a perceived performance problem." This is a subjective measurement but a defined goal. The resident should not be left with any ambiguity about the seriousness of a complaint. If there is a faculty concern about unprofessional behavior for instance, the resident must know that a recurrence of this concern is not acceptable. The resident must know the consequences should a performance issue persist.

All residents (and faculty) have some area that needs improvement. Performance improvement plans need not be reserved for problems that potentially result in failure to advance. Essentially all residents will have some sort of performance plan that is revisited in future evaluations. Serious performance problems however require formal plans with unambiguous consequences if the performance issue is not resolved. Early initiation of formal performance improvement plans for potentially serious problems allow for earlier diagnosis of refractory issues and an earlier resolution should promotion or continuation in residency is inadvisable.

Conclusion

Most serious resident performance problems are identifiable early in residency. We have presented some strategies for identifying and addressing such problems. The earlier these performance problems are identified and addressed, the earlier the refractory and/or serious performance problems can be diagnosed. Emphasis is placed on multiple evaluations from multiple sources and the implementation of performance improvement plans early in residency.

References

1. Williams RG, Roberts NK, Schwind CJ, Dunnington GL. The nature of general surgery resident performance problems. Surgery. 2009; 145: 651-658.
2. Roberts NK, Williams RG. The hidden costs of failing to fail residents. J Grad Med Educ. 2011; 3: 127-129.
3. Gordon MJ. A prerogatives-based model for assessing and managing the resident in difficulty. Fam Med. 1993; 25: 637-645.
4. Hickson GB, Pichert JW, Webb LE, Gabbe SG. A complementary approach to promoting professionalism: identifying, measuring, and addressing unprofessional behaviors. Acad Med. 2007; 82: 1040-1048.
5. Williams RG, Verhulst S, Colliver JA, Dunnington GL. Assuring the reliability of resident performance appraisals: more items or more observations? Surgery. 2005; 137: 141-147.
6. Cianciolo AT, Kegg JA. Behavioral specification of the entrustment process. J Grad Med Educ. 2013; 5: 10-12.
7. Archer JC. State of the science in health professional education: effective feedback. Med Educ. 2010; 44: 101-108.