Evaluation of the residents’ progress toward competence as independently practicing neurologists is an extremely important component of residency training. Similarly, the residency program benefits from resident feedback regarding the program generally and each faculty member. Our program employs an array of evaluation methods as described below.
At the end of each rotation, supervising faculty complete electronic surveys via a website called MedHub. The evaluations include both narrative comments and quantitative assessments of the degree to which the faculty trust residents to carry out certain key responsibilities in each rotation. These key responsibilities are called entrustable professional activities (EPAs). For example, a stroke service EPA is “Can the resident respond to a stroke code and differentiate a true stroke from a stroke mimic?”
Faculty grading of entrustment is then mapped to the various milestones based on the method published by the University of Washington’s internal medicine residency. For example, differentiating a true stroke from a mimic involves history-taking, neurological exam, knowledge of cerebrovascular disease, skill at imaging interpretation, etc. The neurology milestones and the abbreviations used when mapping them are shown in the table below. Please see the individual rotation goals and objectives for details of how each rotation’s EPAs are mapped to the associated milestones.
Note that early in training, it is expected that residents will need greater supervision and that the faculty will trust the residents to function more independently over time as they demonstrate the skills necessary for independent practice. Therefore, junior residents, even very talented ones, will have relatively low EPA and milestone scores as compared to senior residents. This is a significant departure from most undergraduate programs, where grades are referenced to certain norms (e.g., 15% A’s, 50% B’s, etc.) rather than to specific outcomes (e.g., ability to perform an acid-base titration). Junior residents should understand that a relatively low score on the milestones is not the same as a relatively low grade.
|PC2||Neurological Exam||Patient Care|
|PC4||Outpatient Neurology||Patient Care|
|PC5||Inpatient Neurology||Patient Care|
|PC6||Neurologic Emergencies||Patient Care|
|PC7||Brain Death||Patient Care|
|PC12||Psychiatric / Functional||Patient Care|
|MK2||Diagnostic Investigation||Medical Knowledge|
|SP1||Patient Safety||Systems-based Practice|
|SP2||Quality Improvement||Systems-based Practice|
|SP3||System Navigation for Patient-Centered Care||Systems-based Practice|
|SP4||Physician Role in Health Care Systems||Systems-based Practice|
|PL1||Evidence-Based and Informed Practice||Practice-based Learning and Improvement|
|PL2||Reflective Practice and Commitment to Personal Growth||Practice-based Learning and Improvement|
|PR1||Professional Behavior and Ethical Principles||Professionalism|
|PR2||Accountability / Conscientiousness||Professionalism|
|IC1||Patient and Family-Centered Communication||Interpersonal and Communication Skills|
|IC2||Barrier and Bias Mitigation||Interpersonal and Communication Skills|
|IC3||Interprofessional and Team Communication||Interpersonal and Communication Skills|
|IC4||Communication within Health Care Systems||Interpersonal and Communication Skills|
Residents’ professionalism and interpersonal and communication skills are evaluated via electronic survey by patients, other resident colleagues, medical students, and a wide variety of co-workers. This latter category includes nurses, therapists, social workers, case managers, neurodiagnostic technicians, and schedulers.
Residents are directly observed to take histories, examine patients, and formulate cases during Professor Rounds. These are held twice a month, usually on Fridays at 12:00. These exercises allow residents to practice, under the direct observation of our faculty and without the time pressure of a busy clinical service, the fundamental clinical skills of neurology.
Clinical Skills Exams
Residents participate in formal clinical skills exams (NEX), which have replaced the oral board examination previously needed for ABPN certification. All residents are required to pass 5 clinical skills exams for patients unknown to the resident and from the following categories:
- Adult ambulatory
- Adult neurodegenerative
- Child ambulatory
- Critical care
The exams will be conducted by core residency faculty or other active faculty who are board-certified by the ABPN. Residents are expected to have 1 NEX completed by the end of PGY-2, and 2 additional NEX completed by the end of PGY-3. Residents must complete all 5 NEX by December 31 of their PGY-4 year.
Brain Death Exams
Residents must be directly observed in the performance of one brain death exam (see the supervision policy above). This helps to ensure competence in a singularly important neurological task and provides another opportunity for faculty to give direct feedback regarding examination technique generally. The brain death exam may also be used to satisfy the neurocritical care clinical skills exam requirement.
Residents’ general neurological knowledge is assessed each spring via the Residency In-service Training Exam (RITE). Time off from clinical duties and call is provided for the residents in order to take the examination. An additional self-assessment exam sponsored by the American Association of Neuromuscular & Electrodiagnostic Medicine is optional.
Several weeks after each exam is completed, a detailed scoring sheet is provided along with an assessment of where each resident stands compared to peers across the country. The scoring sheets provide sub scores for each subspecialty area of neurology, so that residents can make an educational plan for future learning directed towards specific areas. The exam results are reviewed in the semi-annual evaluation with the program director, but are not used for decisions regarding promotion or graduation.
If a resident is unable to take the RITE, or scores below the 50th percentile, he or she will be asked to complete additional standardized tests, such as the American Academy of Neurology’s Self Assessment Exams (NeuroSAE).
In preparation for the semi-annual Clinical Comptenency Committee meeting and subsequent review with the program director, each resident is asked to complete a self-assessment based on their end-of-rotation and other feedback. This is used to develop an individualized learning plan, help the resident keep track of various requirements such as QI projects, and guide the subsequent discussion with the program director.
At mid-year and year-end, the residency program’s Clinical Competency Committee (CCC) will meet to evaluate each resident’s performance across the various core competencies and milestones; see the CCC description above. In these meetings, the CCC will recommend one of the following actions:
- No action (for mid-year residents in good standing)
- Promotion to the next PGY level (or graduation from the program)
- Promotion to the next PGY level contingent upon remediation
- Remediation without promotion
- Termination or contract non-renewal
Subsequent to the CCC meeting, the program director will meet with each resident to discuss his or her progress individually. The resident’s self-assessment will be reviewed, along with the various evaluations detailed above and the CCC’s report. The template for the semi-annual meeting with the program director can be found here.
Specific Graduation Targets
- Five Clinical Evaluation Exercises:
- Child neurology
- Critical care
- At least three lumbar punctures documented in MedHub
- At least one brain death exam documented in MedHub
- At least 100 EMG reports documented in MedHub
- At least 50 EEG reports documented in MedHub
- Patient safety / quality improvement—at least:
- One M&M presentation and
- One quality improvement project
- Scholarly activity—at least:
- One journal club and
- One grand rounds
Final (Summative) Evaluation
In addition to the usual six-month evaluation for the final six months of training, an additional form is completed by the Program Director attesting that the resident has “demonstrated sufficient competence to enter practice without direct supervision”.
All reports, evaluations, and correspondence will become part of the resident’s permanent personnel file.
Anonymous Resident Feedback
Feedback regarding the residency program and its faculty is derived from several sources. Each year, residents and fellows complete via MedHub an anonymous survey. Ratings and comments are not linked to individual residents, and all adult neurology resident, pediatric neurology resident, and fellow surveys are combined to ensure anonymity. The survey asks about:
- The program. This includes an overall rating and ratings of the clinical, educational, and scholarly components of the program, as well as the facilities and the culture. Each question has a free text comment field as well.
- The rotations. Every rotation is listed, along with a comment field.
- The people. This includes an evaluation of the faculty as a whole, the program director and coordinator, and each attending physician. Again, there’s a comment field for each one.
In addition, as part of the Program Evaluation Committee’s Annual Program Evaluation, the chief resident, in conjunction with the residents, presents a report similar to the above survey, highlighting and expanding on whichever aspects the residents believe deserve the most attention.
Each month, the residents have a Wellness and Administrative Business meeting with the program director, coordinator, and selected associate program directors. This serves as a venue for continuous dialogue among residents and program leadership regarding the structure of the residency program, issues with specific rotations and faculty, and the status of the various program improvement initiatives that are underway.
Twice each year, each resident meets with the program director to discuss their progress through the program, career goals, and other resident-specific matters. In addition, those meetings serve as another venue for residents to provide confidential (in the sense that they don’t have to speak in front of their colleagues) feedback regarding the program.
Justin A. Sattin, MD