Remediating the struggling resident

Sheilah Bernard, MD, APDChair, Clinical Competence Committee

April 8, 2013Wilkins Board Room

Goals

• Gain familiarity with Milestones and Entrustable Professional Activities (EPA's)

• Identify some sources of poor performance

• Understand the different mechanisms by which remediation occurs

Training the successful physician

MODELING

ATTITUDES

SKILLS

KNOWLEDGE

Brainstorm: characteristics of struggling resident

(“It’s like pornography, you recognize it when you see it…..”)

“Heads up”

• PGY1 is disorganized, writes notes late (PC, P, MK)

• PGY1 having difficulty synthesizing medical information (MK, PC)

• PGY2 is quiet, not effectively leading team (IPSC, MK, PC)

• PGY2 is distracted, can’t multitask effectively (PC, PBLI, P)

• PGY3 has “checked out” (P)

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

• Recognize and characterize the problem• Determine the appropriate corrective action plan• Identify the appropriate administrative setting for

the action plan• Determine whether the corrective action plan

has been successful• Attend to due process issues

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

What evaluations are in our “toolkit”?

• Faculty and Peer block evaluation (rarely use NET not enough time). Your observations even over a 1 wk period will still provide useful information

• 360°’s from MS’s, patients, RN’s, discharge planners, Peer review from residents/CMR’s

• Conferences: interactions during morning report, CREX, EBM, Journal Club, Board review, Hopkins modules

• Clinical interaction/Critical Incidence report (STARS): Reportable activity, reflects performance under stress

• Quality Improvement processes: Hospital reporting systems may identify poor performance by resident (Logician Reds, Duty Hour Violations)

• OSCE’s, Sim, Mini-CEX’s directly observed exercises

Infrastructure: Clinical Competence Committee

• Reviews monthly all evaluations < 4 or other concerns brought to Program Office

• Corroborates concerns with other evaluative tools (CMR’s, discussions with evaluators, MiniCEX, peer reviews)

• Advisor (PD/APD) meets and discusses performances, reviews feedback already received

• Sets goals to improve performance, directed to feedback

• Recommends remediation if goals not met

July June

First evaluations identify at risk PGY1’s

Implement further evaluation methods

CMR’s deployed

October

Critical Remediation Timeline

July June

Implement remediation plan if further evals below average

December

Critical Remediation Timeline

July June

Formalize PGY

extension if required

April

October December

Critical Remediation Timeline

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

• Recognize and characterize the problem

Recognize and characterize the problem

• Assumption: Curricular and personal goals are reviewed by residents (in NI under block schedule)– Truth: never reviewed– Phone calls directly to program office

• “Heads up”

• “Red flag”—tardiness, late notes, any single circumstance with adverse outcomes

• Inefficiencies in care• Behavioral issues

Recognize and characterize the problem

• Red flags– A disproportionate amount of attention by training

personnel is required– Grumbling from peers– The trainee’s behavior does not change as a function

of feedback, remediation efforts, and / or time invested (by trainee or program director!!!)

• Faculty and peer evaluations– Document– Document– Document

Peer review

J Grad Med Educ. 2012 March; 4(1): 47–51. doi: 10.4300/JGME-D-11-00145.1

Competence problems that should be documented:

• Lack of or poor judgment • Inadequate clinical skills/patient care• Deficient technical or procedural skills• Ineffective communication skills• Inability and/or unwillingness to acquire and

integrate professional standards into one’s repertoire of professional behavior

• Lack of personal insight or self-awareness• Inability to control personal stress or emotional

reactions that interfere with professional functioning (conduct or emotional problem) and participation in teams

EPA: Demonstrate professional behavior

• Milestone: Responds promptly and appropriately to clinical responsibilities including but not limited to calls and pages

• Milestone: Dress and behave appropriately

• Milestone: Maintain appropriate professional relationships with patients, families and staff

Evaluations

• Any rating less than 5 merits attention by program office (below average)

• No single evaluation will “fail” a resident; be honest and objective within each competence

• Remember, evaluation is SUMMATIVE and judgmental. It should reflect FORMATIVE feedback already provided to the resident

Feedback might reveal:

• Transition issues

• Mild performance anxiety

• Mild discomfort with diverse patient groups or multi-disciplinary team members

• Initial lack of understanding of the attending’s/facility’s standards

• Lack of certain skill sets, but an openness and readiness to acquire them

6 D’s of unprofessional behavior

• Depression

• Deprivation (sleep, food)

• Distraction (finances, family/SO, illness)

• Disability (neurocognitive, physical)

• Disordered personality (ADHD, borderline)

• Drugs (alcohol, narcotics)

Performance Standards

• Outstanding – Resident demonstrates truly outstanding

performance in all competences, achieving milestones before normal trajectory

– Modeling

Performance Standards

• Outstanding--Models• Good/Satisfactory--Teaches/manages

– Resident meets all expectations for performance in all domains over time and demonstrates no deficiencies

– Compensation fallacy: Erroneously labeling residents as “good” who have some strong characteristics (professionalism) that “cancel out” their unsatisfactory characteristics (patient care/judgment)

– Halo effect: some residents are truly outstanding in one or two areas, and deficiencies in other areas are overlooked

Performance Standards

• Outstanding

• Good/Satisfactory

• Marginal – still learning– Resident is not particularly strong and has poor

or ambiguous competencies in some domains– Faculty want to give benefit of doubt– Faculty feel that resident is not incompetent

(ergo must be competent)– Faculty do not want to label resident

Performance Standards

• Outstanding

• Good/Satisfactory

• Marginal

• Unsatisfactory and unsuitable for promotion despite remediation

Why reluctance to grade low?

Problems with post-rotation evaluations

• grade inflation (is 7 average?)

• attending physicians' lack of willingness to document poor performance

• lack of knowledge about how to document performance concerns

• comments section often does not correlate with the numeric ratings

J Grad Med Educ. 2012 March; 4(1): 47–51. doi: 10.4300/JGME-D-11-00145.1

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

• Recognize and characterize the problem

• Determine the appropriate corrective action plan

CCC Remediation with Individualized Educational Plan:

• Identifies deficiencies in context of core competence and EPA/milestone

• Reviews block rotations

• Identifies task-oriented demonstration of skills

Evaluation methods

• MK: Record review, Chart stimulated recall, 360 evals, Simulations, ITE/conf attendance/Hopkins

• PBLI: record review, use of EBM, portfolios • IPSC: OSCE, simulations, Patient Survey• PC: Check list, 360 evals, OSCE, CSR, Record

review • P: OSCE, PS, 360 evals, MiniCEX• SBP: 360, PS, OSCE, portfolios, QI project

CCC Remediation with Individualized Educational Plan:

• Identifies deficiencies in context of core competence and EPA/milestone

• Reviews block rotations• Identifies task-oriented demonstration of skills• Arranges mentor/coach outside of CCC (core

faculty, master clinicians)• Uses different skill sets• Sets timeline with goals • States ramifications of failure to achieve goals• Arranges follow-up to assess progress

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

• Recognize and characterize the problem

• Determine the appropriate corrective action plan

• Identify the appropriate administrative setting for the action plan

Setting for action plan

• Warning• Remediation may or may not be on probation

– Inpatient service• Prewarn service attending• Rotation? Colleagues?

– Ambulatory service– Elective time in areas of deficit– Regular meetings with coach to review notes, discuss

management, identify knowledge– Self-reflection/insight

• Non-promotion, nonrenewal of contract, no credit, extension of training, suspension, withdrawal from program, termination

Case for discussion

• PGY2 first ward block is inefficient, doesn’t recognize sick patients, makes superficial assessments, misses important clinical clues

• Core Competence: Patient care

Individualized learning plan• PC-C1 Clinical Reasoning (12 mos): synthesize

all available data, including interview, physical exam, and preliminary laboratory data, to define each patient’s central clinical problem.– Chart stimulated recall on patients– Observed MiniCEX by preceptor, coach, attending– Chart audit: review and discuss admission notes,

consultations, discharge summary by faculty, CMR’s• PC-B2 (12 mos): accurately track important

changes in the physical examination over time in the outpatient and inpatient settings– Preceptor chart-stimulated recall of changes in

diabetic PE– CMRs discuss changes in CHF exam on CMP

Critical Remediation Targets

• PGY1– Identify 75% of residents requiring remediation– Identify 100% of residents requiring extension

• PGY2– Identify remaining 25% of struggling residents– No residents requiring extension– No “Holy Cow” residents

• PGY3– Rare remediation issues– Patient safety issues trump all other evaluations

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

• Recognize and characterize the problem• Determine the appropriate corrective

action plan• Identify the appropriate administrative

setting for the action plan• Determine whether the corrective

action plan has been successful

Success!

• Successful completion of all remediation steps– Remove from remediation– Continue on rising trajectory

• No further evaluations < 5

• Completes anticipated milestones for promotion

How to identify and remediate the struggling resident

• Develop a comprehensive evaluation infrastructure

• Recognize and characterize the problem• Determine the appropriate corrective action plan• Identify the appropriate administrative setting for

the action plan• Determine whether the corrective action plan

has been successful• Attend to due process issues

Homework handout

backups

Clinical Competence Committee

• ACGME requires a resident to complete an approved 3-year medical residency before sitting for ABIM certification exam– Clinical Competence Committee assesses

noncompensatory competence in other cores of Patient Care, Interpersonal Skills and Communication, Systems based Practice, Practice-based Learning and Improvement, and Professionalism

• Successful passage of ABIM certifying exam reflects residents’ fund of Medical Knowledge

The academic year for PGY’s

Context issues to be considered:

• Separation from support systems

• Adjustment issues to new setting both personally and professionally

• Changes in status (finances or power)

• Impact of significant life events

• Personal risk factors (substance abuse, ADD, other psychiatric disorders, etc.)

Due process

• Meetings with advisor, PD, Chair CCC• Boston Medical Center is obligated to make

reports to the Board of Registration in Medicine (“BORM”) when it takes disciplinary action against a House Officer.

• The hospital must file a report with the BORM when there is a reasonable basis to believe that a House Officer is in violation of any Massachusetts law relating to the practice of medicine or regulations of the BORM.

• Right to appeal to CMO at BMC

Disciplinary actions which must be reported to the BORM:

• Written reprimand or admonition for behavior relating to competence to practice medicine or violation of a law, the regulations of the BORM, or hospital bylaws

• Probation: Such action is taken in accordance with the requirements of the HOA/CIR contract

• Suspension: seriously inappropriate behavior to patients, colleagues or others or significant failure to comply with hospital policies

Reportable to BORM

• Termination or nonrenewal for behavior relating to competence to practice medicine or violation of a law, the regulations of the BORM or hospital bylaws

• Resignation related to competence to practice medicine or violation of a law, the regulations of the BORM or hospital bylaws

• Leave of absence related to competence to practice medicine (administrative leave does not qualify)

Characteristics of rotation evals suggesting need for remediation:

• presence of “outlier” evaluations (>2 SD)

• the amount of comments

• the percentage of negative or ambiguous comments

J Grad Med Educ. 2012 March; 4(1): 47–51. doi: 10.4300/JGME-D-11-00145.1