29
You Be the Judge…
View the following performances
In small groups, take 1 minute to
assign a score from 1-10 using any means.
Be prepared to share your reasoning…
How Did It Go?
What did you notice?
Resident Evaluation
Shawn Alderman, MD
Faculty Development Fellow
Learning Objectives
• Examined why we evaluate
• Identified essentials of evaluations
• Listed common skills to assess
• Reviewed available evaluation tools
• Recognized potential pitfalls and biases
Take Home Points…
• Evaluations are azimuth checks
• Assess specific skills
• Incorporate standards
• Use your tool box
• Be aware of our personal biases
• Departmental communication is essential
Activity One
• Why are evaluations important?
• What problems have you encountered?
Stakeholders
Types of Evaluations
• Formative
• Summative
Activity Two
With the End in Mind
In your groups, discuss the things you consider before writing a learner’s evaluation
The Standard
A Standard is…
• …the yardstick
• …the bare minimum requirement
• …a requirement that must be met
• …a minimum skill set
Standards Must Be
Evaluation Essentials
• Formal - standardized, routine, expected
• Communication – dialogue, frequency
• Documentation - summative, written signed
• Due Process - standards, plans, consequences
Skill Evaluation
Activity Three
In your groups, discuss the skill sets and put them in a ranked order from highest to
lowest in importance
The Tool Box…
• Checklists
• Rating Scales
• Anecdotal Records
• Incident Reports
• RIME Method
• BSQs
• Global Assessments
• OSCEs
• Simulation
• Examinations
• Video Clinic
• 360 Degree Evals
Is the tool sharp?
• Use the right tool
• Inspect the tool
• Tool improvement
Activity Four
In your groups, review the evaluation tools.
How would you improve the tool?
Activity Five
In groups, match the error or bias type with the
appropriate description
Avoiding Error and Bias
• Know thyself
• Consult colleagues
• Use multiple tools
• Faculty development
Recent Incident Bias: Bias because of an isolated recent incident…
Central Tendency: Sitting on the fence …
Extreme Response Bias: Respondents tend to mark extremes …
Affirmation/Yea-Saying Bias: Give positive responses irrespective …
Incompetence Bias: Assign high ratings because of lack of confidence…
Leniency Bias: Being overly charitable
Halo Effect: Overall impression of a person affect rating on each item…
Contrast Bias: Rating against another person's performance…
Stringency Bias: Being overly critical
Our Evaluation System
•Standard focused
•Assessing skills
•Using all tools
•Minimizing error/bias
•Communicating
Commitment
• What are our areas of weakness?
• How can we improve our evaluation process?
• When will we start?
Take Home Points…
• Evaluations are azimuth checks
• Assess specific skills
• Incorporate standards
• Use your tool box
• Be aware of our personal biases
• Departmental communication is essential
