25
Assessment of Competency Milestones AAMC
Assessment of
Competency Milestones
AAMC – Presentation 2014
Lynn J. Romrell, Ph.D.
Florida State University College of Medicine
Florida State University College of Medicine
Overall
Level of
Performance
Assessment of Competence
Achievement of
Competency Milestones
Consistency and Quality of Performance
Most Recent Changes in
the Evaluation System
Reduced Number of Competency
Milestones to 20
Added Quality and Consistency
Assessment
Added Assessment of Student’s
Progression
Progression:
Stage of Development of the Student
Faculty assess the
progression of the
student on a 5-point scale:
Competency/Milestones:
Competency in Four Domains
*Yellow- Highlighted items have
the highest discrimination value.
Consistency and Quality Assessment:
For Aspects of the Competency Milestones
Competency Milestones
judged on a scale from
9 (High) to 1 (Low)
Consistency and Quality Assessment:
For Specific Competency Milestones
Faculty assess the consistency and quality
of aspects of the Competency Milestones
on a scale from 9 (High) to 1 (Low).
OUTCOME: mean is about 8.0
35%
35%
20%
10%
Consistency and Quality Assessment:
Range of the Scores …
Consistency and Quality Assessment:
For Specific Competency Milestones
Level of Development
“Stage of Student Development” by Rotation
of 3rd Year Students – Class of 2015
0
10
20
30
40
50
A B C D E F
Below 3rd Year New 3rd Year
Experienced 3rd Year 4th Year
Resident
Pe
rce
nt
of
Stu
de
nts
Rotations
Level of Development
“Stage of Student Development” by Rotation
of 3rd Year Students … Who Received “Honors”
0
10
20
30
40
50
A B C D E F
Below 3rd Year New 3rd Year
Experienced 3rd Year 4th Year
Resident
Pe
rce
nt
of
Stu
de
nts
Rotation Expectations increased
during the year
Competency Milestones
Reports to:
Regional Campus Deans
AND Students
Statistical Summary and Comments
Includes:
Who Did the Rating,
Amount of Time with Student,
Summary Scores,
Specific scores,
and Comments
Report of Performance to: Regional Campus Deans and Student – Case 1a
Report of Performance to: Regional Campus Deans and Student – Case 1
Report of Performance to: Regional Campus Deans and Student
CASE: 2
Report of Performance to: Regional Campus Deans and Student – Case 2a
Report of Performance to: Regional Campus Deans and Student – Case 2b
Report of Performance to: Regional Campus Deans and Student – Case 2c
Report of Performance to: Regional Campus Deans and Student – Case 2d
Competency Milestones - 8 to 12 Independent Faculty Raters
vs
End of 3rd Year OSCE - 8 Independent Faculty Raters,
SPs and Interstation
Top Performers on End of 3rd Year OSCE - Compared to Competency Milestone Scores
Low Performers on End of 3rd Year OSCE - Compared to Competency Milestone Scores
1 - Variance between the two groups is statistically significant
OSCE vs Clinical Assessment.
2 - Correlation Coefficient = 0.36 for Overall Score vs Clinical Rank
Questions
Key Link: AAMC guidelines: “Core Entrustable Professional Activities
for Entering Residency”
Report Defines 13 entrustable professional activities (EPAs) medical school graduates
should be able to perform on the first day of residency: aamc.org/cepaer
EPA 1: Gather a history and perform a physical examination
EPA 2: Prioritize a differential diagnosis following a clinical encounter
EPA 3: Recommend and interpret common diagnostic and screening tests
EPA 4: Enter and discuss orders and prescriptions
EPA 5: Document a clinical encounter in the patient record
EPA 6: Provide an oral presentation of a clinical encounter
EPA 7: Form clinical questions and retrieve evidence to advance patient care
EPA 8: Give or receive a patient handover to transition care responsibility
EPA 9: Collaborate as a member of an interprofessional team
EPA 10: Recognize a patient requiring urgent or emergent care and initiate
evaluation and management
EPA 11: Obtain informed consent for tests and/or procedures
EPA 12: Perform general procedures of a physician EPA 13: Identify system failures and contribute to a culture of safety and improvement
