Post on 05-Mar-2018
transcript
The Mini Clinical The Mini Clinical
Evaluation ExerciseEvaluation Exercise
(mini(mini--CEX)CEX)
Dr Jeremy MortonDr Jeremy Morton
Medical Teaching Medical Teaching OrganisationOrganisation
University of Edinburgh Medical SchoolUniversity of Edinburgh Medical School
OverviewOverview
•• PerformancePerformance--based assessment (PBA) in based assessment (PBA) in
medical educationmedical education
•• Attitudes and barriers to PBAAttitudes and barriers to PBA
•• minimini--CEX assessment in undergraduate CEX assessment in undergraduate
medical educationmedical education
Reforms of postgraduate medical Reforms of postgraduate medical
education in the UKeducation in the UK
Undergraduate medical educationUndergraduate medical education
•• Shift from Shift from
knowledgeknowledge--based based
to competencyto competency--
based assessment.based assessment.
Assessment of clinical competenceAssessment of clinical competence
Does = Performance Based Testing eg; Mini-CEX, Multi-source Feedback
Shows How = Competency Based Testing eg; OSCE, SP testing
Knows How = Clinical Context Based Testing eg; PMPs, MEQ, EMQ
Knows = Factual Testing eg; MCQ, essay, oral
Miller’s Pyramid
PerformancePerformance--based assessmentbased assessment
Attitudes to Clinical Competence Attitudes to Clinical Competence
Assessments (CCA) Assessments (CCA) –– a survey a survey
CCA CCA –– FurtherFurther
Experiences & AttitudesExperiences & Attitudes
• 75% of clinical teachers indicated that global judgement without direct observation was principal method of rating student
• Reflected in students’ evaluation of assessment practice in Year 5
• 70% of students in favour of summative PBA strategy vs. 30% clinical teacher
• Majority of clinical teachers back a return to more formalised testing
Barriers to PBABarriers to PBA
Clinical Teachers -• Lack of time• Large class sizes• Inadequate resources• Lack of educational training• Conflicting priorities
Students Students --
•• Reticence on the part of their teachers to Reticence on the part of their teachers to
engage in the assessment of observed engage in the assessment of observed
practicepractice
•• Lack of commitment to standardising Lack of commitment to standardising
practicepractice
Is PBA a lost cause?Is PBA a lost cause?
““I think you have to be awareI think you have to be aware of of
the pressure on mainstream the pressure on mainstream
speciality consultants to provide speciality consultants to provide
teaching. I have seven students teaching. I have seven students
for a block of seven weeks four for a block of seven weeks four
times per year and I am their tutor. times per year and I am their tutor.
This is on top of a full clinical This is on top of a full clinical
workload. I struggle to find time to workload. I struggle to find time to
do what I do (and train our own do what I do (and train our own
juniors) so although what I say is juniors) so although what I say is
desirable I dondesirable I don’’t think I can t think I can
achieve it.achieve it.””
minimini--CEXCEX
•• Method for assessing clinical performance in the Method for assessing clinical performance in the workplace developed by ABIMworkplace developed by ABIM
•• Viva examinations abandoned by ABIM in 1972 Viva examinations abandoned by ABIM in 1972 and replaced with the Clinical Evaluation and replaced with the Clinical Evaluation Exercise (CEX).Exercise (CEX).
•• MiniMini--CEX superseded CEX in early to midCEX superseded CEX in early to mid--90s90s
minimini--CEXCEX
•• Postgraduate toolPostgraduate tool
•• Pocketsize cardsPocketsize cards
•• 7 competency 7 competency
domainsdomains
•• Nine point rating scale, Nine point rating scale,
4 defined as marginal4 defined as marginal
•• Snapshot of residentSnapshot of resident--
patient interactionpatient interaction
minimini--CEXCEX
•• Shown to be valid, reliable, feasible, acceptable and Shown to be valid, reliable, feasible, acceptable and
fair assessment in PG & UG practice in the USA.fair assessment in PG & UG practice in the USA.
•• Reproducibility coefficient of 0.8 following 12 Reproducibility coefficient of 0.8 following 12 --14 14
assessments in PG and 8 in UG setting.assessments in PG and 8 in UG setting.
•• Fewer assessments required to reliably identify Fewer assessments required to reliably identify
those who are clearly competent or incompetentthose who are clearly competent or incompetent..
minimini--CEXCEX
•• Strongly Strongly recommended in recommended in assessment of assessment of medical interns, medical interns, USAUSA
•• Adopted as a Adopted as a key element in key element in the FPthe FP’’s s assessment assessment toolkit for Eng.& toolkit for Eng.& WalesWales
minimini--CEX & UKCEX & UK•• FY doctors expected FY doctors expected
to undertake 4 minito undertake 4 mini--CEX assessments in CEX assessments in FY 1 & 6 in FY 2FY 1 & 6 in FY 2
•• Implemented on basis Implemented on basis of published evidence of published evidence relating to practice in relating to practice in USA & experience of USA & experience of pilot schemespilot schemes
•• Limited experience in Limited experience in undergraduate circlesundergraduate circles
minimini--CEX & undergraduate CEX & undergraduate
medical education in the UKmedical education in the UK
•• University of Southampton Medical University of Southampton Medical
School, ASME 2005School, ASME 2005
•• Evaluation of 2340 miniEvaluation of 2340 mini--CEX assessments CEX assessments
in a single yearin a single year
•• Format derived from ABIM Format derived from ABIM
The positivesThe positives
•• Liked by studentsLiked by students
•• Easy for examiners to organiseEasy for examiners to organise
•• Driver for change & improvementDriver for change & improvement
•• Less disruptive to patients and their Less disruptive to patients and their
familiesfamilies
•• Seen to be fairSeen to be fair
The negativesThe negatives
•• Organisational problems in shorter Organisational problems in shorter
attachmentsattachments
•• Less holistic than long casesLess holistic than long cases
•• Perception of lack of consistency in Perception of lack of consistency in
gradinggrading
•• Students with good communication skills Students with good communication skills
but poor overall clinical competence may but poor overall clinical competence may
be able to passbe able to pass
minimini--CEX pilot in EdinburghCEX pilot in Edinburgh
• 202 mini-CEX assessments on 94 students during acute medicine block
• Median no. of assessments per student - 2 (range 1-5)
• Median observation time – 5 minutes
(range 2-90)
Median feedback time – 10 minutes (range 1-60)
• Assessor satisfaction – 119 ratings Median value 8 on 10 point scale.
• Reduced tendency to grade towards middle
Student ExperienceStudent Experience
• 77% of students agreed or strongly agreed that mini-CEX is a fair means of assessment
• Median student satisfaction with
mini-CEX format – 8
(on same 10 point scale)
• Difficult to arrange the time with busy medical staff to carry out mini-CEX assessments
• Recognise the unreliability of a judgement made on one or two assessments
Making miniMaking mini--CEX work in CEX work in
undergraduate medical educationundergraduate medical education
•• Adapt miniAdapt mini--CEX CEX
methodology methodology
coco--operativelyoperatively
•• Encourage integration of Encourage integration of
assessment assessment
•• Set achievable targets Set achievable targets
and consider weighting and consider weighting
the burden of assessment the burden of assessment
to focus on borderline to focus on borderline
candidatescandidates
Making miniMaking mini--CEX work in CEX work in
undergraduate medical educationundergraduate medical education
•• Undertake staff Undertake staff development and quality development and quality control control
•• Factor miniFactor mini--CEX CEX performance into performance into summativesummative assessmentassessment
•• Involve senior training Involve senior training and nonand non--training grade training grade doctors in assessmentdoctors in assessment
•• Reward for excellence in Reward for excellence in teaching and assessmentteaching and assessment
SummarySummary
•• Students and consultant clinical teachers Students and consultant clinical teachers acknowledge the merits of PBA and related acknowledge the merits of PBA and related feedback but doubts persistfeedback but doubts persist
•• minimini--CEX is becoming established as a routine CEX is becoming established as a routine in postgraduate practicein postgraduate practice
•• Experience suggests that miniExperience suggests that mini--CEX testing is CEX testing is acceptable, fair and manageableacceptable, fair and manageable
•• MiniMini--CEX may offer sustainable means of CEX may offer sustainable means of achieving reliable PBA that is practical for achieving reliable PBA that is practical for clinicians and offers quality feedback to studentsclinicians and offers quality feedback to students