Meaningful Evaluation:Framework, Process, Impact

Inis Jane Bardella, M.D., FAAFPAssociate Dean for Faculty Development and Global Health InitiativesDepartment of Family and Preventive Medicine

Lecia M. Apantaku, M.D., FACSAssistant Dean for Clinical Sciences EducationAssociate Professor, Department of Surgery

Chicago Medical SchoolRosalind Franklin University of Medicine and Science

Objectives

Discuss the relationship between evaluation and learning.

Discuss a framework [RIME, Reporter Interpreter Manager Educator] for evaluation.

Discuss the process of evaluation. Apply the RIME framework to achieve

meaningful [accurate, useful, efficient] evaluation of students and residents.

Vision

Effectively prepare medical students and residents to meet the health needs of the people of the United States [especially the poor] with compassion, integrity, high ethical standards and a high level of competence.

Responsibility

Patients, always first Students and Residents Faculty

Clinical teachers – you Clerkship directorProgram director

Education systemCentegraCMS

Definition

Process of making judgments based on factual information and observations in order to rate, rank or assess an individual’s status at a given point in time.

Background – Adult Learning and Teaching

What is different about adults? Self-directed Goal-directed Application-oriented Competency concerns Shared responsibility Moral responsibility Teacher is: Resource, Facilitator, Mentor

Background – Adult Learning and Teaching

How does this influence evaluation?

How should this influence evaluation?

Bottom Line of Evaluation

Evaluation drives learning. Objectives determine evaluation. Evaluation is the second greatest fear

of preceptors/clinical faculty. Evaluation is absolutely necessary to

determine progression toward independent practice.

So, why formal evaluation?

Summarize performance Determine competencies (behaviors) Guide future learning Communicate summary information Determine a grade

Framework – RIME

Reporter Interpreter Manager Educator

Framework – RIME

Reporter

Consistently professional, good interpersonal skills

Reliably obtains and communicates clinical findings

Ownership of clinical findings

Interpreter

Consistently able to analyze and prioritize patient problems

Ownership of explaining things

Framework – RIME

Manager

Consistently proposes reasonable options incorporating patient preferences

Ownership of developing action plan with patient

Educator

Consistent level of knowledge of current medical evidence

Can critically apply knowledge to specific patients

Ownership of evidence for action and sharing

RIME Caveat

Students must be reasonable, residents must be right.

Application – RIME

What is the RIME?

How did you determine the RIME?

Framework – RIME Synthetic model – synthesis

What we do all day long in patient care! Focus is patient – performance in relationship to

patient careProfessionalism requiredHigh level of interpersonal skills required

Descriptive Progressive, building model Competencies (KSA, behaviors) integral Not relative – Expectations clear

A Few More RIME Caveats

Program must determine common problems and uncommon problems.

Program must determine core or essential areas and non-essential areas

BREAK TIME

Application

What is the RIME? What if:

Beginning 3rd year?Middle 3rd year?End 3rd year?Mid 4th year?Beginning intern?Mid intern?2nd year resident?

Process – PIE Cycle

Process - PIE

PlanningClerkship goals and objectivesRequirements/projectsStudent self-evaluateReview the form

Planning

Process - PIE

InstructionObservationFeedbackAction plansGather and record data

• Patient and staff perceptions• Chart documentation• Observation• Student progress notes

Instruction

Process - PIE

EvaluationForm

• Student• You

UnrushedEmphasize objectives, improvementSpecific commentsMeeting

Evaluation

Comments about Comments

Examples? Demonstration Consistency Progression Improvement Recommendations RIME it

Application - Forms

RIME the elements RIME the learner Complete the form using RIME as the

framework to assess behavior, i.g. competency.

Comments

Summary

Evaluation is a continuous process. Evaluation drives learning. Goals & objectives determine evaluation. RIME it.

Determine the student’s starting point.Develop a plan for progression.Determine the final evaluation.

Include comments To teach is to learn again!

ReferencesBardella IJ, ed. Essentials of precepting 2002-2003. 2002, revised 2006.

Ende, J. Feedback in clinical medical education. JAMA. 1983;250:777-781.

Massachusetts statewide area health education center program. Faculty development workbook. 1995.

Pangaro LN. A new vocabulary and other innovations for improving descriptive in-training evaluations. Acad Med. 1999;74:1203-1207.

Pangaro LN. Evaluating trainees in the clinical setting: what does competency in patient care look like? Presented 25 Jan 2012 at RFUMS, North Chicago, IL.

Society of teachers of family medicine pep2 committee. Preceptor education project, second edition: facilitators guide. Society of teachers of family medicine. Kansas City. 1999.

Thank you

inis.bardella@rosalindfranklin.edu