Competency Based Medical Education 2015 03 26 N… · Triple C Competency-based LEARNING...

transcript

Competency Based Medical Education

Lessons from the Trenches - The Basics

Rob Anderson Christina Tremblay Catherine Cervin

Northern Constellations March 28, 2015

Today's Learning Objectives

By the end of this session, participants will be able to:• Describe the two models of CBME that are

being implemented in Canada: CFPC competency assessment program and RCPSC competency by design.

• Build an assessment framework which will support the use of milestones, entrustable professional activities, and skill dimensions.

Presenter

Presentation Notes

Ask 2 – 3 people – you already know what competence is this will help you frame that

The Building Blocks

Design

DEFINED PROGRAM OUTCOMES

Competency-based and guided by CanMEDS-FM

Assess

Triple C Competency-based LEARNING OPPORTUNITIESClinical ExperiencesAcademic ProgramOther Activities

Design and providecurriculum

Triple C Competency-based teaching and learning STRATEGIES

Triple C Competency-based RESOURCESClinical resourcesTeaching Materials Faculty

Outcome-based PROGRAM EVALUATION

ON

GO

ING

ASSESSMEN

T of residents –

based on Evaluation objectives

5

Triple C Competency-based Curriculum

6

The Big Question

• How do we develop competency based training

AND

• ensureAND

• document

End of training competence?

Basic Principles

Define Behaviour Observe Behaviour

Assess BehaviourDecide where learner is on competencetrajectory

Competency Trajectory

Expert

PGY1 Start

PGY2 Start

End Residency

First 3-5 years of Practice

Beginning Professional

Knowledgeable Professional

Novice

TIME

SETT

ING SUMMATIVE

ASSESSMENT

RESIDENTOWNERSHIP!

Development of The Evaluation Objectives

• Asked to describe competence in terms of:

– Patient problems and situations– Clinical decision making and judgment– Other qualities and behaviours– Problem areas

Survey Results / Further Definition • 99 Topics

• 6 Skill Dimensions : Clinical Reasoning, Selectivity, Patient Centered Approach, Communication, Professionalism, Procedural/Psychomotor Skills

• 8 Phases of the Clinical encounter : History, Physical, hypothesis, Investigation, Diagnosis, Management, Referral, Follow-up

Further Definition of TopicsKey Features

• Definition- The critical steps in the resolution of a problem. Focused on the points where we are most likely to make errors and the areas that are the most difficult in practice.

Presenter

Presentation Notes

They identify the essential or critical steps specific to the problem They determine what processes are difficult and most involved in succeeding or not Generated based on practical experience not on theoretical analysis or published references

Skills – Procedural• Decision to act- consider indications,

contraindications, your skills and context ( that day and time ability ), context of the procedure

• Informed Consent• Preparation: Review - anatomy, sequential

technical steps. potential complications and their management, appropriate equipment

• During Procedure: Keep patient informed (decrease anxiety), ensure comfort and safety

• If problems reevaluate ( ? Stop ? Ask for help)

• Aftercare/Follow-up

CANMedsRoles

Clinical Domains

Skill Dimensions

Clinical Contexts

CommunicatorCollaboratorProfessionalScholarManagerAdvocate

Care through the Life CycleWomen’s Health + Maternity

Surgical/Procedural Mental Health

Palliative Care/End of Life CareCare of the Underserved

Care of the Elderly

HospitalHomeOffice

Labour delivery wardNursing Home

Operating RoomEmergency Department

SelectivityClinical ReasoningProfessionalismProcedural skillsPatient CentredMethodCommunicator

FAMILY MEDICINEEXPERT

Community

Levels of competence:

Low levels one well-defined taskdone repeatedly

the same way

High levels multiple tasks, ambiguous,uncertain end-points, partial data,

knows how and why, can justifycan abstract to new situations

Task 1: Picking a competency

• Work with your table to define:• A population• An observable behavior/task• Timeline of success

• For example: Anesthesia residents inserting a labor epidural in a healthy parturient by the end of 2nd year of residency.

“I am not interested in Competency Based Education”

Direct quote from Nov 2011 NOSM anesthesia retreat

slide presented byRob Anderson!

Presenter

Presentation Notes

Start with a bit of history of where we were and talk about some of the experiences that we had that shaped what we have done and where we are going In 2011 the move toward CBME was being talked about everywhere, but no real specifics. Some of the biggest concerns that people had were the “time free” nature of it. The thought of assessing everything, wondering why we need to re-invent the best health care education programs in the world. There was a lot of passion, an none of it was positive. What we did have was a robust FPA and simulation program, experience in RC program with our connection with Ottawa, although that was coming to an end and we needed to decide what to do next, and we had a great group of residents that were mentors and leaders. It was a good time to start an independent program. We were pretty adamant that we would only do a program if it was outstanding. That was our goal. We talked about active learning, simulation, eLearning and PBLs. But not CBME!

What Changed?

• CanMEDs 2015• Time free “softened” to time as a

resource. • ICRE 2012/13• Workshops (Sherbino and UofT)• CFPC SIFP Working group• Unmatched desire to not have to

completely rebuild … ever!

LESSON # 1: DON’T WORRY ABOUT THE EVOLVING DEFINITIONS…THEY WILL JUST TELL US!

Definitions relevant to CBD

• Entrustable Professional Activities (EPA): • A task in the clinical setting that may be

delegated to a resident by their supervisor once competence has been demonstrated

• Milestones: • An observable marker of an individual’s

ability along a developmental continuum

Competency by Design - Lots of Change!

• Increased resident ownership• Mainport ePortfolio• De-emphasize examinations• New accreditation structure• Competency committees and

coaches• Competency frameworks “Back

Office” vs “Front Office”

Back Office

• Comprehensive set of milestones created through a set of workshops at the specialty committee

• Defines what it means to be a competent physician

• Map to EPAs and assessments

• Define duration and content of stages

Front Office

• EPAs and specific milestone assessments

• Each EPA integrates multiple milestones

• Aggregated by ePortfolio• Bank of assessments that have

been identified for the programs• Implementing them is the key!

Early EPA discussion

• We do this anyway!• Assess what is important, not

everything• Multiple milestones captured per

EPA• Key milestones captured across

multiple EPAs

Labor Analgesia in the Healthy Parturient

Presenter

Presentation Notes

Perform a comprehensive assessment and identify the healthy patient (ME, Com) Counsel the patient regarding R&B’s in a compassionate manner (HA, Com) Demonstrate safe procedural technique including (sterile technique, anatomy, equipment…) (ME) Select appropriate medications (ME) Appropriate documentation (Com) and follow up (Man)

Competency Committees

• Promotion is resident driven!• Demonstrate that they have met

the bar• The bar must be clear• EPAs, set by the specialty

committee/working groups, will be mapped to CBD stages

The FPA Journey

• CFPC extending CCC into enhanced skills programs

• Small working group created• Creation of Priority Topics

relevant to anesthesia –validated broadly

• Focus on discriminatory acts, or “key features”

Priority Topics

• General Anesthesia• Post-operative care• Teamwork• Equipment • Neuraxial Anesthesia• Airway: complex• OR Emergencies

and complications• Pre-anesthesia

assessment

• Acutely ill or injured• Vascular access• Acute pain

management• Procedural Sedation• Know and apply

limits of capacity• Self directed

learning

0

20

40

60

80

100

120

OR

emer

genc

ies a

nd…

Airw

ay: r

outin

e an

d co

mpl

ex

Acut

ely

ill o

r inj

ured

Pre-

anae

sthe

tic/s

urge

ry…

Obs

tetr

ical

ana

esth

esia

Regi

onal

ana

esth

esia

Vasc

ular

acc

ess

Pedi

atric

ana

esth

esia

Proc

edur

al se

datio

n

Anae

sthe

sia g

ener

al a

nd e

lect

ive

Post

-ope

rativ

e ca

re

Vent

ilatio

ns a

nd e

quip

men

t

New

born

resu

scita

tion

Pain

man

agem

ent

Anae

sthe

sia a

nd o

besit

y

Med

icat

ions

Tran

spor

t of a

cute

ly il

l or i

njur

ed

VG

WG*6

Top 5*10

Key Features (PT Manage the complex airway)

• Can perform direct laryngosocopy

VS

• Can assess and predict/anticipate the patient with a difficult airway and the stages in which those difficulties may occur.

Or• Perform endotracheal intubation effectively in

elective, urgent and emergent situations that require different approaches

LESSON # 2: YOU DO NOT HAVE TO ASSESS EVERYTHING, JUST THE RIGHT THINGS!

Synergistic messages

• Residents own learning and documentation

• Assess the important things• Subjective and workplace

assessment is going to be vital

Task 2: Defining what is important

• What are the key steps that one must demonstrate to complete the task?

• Keep in mind…• Where do the “incompetents” fall

down.• What is so essential it can’t not be

included

LESSON # 3: FOCUS ON SMALL WINS WHEN IMPLEMENTING CHANGE.

Don’t Try to Drink the Ocean!

Image source: www.istockphoto.com

Presenter

Presentation Notes

Engaging only key people who are already involved in the program to ensure success look at what you have! – Easy wins

http://www.istockphoto.com/

1. Analyze current practices2. Use the big picture to identify

achievable tasks which will move you toward CBD

3. Identify tools & resources required to achieve those tasks

4. Do the work & create the process5. Implement 6. Evaluate & re-evaluate frequently

Look at what you have & what you need

Presenter

Presentation Notes

(Tools) Dr. Moyez Ladhany Mini-MAS – McMaster Already had a lot of the tools in place that they required, just needed to adapt them. Recognize that you will have multiple iterations and some hiccups along the way. Constant evaluation & re-evaluation will stop the curriculum & processes from becoming outdated and dysfunctional.

Achievable Tasks

• Give residents real-time feedback on how they are doing in the program

• Set clear expectations for the residents & faculty

• Assess performance in the clinical setting

Curriculum & Assessment for CBD

Tools for Implementation

1

23

Mul

tiple

iter

atio

ns &

Con

tinuo

us Im

prov

emen

ts

Trying our best to wade through the muddy waters of CBD

Developing & facilitating a culture of real-time feedback

Presenter

Presentation Notes

Muddy overall but used what we did know (tied DEC to $ = almost 100% return) Didn’t like the DEC at the time but helped with the culture change. RLB Lots of mistakes made Flexible & easy to manage!

Resident LogBook Report p.1 of 6

Presenter

Presentation Notes

RLB – great idea – data output not usable Residents & programs don’t know what to do with data

Centralized Learning Space

Presenter

Presentation Notes

Residents access frequently Centralized

Integration with MyCurriculum

Presenter

Presentation Notes

Contextualized it to our program Still not assessing competency but focuses their attention & helps build resident ownership

Progress Reporting

Presenter

Presentation Notes

Can look at future years as well. See how they are doing – if they are ahead on some clinical procedures, talk to the site coordinator to get different exposure.

1

23

Mul

tiple

iter

atio

ns &

Con

tinuo

us Im

prov

emen

ts

Gaining some additional clarity on resident & faculty expectations for CBD

Implementation of the new MicroCEXs in place of our Generic Daily Evaluation Card

Presenter

Presentation Notes

Developed some junior level MicroCEXs following Dr. Sherbino’s talk (looking at upcoming rotations & EPAs) Just get the forms out there and use them where applicable! Other days use DEC Learner driven!!

LESSON # 4: FACULTY BELIEVE IN AND LIKE COMPETENCY BASED ASSESSMENT…IT MAKES SENSE!

Presenter

Presentation Notes

Great face validity More focused assessment adds clarity for everyone

MicroCEX - Checklist

MicroCEX – Global Rating Scale

Key Features inform the Global Rating Scale narratives!

1

23

Mul

tiple

iter

atio

ns &

Con

tinuo

us Im

prov

emen

ts

Further development to

support real time feedback.

Setting clear expectations &

consequences for residents.

Presenter

Presentation Notes

Reporting tools & checklists are being put in place through MyCurriculum Identifying important MicroCEXs to be developed, developing them and mapping them to curriculum ID associated remediational steps if necessary & when these would become necessary.

Incorporating the Assessments

Checklist

Checklists & Reporting

1

23

Mul

tiple

iter

atio

ns &

Con

tinuo

us Im

prov

emen

ts

4?

Presenter

Presentation Notes

Continue to work on processes & culture (e.g. how & when will the residents receive f2f warnings they are behind; strike a competency committee) Keep creating the high yield tools recognizing that some of the content will change when the College releases CBD officially. Transformer

For the Future

• Mapping to FPA priority topics & Royal College EPAs

• Program syllabus• Online assessment forms rather than

paper• Aggregation software to show overall

resident progress (needs assessment to determine program requirements)

Overview of Lessons Learned

Lesson # 1: Don’t worry about the evolving definitions…They will just tell us!Lesson # 2: You do not have to assess everything, just the right things!Lesson # 3: Focus on Small wins when implementing change.Lesson # 4: Faculty believe in and like competency based assessment…it makes sense!

Presenter

Presentation Notes

Small pieces at a time! Not all at once and multiple iterations. Whatever you do will help your residents

What have you learned in this session?

What is one thing you will change because of this talk?

QUESTIONS?

Contact Information

Dr. Rob Anderson Program Director, Anesthesiologyranderson@nosm.ca

Dr. Cathy CervinAssociate Dean, PGMEccervin@nosm.ca

Christina TremblayAssistant Curriculum Instructional Designerctremblay@nosm.ca

mailto:randerson@nosm.ca

mailto:ccervin@nosm.ca

mailto:ctremblay@nosm.ca

Competency Based Medical Education 2015 03 26 N… · Triple C Competency-based LEARNING...

Documents