Best Evidence Medical Education

Hani AlmoallimMBBS, ABIM, FRCPC, FRCPC (Rheum), DipMedEdAssistant Professor, UQUConsultant IM & RheumatologyDiploma in Med Ed – Dundee, Scotland Fellowship in Med Ed – UBC, CanadaHead of Medical Ed Committee - UQU

Objectives

Two major objectives:

To judge the actions that we do as teachers and learners.

To apply the evidence where it is available and relevant.

Teachers should use research to enhance their teaching practice.

Outline

What is BEME? The QUESTS model.

Applications: Teaching EBM. How can we conduct journal clubs? Medical grand rounds.

Tell me and I Tell me and I FORGETFORGET

Teach me and I Teach me and I REMEMBERREMEMBER

Involve me then I Involve me then I LEARNLEARN..

Exercise

Exercise

True or False: Most health care professions are not aware that

there is literature on education.

We do the things we do because that is the way we have been raised ourselves.

Any change or challenge to one’s convictions is an actual challenge to one’s professional integrity.

What is BEME?

It is the implementation, by teachers in their practice, of methods and approaches to education based on the best evidence available. (Harden et al, Medical Teacher 1999, 21(6))

Opinion-based

teaching

Evidence-based

teachingOR

Opinion-based

teaching

Evidence-based

teaching

Do we need it?

“ The evidence base is as important in educating new doctors as it is in assessing a new chemotherapy ” (Harden 1998, Hart 1999, Peterson 1999

in Medical Teacher & BMJ)

Education Trained Doctor

Improved performance

Improved health care

Problems with educational research

A family wants to purchase a tree

Literature

Books

BEME: the simple truth, Harden et al, Medical Teacher, 2000 (22)

What went wrong?

How to grade the evidence? The QUESTS model:

1. Quality How good is the evidence. Is it based on professional judgment,

educational principles, case studies, cohort studies, RCT.

The method itself does not guarantee the quality.

How to grade the evidence? The QUESTS model:

2. Utility: To what extent can the method be

transferred and adopted without modification.

Lectures/ PBL as examples.

How to grade the evidence? The QUESTS model:

3. Extent What is the extent of the evidence? Is the evidence based on a single study or

on metanalysis?

How to grade the evidence? The QUESTS model:

4. Strength How strong is the evidence?

Examples: Simulated patients in medical education

research. Co-operative learning.

How to grade the evidence? The QUESTS model:

5. Target What is the target? What is being measured? The extent to which the outcomes measured

by researcher and outcomes expected by the teacher are similar.

How valid is the evidence?

How to grade the evidence? The QUESTS model:

6. Setting How close does the context or setting

approximate? How relevant is the evidence?

How to use this model?

Quality

Utility

Extent

Strength

Target

Setting

A B C

A B C

A B C

A B C

A B C

A B C

How to use this model?

Quality

Utility

Extent

Strength

Target

Setting

X

X

X

X

X

X

A B C

Truth is rarely pure and never simple.

(Oscar Wilde)

What is the aim then of BEME?

Actions

Medical teachers

Evidence

Improvement

Judge

Think

Judge

Think

Role of Clinical Teachers (Ullian et al 1994)

1. Physician role:

• Knowledgeable and competent.

• Empathy with patients.

• Demonstrate professional attitudes, values and behavior.

Applications

Teaching EBM

Teaching EBM

Should it be part of training programs? Can teaching EBM change practice? Can evidence be applied during clinical

rounds?

Should it be part of training programs?

The Accreditation Council of Graduate Medical Education requires the inclusion of decision making skills by identifying, appraising and applying evidence from clinical research in internal medicine program curricula. (1998)

417 Internal Medicine Program Directors

269 (65%) responded

99 (37%) offered a freestanding EBM curricula

Objectives

• Performing critical appraisal (78%)

• Searching for evidence (53%)

• Posing focused question (44%).

• Applying the evidence in decision making (35%)

97% provided MEDLINE

Green ML, J Gen Intern Med 2000, 15.

Can teaching EBM change practice?

35 attending physician and 12 medical residents

EBM course, syllabus and textbook35 attending physician and 12 medical residents

Month before Month after

Therapies for 483 patients

Significantly more proven therapies based on RCT

62% vs 49% (P=.023)Straus SA et al, J Gen Int

Med,2005;20:340-343

Evidence Cart:CAT, Redbook, RCE,Medline, Cochrane, PE textbook, Radiology textbook,Simuloscop

Clinical rounds:Medical students, house staff, fellows, consultants

Use of evidence in 1 month

•Evidence Cart was used 98 times.

•79(81%) sought evidence that could affect diagnostic and/or treatment decision.

• 71 (90%) were successful.

•37/71 (51%) confirmed their diagnosis and/or treatment

•10.2-25.4 sec to review some resources

•18/71(25%) let to a new skill and management decision.

•16/71 (23%) corrected a previous skill or a management decision

• 5 searches only performed when evidence was needed in 41 patients. (12%)

Sacket DL et al, JAMA 1998, 280

Exercise

Exercise

If extensive EBM course was conducted in your institution do you think that it will change practice?

Discuss with your neighbor!

To …… hospital administration:

“To practice EBM efficiently, residents need access to a range of information resources, like MEDLINE and on-site electronic medical information” Green ML, J Evaluation Clinical Practice 2000, 6(2)

Role of Clinical Teachers (Ullian et al 1994)

2. Teacher role

• Interested in teaching.

• Spends time with learners.

• Explains and answer questions.

• Available to teach.

Journal Clubs

Journal Clubs

Should it be part of training programs? In what format? Are they effective?

Journal Clubs

They are maintained in 95% of internal medicine programs. (Sidorov 1995)

79% of family practice programs. (Heiligman et al 1987)

100% of physical medicine and rehabilitation programs. (Moberg et al 1995).

What is the format?

The most common format: Meeting once a month during working hours. 2-3 articles related to one subject. Only original research were discussed. A distinguished leader. A biostatician may be present.

(Alguire PC, 1998, J Gen Intern Med (13))

What is the ideal format?

An ideal J club format does not exist!

Are journal clubs suitable to teach evidence-based decision making?

Green, M, J Eval Clin Pract, 2000, 6(2)

Exercise:

What do you think the characteristics of J.Club with high attendance and longevity?

1. Mandatory attendance.2. Availability of food.3. Perceived importance by the program director.4. Others: smaller training programs, independent

from faculty journal club, formal teaching in epidemiology…

(Alguire PC, 1998, J Gen Intern Med (13))

Effectiveness?

A systematic review (Cochrance methodology) of all studies on J.Club to evaluate its effectiveness.

Only 7/306 studies met the inclusion criteria. (Ebbert JO et al, Medical Teacher 2001, 23(5))

Results: There was a statistically significant improvement in

epidemiology and biostatistics knowledge, change in reading habits, and increase the use of medical literature in practice.

There was a trend to improve the critical appraisal skills.

Criteria P<0.05 P>0.05

epidemiology and biostatistics knowledge.

Reading habits.

Use of medical literature in practice.

Critical appraisal skills.

Role of Clinical Teachers (Ullian et al 1994)

3. Supervisor

• Provide directions and feedback.

• Delegates responsibilities.

• Involve learners in management.

What do you think about MEDICAL GRAND ROUNDS in your institution?

Re-examining the value of medical grand rounds

A questionnaire was sent 389 US departments of medicine chairs, 300 responded (77%).

Less than 10% were clinical case presentations and 1% were interactive workshops.

High absentee rate.

Do Conferences, Workshops, Rounds, and Other traditional CME activities

Change Physician Behavior or Health Care Outcome?

Davis, D et al, JAMA, 1999, 282,No 9

Conclusion

Didactic sessions do not appear to be effective in changing physician performance.

Interactive CME sessions can effect change in professional practice and, on occasion, health care outcome.

Role of Clinical Teachers (Ullian et al 1994)

4. Supporter

• Friendly, helpful, caring.

• Values learners as individuals.

• Positive attitudes to patients and to own career.

Are you convinced about the need to distribute the concept of BEME?

Exercise

Exercise

How can you distribute the culture of the need to BEME in your institution?

Discuss with your neighbor.

Distributing the culture of BEME

Train the teachers. (who is the effective teacher?)

Train the learners. (how good is the teaching? Do they learn something from us? Do they feel enthusiastic about learning at the end of our teaching?)

Improve a two way approach of assessment. Open discussions to our problems.

Distributing the culture of BEME

Acknowledging and rewarding teachers, Considering excellency in teaching a

prerequisite for promotion and appointments for staff,

A committee for educational development,

Distributing the culture of BEME

Free access to medical education journals, Good library with advanced web-based services, Grand round in medical education, Medical education in journal club sessions, Training courses.

Teachers should use research to enhance their teaching practice.

Thank you