Weighting CanMEDS Roles & Competencies

Marilyn Singer MD, FCFP Director, Clinician Assessment ProgramsBrenda Stutsky RN, PhD, Program Advisor,

Manitoba Practice Assessment Program

Coalition for Physician EnhancementJune 3rd Madison, Wisconsin

Conflict of Interest Disclosure

• None to disclose

Objectives

• Describe the Manitoba Practice Assessment Program (MPAP)

• Describe the results of the CanMEDS weighting research project

• Discuss implications

Mission/Goal of the MPAP

• To provide an assessment of physicians in their individual practices to assist with clarification of licensure status

• To offer an assessment process that is:– Comprehensive– Transparent– Tailored to the physician’s specialty and practice

setting

Assessment Components

• Self-Assessment– Self-Assessment Report– Self-Assessment: Reflective Practice (Matches 360’ Questions)– Self-Assessment: Clinical Practice Skills

• 360 Degree Surveys– Physician Colleagues– Interprofessional Colleagues– Patients

• Chart Audit/Chart Stimulated Recall• Observation of Clinical Practice• Key Interviews• Comprehensive Report

Framework

CanMEDS WeightingProject

Research Questions:1. How do physicians rate the

complexity, frequency, and criticality of CanMEDS roles?

2. How do the CanMEDS roles compare with one another in terms of their relative levels of perceived importance?

Image from Creative Commonshttp://www.flickr.com/photos/davehamster/531768924/sizes/m/in/photostream/

2 Online Surveys

Sample

• Physicians on Scott’s Directories: Canadian Medical Directory– BC, AB, SK, MB, NWT, Yukon, & Nunavut– Final sample size N=88 (23 family practice, 65

specialists) - limited generalizability

Image from Creative Commonshttp://www.flickr.com/photos/mega1210/4948863429/sizes/m/in/photostream/

Analysis

• Index of Practice Importance– Importance = Frequency x Criticality

• Activity Weight– Weight = Sum of Mean Importance/Number of

Competencies (for each of the 7 CanMEDS roles)

ReferencesKane, M. T., Kingsbury, C., Colton, D., Estes, C. (1989). Combining data on criticality and frequency in developing test plans for licensure and certification examinations. Journal of Educational Measurement, 26(1), 17-27.

Raymond, M. R. (2001). Job analysis and the specification of content for licensure and certification examinations. Applied Measurement in Education, 14(4), 369-415.

Results

• No significant difference between family practice and specialists in index of practice importance scores.

• Mean complexity scores:– Medical Expert 4.36– Collaborator 4.04– Communicator 4.03– Health Advocate 3.99– Manager 3.96– Professional 3.85

CanMEDS Weighting

Implications

• Validation of the CanMEDS framework• One set of tools can be used for both family

physicians and specialists• All roles are important when determining

overall competence• Curriculum development for undergrad, post-

grad, and continuing medical education

Thank You!www.umanitoba.ca/cpd

b_stutsky@umanitoba.ca

singer@cc.umanitoba.ca