Medical Education Outcomes ResearchMedical Education Outcomes Research

Frederick Chen, MD, MPHFrederick Chen, MD, MPHCenter for Primary Care ResearchCenter for Primary Care Research

Agency for Healthcare Research and QualityAgency for Healthcare Research and QualityJune 26, 2003June 26, 2003

Setting the ContextSetting the Context

What is the goal of medical education?What is the goal of medical education?

To produce physicians who deliver high quality To produce physicians who deliver high quality carecare

““The quality of care that the public receives is The quality of care that the public receives is determined to some extent by the quality of determined to some extent by the quality of medical education students and residents medical education students and residents receive.” – Commonwealth Report 2002receive.” – Commonwealth Report 2002

The QuestionsThe Questions

How do we demonstrate that medical education How do we demonstrate that medical education is producing physicians who deliver high quality is producing physicians who deliver high quality care?care?

Is the delivery of high quality care related in Is the delivery of high quality care related in any way to medical education?any way to medical education?

How do we introduce the measurement of How do we introduce the measurement of patient-centered clinical outcomes to medical patient-centered clinical outcomes to medical education research?education research?

Why we should study medical Why we should study medical education outcomeseducation outcomes

Accountability in medical educationAccountability in medical education ““American public, policymakers, and private health American public, policymakers, and private health

care managers have a pressing stake in the health of care managers have a pressing stake in the health of our nation’s medical education enterprise.”our nation’s medical education enterprise.”

Government already involved in licensing, financingGovernment already involved in licensing, financing

Why we should study medical Why we should study medical education outcomeseducation outcomes

Medicare is largest supporter of graduate medical Medicare is largest supporter of graduate medical educationeducation– $7.8 billion in 2000$7.8 billion in 2000– >100,000 medical residents in training>100,000 medical residents in training

Over 75% of medical schools receive public subsidiesOver 75% of medical schools receive public subsidies– At least $2 billionAt least $2 billion

Better available methodology and dataBetter available methodology and data

Commonwealth Report 2002Commonwealth Report 2002

Principle: Academic health centers should be Principle: Academic health centers should be held accountable for their performance in held accountable for their performance in educating the nation’s physicians.educating the nation’s physicians.

Finding: The available data are insufficient to Finding: The available data are insufficient to judge the performance of AHCs in discharging judge the performance of AHCs in discharging their education responsibilities beyond their education responsibilities beyond establishing a minimum level of competency.establishing a minimum level of competency.

Commonwealth Report 2002Commonwealth Report 2002

Recommendation: Government should support Recommendation: Government should support research to produce valid and reliable research to produce valid and reliable measures of the costs and quality of medical measures of the costs and quality of medical education.education.

$25M per year to AHRQ to develop and $25M per year to AHRQ to develop and implement improved measures of performanceimplement improved measures of performance

Convene AAMC, ACGME, NBME, ABMS, Convene AAMC, ACGME, NBME, ABMS, HRSA to provide direction…HRSA to provide direction…

IOM - Health Professions EducationIOM - Health Professions Education

Quality ChasmQuality Chasm Recommendation #12 Recommendation #12

Health care professionals need to be better Health care professionals need to be better prepared in order to provide the highest quality prepared in order to provide the highest quality & safest care possible, and to function at & safest care possible, and to function at optimum levels in an ever evolving and optimum levels in an ever evolving and increasingly complex 21increasingly complex 21stst century health system century health system

Research & InformationResearch & Information

AHRQ and foundations should support ongoing AHRQ and foundations should support ongoing research projects focused on the core competencies research projects focused on the core competencies and links with individual/ population health and links and links with individual/ population health and links with educational approacheswith educational approaches

Develop measures reflecting the core set of Develop measures reflecting the core set of competenciescompetencies

AHRQ should issue report cards focused on clinical AHRQ should issue report cards focused on clinical educational institutionseducational institutions

Medical Education ResearchMedical Education Research

What has been the state of medical education What has been the state of medical education research?research?

Authority for medical education historically Authority for medical education historically resides in professionresides in profession

Intrinsic capacity to self-regulateIntrinsic capacity to self-regulate

Medical Education ResearchMedical Education Research

Past 30 years have focused on:Past 30 years have focused on:– Basic research on reasoningBasic research on reasoning– Use of knowledgeUse of knowledge– Problem based learningProblem based learning– Performance assessment (OSCE, std patients)Performance assessment (OSCE, std patients)– Provision of continuing education Provision of continuing education

Norman, G. BMJ 2002;324Norman, G. BMJ 2002;324

Lack of patient outcomes in Lack of patient outcomes in medical education researchmedical education research

Review of 600 research articles published in Review of 600 research articles published in medical education journals (1996-98)medical education journals (1996-98)

Only 4 measured clinical outcomes of patientsOnly 4 measured clinical outcomes of patients Half measured trainee performanceHalf measured trainee performance 34% measured trainee satisfaction34% measured trainee satisfaction

– Prystowsky and Bordage. Med Ed 2001; 35.Prystowsky and Bordage. Med Ed 2001; 35.

Why haven’t we studied patient Why haven’t we studied patient outcomes?outcomes?

Focus on undergraduate educationFocus on undergraduate education Students able to overcome educational interventionsStudents able to overcome educational interventions Not able to conduct randomized, blinded trials Not able to conduct randomized, blinded trials No reliable data on outcomesNo reliable data on outcomes No money for researchNo money for research

Sound familiar?Sound familiar?

Current effortsCurrent efforts

Best Evidence Medical EducationBest Evidence Medical Education International effort to apply evidence-based International effort to apply evidence-based

medicine criteria to medical education researchmedicine criteria to medical education research Campbell CollaborationCampbell Collaboration

Current effortsCurrent efforts

ACGME Outcome Project – Competencies:ACGME Outcome Project – Competencies:– Patient carePatient care– Medical knowledgeMedical knowledge– Interpersonal and communication skillsInterpersonal and communication skills– ProfessionalismProfessionalism– Practice-based learning and improvementPractice-based learning and improvement– Systems-based practiceSystems-based practice

Expert MeetingExpert Meeting

AHRQ - HRSA co-sponsored Expert meetingAHRQ - HRSA co-sponsored Expert meeting July 2001July 2001 AHRQ – lead agency for healthcare qualityAHRQ – lead agency for healthcare quality HRSA – supports training of healthcare HRSA – supports training of healthcare

providersproviders Investigate the link between medical education Investigate the link between medical education

and health care improvementand health care improvement

Role of quality improvementRole of quality improvement

How do physicians learn about CQI?How do physicians learn about CQI? What is effect of medical education uponWhat is effect of medical education upon

– Ability to change and adapt?Ability to change and adapt?– Ability to improve practice?Ability to improve practice?

Do physicians trained in one system bring Do physicians trained in one system bring those skills to another system?those skills to another system?

Multi-method approachesMulti-method approaches

Convergence of medical education researchers Convergence of medical education researchers and health services researchersand health services researchers

Multi-institutional Multi-institutional New statistical methodsNew statistical methods Quantitative and Qualitative assessments of Quantitative and Qualitative assessments of

quality and performancequality and performance

Overarching GoalsOverarching Goals

Making the link between Quality and Medical Making the link between Quality and Medical EducationEducation

Current focus on systems-change but what is Current focus on systems-change but what is influence of individual performance and roles influence of individual performance and roles within systems?within systems?

Evaluation drives CurriculumEvaluation drives Curriculum

Next StepsNext Steps

Need to catalyze research that examines Need to catalyze research that examines linkage between quality health care and linkage between quality health care and medical educationmedical education

Commitment to patient outcomesCommitment to patient outcomes Need to weave medical education back into Need to weave medical education back into

fabric systems improvementfabric systems improvement

Role Of AACOMRole Of AACOM

Interest in developing research among faculty Interest in developing research among faculty and studentsand students

Active partnership in public efforts to improve Active partnership in public efforts to improve medical education – IOM task forcemedical education – IOM task force

Encourage active engagement with AHRQ, Encourage active engagement with AHRQ, HRSA in medical education researchHRSA in medical education research

Past effortsPast efforts

1993 BHPr-AAMC agenda setting conference1993 BHPr-AAMC agenda setting conference Research in medical education: Research in medical education:

– Policies for the FuturePolicies for the Future

Created Centers for medical education Created Centers for medical education research (mostly workforce and specialty research (mostly workforce and specialty choice)choice)