Translating calls for Translating calls for competencies-oriented, competencies-oriented, evidence-based education into evidence-based education into achievable and meaningful achievable and meaningful curricular reform initiatives curricular reform initiatives Frank J Papa, DO, PhD Frank J Papa, DO, PhD Assistant Dean Assistant Dean Curricular Design and Educational Curricular Design and Educational Technologies Technologies UNTHSC UNTHSC
Transcript
Translating calls for Translating calls for competencies-oriented, competencies-oriented,
evidence-based education into evidence-based education into achievable and meaningful achievable and meaningful curricular reform initiativescurricular reform initiatives
Frank J Papa, DO, PhDFrank J Papa, DO, PhD
Assistant DeanAssistant DeanCurricular Design and Educational Curricular Design and Educational
TechnologiesTechnologiesUNTHSCUNTHSC
IntroductionIntroduction
A Time of ChangeA Time of Change CompetenceCompetence
What are our current beliefs and practices What are our current beliefs and practices regarding competence?regarding competence?
What does the literature say about What does the literature say about competence?competence?
Instruction to CompetenceInstruction to Competence Assessment of CompetenceAssessment of Competence Building a competencies-oriented Building a competencies-oriented
curriculumcurriculum
A Time of ChangeA Time of Change
Patient Care arenaPatient Care arena Adoption of Patient Care Competencies:Adoption of Patient Care Competencies:
MEDED arenaMEDED arena Adoption of Educational Competencies:Adoption of Educational Competencies:
Medical KnowledgeMedical Knowledge OMMOMM Patient CarePatient Care ProfessionalismProfessionalism Interpersonal and Communication SkillsInterpersonal and Communication Skills Practice based Learning and ImprovementPractice based Learning and Improvement Systems based PracticeSystems based Practice
BEMEBEME
A Time of ChangeA Time of Change A Time of ConvergenceA Time of Convergence
Merging patient care and educational competenciesMerging patient care and educational competencies EBM literature guides patient care competenciesEBM literature guides patient care competencies Will ‘tradition’ guide the development of educational Will ‘tradition’ guide the development of educational
competencies or will evidence? competencies or will evidence?
What is known about educational competencies?What is known about educational competencies? Competence has an evidence base in selected areasCompetence has an evidence base in selected areas When will BEME really happen?When will BEME really happen?
A Time of ChangeA Time of Change A Timetable for ChangeA Timetable for Change
2010: ___COEB2010: ___COEB How will ___COEB curricula be How will ___COEB curricula be
defined/implemented?defined/implemented?
A Time of ChangeA Time of Change A Time for Leadership and CommitmentA Time for Leadership and Commitment
Who will define COEB curricular reform?Who will define COEB curricular reform? Will Osteopathic Medicine lead or follow?Will Osteopathic Medicine lead or follow? Reality check Reality check
Internal dialogue of the uncommitted (silent Internal dialogue of the uncommitted (silent majority/politically correct/astute)majority/politically correct/astute)
Who are we kidding? Who are we kidding? We’re too small, have too few resources, etc, etc, etcWe’re too small, have too few resources, etc, etc, etc Anything you can do (first) I can do better (later) (“me-too Anything you can do (first) I can do better (later) (“me-too
syndrome”)syndrome”) Isn’t my primary job to keep my job?Isn’t my primary job to keep my job? Was AT Still the first and last Osteopathic leader?Was AT Still the first and last Osteopathic leader?
Is the Osteopathic educational community willing Is the Osteopathic educational community willing to commit to the task of leading COEB reform?to commit to the task of leading COEB reform?
A Time of ChangeA Time of Change The Time for Leadership and Commitment The Time for Leadership and Commitment
is nowis now Someone will define and implement COEB Someone will define and implement COEB
reform!reform! Timetable will be around 2010!Timetable will be around 2010! Will Osteopathic Medicine lead or follow?Will Osteopathic Medicine lead or follow?
Who wants to commit themselves to the Who wants to commit themselves to the task of leading?task of leading?
Osteopathic educators can define and Osteopathic educators can define and implement the first COEB reform implement the first COEB reform
CompetenceCompetence
COMPETENCECOMPETENCE Beliefs and PracticesBeliefs and Practices EvidenceEvidence
CompetenceCompetence
Beliefs and PracticesBeliefs and Practices Competence happens somewhere along a Competence happens somewhere along a
Competence is Intellectual Skills-based Competence is Intellectual Skills-based (heavily predicated upon development of (heavily predicated upon development of problem solving skills (PSS))problem solving skills (PSS))
Time (length of training) determines Time (length of training) determines competencecompetence
Competence is norm referenced (adequately Competence is norm referenced (adequately assessed via traditional exams)assessed via traditional exams)
CompetenceCompetence
EvidenceEvidence Competence: “The Don’ts”Competence: “The Don’ts”
Don’t have to happenDon’t have to happen Don’t have much to do with the Don’t have much to do with the
development of PSSdevelopment of PSS Don’t depend upon length of training timeDon’t depend upon length of training time Don’t have to norm reference itDon’t have to norm reference it
CompetenceCompetence
Evidence: Evidence: Development of Intellectual Skills (IS) such as Development of Intellectual Skills (IS) such as
PSS in the absence of problem and task-specific PSS in the absence of problem and task-specific knowledge will not lead to competence (IS are knowledge will not lead to competence (IS are merely competency antecedents)merely competency antecedents)
Competence is predominately knowledge basedCompetence is predominately knowledge based Knowledge based competencies are Knowledge based competencies are
problem/task-specificproblem/task-specific Develops optimally with deliberate, focused Develops optimally with deliberate, focused
problem/task-specific training and feedbackproblem/task-specific training and feedback To be properly assessed, competencies must be To be properly assessed, competencies must be
measured one problem/task at a timemeasured one problem/task at a time
CompetenceCompetence
Impediments to COEBImpediments to COEB MEDED is OK the way it is!MEDED is OK the way it is!
We’ve been doing it this way for decades (comfort in We’ve been doing it this way for decades (comfort in traditions)traditions)
Everyone else does it this way (comfort in numbers)Everyone else does it this way (comfort in numbers) Same arguments against the adoption of EBM Same arguments against the adoption of EBM
in practicein practice We’ve been doing patient care this way for decades We’ve been doing patient care this way for decades Everyone else practices this wayEveryone else practices this way
COEB AcceleratorsCOEB Accelerators Understanding literatureUnderstanding literature Technology/simulationsTechnology/simulations
CompetenceCompetence
Competence fundamentalsCompetence fundamentals Intellectual Skill antecedentsIntellectual Skill antecedents Knowledge Base antecedentsKnowledge Base antecedents
CompetenceCompetence
Bloom’s Taxonomy Bloom’s Taxonomy A Hierarchy of Intellectual Skills and A Hierarchy of Intellectual Skills and
their development their development (competence antecedents)(competence antecedents)
Information acquisitionInformation acquisition ComprehensionComprehension ApplicationApplication AnalysisAnalysis SynthesisSynthesis GenerationGeneration
CompetenceCompetence
Competence and Bloom’s TaxonomyCompetence and Bloom’s Taxonomy Information acquisition (Naïve student)Information acquisition (Naïve student) Comprehension (Novice student)Comprehension (Novice student) ApplicationApplication AnalysisAnalysis SynthesisSynthesis
Using Bloom’s taxonomy to evaluate Using Bloom’s taxonomy to evaluate curricular reforms curricular reforms What percentage of curricular time is What percentage of curricular time is
devoted to supporting the development devoted to supporting the development of IS-related competence antecedents?of IS-related competence antecedents?
Knowledge Base antecedents to the Knowledge Base antecedents to the development of competence development of competence Competence is heavily predicated upon Competence is heavily predicated upon
how the knowledge base is organized or how the knowledge base is organized or structuredstructured
Knowledge-Based competencies appears Knowledge-Based competencies appears to be organized around:to be organized around: Specific ProblemsSpecific Problems Specific tasks associated with the solving of Specific tasks associated with the solving of
each problemeach problem
CompetenceCompetence
Hundreds of medical problemsHundreds of medical problems Chest pain, dyspnea, melena, vertigo, etcChest pain, dyspnea, melena, vertigo, etc
Three core medical tasks associated Three core medical tasks associated with every problemwith every problem Differential DiagnosisDifferential Diagnosis ExplanationExplanation TreatmentTreatment
CompetenceCompetence
Competence-related IS (application, analysis & Competence-related IS (application, analysis & synthesis) are optimally developed in the context synthesis) are optimally developed in the context of problem and task - specific learning of problem and task - specific learning opportunitiesopportunities
That is, CASE solving activities are the That is, CASE solving activities are the perquisite for applying, analyzing and perquisite for applying, analyzing and synthesizing their evolving problem and task-synthesizing their evolving problem and task-specific knowledge basesspecific knowledge bases
CASES are the catalyst, the grist for the CASES are the catalyst, the grist for the intellectual mill, the context within which IS and intellectual mill, the context within which IS and problem/task-specific knowledge bases are problem/task-specific knowledge bases are organized/structureorganized/structure
CompetenceCompetence
No cases, no IS, no PTSKBNo cases, no IS, no PTSKB The more cases, the greater the IS, The more cases, the greater the IS,
the better organized the PTSKBthe better organized the PTSKB Cases give birth to competenceCases give birth to competence
CompetenceCompetence
Antecedents to the development of Antecedents to the development of competence competence To support the development of both the IS and To support the development of both the IS and
Knowledge-Based antecedents to competence, Knowledge-Based antecedents to competence, you need to have:you need to have:
Problem and task-specific cases sufficient to support Problem and task-specific cases sufficient to support the development of competence-related IS the development of competence-related IS antecedentsantecedents
The curricular content (the learners future The curricular content (the learners future knowledge base) organized around problems and knowledge base) organized around problems and taskstasks
What percentage of curricular time is devoted What percentage of curricular time is devoted to supporting the development of problem and to supporting the development of problem and task-specific competence antecedents?task-specific competence antecedents?
CompetenceCompetenceApplication, Application, Analysis and Analysis and SynthesisSynthesis
Disc/Disc/
SysteSystemsms
PBLPBL PresPres COECOEBB
A case in isolationA case in isolation
A case in the A case in the context of a context of a problemproblem
Multiple cases in Multiple cases in the context of the context of a problem a problem
Multiple cases in Multiple cases in the context of the context of multiple multiple problems and problems and taskstasks
CompetenceCompetenceApplication, Application, Analysis and Analysis and SynthesisSynthesis
Disc/Disc/
SysteSystemsms
PBLPBL PresPres COECOEBB
A case in isolationA case in isolation
A case in the A case in the context of a context of a problemproblem
Multiple cases in Multiple cases in the context of the context of a problem a problem
Multiple cases in Multiple cases in the context of the context of multiple multiple problems and problems and taskstasks
5%5%
0%0%
0%0%
0%0%
CompetenceCompetenceApplication, Application, Analysis and Analysis and SynthesisSynthesis
Disc/Disc/
SysteSystemsms
PBLPBL PresPres COECOEBB
A case in isolationA case in isolation
A case in the A case in the context of a context of a problemproblem
Multiple cases in Multiple cases in the context of the context of a problem a problem
Multiple cases in Multiple cases in the context of the context of multiple multiple problems and problems and taskstasks
5%5%
0%0%
0%0%
0%0%
5%5%
5%5%
0%0%
0%0%
CompetenceCompetenceApplication, Application, Analysis and Analysis and SynthesisSynthesis
Disc/Disc/
SysteSystemsms
PBLPBL PresPres COECOEBB
A case in isolationA case in isolation
A case in the A case in the context of a context of a problemproblem
Multiple cases in Multiple cases in the context of the context of a problem a problem
Multiple cases in Multiple cases in the context of the context of multiple multiple problems and problems and taskstasks
5%5%
0%0%
0%0%
0%0%
5%5%
5%5%
0%0%
0%0%
5%5%
5%5%
5%5%
5%5%
CompetenceCompetenceApplication, Application, Analysis and Analysis and SynthesisSynthesis
Disc/Disc/
SysteSystemsms
PBLPBL PresPres COECOEBB
A case in isolationA case in isolation
A case in the A case in the context of a context of a problemproblem
Multiple cases in Multiple cases in the context of the context of a problem a problem
Multiple cases in Multiple cases in the context of the context of multiple multiple problems and problems and taskstasks
5%5%
0%0%
0%0%
0%0%
5%5%
5%5%
0%0%
0%0%
5%5%
5%5%
5%5%
5%5%
0%0%
0%0%
15-15-25%25%
20-20-30%30%
CompetenceCompetence
Constructing a COEB curriculumConstructing a COEB curriculum Significant reduction in time dedicated to Significant reduction in time dedicated to
information acquisitioninformation acquisition Significant increase in time dedicated to IS Significant increase in time dedicated to IS
provided via the context of multiple cases provided via the context of multiple cases representing multiple problems and tasksrepresenting multiple problems and tasks
Course content organized around problems and Course content organized around problems and taskstasks
Instruction to Instruction to CompetenceCompetence
Three core Knowledge-Based Three core Knowledge-Based CompetenciesCompetencies DDXDDX ExplanationExplanation TreatmentTreatment
DDX DDX The most difficult and resource intensive The most difficult and resource intensive
competence to train and learncompetence to train and learn
Instruction to Instruction to CompetenceCompetence
DDX DDX The most difficult and resource intensive The most difficult and resource intensive
competencecompetence Deliberate practice involving multiple problem and Deliberate practice involving multiple problem and
disease-specific case exposures as basis for disease-specific case exposures as basis for refinement of requisite IS (application, analysis and refinement of requisite IS (application, analysis and synthesis)synthesis)
DDX competence requires development of additional DDX competence requires development of additional IS (pattern matching and pattern discrimination) IS (pattern matching and pattern discrimination)
Training cases selected at appropriate level of Training cases selected at appropriate level of difficultydifficulty
Case-specific feedback oriented to refining Problem Case-specific feedback oriented to refining Problem and Disease-specific Knowledge Base and ISand Disease-specific Knowledge Base and IS
Opportunities for correction of errorsOpportunities for correction of errors
Instruction to Instruction to CompetenceCompetence
Computer-based simulationsComputer-based simulations High fidelity simulationsHigh fidelity simulations
High cost/resource intensive/limited training High cost/resource intensive/limited training opportunitiesopportunities
Demonstrated psychomotor benefitsDemonstrated psychomotor benefits Case availability and appropriateness very Case availability and appropriateness very
limitedlimited Significant feedback limitationsSignificant feedback limitations Faculty have significant limitations in defining Faculty have significant limitations in defining
learning objectiveslearning objectives Problem-specific, Knowledge-Based benefits not Problem-specific, Knowledge-Based benefits not
documenteddocumented
Instruction to Instruction to CompetenceCompetence
Faculty the opportunity to define problem and Faculty the opportunity to define problem and task (DDX)-specific training objectivestask (DDX)-specific training objectives
Multiple cases readily availableMultiple cases readily available Objective and subjective means of selecting Objective and subjective means of selecting
appropriate casesappropriate cases IS and Knowledge-Based feedback inherent to IS and Knowledge-Based feedback inherent to
KBIT training modulesKBIT training modules Student have opportunities to correct and Student have opportunities to correct and
immediately learn from errorsimmediately learn from errors
Instruction to Instruction to CompetenceCompetence
KBITKBIT EvidenceEvidence
KBIT training:KBIT training: Improves student DDX performance up to 25% greater Improves student DDX performance up to 25% greater
than faculty trained studentsthan faculty trained students Effect size-based performance improvements of one Effect size-based performance improvements of one
standard deviationstandard deviation Appears to correlate with, and perhaps support, Appears to correlate with, and perhaps support,
performance improvements in non-DDX tasks (explanation performance improvements in non-DDX tasks (explanation and treatment) (Dr Dubin)and treatment) (Dr Dubin)
Augments traditional coursework leading to increases in Augments traditional coursework leading to increases in student satisfaction indicesstudent satisfaction indices
Perceived as significantly contributing to students evolving Perceived as significantly contributing to students evolving sense of DDX competencesense of DDX competence
Provides instructional approaches and benefits not Provides instructional approaches and benefits not possible via traditional classroom instructionpossible via traditional classroom instruction
Instruction to Instruction to CompetenceCompetence
KBIT DDX training simulatorKBIT DDX training simulator Resource allocationsResource allocations
Minimal time investment of faculty (5-6 hours Minimal time investment of faculty (5-6 hours per module)per module)
Heavily leverages faculty effort /student training Heavily leverages faculty effort /student training ratio (ten to hundreds of times)ratio (ten to hundreds of times)
If one faculty member at one school develops a module If one faculty member at one school develops a module (6 hours development time), all students across all (6 hours development time), all students across all osteopathic schools can have access to the module osteopathic schools can have access to the module (thousands of hours of individualized instruction occur)(thousands of hours of individualized instruction occur)
Provides opportunity for participating faculty Provides opportunity for participating faculty and schools to engage in leading edge and schools to engage in leading edge educational researcheducational research
Accelerates osteopathic educators opportunity Accelerates osteopathic educators opportunity to create and implement COEB curriculato create and implement COEB curricula
Assessment of Assessment of CompetenceCompetence
KBITKBIT EvidenceEvidence
Provides objective measurement of problem Provides objective measurement of problem and disease-specific ‘performance to and disease-specific ‘performance to competence’ (Dr Oglesby)competence’ (Dr Oglesby)
Building a PTSCOEB Building a PTSCOEB curriculacurricula
One problem at a timeOne problem at a time Small reduction of information acquisition in a given Small reduction of information acquisition in a given
discipline or system course (approximately 2-3 days)discipline or system course (approximately 2-3 days) Each school inserts one Problem-Specific moduleEach school inserts one Problem-Specific module
Training to:Training to: DDX via KBITDDX via KBIT ExplanationExplanation Treatment Treatment
Defining and assessing competence performance levels in Defining and assessing competence performance levels in terms of:terms of:
DDX via KBITDDX via KBIT ExplanationExplanation TreatmentTreatment
Sharing module, methods and results across all Sharing module, methods and results across all schoolsschools
A Time of ChangeA Time of Change A Timetable for ChangeA Timetable for Change
Who are we kidding? Who are we kidding? We’re too small – have too few resources We’re too small – have too few resources D.O.’s always follow!D.O.’s always follow!
Isn’t my primary job to keep my job?Isn’t my primary job to keep my job?
A Time of ChangeA Time of Change A Time for Leadership and A Time for Leadership and
CommitmentCommitment Is the Osteopathic educational Is the Osteopathic educational
community willing to commit to the community willing to commit to the task of leading COEB reform?task of leading COEB reform?
A Time of ChangeA Time of Change COEB initiativeCOEB initiative
One problem module to be developed per One problem module to be developed per schoolschool
Twenty modules available to all schools Twenty modules available to all schools after one yearafter one year
100 modules available to all schools by 100 modules available to all schools by 20102010
SOME publishes results of COEB SOME publishes results of COEB initiativeinitiative
A Time of ChangeA Time of Change Come to workshop this Friday eveningCome to workshop this Friday evening Ask questionsAsk questions Dream dreamsDream dreams Make commitment to create tomorrow's realityMake commitment to create tomorrow's reality
Academic MedicineAcademic Medicine Proceedings of the RIME Conference: November, Proceedings of the RIME Conference: November,
20102010 Creation and Implementation of Competencies-Creation and Implementation of Competencies-
Oriented, Evidence-Based Curricula. Authors – SOMEOriented, Evidence-Based Curricula. Authors – SOME New York Times & Wall Street Journal articles state: New York Times & Wall Street Journal articles state:
Osteopathic educators step forward and create first Osteopathic educators step forward and create first 2121stst century curricula; allopathic educators century curricula; allopathic educators acknowledge preliminary data looks promisingacknowledge preliminary data looks promising
