47
G R A C E B R A N T I N G H A M , O M S I V *
B E T H A N Y M U L L I N I X , O M S I V *
S H A R O N C A S A P U L L A , E D D *
L A R R Y H U R T U B I S E , M A ┼
S T E V A N W A L K O W S K I , D O *
WHAT ARE WE TEACHING?:
MAPPING PRE-CLINICAL OMM CURRICULUM TO AACOM CORE COMPETENCIES
*Ohio University – Heritage College of Osteopathic Medicine
┼ Nationwide Children's Hospital
Who We Are
Our Context
Our Research Quest ions
Background
WHO WE ARE
• We are FM/OMM teaching fellows at OU-HCOM Cleveland, we intend to work in underserved settings.
• We are also a part of the Rural and Urban Scholars Program (RUSP).
• OU-HCOM Mission: Train osteopathic primary care physicians to serve Ohio.
• DOs disproportionately work in primary care in rural and underserved areas. - (AOA OMP Report, 2014)
• What’s the secret sauce? We don’t know, but we think OMM education may have something to do with it.
OUR CONTEXT• OU-HCOM’s medical curriculum is scheduled for
transformation in 2018.
• Competency-based education and evaluation is on the horizon.
• We learned about the AACOM Competencies, and felt they would help us map our OMM curriculum. – Ongoing quality improvement
– Future curricular transformation
Competency Based Education
• “complex set of behaviors built on components of knowledge, skills, attitudes, and ‘competence’ as a personal ability.” – Carraccio et al. 2002
• AACOM Competency Report 2012 http://www.aacom.org/docs/default-source/insideome-2012/corecompetencyreport2012.pdf?sfvrsn=0
AACOM Osteopathic Core Competencies for Medical Students, 2012, p. 3
Competency Abbreviation (ours)
Osteopathic Principles and Practices OPP
Medical Knowledge MK
Patient Care PC
Interpersonal and Communication Skills ICS
Professionalism P
Practice Based Learning and Improvement PBLI
Systems Based Practice SBP
Counseling for Health Promotion/Disease Prevention CHP/DP
Cultural Competencies CC
Evaluation of Health Sciences Literature EHSL
Environmental and Occupational Medicine EOM
Public Health Systems PHS
Global Health GH
Interprofessional Collaboration IC
RESEARCH QUESTIONS
• Which OPP performance indicators and objectives are being addressed in our OMM Learning Activities, and how often?
• Which non-OPP competencies, indicators and objectives are being addressed in our OMM Learning Activities, and how often?
• Can a structured survey help an academic department self-evaluate its curriculum in real time?
Methods
Steps in Action Research
Although there are different ideas of what steps action research entails, the general format is:1. Identify the problem area2. Collect and organize data3. Interpret the data4. Develop and implement an
action, based on the data5. Reflect upon the process6. Next steps, if applicable
Methods
• Qualtrics Research Suite
– Created survey utilizing the AACOM Core Competencies verbatim (with approval from AACOM).
– https://ohio.qualtrics.com/SE/?SID=SV_9ZERC4rdj0LQYAJ
• Email post-Learning Activity:– AACOM Report
– Tutorial
• Data collected from presenters and participant observers
http://hsceramics.weebly.com/pinch-pot-wind-chimes.html
Osteopathic Core Competencies for Medical Students, 2012, p. 6
Competency hierarchy
ObjectivesPerformance
IndicatorsCore
Competency
OPP
Approach patient with recognition of bio/psycho/social
context
Recognize and treat whole patient: mind/body/spirit
Obtain consent for all OMM
Listen/communicate effectively when treating dysfunction
Identify potential contraindications
Understand how culture affects presentation of SD
Use relationship between structure and
function to promote health
Integrate OMT as method to improve anatomy
Apply biomedical science to OMT
Form an OMT treatment plan
Connect organ systems and structural findings
Results(preliminary)
Fall Year 1 response rate:
Activity Total Presenter Completion Participant Observer Completion
Lectures 15 13 (86.7%) 15 (100%)
Labs 33 30 (90.9%) 33 (100%)
Total 48 43 (89.6%) 48 (100%)
*Labs are hands-on learning activities taught in OMM lab, lectures are taught in a traditional classroom setting.
124
22
13
4
24
6
0
20
40
60
80
100
120
140
OPP (Total) Medical Knowledge(Total)
Patient Care (Total) InterpersonalCommunication
Skills (Total)
Professionalism(Total)
Other (Total)
Ind
icat
ors
Se
lect
ed
Competencies Addressed
Performance Indicators Selected by Presenters: Fall Yr 1
0
5
10
15
20
25
30
35
40
OP
P I.
1O
PP
I.2O
PP
I.3O
PP
I.4O
PP
I.5
OP
P I.
6O
PP
I.7
OP
P I.
8M
K II
.1M
K II
.2M
K II
.3P
C II
I.1P
C II
I.2P
C II
I.3P
C II
I.4P
C II
I.5P
C II
I.6
ICS
IV.1
ICS
IV.2
ICS
IV.3
ICS
IV.4
P V
.1P
V.2
P V
.3P
V.4
P V
.5P
V.6
P V
.7P
V.8
P V
.9P
BLI
VI.1
PB
LI V
I.2P
BLI
VI.3
PB
LI V
I.4P
BLI
VI.5
SBP
VII
.1SB
P V
II.2
SBP
VII
.3SB
P V
II.4
SBP
VII
.5C
HP
/DP CC
EHSL
EOM
PH
SG
HIC
C
Lear
nin
g A
ctiv
itie
s
Performance Indicators
Performance Indicators Selected by Presenters: Fall Yr 1
OPP Osteopathic Principles and PracticesMK Medical Knowledge PC Patient Care ICS Interpersonal and Communication Skills P Professionalism PBLI Practice Based Learning and ImprovementSBP Systems Based PracticeCHP Counseling for Health Promotion/Disease PreventionCC Cultural competencies EHSL Evaluation of Health Science LiteratureEOM Environmental and Occupational Medicine PHS Public Health Systems GH Global Health ICC Interprofessional Collaboration
0
5
10
15
20
25
30
35
40
OP
P I.
1O
PP
I.2
OP
PI.
3O
PP
I.4
OP
P I.
5O
PP
I.6
OP
P I.
7O
PP
I.8
MK
II.1
MK
II.2
MK
II.3
PC
III.
1P
C II
I.2
PC
III.
3P
C II
I.4
PC
III.
5P
C II
I.6IC
S IV
.1IC
S IV
.2IC
S IV
.3IC
S IV
.4P
V.1
P V
.2P
V.3
P V
.4P
V.5
P V
.6P
V.7
P V
.8P
V.9
PB
LI V
I.1
PB
LI V
I.2
PB
LI V
I.3
PB
LI V
I.4
PB
LI V
I.5
SBP
VII.
1SB
P V
II.2
SBP
VII.
3SB
P V
II.4
SBP
VII.
5C
HP
/DP CC
EHSL
EOM
PH
SG
HIC
C
Lear
nin
g A
ctiv
itie
s
Performance Indicators
Performance Indicators Selected by Presenters: Fall Yr 1
Non-OPP: 69 (36%)
OPP: 124(64%)
159
49
36
410
20
20
40
60
80
100
120
140
160
180
OPP total Medical Knowledgetotal
Patient care total Interpersonal andCommunication
Skills total
Professionalismtotal
Other
Ind
icat
ors
Se
lect
ed
Competencies Addressed
Performance Indicators Selected by Participant Observers: Fall Year 1
0
5
10
15
20
25
30
35
40
45O
PP
I.1
OP
PI.2
OP
PI.3
OP
PI.4
OP
P I.
5O
PP
I.6
OP
P I.
7O
PP
I.8
MK
II.1
MK
II.2
MK
II.3
PC
III.1
PC
III.2
PC
III.3
PC
III.4
PC
III.5
PC
III.6
ICS
IV.1
ICS
IV.2
ICS
IV.3
ICS
IV.4
P V
.1P
V.2
P V
.3P
V.4
P V
.5P
V.6
P V
.7P
V.8
P V
.9P
BLI
VI.1
PB
LI V
I.2P
BLI
VI.3
PB
LI V
I.4P
BLI
VI.5
SBP
SBP
VII.
2SB
P V
II.3
SBP
VII.
4SB
P V
II.5
CH
P/D
P CC
EHSL
EOM
PH
SG
HIC
C
Lear
nin
g A
ctiv
itie
s
Performance Indicators
Performance Indicators Selected by Participant Observers: Fall Yr 1
OPP Osteopathic Principles and PracticesMK Medical Knowledge PC Patient Care ICS Interpersonal and Communication Skills P Professionalism PBLI Practice Based Learning and ImprovementSBP Systems Based PracticeCHP Counseling for Health Promotion/Disease PreventionCC Cultural competencies EHSL Evaluation of Health Science LiteratureEOM Environmental and Occupational Medicine PHS Public Health Systems GH Global Health ICC Interprofessional Collaboration
0
5
10
15
20
25
30
35
40
45O
PP
I.1
OP
PI.2
OP
PI.3
OP
PI.4
OP
P I.
5O
PP
I.6
OP
P I.
7O
PP
I.8
MK
II.1
MK
II.2
MK
II.3
PC
III.1
PC
III.2
PC
III.3
PC
III.4
PC
III.5
PC
III.6
ICS
IV.1
ICS
IV.2
ICS
IV.3
ICS
IV.4
P V
.1P
V.2
P V
.3P
V.4
P V
.5P
V.6
P V
.7P
V.8
P V
.9P
BLI
VI.
1P
BLI
VI.
2P
BLI
VI.
3P
BLI
VI.
4P
BLI
VI.
5SB
PSB
P V
II.2
SBP
VII.
3SB
P V
II.4
SBP
VII
.5C
HP
/DP CC
EHSL
EOM
PH
SG
HIC
C
Lear
nin
g A
ctiv
itie
s
Performance Indicators
Performance Indicators Selected by Participant Observers: Fall Yr 1
OPP: 159(62%)
Non-OPP: 96 (38%)
0
5
10
15
20
25
30
35
40
45
OP
P 1
OP
P 2
OP
P 3
OP
P 4
OP
P 5
OP
P 6
OP
P 7
OP
P 8
MK
1M
K 2
MK
3P
C 1
PC
2P
C 3
PC
4P
C 5
PC
6IC
S 1
ICS
2IC
S 3
ICS
4P
1P
2P
3P
4P
5P
6P
7P
8P
9P
BLI
1P
BLI
2P
BLI
3P
BLI
4P
BLI
5SB
P 1
SBP
2SB
P 3
SBP
4SB
P 5
CH
P/D
P CC
EHSL
EOM
PH
SG
HIC
C
Lear
nin
g A
ctiv
itie
s
Performance Indicators
Presenters v. Participant Observers Selections - Fall Yr 1
Participant Observers Presenters
OPP Osteopathic Principles and PracticesMK Medical Knowledge PC Patient Care ICS Interpersonal and Communication Skills P Professionalism PBLI Practice Based Learning and ImprovementSBP Systems Based PracticeCHP Counseling for Health Promotion/Disease PreventionCC Cultural competencies EHSL Evaluation of Health Science LiteratureEOM Environmental and Occupational Medicine PHS Public Health Systems GH Global Health ICC Interprofessional Collaboration
Interesting Discrepancies
• Medical Knowledge: – 1. Articulate basic biomedical science and epidemiological and clinical science
principles related to patient presentation in the following areas: (A-J)• A. Understand and apply the concepts of osteopathic principles and practices. (Pr 9; Ob 34)
• D. Musculoskeletal System (Pr 10; Ob 34)
• Patient Care– 3. Implement essential clinical procedures. (A-R)
• B. Perform an osteopathic structural examination and osteopathic manipulative therapy as appropriate. (Pr 3; Ob 20)
• Professionalism– 2. HUMANISTIC BEHAVIOR – Demonstrate humanistic behavior, including
respect, compassion, probity, honesty, and trustworthiness. (Pr 11; Ob 6)
Key: Pr= presenter Ob= participant observer
Year 1 Initial Trends • OPP performance indicators accounted for the majority of
those selected for OMM Learning Activities.
• Performance indicators in Medical Knowledge, Patient Care, and Professionalism were the most commonly chosen outside of OPP.
• Mapping tools filled out by faculty show different emphasis than those filled out by participant observers – Participant observers tended to select more OPP and
proportionally more non-OPP performance indicators. – Presenters said professionalism was covered more often than
did participant observers.
Fall Year 2 response rate:
Activity Total Presenter Completion
Lectures 15 11 (73.3%)
Labs 16 12 (75.0%)
Total 31 23 (74.2%)
*Labs are hands-on learning activities taught in OMM lab, lectures are taught in a traditional classroom setting.
76
8
17
3
10
3
0
10
20
30
40
50
60
70
80
OPP MK PC ICS P Other
Per
form
ance
ind
icat
ors
se
lect
ed
Competencies Addressed
Year 2 Performance Indicators Within Each Competency, Selected by Presenters
0
5
10
15
20
25
OP
P 1
OP
P 2
OP
P 3
OP
P 4
OP
P 5
OP
P 6
OP
P 7
OP
P 8
MK
1M
K 2
MK
3P
C 1
PC
2P
C 3
PC
4P
C 5
PC
6IC
S 1
ICS
2IC
S 3
ICS
4P
1P
2P
3P
4P
5P
6P
7P
8P
9P
BLI
1P
BLI
2P
BLI
3P
BLI
4P
BLI
5SB
P 1
SBP
2SB
P 3
SBP
4SB
P 5
CH
P/D
PC
CEH
SLEO
MP
HS
GH
ICC
Lear
nin
g A
ctiv
itie
s
Performance Indicators
Year 2 Performance Indicators Selected by Presenter
OPP: 79(65%)
Non-OPP: 41(35%)
Year 2 Initial Trends • OPP performance indicators accounted for the
majority of those selected for OMM Learning activities.
• Performance indicators in Medical Knowledge, Patient Care, and Professionalism were the most commonly chosen outside of OPP.
• Patient Care performance indicators were more commonly chosen than any other non-OPP performance indicators.
Initial Conclusions
• OU-HCOM OMM Curriculum for Fall Years 1 and 2 addresses performance indicators OPP 1-5 within AACOM Core Competencies with greatest frequency.
• OU-HCOM OMM Curriculum for Fall years 1 and 2 consistently addresses performance indicators in the non-OPP Competencies: Medical Knowledge, Patient Care, Professionalism.
• Action: Given the discrepancy between presenter and observer ratings, our next steps are to encourage our presenters to – Ensure they select all the objectives they are addressing– Consider incorporating other objectives as relevant to their topics.
“I think that’s one of the great things about what you did…used the abilities of [Qualtrics] to make a nested survey. You made it so it was meaningful and easy to approach”
– Presenter Participant
Limitations
Limitations
• Only 1 semester of data
• Presenters self-evaluate
• Nested survey – Less time consuming (pro)
– Reading all objectives is optional (con)
• Recall bias
• Inter-rater reliability
• No student correlations—yet
Future Directions
Future Directions• Assessment of student performance
• Ongoing evaluation and quality improvement
• New methodology of curricular assessment
– Low-cost, high-value self-assessment
– Collaboration welcome
• Rural and Underserved Programs
– Assessment of Residency Didactics
– Most important competencies of interest to residency programs in rural and underserved areas
Thank you!
References:• 2014 Osteopathic Medical Profession Report. 2014. Available at: http://www.osteopathic.org/inside-
aoa/about/aoa-annual-statistics/Documents/2014-osteopathic-medical-profession-report.pdf.
• 2015 Osteopathic Medical Profession Report. 2015. Available at: http://www.osteopathic.org/inside-aoa/about/aoa-annual-statistics/Documents/2015-OMP-report.pdf.
• American Associations of College of Osteopathic Medicine. Osteopathic Core Competencies For Medical Students. Chevy Chase, MD; 2012. Available at: http://www.aacom.org/docs/default-source/core-competencies/corecompetencyreport2012.pdf?sfvrsn=4. Accessed July 8, 2015.
• Laird S, George J, Sanford S, Coon S. Development, Implementation, and Outcomes of an Initiative to Integrate Evidence-Based Medicine Into an Osteopathic Curriculum. JAOA. 2010;110(10):593-601.
• Bandaranayake R. The ethics of student assessment. Medical Teacher. 2011;33:435-436. doi:10.3109/0142159X.2011.579647.
• Bleakley A, Brennan N. Does undergraduate curriculum design make a difference to readiness to practice as a junior doctor?. Medical Teacher. 2011;33(6):459-467. doi:10.3109/0142159x.2010.540267.
• Boud D, Molloy E. Rethinking models of feedback for learning: the challenge of design. Assessment & Evaluation in Higher Education. 2013;38(6):698-712. doi:10.1080/02602938.2012.691462.
• Carraccio C, Englander R, Van Melle E et al. Advancing Competency-Based Medical Education. Academic Medicine. 2015:1. doi:10.1097/acm.0000000000001048.
• Carraccio C, Wolfsthal S, Englander R, Ferentz K, Martin C. Shifting Paradigms: From Flexner to Competencies. Academic Medicine. 2002;77:361-367.
• Dintzner M, Nemec E, Tanzer K, Welch B. Using Radar Plots for Curricular Mapping to Visualize Assessment in a New Doctor of Pharmacy Program. Am J Pharm Educ. 2015;79(8):121. doi:10.5688/ajpe798121.
• Holmboe E, Sherbino J, Long D, Swing S, Frank J. The role of assessment in competency-based medical education. Medical Teacher. 2010;32:676-682. doi:10.3109/0142159X.2010.500704.
References:• Komenda M, Víta M, Vaitsis C et al. Curriculum Mapping with Academic Analytics in Medical and
Healthcare Education. PLOS ONE. 2015;10(12):e0143748. doi:10.1371/journal.pone.0143748.
• Kumagai A. From Competencies to Human Interests. Academic Medicine. 2014;89(7):978-983. doi:10.1097/acm.0000000000000234.
• Laird S, George J, Sanford S, Coon S. Development, Implementation, and Outcomes of an Initiative to Integrate Evidence-Based Medicine Into an Osteopathic Curriculum. JAOA. 2010;110(10):593-601.
• Noll D, King Channell M, Basehore P et al. Developing Osteopathic Competencies in Geriatrics for Medical Students. JAOA. 2013;113(4):276-289.
• McIntosh W. A model for determining allocation of effort and sequencing of the AACOM competencies in a Competency-Based Osteopathic Medical Curriculum. Poster presented at AACOM 2014 Annual Conference: April, 2014; Washington, DC.
• Ramia E, Salameh P, Btaiche I, Saad A. Mapping and assessment of personal and professional development skills in a pharmacy curriculum. BMC Medical Education. 2016;16(1). doi:10.1186/s12909-016-0533-4.
• Schultz K, Griffiths J, Lacasse M. The Application of Entrustable Professional Activities to Inform Competency Decisions in a Family Medicine Residency Program. Academic Medicine. 2015;90(7):888-897. doi:10.1097/acm.0000000000000671.
• Snider K. Establishing a Professionalism Score in an Osteopathic Manipulative Medicine Curriculum. The Journal of the American Osteopathic Association. 2016;116(2):106. doi:10.7556/jaoa.2016.024.
• Walkowski S, Hain S, Miller E, Eland D. In: Entrustable Professional Activities For Osteopathic Graduates.; 2015.
• Zelenitsky S, Vercaigne L, Davies N, Davis C, Renaud R, Kristjanson C. Using Curriculum Mapping to Engage Faculty Members in the Analysis of a Pharmacy Program. Am J Pharm Educ. 2014;78(7):139. doi:10.5688/ajpe787139.
Appendix
Time taken to complete survey
• Year 1: – Presenters: Mean = 9 min, Median = 6 min – Participant Observers: Mean = 19 min, Median = 11
min
• Year 2: – Presenters: Mean = 7 min, Median = 4.5 min
0
5
10
15
20
25
30
OP
P 1
.A
OP
P 1
.B
OP
P 1
.C
OP
P 1
.D
OP
P 1
.E
OP
P 1
.F
OP
P 2
.A
OP
P 2
.B.
OP
P 2
.C
OP
P 2
.D
OP
P 2
.E
OP
P 2
.F
OP
P 2
.G
OP
P 2
.H
OP
P 2
.I
OP
P 3
.A
OP
P 3
.B
OP
P 3
.C1
OP
P 3
.C2
OP
P 3
.D
OP
P 3
.E
OP
P 3
.F
OP
P 3
.G
OP
P 3
.G1
OP
P 3
.H
OP
P 3
.I
OP
P 3
.J
OP
P 4
.A
OP
P 4
.B
OP
P 4
.C
OP
P 4
.D
OP
P 4
.E
OP
P 4
.F
OP
P 4
.G
OP
P 4
.H
OP
P 4
.i
OP
P 5
.A
OP
P 5
.B
OP
P 5
.C
OP
P 6
.A
OP
P 6
.B
OP
P 6
.C
OP
P 6
.D
OP
P 6
.E
OP
P 7
.A
OP
P 7
.B
OP
P 7
.C
OP
P 7
.D
OP
P 8
.A
OP
P 8
.B
OP
P 8
.C
OP
P 8
.D
OP
P 8
.E
Lear
nin
g A
ctiv
itie
s
OPP Objectives
Fall year 1 OPP Objectives Covered according to Presenters
