Curriculum Inventory Administrators’ GroupMarch 8, 2017

Terri Cameron, MADirector, Curriculum Programs

• CI 2015-2016 / 2016-2017 Upload / Chart and Report Updates• March CI in Context: Use of Ultrasound in US Medical Schools• April CI in Context: Social Determinants of Health• CI Reports Issues: Use of Integration Blocks; ‘Summary’ Events• Presentation from WGEA 2017: Systems to Manage Accreditation by Julie Youm, UC-Irvine• Presentation from WGEA 2017: EPA Toolkit for Implementing EPAs in the MD Curriculum,

by Tomo Ito, Oregon Health Sciences University• Medical school highlight: Emory University by Hugh Stoddard• CI for CQI and Accreditation Task Force• CI Research Group• Spring 2017 Conferences• Next meeting: Wednesday, April 12, 1 pm ET

Agenda

CI in Context: March 2017• Use of Ultrasound in US and

Canadian Medical Schools

• Note: Ultrasound is listed as a Resource by 2 medical schools, but 101 schools document in the term ultrasound in CI.

CI in Context: April 2017• Coverage of Social

Determinants of Health in US and Canadian Medical Schools

• Karen Sheehan, Northwestern University

CI Reports Issues• Integration Blocks

• Show themes across the curriculum

• Event Duration

• Range is from 10 minutes to hundreds of hours• ‘Summary’ events (e.g., 4 hours per week x 20 weeks)• Affects ‘Hour’ Reports

WGEA 2017 Presentation

• Systems to Manage Accreditation Julie Youm, UC-Irvine

SYSTEMS TO MANAGE ACCREDITATION 101Julie Youm, PhD, Assistant Dean, Education Compliance Quality, Director Educational Technology

Warren Wiechmann, MD, MBA, Associate Dean, Clinical Science Education and Educational Technology Khanh-Van Le-Bucklin, MD, Senior Associate Dean, Medical Education

Terri Dean, Director, Curricular Affairs

UC Irvine School of Medicine

DISCLAIMEROPINIONS ARE OUR OWN AND NOT THE VIEWS

OF OUR EMPLOYER OR THE LCME

SESSION

OBJECTIVES

• To present an overview of the types of systems and reports that can help prepare a medical school for LCME-accreditation under current standards

• To describe three examples of systems applied to address specific LCME standards and elements

• To outline the benefits and challenges of meeting the new accreditation standards with new and legacy systems most commonly found in medical schools

OVERVIEW

ELEMENT 1.1

LCME Element 1.1: Strategic Planning and Continuous Quality Improvement

“A medical school engages in ongoing planning and continuous quality improvement processes that establish short and long-term programmatic goals, result in the achievement of measurable outcomes that are used to improve programmatic quality, and ensure effective monitoring of the medical education program’s compliance with accreditation standards.”

OVERVIEW

ELEMENT 1.1

LCME White Paper (October 2016) “Implementing a System for Monitoring Performance in LCME Accreditation Standards” • Policy • Personnel • Resources • Selecting Elements to be Monitored• Managing Data Collection and Review

OVERVIEW

ACCREDITATION SYSTEMS

Curriculum Management

Evaluation

Medical School Management

Reporting

OVERVIEWCURRICULUM MANAGEMENT

• Curriculum mapping to identify gaps and (un)planned redundancies

• Annual AAMC CIR upload • MedBiquitous Standard

LCME • Element 6.1: Program and Learning Objectives • Standard 7: Curricular Content • Element 8.1: Curricular Management

OVERVIEW

EVALUATIONS

• Faculty + Student + Course/clerkship

LCME • Element 9.7: Formative Assessment and Feedback • Element 9.8: Fair and Timely Summative Assessment

OVERVIEW

MEDICAL SCHOOL MANAGEMENT

• Student records • Case logs/required clinical experiences • Duty hours

LCME • Element 8.6: Monitoring of Completion of Required Clinical

Experiences • Element 8.8: Monitoring Student Time • Element 9.7: Formative Assessment and Feedback • Element 9.8: Fair and Timely Summative Assessment

Canvas Learning Management System

+STUDENT ROSTER +FACULTY ROSTER +STAFF ROSTER

+COURSE SCHEDULE STUDENT GRADES

Ilios Curriculum Management

System +STUDENT ROSTER +FACULTY ROSTER +STAFF ROSTER

-COURSE SCHEDULE

UCI Evaluations +STUDENT ROSTER +FACULTY ROSTER +STAFF ROSTER

+ROTATION SCHEDULE

UCI Student Database STUDENT DEMOGRAPHICS

STUDENT GRADES -STUDENT ROSTER

+ROTATION SCHEDULE

New Innovations +STUDENT ROSTER -EVALUATIONS -CASE LOGS -WORK HOURS

UCI Scheduling +STUDENT ROSTER +FACULTY ROSTER

-ROTATION SCHEDULE

Central SOM Database

+STUDENT ROSTER -DASHBOARD -REPORTING

LEGEND: +Input -Output

UCI School of Medicine Systems Overview

ADMISSIONS

EXAMSOFTB-LINE

OVERVIEW

REPORTING

• See and understand data • Dashboards • View data from multiple sources

LCME • School-reported data throughout the DCIs

Canvas Learning Management

System +STUDENT ROSTER +FACULTY ROSTER +STAFF ROSTER

+COURSE SCHEDULE STUDENT GRADES

Ilios Curriculum Management

System +STUDENT ROSTER +FACULTY ROSTER +STAFF ROSTER

-COURSE SCHEDULE

UCI Evaluations +STUDENT ROSTER +FACULTY ROSTER +STAFF ROSTER

+ROTATION SCHEDULE

UCI Student Database

STUDENT DEMOGRAPHICS STUDENT GRADES -STUDENT ROSTER

+ROTATION SCHEDULE

New Innovations +STUDENT ROSTER -EVALUATIONS -CASE LOGS -WORK HOURS

UCI Scheduling +STUDENT ROSTER +FACULTY ROSTER

-ROTATION SCHEDULE

Central SOM Database

+STUDENT ROSTER -DASHBOARD -REPORTING

UCI School of Medicine Current Reporting Methods

Ilios Reporting Tool (Developed by ETG)

Excel

Built-in Reports Built-in Reports

EXAMPLE #1FAIR AND TIMELY SUMMATIVE ASSESSMENT (ELEMENT 9.8)

• Canvas Learning Management System • New Innovations

Availability of Final Grades AY 2014-15

EXAMPLE #2PROGRAM AND LEARNING OBJECTIVES (ELEMENT 6.1, 8.2, 8.3)

• Ilios, Curriculum Management System • Program, course and session-level learning objective mapping • Ilios Reporting Tool

EXAMPLE #3MONITORING REQUIRED CLINICAL EXPERIENCES (ELEMENT 6.2 AND 8.6)

• New Innovations • Weekly reports to Clerkship Directors

Ob/Gyn Surgery InptMedicine Ambulatory

Pediatrics Psychiatry Neurology EM FamilyMed

ACCREDITATION SYSTEMS

BENEFITS

• Ability to achieve CQI and compliance monitoring • Efficient collection of data • Robust analysis of data

ACCREDITATION SYSTEM

CHALLENGES

• Disparate/siloed systems • Lack of standardization and interoperability • System limiting process

ACCREDITATION SYSTEM

FUTURE

• Standardization • Interoperability

From Words to Action:Introduction of Core EPA Toolkits

AAMC WGEASLC, Utah

February 27, 2017

Tracy Bumsted, MD, MPHAssociate Professor, PediatricsAssociate Dean, UME

Tomo Ito, MS.EdAdministrative Manager, Curriculum & Student Affairs

We Have No Financial Disclosures or Conflicts of

Interest to Report

Institution Members of the AAMC Core EPA Pilot1. Columbia University College of Physicians and Surgeons2. Florida International University Herbert Wertheim College of

Medicine3. Michigan State University College of Human Medicine4. New York University School of Medicine5. Oregon Health & Science University School of Medicine6. University of Illinois College of Medicine7. University of Texas Health Sciences Center at Houston8. Vanderbilt University School of Medicine9. Virginia Commonwealth University School of Medicine10. Yale School of Medicine

https://www.aamc.org/initiatives/coreepas

Core Entrustable Professional Activities for Entering Residency Curriculum Developers’ Guide

https://members.aamc.org/eweb/upload/core%20EPA%20Curriculum%20Dev%20Guide.pdf

First, A Few DefinitionsEPA:

Units of professional practice, defined as tasks or responsibilities that trainees are entrusted to perform unsupervised once they have attained sufficient specific competence.

Competency:An observable ability of a health professional, integrating multiple components such as knowledge, skills, values, and attitudes.

Milestones:A behavioral descriptor that marks a level of performance for a given competency

13 EPAs• EPA 1: Gather a history and perform a physical examination

• EPA 2: Prioritize a differential diagnosis following a clinical encounter

• EPA 3: Recommend and interpret common diagnostic and screening tests

• EPA 4: Enter and discuss orders and prescriptions

• EPA 5: Document a clinical encounter in the patient record

• EPA 6: Provide an oral presentation of a clinical encounter

• EPA 7: Form clinical questions and retrieve evidence to advance patient care

• EPA 8: Give or receive a patient handover to transition care responsibility

• EPA 9: Collaborate as a member of an interprofessional team

• EPA 10: Recognize a patient requiring urgent or emergent care and initiate evaluation and management

• EPA 11: Obtain informed consent for tests and/or procedures

• EPA 12: Perform general procedures of a physician

• EPA 13: Identify system failures and contribute to a culture of safety and improvement

Charge to Core EPA Drafting Panel

To delineate those activities that all entering residents should be expected to perform on day 1 of residency without direct supervision, regardless of specialty.

Direct SupervisionIndirect Supervision (2 levels):

Direct Supervision Immediately AvailableDirect Supervision Available

Elements of EntrustmentBased largely on the work of Tara J. Kennedy, MD, MEd

Supervisor’s assessment of a trainee’s trustworthiness for independent clinical work takes into consideration four dimensions:

1. Knowledge, Skills, Values, Attitudes 2. Discernment3. Truthfulness4. Conscientiousness

Kennedy et al, Acad Med 2008;83(10Suppl):S89-92.

EPA Domain of Competence Competencies Milestones

Purpose of Core EPA ToolkitsTo create an accessible format that can support:

• Faculty development• Learner development• Design of curricular and assessment tools

How OHSU is Thinking About Implementing EPAs

(Still a Work in Progress!)

43 OHSU Competencies for MD degree

Students will need to be “entrusted” for all 43 in order to graduate

Plus AAMC 13 Core EPAs

What information will be used to award the EPA badges?

Entrustment Group: Award EPA Badges for Programmatic Entrustment

Thank You!

Curriculum InventoryFramework Used at Emory SOM

Hugh Stoddard, M.Ed., Ph.D.Assistant Dean for Medical Education Research

Outcomes Based Education

• Emory “Student Physician Activities” (SPAs)– Built on the conceptual basis of Entrustable Professional Activities– Connect to PCRS (national)– Map to the 6 ACGME Core Competencies– Encompass and expand Core EPAs

Graphic Organizer

Emory SPAs

Take a patient-centered history (focused and complete) SPA01 Recognize and address ethical dilemmas SPA15Perform a physical examination (focused and comprehensive) and recognize normal and abnormal

SPA02 Protect patient information SPA16

Apply principles of medical science to interpret clinical information

SPA03 Fulfill the professional role of a physician SPA17

Apply principles of medical science to patient care to develop a problem list, working diagnosis, etiologic evaluation, and management plan

SPA04 Manage time SPA18

Develop a patient care plan SPA05 Be a leader SPA19Perform technical procedures SPA06 Use feedback to improve one's own practices SPA20Communicate with patients and their support system regarding their care

SPA07Demonstrate trustworthiness to patients, colleagues, and other healthcare personnel

SPA21

Participate in difficult conversations with patients and their families

SPA08 Treat patients while understanding own biases SPA22

Document patient findings and treatment plans SPA09 Treat patients without regard to personal advantage SPA23Explain clinical decisions using scientific reasoning SPA10 Work in interprofessional teams SPA24Use electronic medical records SPA11 Identify personal limitations and seek assistance as SPA25Formulate questions and search the literature to resolve knowledge gaps

SPA12 Teach peers and team members SPA26

Contribute to generalizable medical knowledge SPA13 Serve the community SPA27Apply best evidence to the care of individual patients SPA14 Contribute to healthcare quality and safety initiatives SPA28

Student Outcomes (SPA) Code Student Outcomes (SPA) Code

Framework for Counting

• Categorize the content that lead to achieving the SPAs• Use content categories to align teaching and assessment• EUSOM Content Categories (in addition to SPAs)

– ICD9 organ systems– USMLE organizers

• National consensus of required knowledge

– LCME topics– EUSOM priorities

USMLE(mostly academic disciplines)

NA Prin00

Biochemistry and molecular biology

Prin01

Tissue response to disease(Pathophysiology)

Prin02

Normal structure and function(Gross anatomy & physiology)

Prin03

Histology and Biology of cells Prin04

Human genetics Prin05

Human development, lifecycle, & aging

Prin06

Immune responses Prin07

Microbial biology and infection Prin08

General Principles CodePharmacodynamic and pharmacokinetic processes

Prin09

Pharmacotherapeutics Prin10

Population health Prin11

Healthcare delivery and systems Prin12

Healthcare economics and finance

Prin13

Research methods and quantitative analysis

Prin14

Ethics Prin15

Jurisprudence Prin16

Medical humanities (philosophy, literature, & arts)

Prin17

Social science (anthropology, sociology, & psychology)

Prin18

General Principles Code

Special Topics (LCME and EUSOM)

NA Top00

Primary Care Top01

Wellness, Prevention and Health Promotion

Top02

Behavioral and Social Determinants of Health

Top03

Healthcare Disparities Top04

Societal Problems Top05

Sexuality Top06

Special Topics Code

Newborn & Pediatric Top07

Women's health Top08

Geriatrics Top09

End-of-Life care Top10

Rehabilitative Care Top11

Anesthesia and Anesthetics Top12

Transplant medicine Top13

Special Topics Code

Patient ProblemsPatient Problems Code

chronic conditions highlighted

NA Pt00

abdominal pain Pt001

accidents Pt002

acid base imbalance Pt003

acute coronary syndromes Pt004

acute kidney injury Pt005

Alzheimer's disease Pt006

amenorrhea Pt007

anemia Pt008

anxiety disorders Pt009

arthralgia Pt010

asthma Pt011

back pain Pt012

bipolar disorder Pt013 . . . et cetera

CI for CQI and Accreditation (CICA)• March 1st meeting:

• Reviewed generated lists of sequence blocks and competency objects from participants’ CI submissions for responses to 7.2. Preliminary results were met with general approval and indicate we have a working method to address 7.2 and questions like it.

• Participants continue to provide lists of search terms they have used to find question-appropriate content and Walter uses those terms to identify like-terms and concepts so that the results for any CI participant will be satisfactory.

Curriculum Inventory Research Group• Reviewing data for three 2017 projects:

• Review of Longitudinal Integrated Clerkships / Clerkship Continuity Experiences

• Longitudinal Study of how Boot Camps / Transition to Residency / Individual school measurements prepare students for residency

• Review of CI aggregate submissions for schools that implemented new curricula in 2016 – can we find new trends and initiatives from the data?

• Regional GEA Meetings • WGEA: February 25-27, Salt Lake City, Utah

• Best Practices for Using Your Curriculum Management System(Souza, Janes, Standley, Stone, Cameron)

• Leveraging the National Curriculum Inventory for Large-Scale Research Opportunities (Colbert-Getz, Teherani, Cameron)

• CGEA: March 29-31, 2017, Chicago, Illinois• Best Practices for Using Your Curriculum Management System

(Noiva, Rivest, Reed, Cameron)• Leveraging the National Curriculum Inventory for Large-Scale Research

Opportunities (DuFault, Farnan, Mavis, Cameron)

Spring/Summer Education Meetings

• Regional GEA Meetings • SGEA: April 19-22, 2017, Charlottesville, Virginia

• Best Practices for Using Your Curriculum Management System(Gibson, Lopez, Stoddard, Fitz-William)

• Leveraging the National Curriculum Inventory for Large-Scale Research Opportunities (Grochowski, Cameron)

• NEGEA: 2017 May 4-6, 2017, Rochester, NY• Best Practices for Using Your Curriculum Management System

(Vaughan, Hueppchen, Mallott, Fitz-William)

Spring/Summer Education Meetings

Spring/Summer Education Meetings• American Association of Colleges of Osteopathic Medicine (AACOM):

April 26-29, Baltimore, Maryland• Comparing and Benchmarking Curricula Using the Curriculum Inventory

(Watts, Wimsett, Moscatello, Fitz-William)• Curriculum Inventory: A Tool for Data-Driven Discussions (Fitz-William)

• Canadian Conference on Medical Education (CCME): April 29 – May 2, Winnipeg, Manitoba, Canada• Pre-Conference Workshop: The Curriculum Inventory – A Tool for Data-

Driven Decisions (Fitz-William)• Comparing and Benchmarking Curriculum using the Curriculum Inventory

(Simpson, Pennell, Fitz-William)

Spring/Summer Education Meetings• AAMC Group on Information Resources(GIR)/Group on Business

Affairs (GBA) /Group on Institutional Planning (GIP): May -12, Atlanta, Georgia • Best Practices for Using Your Curriculum Management System

(Gerber, McCarthy, Simpson, Fitz-William)• Curriculum Inventory Workshop / CI Support of CQI and Accreditation

(Fitz-William, Lawton) POST-CONFERENCE WORKSHOP

• MedBiquitous Conference:June 4-6, Baltimore, Maryland• Best Practices for Using Your Curriculum Management System

(Gerber, Fitz-William)• Curriculum Inventory Workshop (Fitz-William, Lawton)

Spring/Summer Education Meetings• International Association of Medical Science Educators (IAMSE):

June 10-13, Salt Lake City, Utah• Using the Curriculum Inventory for International Medical Education

Benchmarking (Cameron)• Using a National Curriculum Database for Medical Education Research

(Farnan, DuFault, Cameron)• Documenting and Reporting on Trends in Medical Education (Growchowski,

Bonaminio, Cameron)• Documenting, Reporting and Benchmarking Evidence-Based Content in

Medical Education Curricula (Simpson, Ma, Helf, Cameron)• Please let us know if you have any proposals accepted that relate to

Curriculum Management / Mapping

• Association for Medical Education in Europe (AMEE):August 26-30, Helsinki, Finland• No Proposals Due Submitted by AAMC Staff – please let us know if you have

any proposals accepted that relate to Curriculum Management / Mapping

• Wednesday, April 12, 1 pm ET• (Second Wednesday of each month, 1 pm ET)• Registration Links posted in Training and Resources section of

www.aamc.org/cir• Please send agenda items to tcameron@aamc.org

Next meeting: