“e-indicators” of Ambulatory Care Quality: New Paradigms for

Measuring Clinical Performance Using Electronic Health Records

Presented at the Academy Health, Washington DC June 10 2008

Jonathan P. Weiner, DrPHDepartment of Health Policy & ManagementJohns Hopkins Bloomberg School of Public Healthjweiner@jhsph.edu

2

Topics to be discussed:

• Ideas and paradigms regarding health IT (HIT) as applied to ambulatory / population based quality performance measurement and reporting.

• Findings from a multi-faceted collaborative project to develop electronic health record (EHR) based “e-indicators.”

• Future challenges and opportunities of EHR based quality measurement.

3

Embracing new HIT capabilities to measure quality in the ambulatory care sector: The “e-indicator” project

Goal: To develop innovative measures of quality that take advantage of new HIT capabilities and data sources.

Development effort based at Johns Hopkins and Park Nicollet Institute: – Collaboration involving leading edge “wired” integrated delivery

systems: Park Nicollet (MN), Kaiser Permanente (OR), Geisinger (PA), Health Partners (MN) Billings (MT), Dupont/Nemours (DE), Boston CHC network (MA).

– Involves both an “adult” project focusing on chronic illness and a “child project” focusing on screening and development

– Advisors include key quality groups, NCQA, NQF, AMA, Medicare, VA, ONCHIT, AHRQ, medical specialty societies

– Funded by Commonwealth Fund, Robert Wood Johnson, US AHRQ, Nemours

– Methods include lit review, survey of experts & early adopters, case study, development of starter-set of “e-indicators”.

4

Applications of HIT-based performance measures

• Quality improvement for organizations

– Real time (safety / care management)

– Retrospective evaluation

• “Pay for Performance” (P4P) incentives

• Community / regional reporting and strategic planning

5

HIT will transform performance measurement

• e-Indicators will be essential tool for:

– Provider / clinician teams /organizations

– Health plan / sponsors

– Government / communities / public health

– Outcomes researchers / scientists

6

Data sources and types of quality / performance measures

Type of Measure

Data Source: Denominator Process Outcome Pt-Cent. Cost

Electronic / HIT

PH records / registry X

Insurance files X X X X

EHR X X X X

CPOE (order entry) X

PHR / Pt. “web portal” X X X

CDSS (clinical support) X X

Non-electronic

Paper medical record X X

Surveys X X

7

A proposed typology for HIT based “e-indicators” of quality

1) Translational: Traditional (e.g., paper record and claims) measures translated for use on HIT platforms

2) HIT-facilitated: Measures that while not conceptually limited to HIT, would not otherwise be feasible.

3) HIT-enabled: Measures that generally would not be possible outside of EMR/EHR context.

4) HIT system management: Measures needed to implement, manage and evaluate HIT systems

5) “e-iatrogenesis”: Measures of patient harm caused at least in part by application of HIT.

8

Examples of each type of e-indicator

1) Translational:- Computerization of existing HEDIS measures

2) HIT-facilitated: - % of children > BMI of x receiving intervention

- % of patients with e-prescriptions who did not pick up their Rx within x days

- % of clinicians reviewing and responding to abnormal lab value with x hours

3) HIT-enabled:- % of consumer generated web-based shared care plans accessed

by both generalist & specialists within 6 months

- % of in scope care that is routed through CDSS supported workflow algorithm

9

Examples of each type of e-indicator - cont.

4) HIT system management:

- Attainment of EHR interoperability targets

- % of CDSS alerts ignored by clinicians

- % of allergy lists updated by patient annually

5) e-iatrogenesis:

- % of e-prescriptions that result in wrong drug

- % of patients needlessly exposed to imaging radiation due to inappropriate use of CDSS module

10

Broad observations re EHRs and quality

• The needs of quality measurement have not been adequately addressed by most current EHR / HIT systems.

• The most useful quality measures will be hard-wired into HIT “workflow” or “streamed” automatically.

• Integration of real-time clinical decision support and provider order entry (e.g., e-prescribing) represent a major paradigm shift for quality measurement.

• To date, wired organizations have replicated paper records and claims data methods and approaches.

• Outside of IDSs, e-indicators at person and population level are not feasible without EHRs “interoperability.”

11

Advantages of EHR based measures

• Data elements that are “streamed” or part of structured e-workflows are likely to be most accurate.

• 100% electronic census will reduce bias associated with sampled chart reviews and surveys.

• Abstraction errors due to poor handwriting will be eliminated.

• There are rich new sources of information in the domains of time and information flow.

12

Challenges of EHR based measures

• The process of entering, transferring, archiving, and analyzing EHR data introduce numerous opportunities for inaccuracy.

• There are new types of errors associated with provider behaviors (e.g., cut and paste “plagiarizing.”)

• CDSS or workflow algorithms can introduce and perpetuate measurement errors by systematizing them.

• For foreseeable future, analysis of free-text section of EHR will be fraught with difficulty.

13

Some recommendations moving forward• Quality measurement must be part of workflow and HIT system design from the start.

• We need to develop a better understanding of how new paradigms and data sources affect reliability and validity.

• For foreseeable future HIT “systems management” indicators should be central part of IT CQI process.

• EHR / HIT systems offer great opportunities for population and person based measures (moving beyond those that are provider or patient based).

• Although there will likely be considerable positive benefit, we must learn how to measure and monitor “e-iatrogenesis”

14

Further information / Acknowledgements

If you wish to receive copies of our project manuscripts please give me your card or send me an e-mail at jweiner@jhsph.edu

I would like to acknowledge co-authors of this paper:

– Jinnet Fowles

– Kitty Chan

– Betsy Kind