HIT Policy CommitteeHIT Policy Committee

METHODOLOGIC ISSUESTiger Team SummaryTiger Team Summary

Helen BurstinNational Quality Forum

Jon WhiteAgency for Healthcare Research and Quality

October 28, 2010

Methodologic Issues Tiger Team Members

• Helen Burstin (Co-chair)• Jonathan White (Co-chair)• Mark Weiner• Ross Lazarus• Bob Dolin• Phil Renner• Danny Rosenthal• Mitra Rocca• Abel Kho• John Moquin• Joanne Cuny• Dan Malone• Dave Baker

2

Methodologic Issues

• The Methodologic Issues tiger team was tasked to identify the critical methodologic issues that should be considered in the following areas:– Longitudinal measurement - the use of measures that assess patient

experience and quality across different settings and providers across time.

– Measures that demonstrate change over time (“delta” measures) - the use of measures that assess change in outcomes across time, rather than only achievement of a threshold.

– Adverse event reporting systems, with emphasis on medication safety – relates to the use of IT systems for automatic collection and reporting of patient-level adverse events within clinical workflow.

– Other issues that emerge from the prioritized measure concepts from the other Tiger Teams.

• Methodologic issues associated with patient reported outcomes have been addressed by the Patient and Family Engagement Tiger Team

3

Longitudinal Measurement: Methodologic Issues

• Standardized data collection and harmonization of measures across data platforms

• Need for a shared patient identifier• Difficult to attribute results of care to specific providers• Reliance on the problem list for assessment of conditions/diagnoses over time• Lack of interoperability limits tracking across care settings• All measure concepts are not equally amenable to longitudinal data capture-

avoidance of “check box” measures• Consider other approaches to measurement of key processes:

– Structural approaches may provide better information (e.g., exchange of medication lists across providers)

– Patient experience may be a more practical and valid approach– Claims data may be a better indicator for some indicators for longitudinal

follow-up (e.g., first follow-up after hospitalization)

4

Delta Measurement: Methodologic Issues

• Consider measures most amenable to delta measurement• Determine appropriate points in time for baseline and follow-up• Consider standards for the degree of change (e.g., % gap to threshold value

achieved)• Ensure completeness of data across different data systems• Account for outcomes that do not have a linear trajectory and may include a

lower limit associated with harm• Unclear if delta measures present a valid alternative to threshold measures that

assess performance• Additional testing on delta measures necessary. Do they add value?• Difficult to track untoward clinical effects of improving clinical endpoints (e.g.,

dizziness resultant from blood pressure medications)• Accounting for tests not performed for patients regardless of interaction with

healthcare system (population health related)• Strong interest to capture health status before and after preference-sensitive

procedures (e.g., cataracts, back surgery)

5

Adverse Event Reporting: Methodologic Issues

• Numerous barriers to routine medication adverse event reporting include:– Limitations in coding and capture of medication-related adverse events

– May be difficult to link event reporting back to specific drugs. • Integrate adverse event reporting into clinician workflow, considering time• Protect patient privacy when developing an adverse event reporting

program• Adverse event reporting programs requires data from disparate systems• Computer algorithms that analyze data and identify adverse events are

not very high-tuned to detect events• Level of harm from adverse events is not well documented• Measures can target high impact reactions to certain medications (e.g.,

warfarin)• Alert-fatigue is an issue and a medication-related event may be missed if

alerts are turned off

6

General Methodologic Issues

• Incremental quality measurement to ensure that the complexity of quality measures is in step with the capacity of available EHRs

• Integration of standardized EHR context into measurement

• Utilizing HIEs to capture needed data across providers (efficiency)

• Interoperable systems that can assure that original results are available (efficiency)

• Patient data accurately tracked across different providers and data platforms (population health)

• Standardizing the strata for race, ethnicity, language to allow for measure stratification (equity)

• Emerging technology that addresses health literacy (e.g., talking touch screens) (patient-reported outcomes)

• Inconsistencies in coding between types of providers (e.g., primary care and psychiatrists)

7

Methodologic Issues/Recommendations

Problem List

Accuracy of coding is highly variable depending on type of diagnosis

• Need for standards for coding problem list – e.g., active/inactive designation of a problem; date of onset for a given disease process

• Need for clinicians to consider the problem list as a quality measurement reporting tool

• Need conventions for how to deal with problems that have resolved and been deleted from problem list

• Need to establish rules for consistent use of problem list and past medical history

• Need guidance for proper use of problem list for reporting of new conditions

8