Updated validation of AHRQ Prevention Quality Indicators in the USA
Patrick S. Romano, MD MPH
UC Davis Center for Healthcare Policy and Research
Organization for Economic Cooperation and Development
October 22, 2009
PQIs: Potentially Avoidable Hospitalizations
Admissions for diagnoses that may have been prevented or ameliorated with access to high-quality outpatient care
Two independently developed measure sets described in the 1990s literature– John Billings
– Joel Weissman
Strong independent negative correlations between self-rated access and avoidable hospitalization
Correlations between avoidable hospitalization and:– household income at zip code level (neg)
– uninsured or Medicaid enrolled (pos)
– maternal education (neg)
– Primary care physician to population ratio (neg)
– Weaker associations for Medicare populations
Current uses of the PQIs in the USA:Four examples
National Healthcare Quality Report, Commonwealth Fund, California:
Public health agencies tracking and comparing health system performance across counties, states
Wisconsin Medicaid program:
Comparing performance of managed care plans for low-income persons
Dallas-Fort Worth Hospital Council:
Exploring how community health affects hospitals
General Motors:
Estimating potential “return on investment” with improving primary care access in communities with GM employees.
Measures recommended for state Medicaid programs by the Foundation for Accountability
Ambulatory care sensitive conditions (“potentially avoidable hospitalizations”) Angina
Adult asthma/pediatric asthma
Chronic obstructive pulmonary disease
Congestive heart failure
Diabetes (short-term and long-term complications, uncontrolled)
Lower extremity amputation with diabetes
Hypertension
Other potentially avoidable conditions Perforated appendix
Low birth weight
Evaluating Medicaid managed care programs in Wisconsin
% ICare Enrollees with CHF Hospitalized for CHF
20.8
15.1
0
5
10
15
20
25
1998 2000
Pe
rce
nt
% ICare Enrollees with Asthma Hospitalized for
Asthma: 1998 & 2000
3.9
2.8
0
2
4
6
8
10
12
1998 2000
Pe
rce
nt
% ICare Enrollees with COPD Hospitalized for COPD
5.84.7
0
2
4
6
8
10
12
1998 2000
Pe
rce
nt
% ICare Enrollees with Diabetes Hospitalized for
Diabetes: 1998 & 2000
2.9
2.5
0
0.5
1
1.5
2
2.5
3
3.5
1998 2000
Perc
en
t
Comparison of total E.R. visits and ambulatory care sensitive condition (ACSC) E.R.
visits for I-Care and matched FFS recipients, 1999
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
I-Care (n=1722) Comparable FFS (n=1722)
No ER visit
ER, not for ACSC
ER for ACSC
Reducing ED visits for PQIs in Medicaid managed care
Community Health Assessments using PQIs to understand role of hospitals
Risk Adjusted Rates per
100,000 PopulationNamed counties without shading have a Risk
Adjusted rate of zero.1 to 117.0
117.1 to 283.0
283.3 to 399.2
399.3 to 565.2
> 565.2
DI Hospitals
Congestive Heart Failure Admission Rate
Congestive heart failure (CHF) can be
controlled in an outpatient setting for the most
part; however, the disease is a chronic
progressive disorder for which some
hospitalizations are appropriate.
AHRQ Prevention Quality Indicators
Congestive Heart Failure Admission Rate - 2003
+ = County’s RA rate significantly lower than State RA rate
- = County’s RA rate significantly higher
o = No statistical difference
Texas Hospital Inpatient Discharge Public Use Date File, FY2002. Texas
Health Care Information Council, Austin, Texas. December, 2003.
© 2004, Dallas-Fort Worth Hospital Council - Data Initiative For Hospital Internal Use Only – April, 2005
2003 (PQI 08) Rates per 100,000 Cases
County
Numerator
(Outcome)
Denominator
(Population) Observed
Risk
Adjusted
Confidence Interval
(95%)
Stat.
Sig.
State of Texas 66,822 15,882,253 420.7 504.5
BOSQUE 21 13,486 155.7 0.0 ( 0.0, 0.0 )
CAMP 80 8,567 933.8 860.3 ( 664.7, 1055.9 ) -
COLLIN 911 429,184 212.3 410.7 ( 391.6, 429.8 ) +
COMANCHE 117 10,233 1,143.3 954.4 ( 766.0, 1142.8 ) -
COOKE 60 28,036 214.0 142.7 ( 98.5, 186.9 ) +
DALLAS 6,412 1,631,345 393.0 527.8 ( 516.7, 538.9 ) -
DELTA 32 4,172 767.1 622.7 ( 384.0, 861.4 ) o
DENTON 876 369,935 236.8 456.8 ( 435.1, 478.5 ) +
EASTLAND 26 14,031 185.3 0.0 ( 0.0, 0.0 )
ELLIS 367 88,785 413.4 518.2 ( 471.0, 565.4 ) o
ERATH 97 24,993 388.1 376.7 ( 300.8, 452.6 ) +
FANNIN 171 25,024 683.3 610.2 ( 513.7, 706.7 ) -
FRANKLIN 48 7,616 630.3 505.7 ( 346.4, 665.0 ) o
GRAYSON 568 86,204 658.9 612.4 ( 560.3, 664.5 ) -
HAMILTON 19 6,241 304.4 23.7 ( 0.0, 61.9 ) +
HENDERSON 386 58,796 656.5 560.5 ( 500.2, 620.8 ) o
HILL 220 25,615 858.9 776.5 ( 669.0, 884.0 ) -
HOOD 149 34,883 427.1 330.1 ( 269.9, 390.3 ) +
HOPKINS 37 24,315 152.2 84.1 ( 47.7, 120.5 ) +
HUNT 362 59,959 603.7 630.8 ( 567.4, 694.2 ) -
JACK 12 6,916 173.5 135.9 ( 49.1, 222.7 ) +
JOHNSON 564 100,860 559.2 664.9 ( 614.7, 715.1 ) -
KAUFMAN 390 59,401 656.6 750.8 ( 681.4, 820.2 ) -
LAMAR 260 36,763 707.2 619.4 ( 539.2, 699.6 ) -
MONTAGUE 21 14,900 140.9 0.0 ( 0.0, 0.0 )
MORRIS 64 10,014 639.1 507.3 ( 368.2, 646.4 ) o
NAVARRO 216 34,415 627.6 596.6 ( 515.2, 678.0 ) -
PALO PINTO 22 20,368 108.0 2.9 ( 0.0, 10.3 ) +
PARKER 210 72,541 289.5 361.6 ( 317.9, 405.3 ) +
RAINS 38 8,472 448.5 396.8 ( 262.9, 530.7 ) o
ROCKWALL 106 39,433 268.8 394.3 ( 332.4, 456.2 ) +
SOMERVELL 9 5,416 166.2 139.0 ( 39.8, 238.2 ) +
STEPHENS 6 7,147 84.0 0.0 ( 0.0, 0.0 )
TARRANT 3,688 1,118,382 329.8 456.1 ( 443.6, 468.6 ) +
TITUS 40 19,796 202.1 198.5 ( 136.5, 260.5 ) +
VAN ZANDT 209 38,329 545.3 458.6 ( 391.0, 526.2 ) o
WISE 58 39,967 145.1 209.0 ( 164.2, 253.8 ) +
WOOD 235 30,845 761.9 599.1 ( 513.0, 685.2 ) -
YOUNG 15 13,604 110.3 0.0 ( 0.0, 0.0 )
General Motors: Impact of PQI admissions on employer-financed health care
QI Name
Potential cost savings if
number of admissions
were reduced by specified
percentage
County name (all counties in MI listed),
average cost of admission for QI
specified, total number of cases, and
total cost
General Motors mapped PQI data
Name of Indicator
and Data Year in
Map Title
Data quintiles.
Green is the lowest
20% or the lowest
rates. Red is the
highest 20% or the
highest rates.
Symbol indicating
number of GM covered
beneficiaries, number
below is average in the
group.
Counties with
high indicator
rates and
higher number
of beneficiaries
Using GM-mapped data
Integrate action plans with other Community Initiatives projects
Pay for Performance for providers in specific counties to reduce admission rates
Coordination with other Community Stakeholders to achieve desired improvement
Funding to implement projects at a community level
Focus on indicators with highest potential cost savings
Potential uses of PQIs
QI
Comparative
Reporting
Pay for
Performance
Area X
Payor X X
Provider X X X
1 We initially assessed the internal quality improvement application for large provider groups. Following our initial rating
period, panelists expressed interest in applying select indicators to the long term care setting and these applications were
added to our panel questionnaire.
Current application
Extended applications
Revalidation methods
Clinical Panel review using new hybrid Delphi/Nominal Group technique
Two groups: Core and Specialist
– Core assesses all; Specialist only those applicable to their specialty
Three indicator groups: Acute, Chronic, Diabetes
Two panels:
– Delphi
– Nominal Group
Delphi vs. Nominal
Delphi group
– Advantages: Larger,
hence better reliability,
more points of view,
less chance for one
panelist to pull the group
– Disadvantage: Less
communication and
cross-pollination across
panelists, less ability to
discuss and refine
details of
indicators/evaluation
Nominal group
– Advantages: Can
discuss details, facilitate
sharing of ideas
– Disadvantages: Limited
in size and therefore in
representation, one
strong panelist can
flavor group and
therefore poorer
reliability
Composition of panels
Characteristic Delphi Group (n = 42)
Nominal Group (n = 23)
Gender
Male 62.8 73.9
Female 37.2 26.1
Urban/Rural1
Urban 32.6 30.4
Suburban 14.0 13.0
Rural 7.0 8.7
Multiple/All areas served 16.3 30.4
Academic Affiliation1
Academic practice 27.9 47.8
Non-academic practice 34.9 30.4
Any academic affiliation 69.8 87.0
Underserved population
in practice1 46.5 69.6
Funding1
Public 27.9 34.8
Private and/or Non-profit 20.9 39.1
Multiple sources 7.0 0
Specialties representedSpecialty Delphi Panel Nominal
Panel
Internal Medicine 5 3
Family Medicine 4 1
Geriatric Medicine 2 2
Public Health Physician 4 0
Emergency Medicine 3 2
General Nurse Practitioner 2 1
Endocrinology 4 2
Vascular Surgery 2 1
Diabetes Outpatient Management
1 1
Nephrology 0 1
Cardiology 4 3
Pulmonology 3 2
Asthma Specialist 1 0
Pulmonary Rehabilitation 1 0
Infectious Disease 2 2
General Surgery 3 2
Urology 1 0
Delphi Dephi rating
Results: initial rating
Delphi comments
Nominal comment
Nominal Nominal rating
Results: Initial rating
1st round results to panelists prior to call
Diabetes call
Acute call
Chronic call
Nominal panel re-rates
Call summaries to panels
Final ratings
Delphi panel re-rates
Panel Process: Exchange of Information
Quality Improvement Applications
Indicator Provider
COPD and Asthma (40 yrs +) ▲▲ +
Asthma ( < 39 yrs) ▲▲▲▲
Hypertension ▲▲ +
Angina ▲▲
CHF ▲▲▲▲
Perforated Appendix ▲+
Diabetes Short Term Complications ▲▲▲▲
Diabetes Long-Term Complications ▲▲+
Lower Extremity Amputation ▲▲+
Bacterial Pneumonia ▲▲
UTI ▲▲
Dehydration ▲+
▲ Major Concern
Regarding Use ,
▲▲Some Concern
▲▲▲General Support
▲▲▲▲Full Support
+ Either Delphi or
Nominal Panel
reported higher level of
support for measure
than shown
Comparative Reporting Applications
Indicator Area Provider
COPD ▲▲ ▲▲+
Asthma ( < 39 yrs) ▲▲+ ▲▲+
Hypertension ▲▲+ ▲▲
Angina ▲▲ ▲
CHF ▲▲+ ▲▲▲▲
Perforated Appendix ▲+ ▲+
Diabetes Short Term ▲▲ ▲▲+
Diabetes Long-Term ▲▲+ ▲▲
LE Amputation ▲▲▲▲ ▲▲
Bacterial Pneumonia ▲▲ ▲▲
UTI ▲▲ ▲▲
Dehydration ▲▲ ▲
▲ Major Concern
Regarding Use ,
▲▲Some Concern
▲▲▲General Support
▲▲▲▲Full,Support
+ Either Delphi or
Nominal Panel
reported higher level of
support for measure
than shown
Potential interventions to reduce hospitalizations
Acute Chronic
Area Access to primary
care/urgent care
Access to care
Lifestyle modifications
Payor Coverage of medications
Coverage of auxiliary
health services (e.g. at
home nursing)
Access to primary
care/urgent care
Coverage of medications
Coverage of comprehensive care
programs
Coverage of auxiliary health
services (e.g. at home nursing)
Disease management programs
Lifestyle modification incentives
Provider Quality nursing triage
Patient education
Accurate/rapid diagnosis
and treatment
Appointment availability
Outpatient treatment of
complications
Education, disease management
Lifestyle medication interventions
Comprehensive care programs,
care coordination, auxiliary health
services
Potential improvements
Defining the numerator– One admission per patient per year?
– Using related principal diagnoses with target secondary diagnoses (i.e., pneumonia with COPD)
– Consider excluding first hospitalization before chronic condition diagnosed
Defining the denominator– Identifying patients with chronic diseases (mulitple
diagnoses from outpatient claims, population survey rates, pharmaceutical data
– Requiring minimum tenure with payor or provider or minimum duration of residence in an area
Risk adjustment
Demographics– Age and gender highly rated as important
– Race/ethnicity depending on indicator (i.e., diabetes)
Disease severity– Historical vs. current data
Comorbidity
Lifestyle associated risk and compliance– Smoking, obesity
– Pharmacy records
Socioeconomic status– May mask true disparities in access to care
– Panel felt benefits of inclusion outweighed problems
Policy implications
Ensuring true quality improvement
– Shifting risk (adverse selection)
– Manipulating coding
Cost/burden of data collection to support better risk adjustment
Different perspectives of different stakeholders
Does avoiding hospitalization really reflect the best quality and value?
Acknowledgments
Project team:
Sheryl Davies, MA (Stanford)
Kathryn McDonald, MM (Stanford)
Eric Schmidt, BA (Stanford)
Ellen Schultz, MS (Stanford)
Olga Saynina, MS (Stanford)
Jeffrey Geppert JD (Battelle)
Patrick Romano, MS, MD (UC Davis)
This project was funded by a contract from the Agency for Healthcare Research and Quality (#290-04-0020)