“The public is concerned about hospital quality but

has less information about selecting a hospital than

they do when purchasing a toaster or a car” (Malone, 1999)

“Puffery” “…representation, statement, or

conduct that clearly over-exaggerates…

…is not intended to be an offer to be relied upon.”

Consumer protection

(2000 Legal on-line Dictionary)

Methods

1. Solucient’s “Top 100 Hospitals”

2. US News & World Report’s “Best Hospitals”

3. American Nurses Credentialing Center’s “Magnet Recognition Program”

Descriptive statistics Spatial mapping Review of empirical literature

US News & World Report’s “Best” Hospitals

“If you are looking for the ‘best’ in medical care…”

—web-based guide.

“designed to assist you in your search for the highest level of medical care.”

(US News & World Report)

US News: Selecting the “Best” National Opinion Research Center Each July since 1990 Ranked by specialties (17) “Only rigorous assessment of all US hospitals” Eligible -- “tertiary care hospitals”

COTH membership or Affiliation with medical school or Availability of 9 out of 17 prescribed

technologies Hospital Quality Index (HQI) Honor Roll – score well in at least 6 specialties

US News continued…

Measures: produce HQI Structure = AHA Annual Survey data

• FTE “on-staff” Nurse to bed staffing ratios• Services available (technology index)

– # of specialty-specific technology services Process = (“proxy measure”)

• Physician nominations• Newest specialties reputation only

Outcomes = • Medicare risk-adjusted mortality by specialty• Pool most recent 3 years• Mortality ratio = # inpatient deaths/expected

Solucient’s “Top 100” Hospitals

“Identifies industry benchmarks by recognizing hospitals that demonstrate superior clinical, operational, and financial performance.”

Selecting the “Top 100” Since 1993 -- December “Management results that separate these

hospitals from the pack.” “Which hospitals offer superior care at lower

costs.” “Data obsession--common trait.” Enterprise = benchmark reports for purchase Eligibility restricted: 5 categories (n=20) Overall + select specialties (ICU, CV, Ortho, Stroke

benchmarks) Data source – Medicare Cost Report

Solucient “TOP” – 5 strata

Major Teaching (400 or more beds & resident to bed ratio of at least .25)

Teaching (at least 5 residents or resident to bed ratio of .01-.24)

Large Community (250 or more beds) Medium Community (100-249 beds) Small Community (25-99 beds)

“Top 100”selection… Ranked by strata on following measures—20 best rankings per

strata selected Financial management =

• Expense/adjusted discharge: case mix and wage adjusted

• Profitability• Proportion of outpatient revenues• Index of total facility occupancy

Clinical performance =• Medicare risk-adjusted mortality index• Risk-adjusted complication index

Operational performance =• Severity-adjusted average LOS• Case-mix and wage-adjusted expense per discharge

ANCC Magnet Recognition Program for Excellence in Nursing

(1994)

ANCC Recognizes “excellence” in 4 target areas

1. Management philosophy & practices

2. Adherence to standards for improving the quality of care

3. Leadership of the CNO in supporting professional practice and continued nursing competence

4. Attention to the cultural and ethnic diversity of patients, their significant others, and providers

Magnet Recognition Process

Self evaluation: ANA’s Scope & Standards ANA quality indicator data (NDNQI) Site visit Community input Could range from $30,000 - $75,000 Reevaluated every 4 years or more often with

leadership change

Continuous Process: submit data annually

Diffusion of “Magnetism”by year and total number recognized

0

20

40

60

80

100

120

1994 1996 1998 2000 2002 2004

“Creep” & Overlap - 2001 & 2003

Designation 2001 2003

Best 173 203

Top 101 101

Magnet 47 91

Best & Magnet 14 25

Best & Top 15 13

Top & Magnet 2 12

Best, Top & Magnet

1 6

Best, Top, & Magnet

1. University of CA Irvine2. Poudre Valley Hospital, Fort Collins

CO3. Mayo Clinic, Rochester MN4. NC Baptist, Wake Forest Un., NC5. Methodist Hospital, Houston, TX6. Inova Fairfax Hospital, Falls Church

VA

Summary – Best, Top & Magnet

Best (14yrs) Top (11yrs) Magnet (10yrs)

Measurement Framework

norm referenced norm referenced

criterion referenced

Focus structure, process, outcomes

outcomes structure, process

Self-nomination no no yes

Self-appraisal no no yes

Site visit no no yes

Nursing yes no yes

Purpose “inform consumer choice”

“excellence in management”

“excellence in nursing care”

Empirical evaluations

3 3 4

U.S. News “Best Hospitals” - 2003

Solucient’s “Top 100 Hospitals” - 2003

ANCC’s Magnet Hospitals - 2003

Best, Top, & Magnets - 2003

Regional dominance = “Pockets of excellence” ?? High preponderance – northeast, south 6 states – “empty” Mountain, west north central, west south

central regions Some states in highly designated regions

had fewer hospitals than those in the “empty regions” CA -- 450 hospitals (3 magnets) = .7% NJ – 74 hospitals (14 magnets) = 19%

The US NEWS “Best” -- 3 Empirically based publications

Different rankings when different decision models used (Teasley, 1996)

CABG outcomes in PA and NY (Hartz et al., 1997) - “Best did not perform better than rest”

Significantly lower AMI mortality rates (Chen et al. (1999)Clinical practice processesAsprin & beta blockers

Numerous reports of methodological “flaws” & conceptual issues What is examined Who does the nomination “best colleges” “best party schools”

Solucient “Top” -- 3 empirically based publications

1. CHEN et al., 1999• AMI patients over 65 Significantly lower LOS and

costs than “rest”• No significant difference in mortality rates

2. Griffith & Alexander 2002• Found that 8 of the Solucient measures of

hospital performance were valid and reliable

3. Griffith, Knutzen & Alexander, 2002• “a disjuncture” between “Top” outcome

measures and JCAHO evaluations in non-federal hospitals

ANCC Magnet Hospitals – 4 Empirically based publications

Aiken, Havens, & Sloane, 2000 support for nursing practice nurse reported quality nurse satisfaction better nurse staffing ratios more highly educated nurses nurse burnout

Havens, 2001 strong departments of nursing quality ratings organizational support for nursing practice difficulty recruiting nurses patient and family complaints

Upenieks, 2002, 2003 nurse satisfaction in ANCC magnet

hospitals -- linked to stronger nurse manager leadership

levels of RN reported empowerment

Havens, (Forthcoming) use of “outside” nurses vacancy rates

Comparison of JCAHO HospitalEvaluation Score Means

(2002 JCAHO file)

Hospital Type mean score

SD

All Community Hospitals (Federal, psych and other specialties eliminated)

90.84 4.43

U.S. News “Best” 91.66 3.68

Solucient “Top 100” 93.13 3.05

ANCC Magnet Hospitals 94.43 1.27

Hospitals with Multiple Designations 96.34 2.04

JCAHO “Commended” 96.94 1.61

Overall Evaluation ScoresComparative Data - 2002

Do JCAHO scores contribute to designation?

No statistically significant differences between designated hospitals and the rest on JCAHO scores (t-test)

Logistic regression models of designation (does JCAHO score predict designation or not?)

Unrelated to receiving best, top, or magnetHigher JCAHO scores only slightly (and not

significantly) predict being designated. Model fit is poor.

Havens, Vasey, & Kellogg 2003,Unpublished

What is the take home? Concern about the quality of hospital

care Adverse events and staffing deficiencies

cause alarm Scant information to assess performance No universal, “consumer friendly”

measure of quality Growing # of ratings / rankings Competition and marketing campaigns

AHRQ advice re: selecting quality care…

“Look for a hospital that: Is accredited by JCAHO.

Is rated highly by the state, consumer groups, or other organizations.”

http://ahrq.gov/

“PR Bonanzas”

“Americans have a love affair with rankings.” “Public relations goldmine” Marketing campaigns News media pick up Web-sites “Get a 2nd opinion from the ‘best’ hospitals” “Helps our development efforts” US News – “hottest marketing tool in the healthcare

industry” Financial boon for awarding organizations… And ….more

Research Implications Paucity of research Mixed findings Are the “Best really better than the rest”? Are some of the best better than other bests? Best on what? Is this meaningful to consumers - The big picture

paradigm? Reliance on structure and process only What are the key questions?

Safety Patient perspectives

Policy Implications A case for moving research to inform policy Consumer info, mis-info, or dis-info? As competition grows—release of comparative information will

increase “Desire to hold healthcare organizations accountable…need

accurate measures.” (Devers) How to uniformly gauge quality? Some argue that public does not understand or will not use…

the public is requesting “top 10” is part of American culture How to depoliticize – big $ maker for magazines Use of e-health sites is growing Being touted as a decision guide for consumers

Web sites Ladies Home Journal Reader’s Digest

We are on the job!

Stay tuned