THE UNIVERSITY OF

WESTERN AUSTRALIA

Using routine data to measure and promote safety and quality in hospitals  

Stephen DuckettAdjunct Professor, ACERH/UQ(Joint work with Dr Terri JacksonAssociate Professor, ACERH/UQ)

Overview

What do we know about patient safety and patient harms in Australia?

What do we count as patient harm? How can we measure and monitor it? How much does it cost? How can use of routine data help us

understand the economics of improving patient safety?

Payment policy and patient safety

What do we know about Australia?

Landmark 1995 QAHC study:• Careful (expensive) methods• Incidence: 16.6 % of multi-day inpatient stays• Revised down to 10% (comparative US study)• Costs to system: $900 mil pa

The newspapers:• Bundaberg• The NSW ‘Cam affair’ (Cambelltown & Camden) • WA’s Royal Commission

Australian Commission on Safety & Quality in Health Care (Building on Aust Council for S&QHC (1999-2005)

What and how you ‘count’ depends on why you want to count it

WHY count?

We need to monitor rates routinely: using hospital discharge data

ICD-10-AM has specific codes:• T 80.0-88.9 ‘Complications of surgical &

medical care’• ‘End of chapter’ codes• Y 40-84.9 ‘External cause--complication of

surgical or medical care’

Australia has world-class quality hospital data

Victorian (and now Oz) ‘condition-onset flag’ (C-prefix) denoting:• Condition required treatment or extended LOS• Condition was not present on admission

Improving patient safety requires hospital-acquired incident cases

Problem of distinguishing ‘comorbidities’ from ‘adverse events’

Incident vs prevalent cases• Not GP, nursing home, admission from another

hospital• Overall rate in Victoria (2000/01): 8.25% (15.9% for

multi-day stays)• Only two-thirds (5.61%) were ‘Incident’ cases

The ‘C-prefix’ adds valuable information• 41% of all hospital-acquired diagnoses were missed

by ICD alone, eg, UTI, atrial fibrillation, pneumonia• Result: new national ‘Condition Onset’ flag on all Dx

Jackson TJ, Duckett SJ, Shepheard J, & Baxter KG. ‘Measurement of adverse events using ‘incidence flagged’ diagnosis codes’ Journal of Health Services Research and Policy, 11 (1):21-25; 2006.

Strengths of these data

Strengths• Timely & cheap data collection• Standardised definitions and coding

rules• Includes both dramatic and mundane• Independent reporting (not bedside)• Current payment incentives for

thorough coding

Weaknesses of these data

• May miss same-day patient harms Less coding investment Symptoms appearing post-discharge

• Prefix not currently audited• Coders may miss clues clinicians

could spot• No judgement about ‘preventability’

Notorious inter-rater reliability problems in this judgement

May be a ‘strength’ as today’s routine complication becomes tomorrow’s ‘preventable’ adverse event…

Risk factors and outcomes for hospital-acquired diagnoses (HA Dx)

Age is an important predictor of HA Dx • 29% of patients over 85 vs 9% in the 5-9 age group

Risk of HA Dx varies considerably by medical specialty• Gastroenterology 86.0% vs ENT surgery 10.5%

ALOS strongly associated with HA Dx • 11.2 days with HA Dx vs 6.9 days without

ALOS both a risk factor and an outcome In-hospital mortality associated (not caused)

• 4.09% with HA Dx vs 1.75% without

Duckett SJ, Jackson TJ, Hong Son Nghiem (2008). Risk factors and outcomes for incident complications in multi-day admissions to Victorian hospitals (manuscript).

Voluntary Sentinel Event reporting compared with routinely-coded hospital-acquired diagnoses*

Study Objective: To compare two sources of data on hospital sentinel events to evaluate strengths and weaknesses of both

Problems with voluntary reporting:• Stigma and blame attached to involvement in adverse

outcomes• Resulting reluctance to report• Safety-aware hospitals appear to have ‘worse’ outcomes• Focus on single events rather than rates reinforces

individual rather than system causation

Jackson T, Moje C, Shepheard J, and McMillan A. Poster presented to the 2007 Australian Conference on Safety and Quality in Healthcare, August, 2007.

Using routine data to identify sentinel events

Results

This SE Reporting

Study Victoria National

  05/06 05/06 04/05

SE1 Wrong patient or body part 13 25 53

SE2 Suicide in an inpatient unit 3 7 25

SE3 Retained instruments /material 53 6 27

SE4 Intravascular gas embolism 6 0 1

SE5 Transfusion reaction 0 0 1

SE6 Medication error 20 2 7

SE7(1) Maternal death (O95, O96, O97) 0    

SE7(2) Any maternal death 0 2 16

Public Reporting

Risk adjustment important for public comparisons• Reputation risk• Risk of patients being denied treatment• Fairness to providers

Public officials expected to intervene when quality declines

What do you report?

• What providers have ‘done’ or• What they are now doing to improve?

‘GOLD STANDARD’Identified Positive

Identified Negative

True Positive False Negative: harm to future

patients

True NegativeFalse Positive:

harm to reputations

Any measurement approach mustbalance the risks

Problems with public reporting using routine clinical data

Timeliness – data provided on an annual

basis, about 15 month delay in publication

Mistrust of ‘administrative’ data• Not ‘risk adjusted’• Insufficient clinical detail

Aggregated Data – unable to detect runs Nothing about cause of differences in

rates

Queensland Health’s Variable Life Adjusted Display (VLAD)

Plot of the cumulative difference between expected and actual

outcomes over a period of timeAMI VLAD - ( July 2003 - March 2006 )

0

1

2

3

4

5

6

7

0 20 40 60 80 100 120 140 160 180Case number

Esti

mate

d S

tati

sti

cal

Liv

es G

ain

ed

VLAD

Coory M, Duckett SJ, Sketcher-Baker K. Using control charts to monitor quality of hospital care with administrative data. International Journal for Quality in Health Care 20(1): 31-39..

Simplified Variable Life Adjusted Display (VLAD)

Patient survives - VLAD increases by the probability of the patient dying

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 1 2

Case number

Esti

mate

d S

tati

sti

cal L

ives

Gain

ed

Simplified Variable Life Adjusted Display (VLAD) / 2

Patient dies: VLAD decreases by the probability of the patient surviving

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 1 2

Case number

Esti

mate

d S

tati

sti

cal L

ives G

ain

ed

Simplified Variable Life Adjusted Display (VLAD) / 3

2nd patient survives: VLAD increases by the probability of the 2nd patient dying

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 1 2 3

Case number

Estim

ate

d S

tatistical L

ives G

ain

ed

Simplified Variable Life Adjusted Display (VLAD) / 4

2nd patient dies: VLAD decreases by the probability of the 2nd patient surviving

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

0 1 2 3

Case number

Estim

ate

d S

tatistical L

ives G

ain

ed

Variable Life Adjusted Display (VLAD)Plot of the cumulative difference

between expected and actual outcomes over a period of time

AMI VLAD - ( July 2003 - March 2006 )

0

1

2

3

4

5

6

7

0 20 40 60 80 100 120 140 160 180Case number

Esti

mate

d S

tati

sti

cal

Liv

es G

ain

ed

VLAD

Lessons:

Sometimes statisticians’ methods are somewhat dubious

Be wary of replicating them, even if you think you understand them

Specific Clinical Indicators

Developed in consultation with clinical expert groups

Indicators reviewed annually and refined on basis of feedback

Data from the Queensland Hospitals Admitted Patients Data Collection and the Perinatal Data Collection

Sex, age and comorbidities used to risk-adjust for illness severity and co-existing conditions

Clinical Indicatorsbeing developed

Readmission Acute Myocardial Infarction Heart Failure Knee Replacement Hip Replacement Depression Schizophrenia Paediatric Tonsillectomy and Adenoidectomy

Mortality Acute Myocardial Infarction Heart Failure Stroke Pneumonia Fractured Neck of Femur

O&G Caesarean Section Rate Std Primiparae Induction of Labour Rate Std Primiparae Perineal Tears (3rd or 4th

Degree) Hysterectomy – on women < 35 yo

Complications of Surgery Laparoscopic Cholecystectomy Vaginal Hysterectomy Abdominal Hysterectomy Fractured Neck of Femur Colorectal Carcinoma Knee Replacement Hip Replacement Prostatectomy

Future Falls Pressure ulcers Overall Surgical Complications AHRQ Patient Safety Indicators

Web-based documentation of VLADs

How bad is a ‘bad run’?

AMI VLAD - (01 J uly 2003

-14

-12

-10

-8

-6

-4

-2

0

2

0 50 100 150 200

Case number

VLAD

Variable Life Adjusted Displaywith control limits

-20

-15

-10

-5

0

5

0 50 100 150 200

Case number

Sta

tistical L

ives G

ain

ed

VLAD

Lower Control limit

Upper Control limit

Flagging criteria

• 30% higher than expected mortality– automated message emailed to the district manager and clinical lead, encouraging internal investigation and report to Area Clinical Governance Unit (50% for non-mortality flags)

• 50% higher than expected – flagged to Area Clinical Governance Units to ensure they are involved in further investigation (75% non-mortality)

• 75% higher than expected - identified to State Patient Safety and Quality board and in public reporting as being statistically significantly different from the average (100% non-mortality)

Duckett SJ, Coory M, Sketcher-Baker K. Identifying variations in quality of care in Queensland hospitals. Medical Journal of Australia 187 (10): 571-575.

VLAD charts allow tracking to specific case files

Detailed data underpinning the VLAD system

QH Pyramid Model of Investigation

Professional

Process of Care

Structure of Resource

Patient Case Mix

Data

Figure 1 : Pyramid Model of Investigation

Key strength of routine data: links to patient costs

Victorian and Queensland CWS data• n≈ 1.6 mil records pa• n≈ 100 large public hospitals• Validated for use in hospital funding

What can cost data tell us?• Economic burden of adverse events• Economic priorities for prevention efforts• Business case for:

prevention efforts medical research

• Cost-effectiveness analyses for: prevention programs new patient safety devices and procedures

Incident cases represent a large economic burden to the health care system

Patients with at least one C-prefixed adverse event: • Stay nearly 10 days longer than other patients• Cost $ 6826 more per episode (controlling for DRG,

age and co-morbidity) Extrapolated to entire hospital system:

• At least $511.5 mil additional cost in Victoria (2003/04)

• Adds 18.6% to hospital expenditures• Around $2 bil pa nationally• Even if only 40% preventable: $200 mil pa could be

saved in Vic; $800 mil nationally

Ehsani JE, Jackson TJ and Duckett SJ. ‘The incidence and cost of adverse events in Victorian hospitals, 2003-04’ Medical Journal of Australia, 184;11; 5 June 2006

Re-admissions* add to this cost burden

Current work on Victorian separations with a PDx in the T80-88.9 range of ICD-10-AM:• 16,734 admissions with a PDx of a

‘complication of surgical or medical care’• $70.6 mil pa additional public expenditure

on these cases in Victoria**

*Includes admissions for HA Dx from primary care and nursing homes

**McNair P, Jackson TJ, Borovnicar D. ‘Costs of Victorian admissions for treatment of adverse-event principal diagnoses, 2005/06’ submitted for publication, August 2008.

Queensland Health P4P approach

Incentives at margin ($8M out of $7B budget, 0.1% of total, larger % of specialty)

Range of specialties, data collection approaches, foci

Specialties:• Mental health• Stroke• Emergency department care• COPD• Medication

Duckett, S.J et al (2008) ‘P4P in Australia: Queensland’s New Clinical Practice Improvement Payment’ Journal of Health Services Research and Policy 13:174-177

Mental health indicators

Indicator Payment amount etc

Patients with the DRG Schizophrenia seen by a community mental health professional within 7 days following discharge from the same district mental health service provider.

$100Volume: 4518Electronic data collection in place

Recording of antipsychotic injection (depot) medication on iPharmacy for DRG Schizophrenia

$100Volume: 4518Electronic data collection in place

Acute stroke indicators

Indicator Payment amount etc

Acute Stroke patients with acute ischaemic stroke receiving antiplatelet therapy within 48 hours

if clinically appropriate

$50Volume: 6777Manual data collection system in place

Acute Stroke patients receiving dysphagia screen (minimum requirement) within 24 hours

$125Volume: 8699Manual data collection system in place

Emergency indicator

Indicator Payment amount etc

All patients aged 65 years (or 50 years if ATSI) and over who are admitted to and discharged from an emergency department to home/nursing home have evidence of

communication back to the GP/LMO

$100Volume: 28962Electronic data collection in place

Medication indicator

Indicator Payment amount etc

Electronic Discharge Medication Record completed where patient is over 65 years and has a complex medication regimes

$100Volume: 70000Electronic data collection in place

COPD indicator

Indicator Payment amount etc

Pulmonary rehabilitation program meets an acceptable standard as defined by the Statewide Chronic Obstructive Pulmonary Disease (COPD) Clinical Network.

$10 000Volume: 20 (review of unit)12 month audit of all sites (one off data collection)

Queensland Health approach

Moral suasion will be used to encourage most of (80%) CPIP payment to flow to clinical unit where indicator activity is being undertaken.

CPIP only to be used for non recurrent expenditure items

CPIP to be used to enhance not expand service (eg increase number of operations performed)

CPIP - summary

Started in January 2008 Experiments with very different

approaches initially Mostly Pay for process

adherence/reporting rather than outcomes

Can be rolled out to other indicators, add/drop indicators

Stephen_duckett@health.qld.gov.au