30-day mortality for five clinical conditionsNSW hospitals, July 2009 – June 2012

Variation in mortality – international

“What is firmly established based on the evidence I have seen is that the various mortality statistics were sufficiently significant to require an in-depth investigation of the areas of service apparently involved.”

Francis Report, Paragraph 58, page 369

The mortality data paradox

There is a particular problem with publishing data about something everyone understands – death – manipulated statistically into an indicator almost no-one understands – a standardised ratio.

Why report mortality?

• Mortality statistics are an important element in monitoring quality

• For the first time, hospital-level data, adjusted to make fair comparisons, are available in NSW for the five clinical conditions

The Insights Series: 30-day mortality in NSW

• This report draws on 12 years of information to provide our analyses of 30-day mortality following hospitalisation for five clinical conditions.

• The report includes deaths in and outside hospital within 30 days, in line with international best practice.

• This captures both in hospital care as well as the broader healthcare system following discharge from hospital.

• This is the first time in Australia these mortality measures are being published for individual hospitals for these conditions.

Why are we reporting this measure?

• Mortality following hospitalisation is reported internationally to identify if there are required improvements.

• Reporting this measure ensures that care is of the best possible quality for patients.

• Mortality ratios provide a piece of the picture about hospital performance and are complementary to other quality and safety measures currently used.

The five clinical conditions – infographic

What is the measure about?

• It compares the number of deaths that occurred in the 30-days following admission to hospital with the number of ‘expected’ deaths.

• A statistical model is used to calculate the ‘expected’ number of deaths based on the characteristics of the patients presenting at each hospital.

• Hospitals with patients that are or sicker than the average hospital will have a higher ‘expected’ mortality.

• The findings are not appropriate for comparing or ranking hospitals or for identifying avoidable deaths.

Information publicly released • The Insights Series: 30-day mortality report

• Results presented for five clinical conditions

• NSW results and variation within the state

• Hospital profiles

• Up to 21 pages of content for each hospital

• Technical supplement

• Spotlight on measurement

• Discussion of the approach and sensitivity analyses

Cohort – identifying cases and deaths

• Privilege of linked data

• Cases identified by ICD-10 codes for principal / primary diagnosis (in line with international practice)

• Multiple episodes considered in a single period of care

• Fact of death

• Deaths in hospital and outside within 30 days of hospitalisation (captures discharge practices)

Risk adjustment and attribution

• Modelling - random intercept logistic regression

• International best practice for nominal reporting

• Allows control of multiple variables

• Smaller hospitals are reportable

• Attribution

• 2 attribution approaches (first / last hospital) – the Bureau chose to attribute deaths to the first hospital

• Transfers - extensive investigation and sensitivity analyses

• Implications reported in technical document

Due diligence

• Fairness

• Unadjusted mortality ratios

• Age-sex adjusted ratios

• Risk-standardised mortality ratios (RSMRs)

• Precise and thorough

• 4 modelling approaches

• 3 comorbidity methods (Charlson, Elixhauser, Australian Commission on Safety and Quality in Health Care - ACSQHC)

• Index; 1-year; 3-year; and 5-year look back

• Sensitive to context

• Drew on the expertise of a wide range of experts including clinicians, statisticians, policy developers and health service researchers to ensure relevance, validity and fairness

Nominal reporting rules

• All patients included in NSW-level data

• Hospitals with < 50 patients in the three-year period July 2009 – June 2012 are not reported nominally (not named)

• Internal communication to LHDs on all outliers < 50 patients

Understanding the results

• Random intercept logistic regression model calculates ‘expected’ mortality for each hospital, given its case mix  

• ‘Observed’ mortality is compared to ‘expected’ mortality [O/E] to form a ratio

• Takes into account a range of patient level factors that have been shown to influence the likelihood of dying

• A ratio less than 1.0 indicates lower-than-expected mortality, and

a ratio higher than 1.0 indicates higher-than-expected mortality

• Small deviations from 1.0 are not considered to be meaningful

• A mortality ratio of 1.25 may be within the control limits for one hospital but outside the limits for another hospital

Interpreting the results

• RSMRs are screening tools. They provide an indication of where further assessment may be needed

• Mortality data are not standalone indicators of quality or performance

• Mortality ratios are not be used to compare performance between hospitals

• RSMRs are not a reflection of the number of avoidable deaths

• Mortality data reflect performance of the healthcare system, not just hospitals

• The goals for NSW should be to ‘shift the curve’; and to reduce unwarranted variation

Mortality in hospital and following discharge

Hospitals with higher than expected results

Hospitals with one or more results higher than expected

Key findings

• In all five conditions mortality has decreased in NSW since 2000

• NSW compares well in the international context

• Among 80 referral, major and district hospitals, 58 did not have higher than expected mortality for any of the five conditions and no hospital had higher than expected mortality for all five conditions

• 18 had higher than expected mortality for only one of the five conditions

• 3 had higher than expected mortality for two conditions

• 1 had higher than expected mortality for four conditions

• Hospitals with higher than expected mortality were found in urban and rural settings

What’s next?

• The Bureau of Health Information has a mandate to inform the population, clinicians, policy-makers and members of Parliament in a way that both supports improvement in the system and promotes accountability.

• Hospitals should consider their results and identify where improvements can be made.

• Together with these results, other quality and safety measures,

such as clinical audit and review panels, can further assess models of care.