© Nuffield Trust

Predictive Risk 2012: Context

Predictive R13 June 2012

Martin Bardsley Head of ResearchNuffield Trust

© Nuffield Trust

Predictive modelling

• BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher-quality healthcare than the NHS at a lower cost. Kaiser identify high risk people in their population and manage them intensively to avoid admissions

• Modelling aims to identify people at risk of future event

• Relies on exploiting existing information

+ve: systematic; not costly data collections; fit into existing systems

-ve: information collected may not be predictive

• Use pseudonymous, person-level data

• In health sector a number of predictive models are available e.g. PARR++ and the combined model.

• *Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143

© Nuffield Trust

Uneven distribution of costs

The proportion of total costs spent on patients with category of annual costs (area of shape) with the proportion of all patients in annual cost band (dots)

Around 3% of patients are responsible for nearly half the total patient costs

© Nuffield Trust

Predicting admissions in advanceChange in average number of emergency bed days

Predictive models try to identify people here

© Nuffield Trust

Health and social care event timeline

© Nuffield Trust

Patterns in routine data to identify high-risk people next year

© Nuffield Trust

Distribution of Combined Model risk scoresImportance of risk adjustment

General population

Top 0.5%

0.5% - 5%

5% - 20%

20% - 100%

WSD participants – receiving telehealth or telecare

Top 10%

10% - 45%

45% - 85%

85% - 100%

Very high risk

High risk

Moderate risk

Low risk

© Nuffield Trust

Applications of predictive risk

• Case finding for people at high risk of admission seen as increasingly important for people with LTCs and complex conditions

• Evaluation and risk adjustment eg WSD

• Predicting future costs eg work on resource allocation

Related: Scope to make the most of linked data sets in describing care pathways

© Nuffield Trust

Choosing the predictive model bit

• What event should we be aiming to predict?

• What models and tools are available?

• What data do I need and how often?

• How often do predictive models need to be run?

• How accurate is the model?

• How much does it cost?

© Nuffield Trust

(1) Predictive tool = Predictive model + Software platform

Inpatient data

Outpatient

GP data

Population data

Tools to organise input data

Predictive model

Presentation and analysis tools-Gaps in care-Priority lists

Patient lists with risk score

InputsProcessing

Outputs

Users

© Nuffield Trust

Age distribution and mean risk scores within a diagnostic categories

© Nuffield Trust

Testing for gaps in care

© Nuffield Trust

(2) Key metrics for performance of a model (PPV and sensitivity)

Pooled 4-site 1k model

SensitivityWhat proportion of all events will the model detect?

Positive predictive valueWhen the model says high risk how often is it right?

© Nuffield Trust

Typical performance of models – predicting events next year

Predicting ... How many positives are correct (PPV)

What proportion of all events are found (Sensitivity)

Readmission based on prior admissions eg PARR

50%-75% 30-50%

Admission to hospital from a general population

20-50% 5%-15%

As above but just for highest risk groups (top 10%)

70-80% 5-10%

Changes in social care use 20-50% 5-15%

© Nuffield Trust

(3) Emerging market in England

• August 2011, the Department of Health announced that it had no plans to commission national updates of the latest Patients at Risk of Re-hospitalisation tool (PARR++) or the Combined Predictive Model

• Range of new/established commercial organisation developing risk tools

• Creation of new commissioning groups and new markets

• Increasing ease of accessing GP data

• Continuing financial pressures and the search for ways to reduce emergency hospital care.

© Nuffield Trust

Examples of case finding models available (with or without software platforms)

SPARRA PARR (++)

SPARRA MD Combined Predictive Model

PRISM PEONY

AHI Risk adjuster LACE

ACGs (Johns Hopkins) MARA (Milliman Advanced Risk Adjuster)

DxCGs (Verisk) Dr Foster Intelligence

SPOKE (Sussex CPM)LACE

QResearch models eg QD score

RISC

Variants on basic admission/readmission predictions:Short term readmissions Social careCondition specific tools Costs

© Nuffield Trust

(4) The model by itself doesn't change anything...Choosing an application

• Which people should I target?

• What interventions should we use?

• Who will use it and how? What clinical staff need to see results?

• Will some patients benefit more than others?

• When can I expect to see a return on investment?

© Nuffield Trust

Summary

• Predictive modelling is a practical case finding tool for identifying high risk patients

• Growing market for predictive models – extending beyond simple annual predictions of readmissions

• Ability to look at linked data valuable for other analyses

• Technical details of model performance is important – but so how is the way the model is implemented

• We hope today's conference will help you learn more about peoples’ experience of using these models.

© Nuffield Trust

The day ahead

• A review around the UK

• Examples of different ways that risk models have been applied in the NHS

• A view from outside the UK Germany and US.

• Developments in modelling

• Open session...share your experiences.

© Nuffield Trust

www.nuffieldtrust.org.uk

Sign-up for our newsletterwww.nuffieldtrust.org.uk/newsletter/login.aspx

Follow us on Twitter(http://twitter.com/NuffieldTrust)

© Nuffield Trust