© Nuffield Trust 15 March 2012
Analysis of Virtual Wards: a multidisciplinary
form of case management that integrates social
and health care
Geraint Lewis FRCP FFPH
Senior Fellow
Nuffield Trust
Co-authors: Martin Bardsley, Jennifer Dixon, John Billings and Rhema Vaithianathan
© Nuffield Trust
Why research this topic?
Challenge
Ageing population
Rising prevalence of chronic disease
Financial challenge
Opportunity
5% of patients account for 49% of emergency bed days
Expensive
Undesirable
Potentially avoidable
© Nuffield Trust
Historical Context
BMJ in paper* in 2002 showed Kaiser Permanente in California seemed to provide higher quality healthcare than the NHS at a lower cost
*Getting more for their dollar: a comparison of the NHS with California's Kaiser Permanente BMJ 2002;324:135-143
Kaiser identify high risk people in their population and manage them intensively to avoid admissions
Inaccurate Approaches:
Clinician referrals
Threshold approach (e.g. all patients aged >65 with 2+ admissions)
© Nuffield Trust
Kaiser Pyramid
Pyramid
represents the
distribution of
risk across the
population
Small numbers of
people at very
high risk
Large
numbers of
people at
low risk
© Nuffield Trust
Inpatient
data
A&E data GP
Practice
data
Outpatient
data PARR Combined
Model
Census
data
£0
£500
£1,000
£1,500
£2,000
£2,500
£3,000
£3,500
£4,000
£4,500
0 10 20 30 40 50 60 70 80 90
Predicted Risk (centile rank)
Actu
al
Avera
ge c
os
t p
er
pa
tien
t
Predictive accuracy of the Combined Model
Virtual Ward B Community Matron
Nursing complement
Health Visitor
Ward Clerk
Pharmacist
Social Worker
Physiotherapist
Occupational Therapist
Mental Health Link
Voluntary Sector Link
Virtual Ward A Community Matron
Nursing complement
Health Visitor
Ward Clerk
Pharmacist
Social Worker
Physiotherapist
Occupational Therapist
Mental Health Link
Voluntary Sector Link
Specialist Staff
•Specialist nurses
•Asthma
•Continence
•Heart Failure
•Palliative care team
•Alcohol service
•Dietician
GP Practice 1
GP Practice 2
GP Practice 3
GP Practice 5
GP Practice 4
GP Practice 6
GP Practice 7
GP Practice 8
© Nuffield Trust
100 patients per virtual ward
“Weekly”
35
Patients
“Monthly”
60 Patients
5 (35 5) (60 20)
= 5 + 7 + 3
= 15 patients for discussion each day
“Daily”
5 Patients
© Nuffield Trust
Ward rounds
20 minutes
PCT offices or GP practice meeting rooms
Tele-conferencing facility
© Nuffield Trust
Discharge
NIHR Management Fellow project
Options:
• Never discharge
• Fixed duration
• Clinical opinion
• Fixed goals
• Predictive score
• Front burner/back burner
© Nuffield Trust
Adaptations and Evaluations
Site Feature Evaluation
Croydon Nurse-led
Wandsworth VWGPs
Devon Practice-based
New York Homeless
Toronto Discharge Virtual Ward
North Somerset Clinical referrals
© Nuffield Trust
Adaptations and Evaluations
Site Feature Evaluation
Croydon Nurse-led Current study
Wandsworth VWGPs Current study
Devon Practice-based Current study
New York Homeless RCT
Toronto Discharge Virtual Ward RCT
North Somerset Clinical referrals Formative study
© Nuffield Trust
Evaluation Methods
Unreliable Reliable
Pre/Post comparison .
Area-based analyses Regression discontinuity
analysis
Clinical opinion .
© Nuffield Trust
Evaluation Methods
Unreliable Reliable
Pre/Post comparison Prognostic score matching
Area-based analyses Regression discontinuity
analysis
Clinical opinion Randomised controlled trial
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Nu
mb
er
of
em
erg
en
cy h
osp
ital
ad
mis
sio
ns
pe
r h
ead
pe
r m
on
th
Month
Intervention
Start of intervention
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Nu
mb
er
of
em
erg
en
cy h
osp
ital
ad
mis
sio
ns
pe
r h
ead
pe
r m
on
th
Month
Intervention
Start of intervention
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Nu
mb
er
of
em
erg
en
cy h
osp
ital
ad
mis
sio
ns
pe
r h
ead
pe
r m
on
th
Month
Intervention
Start of intervention
0.0
0.1
0.2
0.3
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4 5 6 7 8 9 10 11 12 Nu
mb
er
of
em
erg
en
cy h
osp
ital
ad
mis
sio
ns
pe
r h
ead
pe
r m
on
th
Month
Control Intervention
Start of intervention
© Nuffield Trust
Research Project Overview
Start date: May 2010
Publication date: May 2012 (estimated)
Project Components
1. Permissions and data collection
2. Propensity Score Matching
Local
Matched areas
3. Economic evaluation
4. Synthesis
© Nuffield Trust
Research Protocol
Lewis et al. (2011)
International Journal of
Integrated Care, 11(2)
© Nuffield Trust
Performance of HES data linkage
March 2011
1727
122 228
1725
122 222
1656
114 219
Croydon Devon Wandsworth
Records received
Records linked to HES
Unique patients linked to HES
© Nuffield Trust
0
50
100
150
200
250
300
Jan 2006
Jul 2006
Jan 2007
Jul 2007
Jan 2008
Jul 2008
Jan 2009
Jul 2009
Jan 2010
Jul 2010
Croydon
Devon
Wandsworth
Admission dates to Virtual Wards (analysis based on patients linked to HES)
March 2011
Are start dates accurate?
Did the definition change?
© Nuffield Trust
Age of virtual ward patients (analysis based on patients linked to HES)
March 2011
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
50%
Under 20
20 - 29 30 - 39 40 - 49 50 - 59 60 - 69 70 - 79 80 - 89 90 - 99 Over 100
Pro
po
rtio
n o
f p
art
icip
an
ts
Croydon Devon Wandsworth
© Nuffield Trust
Diagnoses recorded in inpatient data sets (analysis based on patients linked to HES)
March 2011
0%
10%
20%
30%
40%
50% Croydon (N=1656) Devon (N=114) Wandsworth (N=219)
© Nuffield Trust
Emergency hospital admissions per person per quarter (Note: no controls yet)
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4
Croydon (N=1394) Devon (N=72) Wandsworth (N=71)
Date of admission to VW
Analysis based on patients linked to HES admitted before Sep 09
© Nuffield Trust
Elective hospital admissions per person per quarter (Note: no controls yet)
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4
Croydon (N=1394) Devon (N=72) Wandsworth (N=71)
Date of admission to VW
Analysis based on patients linked to HES admitted before Sep 09
© Nuffield Trust
Outpatient attendances per person per quarter (Note: no controls yet)
0.00
0.50
1.00
1.50
2.00
2.50
3.00
3.50
-12 -11 -10 -9 -8 -7 -6 -5 -4 -3 -2 -1 1 2 3 4
Croydon (N=1394) Devon (N=72) Wandsworth (N=71)
Date of admission to VW
Analysis based on patients linked to HES admitted before Sep 09
© Nuffield Trust
Economic Analysis
Aims:
Calculate the direct costs of the VWs in each site
Perspectives of the NHS and the local authority.
If the intervention is effective, calculate the cost per averted hospital
admission
© Nuffield Trust
Economic Analysis: Step 1
Establish the cost of routine care
• Use data on the utilization of health and social care services,
including GP visits, hospital costs (inpatient, outpatient and A&E) and
council-funded social care.
• Calculate unit costs as the actual costs to the NHS (hospital tariff, GP
hourly rates etc.) and social care (PSSRU unit costs)
© Nuffield Trust
Economic Analysis: Step 2
Establish the cost of the intervention
• Financial data
• Validated with staff interviews, questionnaires and work diaries
Costs considered:
Staff costs
Travel costs
Land, computers and fixed capital costs
Management costs
Ignore:
Pharmaceutical
Laboratory costs
© Nuffield Trust
Prognostic controls
Local Controls
• Hospital data (inpatient, outpatient, A&E)
• GP data
• Community health services data
• Social care data
National Controls (matched areas)
• Hospital data (HES)
© Nuffield Trust
What next?
Planned matched control areas VW site Comparison areas
Croydon Enfield Waltham Forest Greenwich
Teaching
Redbridge
Devon* Somerset Cornwall & Isles
of Scilly
Shropshire
County
Herefordshire
Wandsworth Hammersmith &
Fulham
Camden Islington Westminster
* Dorset PCT and North Yorkshire &York PCT were excluded because they had virtual
ward schemes or equivalent during the comparison period
Predictive
Risk Model Impactibility
Model
Whole Population
People at Risk
People at
Risk who
will benefit
© Nuffield Trust
Approaches to Impactibility Modelling
Approach to Impactibility
Modelling
Efficiency Equity
Prioritise patients with ACS
conditions
Prioritise patients with high
“gap scores”
Exclude “difficult” patients
© Nuffield Trust
Acknowledgements
This work was funded by the National Institute for Health Research (NIHR) Service Delivery and Organisation (SDO) programme. Project number 09/1816/1021.
The views and opinions expressed here are those of the authors and do not necessarily reflect those of the NIHR SDO programme or the Department of Health.
We are grateful to the support and guidance of staff in our three study sites, and in particular our site representatives:
• Paul Lovell (Devon)
• David Osborne (Croydon)
• Seth Rankin (Wandsworth)
© Nuffield Trust 15 March 2012
www.nuffieldtrust.org.uk
Sign-up for our newsletter www.nuffieldtrust.org.uk/newsletter/login.aspx
Follow us on Twitter (http://twitter.com/NuffieldTrust)
© Nuffield Trust
© Nuffield Trust
Regression discontinuity analysis
90 Predictive risk score
Un
pla
nn
ed
ad
mis
sio
n r
ate
© Nuffield Trust
Outline
Background
Predictive modelling
Virtual wards
Research methods
Propensity score matching
Economic analysis
Preliminary findings
Next Steps
© Nuffield Trust
Why do this twice?
Local controls (SUS/GP/Social Care/Community services):
Can match on characteristics from all data sets
Can examine impact on use of wider range of services
Consistent coding from providers of care
But:
If intervention widespread, scope to find controls may be limited
We might not link every participant to routine data
May be spill over effects from intervention
© Nuffield Trust
Why do this twice?
Controls from matched areas (HES):
More plentiful source of controls
Comparison with other benchmarks of “usual care”
Less susceptible to incomplete data about participants
But:
Restricted in focus to hospital use (for matching and metrics)
Unexplained variation in hospital use around England (use of matched control areas helps to an extent)
Variation in coding practices by providers
© Nuffield Trust
What next?
Building local predictive models for each month
0
50
100
150
200
250
300
Jan 2006
Jul 2006
Jan 2007
Jul 2007
Jan 2008
Jul 2008
Jan 2009
Jul 2009
Jan 2010
Jul 2010
Croydon
Devon
Wandsworth
... To predict 12
months ahead
Predictor variables taken from two previous
years....
© Nuffield Trust
What next?
Selecting controls Should be balanced on:
• Age, sex, ethnicity, area-level deprivation score
• Recorded diagnoses of health conditions (social care needs?)
• Prior use (inpatient, outpatient, A&E, primary care, social care)
• Predictive risk score
Problem: this could be very numerically intensive
• Number of participants = ~1,800
• Number of potential controls = 3 million (matched control areas)
• Number of distinct categories of individuals =
(10 age bands) x (2 sexes) x (5 ethnic categories) x (15 diagnosis
categories) x ..... = lots!
© Nuffield Trust
What next?
Selecting controls Matching techniques :
• Propensity score matching: Rosenbaum and Rubin (1983)
• Model probability that an individual receives treatment (p-score)
• Match on this single score rather than on individual variables
• Prognostic (“predictive risk”) score matching: Hansen (2008)
• Model probability of experiencing outcome (predictive risk score)
• Inverse probability weighting: Hernán et al (2000)
• Use inverse probability of treatment in weighted regression
• Genetic matching: Diamond and Sekhon (2005)
• Match on individual variables using numerical optimisation routine