Aligning clinician and patient reported outcomes

Tales from the Beautiful SouthJuly 2015

1

Liz Vernon-Wilsonelizabeth.vernon-wilson@southernhealth.nhs.uk

How do we know what we’re doing well?– With an individual with our caseload?– Or as a team/service/provider?

Do we understand case mix, resource use and effect?

Businesses use data from different sources to assess their market, and evaluate how well they’re meeting market demands.Staff and consumers can contribute to the information pool anddrive improvement of understanding and service if the data is contextualised and used….

Quality & Outcomes

What are our clients’ needs?

How much of this and how much of that do we need?

Creating Outcomes DataLocal CQUINs followed national directive to collect CROM, PROM & PREM and other quality indicators to support PbR development (CPA etc).

Care review Discharge

Initial community contact

PROM

HopeAgencyOpportunityWorking relationships

Recovery

Personalisation

Outcomes

Evidence

Partnership

Local choices of PROMs reflect an interest in understanding howRecovery orientated our service are (working aged adult MH)Holistic our interventions are (integrated older adult MH services).

Using Outcomes Data

We aim to integrate data from different sources to pull together a narrative about clinical effectiveness and patient experience

Validate with what the weather/health was like & move towardsPredicting what it will be like? Role of interventions?

The challenges are– to validate tools– build clinically relevant scenarios– Get clinicians asking questions

rather than being recipients of deductive analysis

What can PROM and HoNOS show?

They measure different things, but some scales are related and create an overall picture of well being.

5

EQ-5D scale for depression/anxiety relates to HoNOS65+ Mood and Other scales (7 & 8).

HoNOS Scale 10 (Activities of Daily Living) and EQ-5D Self Care, Usual activities

EQ-5D Visual analogue scale records change but EQ domains are unchanged.

HoNOS 4-factor shows effect size change, when individual scales are aren’t significant alone.

How does this help understand clinical effectiveness?• Benchmarking to understand what change is possible/likely..• Reflective, constructive comparison between teams….

The sensitivity of scales may be appropriate to some, but not all service user groups/points in the care pathway.

Outcomes for service users in Cluster 4Non-psychotic problems (moderate*/severe) of depression/anxiety/other NP. They may experience disruption to function in everyday life and increasing likelihood of significant risks.What can cluster assessment data show about what happens?

AMH n=425OPMH n=105

First, 50% of service users in AMH, but only 33% in OPMH have cluster 4 renewed. Service users are more likely to move to cluster 3 in OPMH. Do OPMH do a better job?

Number making transition to cluster 3 is n=93 AMH; n=53 OPMH.Average HoNOS/(65+) profiles from first cluster assessment in pair shown.*Starting points are significantly different for scales 5 & 9.Mean duration of these pairs was 200 days AMH, 230 days OPMH; not quite significantly different (p=0.057). However, over all cluster 4 reviews are done quicker by AMH than OPMH (182 days compared to 214; p=0.0013).

Understanding Cluster 4:3 transition data

All cluster 4 pairs Clinical change observed needs to be contextualised by time taken

**

Cluster 4:3 Clinical outcomesThe mean scores for HoNOS /(65+) recorded at cluster transition are shown.AMH pairs show sig difference* on scales 2, 7-10 (paired T test p<0.05).OPMH pairs show sig change on scales except 3 & 5.

What change do service users report?What interventions are used?

Mean changes recorded for OPMH service users are greater than AMH, noticeably on scales 1,7 & 8 which rate behaviour, mood & other problems. But the difference between AMH & OPMH is only significant on scale 7 (mood).

*

* * * *

Good & Bad News…

No equivalent cluster 4 to cluster 3 transition pairs available for PROM,But we can investigate – Cluster 4 renewal pairs (and they show some improvement too…..)– PROM responses of those that have scale 7 & 8 ratings similar to those

indicated for cluster 4:3 transitions.

n =240 n =35

What AMH service users in cluster 4 report…

Trend data from Hope, Agency & Opp PROM suggests service users feel more hopeful etc. through the care pathway.Do cluster transitions follow?Our practice doesn’t yet follow clear temporal correlation between when PROM is offered and when cluster is reviewed. Mean duration of PROM pairs here 109 days compared to cluster review duration of 182 days.

Limited pairs (n=20), all were renewals, mean duration 109 days

Severe problem

Improvement

Deterioration

HopelessHopeful

Hopeless

What OPMH service usersin cluster 4 report…

Improvement

Deterioration

Improvement

Deterioration

Older aged adults, all cluster 4 pairs;n=32. Mean duration 6 months

Trend data and paired data show improvement on the VAS (0-100 score) and individual domains (summarised as index score). Paired t test show statistical significance.Effect size?

Mean duration 6 months is closer to cluster review (mean 7 months) than AMH PROM pairs.

EQ index VAS Sample size

All pairs Medium (0.5)

Medium (0.7)

32

New referral to review or discharge

Medium (0.6)

Large (1.1) 13

PROMs for other clustersEQ-5D shows similar results for service users’ completed episodes in clusters 3 & 4, but cluster 18 data is different. The VAS & index profile recorded at initial assessment is sig diff for all three groups. Cluster 3 Cluster 4 Cluster 18

VAS 55 46 71

EQ- index 0.54 0.45 0.72

Starting point differs, as does “outcome”. Cluster 18 pairs show no change, cluster 4 show improvements.

HAO- Clusters 12 & 4 comparedComparison of trend scores for cluster 12 and cluster 4 show differences in the range of HAO scores recorded and variation in observed change.

Paired scores suggest that there are clinically significant effects for service users in cluster 4, that are not apparent in cluster 12, although sample size is low. Cluster 4 Cluster 12

Initial 135 56Review 69 112Discharge 40 30

Correlating CROM & PROMService users rated as having mood or anxiety problems (HoNOS) have more HAO problems (any cluster).Improvement in mood / anxiety could trigger cluster change.Improved mood is likely to correlate with improved PROM.Improvement in anxiety is less well correlated.

Improvement in hallucinations & delusions has no impact on HAO rating.

Count Mood Anxiety OccupationNo Problem 1048 733 1229Mod Sev Problem 773 472 546

n=1169 no, or minor problems not needing interventionn= 305 mod/severe hallucinations & delusions

Outcomes Information for whom?

Cluster 12 results illustrate that outcomes for service user are wider than anything we measure, CROM changes but recovery PROM doesn’t.Service’s aspirations for outcomes information can be different to one another (AMH/OPMH) and individual clinicians’ and service users.But there is overlap… good outcomes for all;

• To improve quality of life• Prevent relapse• Help improve quality of services

Measure at contact

Supportive relationships

Prevent relapse

Physical well being

Mental well being Work

Self- management

Home

Our information represents sampling at point of contact

Summary what does the data show?

HoNOS pairs give the clinicians’ perspective of change. Clustering has driven more regular collection.PROMS EQ-5D & HAO show the service users’ views, but cover different areas & don’t always correlate with cluster. They do help with engagement & evidence personalisation.When CROM & PROM coincide, they enrich our understanding of outcomes at clinically significant points.Different groups show different changes, e.g. cluster 18 and cluster 4 are different.Change varies depending on duration of pair and starting point.Clinician and service user report similar changes (or stabilisation). Suggesting we don’t engage with “gaming” (quality in PbR world).

(Some of) the Challenges ahead

Validation (tools & models)More data! Quantity & quality pleaseInformation/ knowledge quality of contextualisationUsing the data reflectively & more– Do cluster transitions help understand resource use?– Does benchmarking HoNOS/PROM show difference in

practice?– help understand what changes we’d hope to see from

interventions?– or encourage service improvement?

17

More questions?

18

Outcomes for service users

We’ve tried to measure some of what everyone needs, but do clinical pathways now direct us towards more specific tools?

HoNOS

Hope, Agency, & Opportunity PROM

Hope, Agency, & Opportunity PROM

http://www.mednetconsult.co.uk/imhsec/index.php/therapeutic-aims-started

Section 2

Relating PROM to Clinician Reported Outcomes

Service users who rated themselves as having a severe or disabling problem were compared to those who rated themselves as having no problem (2 domains chosen).Average HoNOS profiles for the two groups were compared.EQ-5D depression/anxiety maps to HoNOS mood and Other scale.Usual activities maps to more HoNOS scales..

FindingsHoNOS65+ reports on similar problems to EQ-5D domains indicating clinician and patient reported outcomes should be comparable. The VAS is a more holistic measure, affected by a broader range of factors, some of which may not be captured by HoNOS65+. It represents a different way of expressing outcome, relating to overall well-being.

Appendix I …PROM

Appendix 1 Hope, Agency & Opportunity Questionnaire

22