Methods to improve the efficiency of screening for...

transcript

Methods to improve the efficiency of screening for multiple mental disorders

Phil Batterham Centre for Mental Health Research

The Australian National University

Hierarchical screening

General psychological distress

Internalizing disorders

Depression Anxiety

Social phobia Generalized

anxiety Panic disorder

Externalizing disorders

Alcohol abuse Substance abuse Adult ADHD

Psychological distress (e.g. K6)

Depression (PHQ-9)

GAD (GAD-7)

Social phobia (SOPHS-5)

Panic disorder

(PHQPAN-5)

Finish 4 disorders, 26 items

First-phase screening approaches 1. No hierarchy (control)

2. K6 score

3. Psychological distress decision tree

4. Disorder-specific decision tree

5. Gating items

Method 1: No hierarchy (control)

Depression (PHQ-9)

GAD (GAD-7)

Panic disorder (PHQ-Panic-5)

Method 2: K6 hierarchy

K6 score

Full screen

Depression (PHQ-9)

GAD (GAD-7)

Panic disorder (PHQ-Panic-5)

Method 3: Distress decision tree

• Choose distress items that best discriminate absence of disorder

• Subgroups least likely to have disorder escape further screening

• Chi-Square Automatic Interaction Detection (treedisc macro in SAS)

Chi-Square Automatic Interaction Detection (CHAID) • Divides the sample into subsamples with

different risks of outcome

• Diagram with leaves and branches

• Categorical items

• Branching based on item that best differentiates on the basis of the outcome

• Smallest p-value from a chi-square statistic

Chi-Square Automatic Interaction Detection (CHAID) • Splitting stops when:

– There is a small number of observations in a leaf (20 observations)

– No split would result in a significant 2 value (=.2)

– A specified level of branching is reached (6 levels)

K7 (depressed)

MWBS9 (close to others)

K18 (tense/shaky)

FULL SCREEN

0 1 3,4

K18 (tense/shaky)

K11 (good mood)

K18 (tense/shaky)

MWBS11 (decisive)

K8 (restless)

FULL SCREEN

STOP (n=23)

STOP (n=95)

FULL SCREEN

STOP (n=14)

FULL SCREEN

K2 (nervous)

FULL SCREEN

K19 (angry)

FULL SCREEN

STOP (n=37)

2,3,4 2,3,4 0,1

3,4 0,1,2

0-2 3,4

4 0-3 0 1-4

0,1 2,3,4

MWBS2 (useful)

FULL SCREEN

K1 (so sad...)

STOP (n=11)

K3 (restless)

FULL SCREEN

STOP (n=10)

0 1 2,3,4

2,3,4 0,1

Method 4: Disorder decision tree

PHQ-2 (down/

depressed)

SOCPH-4 (suffered social

occasions)

SOCPH-5 (social fear disruption)

PHQPAN-1 (anxiety attack)

GAD7-1 (nervous/ anxious)

PHQ-1 (little

interest)

FULL SCREEN

STOP (n=425)

FULL SCREEN

STOP (n=52)

FULL SCREEN

GAD7-3 (worrying)

FULL SCREEN

0 1 2,3

1,2 3,4,5

Yes No

No 3,4

0,1 2,3

FULL SCREEN

PHQ-1 (little

interest)

FULL SCREEN

PHQPAN-1 (anxiety attack)

STOP (n=189)

FULL SCREEN

No Yes

0,1 2,3

Method 5: Gating items

PHQ-9 Items 1 & 2

PHQ-9 Items 3-9

GAD-7 Item 1

GAD-7 Items 2-7

PHQ-panic Item 1

PHQ-panic Items 2-5

SOPHS Item 1

SOPHS Items 2-5

PHQ1 ≥ 2 or PHQ2 ≥ 2 PHQ1<2 and PHQ2<2

GAD1 > 2 GAD1 < 2

PHQ-pan1 = “No” PHQ-pan1 = “Yes”

SOPHS1 ≤ 2 SOPHS1 > 2

Testing the hierarchies

• Efficiency

– Mean number of items presented

• Precision – Sensitivity relative to control

Validation samples

• Two community-based samples

• N1 = 1360; N2 = 668

• Recruited through Facebook ads

• Australia-wide, 18+

Sample 1 (N=1360)

0% 10% 20% 30% 40%

60 and over

Refused

Metropolitan Regional Rural Remote Refused

Sample 2 (N=668)

0% 10% 20% 30% 40%

60 and over

Refused

Metropolitan Regional Rural Remote Refused

Samples: Psychopathology

MDD 34%

GAD 26%

PD 22%

MDD 25%

GAD 20%

PD 14%

SP 16%

None: 63% None: 53%

N = 1360 N = 668

(Not to scale)

Results: Efficiency gains

22.0 21.8

No hierarchy K6 hierarchy Psychologicaldistress items

Disorder-baseditems

Gating items

1% 16% 34% 46%

Results: Projected efficiency gains

Disorder-baseditems

Gating items

17% 26% 40% 54%

Results: Summary

99.7% 99.9% 100.0%

Disorder-baseditems

Gating items

Results: Summary

• Two-phase hierarchical screening was efficient and precise

• Using gating items had most efficiency gain (up to 54%)

• Using decision trees also had large efficiency gains (up to 40%)

• K6 did not improve screening efficiency

Considerations

• The K6/K10 were designed to “rule in” not “rule out”

• Hierarchical screening works better for:

– Low rates of psychopathology

– Longer screening scales (60% fewer items)

• Tested with other disorders/outcomes

– PTSD, adult ADHD, alcohol abuse, suicidality

Considerations

• Purpose of screening

• Brevity vs. need for data

• Ease of implementation vs. efficiency

– Gating only works for scales with gated scoring criteria

– Pencil and paper vs. computer-based

Future research: Adaptive screening

• Fully adaptive measures

– Each response determines next item presented

• PROMIS measures

– IRT-calibrated item banks

– PROMIS-depression 5-item adaptive screener more precise than 20-item CES-D

From: Pilkonis PA, et al. Item Banks for Measuring Emotional Distress From the Patient-Reported Outcomes Measurement Information System (PROMIS®): Depression, Anxiety, and Anger. Assessment 2011 18: 263-283

From: Choi SW et al. Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms. Qual Life Res, 2010, 19: 125-136.

• Assess severity level, not clinical criteria

→ Test against DSM criteria using decision tree approach

• Validated in US

→ Validate internationally

• Limited array of mental health problems

→ Develop item banks for other disorders

Psychological distress

Psychosis Internalising

Depression Generalised

anxiety

Panic / agoraphobia

Social phobia

PTSD OCD

Externalising

Alcohol use/abuse

Substance use/abuse

Adult ADHD Anger

Suicidality

Emergency referral

Referral to MH professional

Referral to phone service

Referral to primary care

Low intensity online CBT

Psychoeducation

Watchful waiting

Item bank

Adaptive screening tools: •Anxiety •Depression •Suicidality, etc

Assessment: •Resources •Context •Preferences

Care decision

Ongoing monitoring

VIRTUAL CLINIC

STEPPED CARE SERVICE

Conclusions

• Hierarchical screening for multiple disorders can result in large efficiency gains without sacrificing accuracy

• Disorder-specific items more useful than general distress items

• Much promise in adaptive screening

Conclusions

• May be applied to – Virtual clinics

– Primary care screening

– Research

– School-based screening

– Other service contexts

Collaborators

Prof Helen Christensen Black Dog Institute, The University of New South Wales, Sydney Australia

Dr Alison Calear Centre for Mental Health Research, The Australian National University, Canberra Australia

Dr Matthew Sunderland and Dr Natacha Carragher National Drug and Alcohol Research Centre, The University of New South Wales, Sydney Australia

Prof Andrew Mackinnon Orygen Youth Health Research Centre, The University of Melbourne, Melbourne Australia

Acknowledgements

The College of Medicine, Biology and Environment, The Australian National University

Australasian Society for Psychiatric Research

National Health and Medical Research Council (Australia)

philip.batterham@anu.edu.au

Methods to improve the efficiency of screening for...

Documents