Class 11 -12

Chapters 5 & Elkins (1989)

Elkin et al: Purpose

Test feasibility of the collaborative clinical trial model

Examine relative efficacy of CBT, IPT, and Medication for Depression

2

NIMH Treatment of Depression Collaborative Research Program

U. of Pittsburg George Washington U. U. of Oklahoma 250 Patients: Major depressive disorder 28 therapists: years experience 2 -27; 71% male

10 psychologists 18 psychiatrists

3

4

Experimental Between-Group Designs

1. Post-Test Only Control2. Pre-Test -- Post-Test Control3. Solomon Four Group (combination of 1 and 2 above)

Factorial Design more than one independent variable; interactions treatment X therapist

or patient characteristic

Dependent Sample Design (Matching)

5

Experimental Between-Group Designs

1. Post-Test Only Control2. Pre-Test -- Post-Test Control3. Solomon Four Group (combination of 1 and 2 above)

Factorial Design - Post Hoc more than one independent variable; interactions

treatment X patient characteristic (depression level at intake)

Dependent Sample Design (Matching)

6

IVs: Experimental Groups:

Cognitive Behavioral Therapy Interpersonal Therapy

16 individual sessions/ 50 min.Medication + Clinical Management* Pill-Placebo + Clinical Management*

1st session 55 min.; then 20 to 25 min.

* Minimal supportive therapy condition

Dependent Variables

Clinical Evaluator

Self Report

7

Dependent Variables

Clinical Evaluator

• Hamilton Rating Scale Depression (HRSD)

• Global Assessment Scale (GAS)

Self Report

• Beck Depression Inventory (BDI)

• Hopkins Symptom Checklist (HSCL-90)

8

Outcome Research Strategies

Primary Analyses

Secondary Analyses (Post-Hoc)

9

Outcome Research Strategies

Primary AnalysesTreatment packageComparative

Secondary AnalysesClient Variation -moderation effect?

10

Outcome Research Strategies

Secondary AnalysesClient Variation -moderation effect

depression level at intake as moderator of relation of treatment groups to outcome

Were outcomes across treatment types different for patients with higher versus lower levels of depression at pre-test?

11

Control Groups

CBT IPTMedication + Clinical Management* Pill- Placebo + Clinical Management*

* Minimal supportive therapy condition

12

Treatments & Therapists

Cognitive Behavioral Therapy

Interpersonal Therapy

Different group of experienced therapists

Medication + Clinical Mngmnt

Pill-Placebo + Clinical Mngmnt

Same therapists - psychiatrists

13

Ensure Valid Treatments

Specify the treatment(s)

Therapist training/monitoring

Fidelity Checks

14

Ensure Valid Treatments

Specify the treatment(s)Manuals

Therapist training/monitoringFidelity Checks- therapy tapes

Collaborative Study Psychotherapy Rating Scale (CSPRS)

Treatments could be discriminated 95% of the time

15

Attrition (>15 sessions or 12 weeks)

Total: 77/239 32%CBT 32%IPT 23%Meds/CM 33%Placebo/CM 40%

16

Which group to use in outcome analysis??

Total N = 239 CompletersN = 155

15 weeks or12 sessions

End-Point N = 204

At least 3.5 weeks or4 sessions

End PointN = 239

Intent to Treat Group(last assessment or pre-test)

17

Assessment Times

Pre treatment

Post Treatment4, 8, 12 weeksTermination – 15 weeksFollow up: 6, 12, 18 months

18

19

Analyses of Pre-test/Post-test (1)

Paired T-Test to examine differences between pre-test and post-test scores (p. 974)

How Many ??

Table 1 Completer Group: At least 12 sessions; n=155 (page 975)

20

21

Analyses of Pre-test/Post-test (1)

Paired T-Test to examine differences between pre-test and post-test scores (p. 974)

How Many ??

4 Treatment groups X 4 Outcome measures CBT HRSD IPT GAS IMI-CM BDI Pla-CM HSCL-90 X 3 Samples – Completers; End Point 204; 239

Findings – T-Tests

22

Analyses of Post-test scores Use pre-test as a covariate in analyses of

co-variance to compare mean post-test scores across the 4 treatment groups

Calculate a residualized change scores – amount of variability in the post-test that is not associated with the pre-test score

Used a P<.10 in ANCOVAS and p = .10/.06 = .017 for pair-wise comparisons (p.974)

23

24

ANCOVAS: Post test scores

Statistically significant differences between groups in scales at post-test Four 3 X 4 ANCOVAS: differences across

treatments in Post-treatment scores in: HRSD, GAS --- BDI, HSCL90

3 (sites) X 4 (treatment groups) Analyses reported only for treatment groups

combining them across sites

Co-VariatesPre-test scores Marriage Status (1,2)

Why not MANCOVAS? P.973

25

Table 1 Completer Group: At least 12 sessions; n=155 (page 975) p<.10

26

Table 1 End Point 239 Group CBT IPT IMI-CM PLA-CM p<.10

27

Findings Pair-wise ComparisonsSample Clinical Evaluator Self-Report

Completer N = 155

BDI Pairwise NSHSCL-90-T p=.006

IMI-CM<PLA-CM

EP-204GASIMI-CM<PLA-CM (trend p=.020)

EP-239HRSDep IPT, IMI-CM<PLA-CMGAS p =.010 IMI-CM<PLA-CM

(trend p=.017,.018)

28

29

Measuring Change Elkin et al. 1989

Statistical significance

Clinical significance

30

Measuring Change Elkin et al. 1989

Statistical significance Differences between groups in scales at

post-test controlling for pre-test scores

Clinical significance Percentage of participants that changed

from dysfunctional to functional level (using cut-off scores)

Clinical Significance Recovery Analysis

Proportion of patients who improved vs. not improved

Cut Off Scores Not Depressed HRSD < 6 and BDI < 9 Depressed HRSD > 6 or BDI > 9

Statistical Analyses Chi square: Proportion of depressed and non-

depressed patients across treatment groups at termination.

32

33

34

End Point 239 HRSD p = .04 CBT IPT IMI-CM P-CM

Chi Square (Χ2) tests to what extent the proportion in each group is what may be expected by chance or if it is larger or smaller than expected…….

IPT = IMI-CM>Placebo-CM CBT - % comparison was not sig. for any group

Proportion of cases that met recovery criteria

36%(ns)

Proportion of cases that met recovery criteria

43%

Proportion of cases that met recovery criteria

42%

Proportion of cases that met recovery criteria

21%

Secondary Analyses

To examine effect of pre-treatment severity (HRSD/GAS) on outcome by treatment groupDVs: Post-treatment scores Severity Criteria

HRSD>20 44% of sample GAS<50 41%

Covariate Marital Status

35

2X4 ANCOVA (severity x treatment) DVs- Post Test HRSD, GAS, BDI, HSCL-90

Main Effect for

Main Effect for

(Interaction term)***

36

2X4 ANCOVA (severity x treatment) DVs- Post Test HRSD, GAS, BDI, HSCL-90

Main Effect for Severity More Severe Pre-Test HRSD>20; GAS<50 Less Severe Pre-Test

Main Effect for Treatment CBT IPT IMI-CM P-CM

Severity X Treatment (interaction term)*******37

38

Interaction Effect HRSD Severity x TG Dependent Variables: HRSD* GAS, BDI, HSCL-90 (p.976)

Completer S BDI IPT IMI-CM P-CMHigh Depression

Low Depression

Completer S BDI IPT IMI-CM P-CMHigh Depression

Low Depression

Completer* CBT IPT IMI-CM P-CMHigh HRSD Low HRSD

End Point 239^ CBT IPT IMI-CM P-CMHigh HRSDLow HRSD

4 sets of 3 2X4 Ancovas: 4DVs, 3 sample subgroups *p<.10; ^p<.11

End Point 204* CBT IPT IMI-CM P-CMHigh HRSD

Low HRSD

39

Interaction Effect GAS Severity x TG: Dependent Variables: HRSD GAS, BDI, HSCL-90

Completer S BDI IPT IMI-CM P-CMHigh Depression

Low Depression

Completer S BDI IPT IMI-CM P-CMHigh Depression

Low Depression

Completer** CBT IPT IMI-CM P-CMHigh GASLow GAS

End Point 239* CBT IPT IMI-CM P-CMHigh GASLow GAS

End Point 204**** CBT IPT IMI-CM P-CMHigh GAS

Low GAS

40

Treatment by Severity Interaction/end-point 204 sampleHigher score Negative Outcome Higher Score Positive Outcome

Summary All Pairwise analyses following interaction effects p.976

Less severe groups: no differences across treatment groups

More severe groups IPT more effective than PLA-CM in 3

instances all in the HRSD measure in the END Point Sample 204 (3 out of 4 comparisons)

IMI-CM more effective than PLA-CM across a number of measures (8 out 10 comparisons)

41

42

Figure 2Recovery Rates (%) endpoint /204 sample

Figure 2 Recovery Rates (%) endpoint /204 sample for severity groups (p.977)

Less severe subgroups: NS differences among treatments for all samples with HRSD or GAS.

More severe subgroups for HRSD and GAS: Consistent findings across the three samples IPT>PLA-CM 5/6 and IMI-CM>PLA-CM 6/6

43

Threats to Statistical Conclusion ValidityAre the observed relations among variables accurate?

Power

Unreliability of Measures

Unreliability of Treatment Implementation

Extraneous Variance in the Experimental Setting

Heterogeneity of Participants

44

Threats to Statistical Conclusion ValidityAre the observed relations among variables accurate?

Power • Large N by group range 34-62 +• Outcome measures are well-known +• Power analyses 81-95% for medium effects +• p<.10 for Mancovas and .10/6 for pairwise comp

Unreliability of Measures

Unreliability of Treatment Implementation

• Experienced Therapists – 2-27yrs Mean = 11 +• Manuals, training per treatment group +• Closely monitored +• Taped sessions – 95% correctly classified +

Extraneous Variance in the Experimental Setting

• Not known for the most part -• 28 therapists from 3 – 11 patients each -• no way to control for therapist effects P. 980 -• one site CBT another site IPT similar to

Meds/CM

Heterogeneity of Participants

• Random assignment to groups +• Only included 45% of those screened. + • Mostly women 70% female +• 89% white participants + 45

Threats to Internal ValidityCan we conclude that there is a causal relation between the IV and the DV? Did treatment cause differences in DV across groups? Selection Who gets assigned to which group?

History

Attrition What do we know about drop-outs?

Repeated Testing Effects

Reaction to Control Group Assignment

46

Threats to Internal ValidityCan we conclude that there is a causal relation between the IV and the DV?Selection Used Randomization-

See factors under Heterogeneity of Participants

History Time frame of study not reportedDid therapy happen at about the same time for everyone?

Attrition Relatively high attrition rates - 32% -- about 25% was for negative reasons related to treatment- (-)Early terminators were more depressed at intake (-)

Repeated Testing Effects

Tested at frequent intervals –’ pre-test, 4, 8, 12, weeks, termination 6 12 and 18 months follow-up

Reaction to Control Group Assignment

Not known – but could be the case. Placebo/CM experienced the highest attrition –  32% CBT—23% IPT – 33% Meds/CM -- 40% Placebo/CM

47

Threats to Construct Validity

To what extent variables capture desired constructs

Mono-Operation Bias(Instruments)

Mono-Method BiasExperimenter Expectancies

48

Threats to Construct Validity

To what extent variables capture desired constructs

Mono-Operation Bias(Instruments)

• Used 4 different outcome measures HIRSD, BDI, GAS, HCSL-90 +

• Measures of well-known psychometric properties +

Mono-Method Bias• Used both patient self report and clinician

completed measures +• Measures of well-known psychometric

properties +

Experimenter Expectancies

• Clinicians not blind to therapy modality-• Psychiatrist blind to Med condition +

49

Threats to External ValidityCan we generalize observed relations across persons, settings and times

Person-Units

Outcome Measures

Settings

50

Threats to External ValidityCan we generalize observed relations across persons, settings and times

Person-Units• Highly selected sample (-)• Only 45% screened were selected (-)• Generalizable to white (89%) women (70%)

highly educated (75% coll degree or some coll) who were less severely depressed (p.974)

Outcome Measures• Interview and self –report measures +• Clinical significance recovery rates +• Statistically significant findings were not

consistent across measures – HRSD detected more differences in depression that BDI -

Settings Empirical Question ????

51

52

Results: Summary 1/3

Paired T test showed stat. sig. differences (p<.001) in Pre- Post scores in all measures for all three groups of participants (even placebo pill/CM) Intent-to treat Completers Minimum

3.5<Sessions Completers of all or most sessions

At least 12<sessions > 15 (n=155)

Results: Summary 2/3

ANCOVAS showed no stat sig differences in pre-test scores in

any measure for any treatment group

Stat sig differences in post-test BDI/HSCL90 Completers HSRD/GAS Total Group (239)

54

Results: Summary 3/3

Pairwise Follow-up ANCOVA HSCL-90 IMI-CM> PLA-CM (Completer) GAS -- IMI-CM>PLA-CM (Total 239 group) HRSD IPT, IMI-CM>trend PLA-CM (Total 239)

Recovery Findings (Clinical Significance) IPT, IMI-CM > PLA-CM ( End-Point 239) 43% 42% 21% Post-test HRSD<6 CBT = 36% NS