Simulation methods for calculating the conditional power in interim

analysis: The case of an interim result opposite to the initial hypothesis in a life-threatening

disease.

Somatostin plus Isosorbide-5-Mononitrate vs Somatostatin in the control of acute gastro-oesophageal

variceal bleeding:

a double blind, randomized, placebo-controlled clinical trial.

Junquera F, et al.GUT 2000; 46 (1) 127-132.

Design• Disease

– Acute variceal bleeding in cirrhotic patients

• Objective

– To test whether the addition of oral Isosorbide 5-Mononitrate (Is-5-Mn) improved the efficacy of Somatostatine (SMS) alone in the control of bleeding.

Design• Treatments

– Group 1: SMS + PLB (Control)

– Group 2: SMS + Is-5-Mn (Experimental)

• Working hypothesis

– The rate of success would increase from 60% to 90%.

Sample size: Pre-determination

n=n per group

2 = variance

= effect size

f( , ) = function of type I and II errors

n = 2 / 2 * f( , )

Statistical errors: f(,)f(,) = (U + U)2

(1 tail) 0,050 0,025 0,005

(2 tails) 0,100 0,050 0,010

0,200 6,183 7,849 11,679

0,100 8,564 10,507 14,879

0,050 10,822 12,995 17,814

Fixed sample size

ALPHA = 0.05

POWER = 0.90

P1 = 0.90

P0 = 0.60

Case sample size for uncorrected chi-squared test: 42

Introduction: interim analyses

• Often ethical concerns on these situations, specially in life-threatening diseases.

• Sometimes, pre-defined working hypothesis may not adjust to reality.– Treatments may be better than expected

– Treatments may be worse than expected (safety and/or efficacy)

• Long studies or big sample sizes make advisable some kind of interim control.

Introduction

• At some fixed times, cumulated data can be analysed and decisions may be taken in base to the findings.

• Multiple analysis can lead to statistical errors and mistaken clinical decisions.

• Several methods deal with multiplicity issues.

Design• For ethical reasons the design allows an

interim analysis, when half of the sample size is recruited.

• Pocock’s group sequential method (1977)

= 0.05

= 0.1 (power 90%)

p0= 60%, p1=90%

K z ' z ' z '1 2.782 0.005 2.576 0.010 2.178 0.0292 1.967 0.049 1.969 0.049 2.178 0.029

1 3.438 0.001 2.576 0.010 2.289 0.0222 2.431 0.015 2.576 0.010 2.289 0.0223 1.985 0.047 1.969 0.049 2.289 0.022

1 4.084 0.000 3.291 0.001 2.361 0.0182 2.888 0.004 3.291 0.001 2.361 0.0183 2.358 0.018 3.291 0.001 2.361 0.0184 2.042 0.041 1.969 0.049 2.361 0.018

1 4.555 0.000 3.291 0.001 2.413 0.0162 3.221 0.001 3.291 0.001 2.413 0.0163 2.630 0.009 3.291 0.001 2.413 0.0164 2.277 0.023 3.291 0.001 2.413 0.0165 2.037 0.042 1.969 0.049 2.413 0.016

O'Brien & Fleming Peto Pocock

Group Sequential Methods

adjusted sample size

ALPHA = 0.029

POWER = 0.90

P1 = 0.90

P0 = 0.60

Case sample size for uncorrected chi-squared test: 48

•Digestive System Research Unit

•Liver Unit

• Pharmacist

• Statistician

• Clinical Pharmacologist

Internal Participants

Monitoring Comittee

50% Sample size with evaluated outcome

Statistical analysis:

50 patients finalised

Data for Interim analysis

Interim analysis

Sucess 21 87.5% 18 69.2%

Failure 3 12.5% 8 30.8%

24 100.0% 26 100.0%

Control Exp

Chi-square=2.427, p-value=0.119

OR1 (observed): 3.11 (0.72 –13.51)

ORr (design): 0.17

Problem statement

• Evidence from interim analysis against working hypothesis

• Although no statistical evidence supporting study termination, clinical criteria suggested so.

• Search for objective support to clinical intuition.

50% Sample size with evaluated outcome

Statistical analysis:

50 patients finalised

Data for Interim analysis

Recruitment:

10 patients

Possible solutions

1) Group sequential methods

2) Alpha spending function approach

3) Repeated confidence intervals

4) Stochastic curtailing methods

5) Bayesian methods

6) Boundaries approach (likelihood function)

Conditional power

• Negative results:

– CAST (I-II) study. NEJM (1989 & 1992)

• Group sequential testing using permutation

distribution & stochastic curtailment methods

– HPMPC trial, Ann Intern Med 1997

– ACTG Study 243. NEJM 1998

Conditional power

• Positive results:– CRYO-ROP Arch Ophthalmology,1988

– Grable el al. Am J Obstet Gynecol, 1996

• Extension of trial: – Proschan MA, Biometrics, 1995

Stochastic curtailment

Lan, Simon y Halperin (1982)

Stop if in i inspection:

0, P(reject H0 | ) is high at the end

0, P(reject H0 | ) is small at the end

Application to real data• design:

p(ctr) = 60% p(exp) = 90%

• 1st Inspection (50 patients):p(ctr) = 87.5% p(exp) = 69.2%

• Probability of proving the working hypothesis at the end (100 patients)

projecting the results from this inspection

Methods:

• OR design: 0.17 => r = log(OR) = -1.792

• Simulations:

– Fortran 90

– 1,000,000 studies =>precision < 0.01%

– 15 possibilities ranging from –1.5xr to +1.5xr

Effect Size

0-1.5 x r+1.5 x

r

x rx r

Observed Design

/r

ORr design: 0.17 r = log(OR) = -1.79

p(Exp) p(Ctr) OR / R

absolute

diff .

1 90.0% 99.25% 14.70 2.688 -1.50 9.2%

2 90.0% 98.83% 9.39 2.240 -1.25 8.8%

3 90.0% 98.18% 6.00 1.792 -1.00 8.2%

4 90.0% 97.18% 3.83 1.344 -0.75 7.2%

5 90.0% 96.55% 3.11 1.135 -0.63 6.6%

6 90.0% 95.66% 2.45 0.896 -0.50 5.7%

7 90.0% 93.37% 1.57 0.448 -0.25 3.4%

8 90.0% 90.00% 1.00 0.000 0.00 0.0%

9 90.0% 85.19% 0.64 -0.448 0.25 -4.8%

10 90.0% 78.61% 0.41 -0.896 0.50 -11.4%

11 90.0% 74.31% 0.32 -1.135 0.63 -15.7%

12 90.0% 70.13% 0.26 -1.344 0.75 -19.9%

13 90.0% 60.00% 0.17 - 1.792 1.00 - 30.0%

14 90.0% 48.94% 0.11 -2.240 1.25 -41.1%

15 90.0% 37.98% 0.07 -2.688 1.50 -52.0%

H0

Obs

H1

Conditional power calculationOR / r % Sig.

Studies% Sig.

Studies Exp.

1 14.70 -1.50 2.69 65.402 0.0002 9.39 -1.25 2.24 62.304 0.0003 6.00 -1.00 1.79 57.597 0.0004 3.83 -0.75 1.34 50.526 0.0005 3.11 -0.63 1.13 46.298 0.0006 2.45 -0.50 0.90 40.516 0.0007 1.57 -0.25 0.45 28.147 0.0008 1.00 0.00 0.00 15.609 0.0009 0.64 0.25 -0.45 6.042 0.00010 0.41 0.50 -0.90 1.417 0.00311 0.32 0.63 -1.13 0.543 0.03012 0.26 0.75 -1.34 0.327 0.15813 0.17 1.00 -1.79 2.663 2.65514 0.11 1.25 -2.24 17.072 17.07115 0.07 1.50 -2.69 48.374 48.374

Conditional power calculation

0

20

40

60

80

100

-1.50 -1.00 -0.50 0.00 0.50 1.00 1.50

r

Po

we

r (%

)

1 (1st inspection)

r (design)

P( < 1 | /r= 1.00) = 53/1,000,000

P( < 1 | /r= 1.25) = 2/1,000,000

P( < 1 | /r= 1.50) = 0/1,000,000

Interim analysis after completion of 10 more patients

Chi-square=4.794, p-value=0.029

OR1’ (observed): 4.00

ORr (design): 0.17

Success 21 87.5% 18 69.2%

Failure 3 12.5% 8 30.8%

24 100.0% 26 100.0%

Control Exp

Final Interpretation

• The study was interrupted not based in the sequential pre-defined rule.

• The clinical intuition was confirmed by the conditional power calculation.

• The study was finished due to: – The low likeliness of the working hypothesis

– The high probability of a worse outcome with the experimental treatment

Conclusions

•Simulations may be a very useful tool in some design and analysis situations, as it has been shown in this case of the conditional power calculation.