Topics in Clinical Trials (8) - 2012
J. Jack Lee, Ph.D.Department of BiostatisticsUniversity of Texas M. D. Anderson Cancer Center
Monitoring Response Variables
Monitor toxicity: ethical considerations, protect the safety of participantsMonitor the primary endpoint
Shows intervention is clearly better Early stopping due to efficacy
Shows intervention is harmful Early stopping due to adverse effect
Shows intervention is unlikely to be beneficial Early stopping due to futility
No clear indication one way or the other Continue the trial as planned Adjust the sample size based on observed effect, etc.
Data monitoring committee (DMC) / Data Safety and Monitoring Board (DSMB)
Fundamental Point
During the trial, response variables need to be monitored for early dramatic benefits or potential harmful effects.Preferably, monitoring should be done by a group of independent investigators.Although many techniques are available to assist in monitoring, none of them should be used as the sole basis for the decision to stop or continue the trial.
Data Monitoring Committee (DMC)
Composition Why not include the study investigator?
Conflict of interest, lack of objectivity Knowing the interim result may affect the clinical
equipoise and the subsequent trial conduct Credibility
Independent group of experts in the field including clinicians, statisticians, patient advocates, etc.
Institutional DMC or a special DMC assembled for the trial
Responsibility of DMC
Ensure the safety of participantsEnsure the integrity of the trial monitor the accrual of the trial Examine the randomization process Evaluate the compliance status
Make decision to continue or terminate the trial based on all data and informationOversee the trial on behalf of the sponsorProvide a service to the regulatory agency such as NCI or FDA
Operation of DMC
Meeting schedule Before or in the early phase of the trial Correspond to the time of pre-defined interim
analyses End of study
Format and Confidentiality Open session
All invited: accrual, logistic, data quality, adherence, toxicity, etc.
Closed session DMC + study statistician DMC only
Executive session DMC + study PI
Decisions During the Trial
Planned Interim Analysis Group sequential design
Unplanned Interim Analysis Group sequential design – alpha spending function Conditional Power Stochastic Curtailment Predictive Power / Predictive Probability
Interim Decisions Stop the trial due to efficacy
superiority, one way or another Stop the trial due to futility
Lack of efficacy, no difference between groups Continue the trial as planned Sample size re-estimation
Conditional Power ApproachExample 1: Compare 2 Antibiotics
Goal: compare the response rate of pipracillin (tx1) vs. clindamycin (tx2)
Design: RCT with 47 pts/arm. 80% power to detect an absolute difference of 25% (85% vs. 60%) with 1-sided a= 5% (2-sided a= 10%)Interim result:
ResponseNo Yes
Tx 1 4 22 26
Tx 2 3 18 21
7 40 47
Question: Can the trial be stopped early?
Estimation
The 95% CI of P1 – P2 covers 0.25. If the goal of this trial is to detect a 25% difference in response rate, cannot stop the trial due to “no difference.”CI calculation does not reflect the nature of interim analysis. If use “repeated CI” (e.g. Jennison & Turnbull), the CI will be wider.
1
2
1 2
ˆ 22 / 26 0.846
ˆ 18 / 21 0.857
ˆ ˆ 0.011, 95% exact CI = (-0.33, 0.27)
P
P
P P
Conditional Power by B-ValueLan & Wittes, Biometrics 44:579-585, 1988.
1 2
0 a
1
Let , ,..., ( , 1)
For testing H : 0 versus H : 0,
Let / be the test stat. with pts
/ be the information time,
( ) and ( )
Key equation : (1) ( ) [ (1) ( )]
1. ( ) an
N
n
n ii
n N
X X X N
Z X n n
t n N
B t Z t E Z N
B B t B B t
B t
d (1) ( ) are indep. normally distributed.
2. [ ( )] , [ (1) ( )] (1 )
3. [ ( )] , [ (1) ( )] (1 )
B B t
E B t t E B B t t
Var B t t Var B B t t
/ 2 / 2
The conditional power (1) | ( ), is
( ) Pr( (1) | ( ), ) Pr( ( ( ) (1 ),1 ) )
( ) (1 ) 1 ( )
1For two-sided test
( ) (1 ) ( ) (1 )( ) 1 ( ) ( )
At e
1
t
1
h
p
p
B Z B t
C B Z B t N B t t t Z
Z B t t
t
Z B t t Z B t tC
t t
1 2
47
interim, we have
(.846)(.154) (.857)(.143)~ (.846, ), ~ (.857, ),
26 21.011
.106(.846)(.154) (.857)(.143)
26 21( .106) 0.006, ( 0) 0.008
.85 .60( 2.83 or 2.8
(.85)(.15) (.60)(.40)
47 47
p p
p N
P N P N
Z
C C
C Z
3) 0.34
Stochastic CurtailmentLan, Simon, Halperin: 82 Comm in Stat.-Seq:1:207-
219Davis, Hardy: 94, J. Clin. Epi 47:1033-1042
One might stop the trial early and Reject Ho if cond. Prob P(Z(1) R| Z(t), Ho) ≥ g Accept Ho if cond. Prob P(Z(1) A| Z(t), H1) ≥ g’
Early termination can be due to efficacy or futility Overall type I error rate ≤ /a g Overall type II error rate ≤ /b g’ With a small number of looks, the error rates will be even
less
Boundaries of stochastic curtailment
1
1 θ(1 )( ) ,
Under : θ 0, Under : θo
Z Z t tZ t
tH H Z Z
Stochastic Curtailment and Conditional Power
Under H1: = 1.960 + 1.282 = 3.242, cannot stop before t=0.64
0.0 0.2 0.4 0.6 0.8 1.0
-10
-8
-6
-4
-2
0
2
4
6
8
10
0.0 0.2 0.4 0.6 0.8 1.0
-10
-8
-6
-4
-2
0
2
4
6
8
10
Information Time
Z V
alu
eStopping boundary for 2-sided alpha= 0.05 power= 0.9 Condition Power= 0.9
Stop, Conclude H1
Stop, Conclude H1
Stop, Conclude Ho
Early Stopping Using Predictive Power
Similar to the conditional power approach but integrate over the possible values of the conditioning parameter
Predictive Power = ( ) ( | )p nC f X d
Early Stopping Using Predictive Probability (PP)
At interim, compute the predictive probability, i.e., a positive result at the end of trial if the current trend continues given the current data. If PP is very high, stop the trial now
and declare treatment is working, If PP is very low, stop the trial now and
declare treatment is not working, Otherwise, continue.
Frequentist Approach
Bayesian Approach
==
Example 2: Predictive Probability in Breast Cancer
Buzdar AU et al, JCO 23, 3676-3685, 2005
Disease: HER-2 (+) Breast Cancer
Agent: A: 4 cycles of paclitaxel + 4 cycles of fluorouracil, epirubicin, and
cyclophosphamide B: same as A + weekly trastuzumab for 24
weeks
Statistical Design: phase II
Sample Size: 164 (maximum)
Primary Endpoint: pathological complete response (pCR)
Method: Standard two-stage design with Predictive Probability Interim Monitoring by
DSMB Results: After 34 pts completed therapy, DSMB stopped the trial.
With 34 Patients
Arm CR rates
A 4 / 16 (25%)
B 12/ 18 (67%)
With 42 Patients
Arm CR rates
A 5 / 19 (26%)
B 15/ 23 (65%) P=0.016
Review of Randomized Phase II TrialsLee and Feng, JCO 23, 4450-4457, 2005
56%
3%
Comparing Fixed vs. Random P
Assume we observed x/n successes.What is the probability of observing X/N successes in the future? Assuming the estimated P is
fixed random
What are the differences?Which one do you prefer?
0 1 2 3 4 5 6 7 8 9 10
0.0
0.0
50
.10
0.1
50
.20
0.2
50
.30
Observed x/n = 5/10, Fixed P
X in future N=10
Pro
ba
bili
ty
0 1 2 3 4 5 6 7 8 9 10
0.0
0.0
50
.10
0.1
50
.20
0.2
50
.30
Observed x/n = 5/10, Prior=Beta(.5, .5)
X in future N=10
Pro
ba
bili
ty
0 1 2 3 4 5 6 7 8 9 10
0.0
0.0
50
.10
0.1
50
.20
0.2
50
.30
Observed x/n = 5/10, Prior=Beta(.5, .5), Fixed P (blue), Random P (red)
X in future N=10
Pro
ba
bili
ty
0 1 2 3 4 5 6 7 8 9 10
0.0
0.0
50
.10
0.1
50
.20
0.2
50
.30
Observed x/n = 8/10, Fixed P
X in future N=10
Pro
ba
bili
ty
0 1 2 3 4 5 6 7 8 9 10
0.0
0.0
50
.10
0.1
50
.20
0.2
50
.30
Observed x/n = 8/10, Prior=Beta(.5, .5)
X in future N=10
Pro
ba
bili
ty
0 1 2 3 4 5 6 7 8 9 10
0.0
0.0
50
.10
0.1
50
.20
0.2
50
.30
Observed x/n = 8/10, Prior=Beta(.5, .5), Fixed P (blue), Random P (red)
X in future N=10
Pro
ba
bili
ty
Available Software
East (www.cytel.com)ADDPLAN (www.addplan.org)PEST http://www.rdg.ac.uk/mps/mps_home/software/pest4/pest4.htm
PASS http://www.ncss.com/passsequence.html
S+SeqTrial
http://www.statsci.com/products/seqtrial/default.asp
GLUMIP for internal pilot studyhttp://www.soph.uab.edu/coffey
M.D. Anderson (biostatistics.mdanderson.org)
PEST 4 (Planning and Evaluation of Sequential Trials)
This package includes facilities for:A range of designs to detect: superior efficacytreatment equivalence or non-inferiorityfutilitysafety concerns Flexible interim monitoring information-based to guarantee power Test statistics calculated within PEST 4 and displayed with the design boundariesAdjustment for prognostic factors using stratification or covariate adjustmentFinal analysis providing valid p-values, estimates and confidence intervalsSimulation for illustration and exploration of accuracy
Derry FA et al. Efficacy and safety of oral sildenafil (Viagra) in men with erectile dysfunction caused by spinal cord injury. Neurology. 1998 Dec;51(6):1629-33.
Figure 1. Single triangular sequential design shows original and adjusted continuation regions. Z is a measure of the observed advantage of sildenafil versus placebo and its expected value is the log odds ratio ([THETA]) multiplied by the information (V) collected up to that point. V is the information collected up to the interim analysis point about [THETA], as contained in the current value of Z. The expected value of the variance of Z is V. The arrow marks the critical point along the bottom boundary. To the left of this point sildenafil is significantly worse than placebo; to the right of this point there is evidence of no difference.
GROUP SEQUENTIAL TESTS in PASS
Group Sequential Tests of MeansThis module calculates sample size and power for group sequential designs used to compare two treatment means. The program allows you to vary the number and times of interim tests, the type of alpha spending function, and the test boundaries. It also gives you complete flexibility in solving for power, significance level, sample size, or effect size. The results are displayed in both numeric reports and informative graphics.
Group Sequential Tests of ProportionsThis module calculates sample size and power for group sequential designs used to compare two proportions. The program allows you to vary the number and times of interim tests, the type of alpha spending function, and the test boundaries. It also gives you complete flexibility in solving for power, significance level, sample size, or effect size. The results are displayed in both numeric reports and informative graphics.
Group Sequential Tests of Survival CurvesThis module calculates sample size and power for group sequential designs used to compare two survival curves. The program allows you to vary the number and times of interim tests, the type of alpha spending function, and the test boundaries. It also gives you complete flexibility in solving for power, significance level, sample size, or effect size. The results are displayed in both numeric reports and informative graphics.
East East is software for the design, simulation and monitoring of clinical trials using group sequential and adaptive methodologies. East allows investigators to easily design superiority, futility only, and non-inferiority trials, for all endpoints, with complete confidence that the type-1 error and power of the study will be protected. East's simulation capability aids clinicians and statisticians in understanding the trade-offs between trial designs so that designs can be compared and investigators can choose the best one. And East's interim monitoring module will perform all the necessary calculations for exact inference at an interim analysis. By designing, simulating and monitoring clinical trials using East, investigators can take advantage of flexible approaches that will allow them to identify futile trials and terminate them, fast-track effective therapies, and salvage underpowered studies by performing sample-size reassessment, without jeopardizing the statistical integrity of the trial.
S+SeqTrial
S+SEQTRIAL offers a complete computing environment for applying group sequential methods, including: A fully object-oriented language with
specialized objects (such as design objects, boundary objects, and hypothesis objects)
and methods (such as operating characteristics and power curve plots);
Easy comparative plots of boundaries, power curves, average sample number (ASN) curves, and stopping probabilities;
User-selected scales for boundaries: sample mean, z-statistic, fixed sample p-value, partial sum, error spending, Bayesian posterior mean, and conditional and predictive probabilities;
Stopping Rule Computation
The unified family of group sequential designs, which includes all common group sequential designs: Pocock (1977), O’Brien & Fleming (1979), Whitehead triangular and double triangular (Whitehead & Stratton, 1983), Wang & Tsiatis (1987), Emerson & Fleming (1989), and Pampallona & Tsiatis (1994);A new generalized family of designs. S+SEQTRIAL includes a unified parameterization for designs, which facilitates design selection, and includes designs based on stochastic curtailment, conditional power and predictive approaches;Applications including normal, Binomial, Poisson, survival, one-sample and two-sample; One-sided, two-sided, and equivalence hypothesis tests, as well as new hybrid tests;Specification of the error spending functions of Lan & DeMets (1989) and Pampallona, Tsiatis, & Kim (1993);Arbitrary boundaries allowed on different scales: sample mean, z-statistic, fixed sample p-value, partial sum, error spending, Bayesian posterior mean, and conditional and predictive probabilities; Exact boundaries computed using numerical integration.
Meta-Analysis
Cochrane data: randomized trials before 1980 of cortico-steroid therapy in premature labor and its effect on neonatal death
• What is the overall conclusion of the effect of corticosteroid in reducing neonatal death by combining the data from all trials?
No. Trial ev.trt n.Trt ev.ctrl n.ctrl
1 Auckland 36 532 60 538
2 Block 1 69 5 61
3 Doran 4 81 11 63
4 Gamsu 14 131 20 137
5 Morrison 3 67 7 59
6 Papageorgiou
1 71 7 75
7 Tauesch 8 56 10 71
Models: Assume Yi is the measure of treatment effect (e.g., log(OR) for Trial i)
Fixed-effect model: A common, fixed q2( , )i iY N s
2
2
| , ( , )
| , ( , )
i i i i i
i
Y s N s
N
Random-effect model: Random qi from each trial
2 2( , )i iY N s 2 2 2 2| Y, , ( (1 ) , (1 )) where /( )
is a shrinkage factor for study
i i i i i i i i i
th
N B B Y s B B s s
i
Estimation: Fixed Effect Model
Frequentist’s method When is assumed known
Bayesian method
2is
2
1 1
1
ˆ / with 1 /
ˆ ( ,1/ )
k k
MLE i i i i ii i
k
MLE ii
WY W W s
N W
20
2 2 20 0 01 1 1
201 1
Assume the prior for is (0, )
| , , ~ ( /( + ), 1 /( + ))
/( + ) is the posterior mean
k k k
i i i ii i i
k k
B i i ii i
N
Y s N WY W W
WY W
Fixed Effects: Mantel-Haenszel Method
windows(record=T)library(rmeta)data(cochrane)steroid.MH <- meta.MH(n.trt, n.ctrl, ev.trt, ev.ctrl,names=name, data=cochrane)summary(steroid.MH)Fixed effects ( Mantel-Haenszel ) meta-analysisCall: meta.MH(ntrt = n.trt, nctrl = n.ctrl, ptrt = ev.trt, pctrl = ev.ctrl, names = name, data = cochrane)------------------------------------ OR (lower 95% upper)Auckland 0.58 0.38 0.89Block 0.16 0.02 1.45Doran 0.25 0.07 0.81Gamsu 0.70 0.34 1.45Morrison 0.35 0.09 1.41Papageorgiou 0.14 0.02 1.16Tauesch 1.02 0.37 2.77------------------------------------Mantel-Haenszel OR =0.53 95% CI ( 0.39,0.73 )Test for heterogeneity: X^2( 6 ) = 6.9 ( p-value 0.3303 )
Fixed Effects: Mantel-Haenszel Method
StudyAucklandBlockDoranGamsuMorrisonPapageorgiouTauesch
Summary
Deaths/N(steroid)36 / 532
1 / 694 / 81
14 / 1313 / 671 / 718 / 56
Deaths/N(placebo)
60 / 5385 / 61
11 / 6320 / 137
7 / 597 / 75
10 / 71
OR0.580.160.250.700.350.141.02
0.53
0.1 0.5 1.0 1.5 2.0
Forest Plot
Funnel Plot
-2.0 -1.5 -1.0 -0.5 0.0
log(OR)
Siz
e: 1
/se
-4 -3 -2 -1 0 1
log(OR)
Siz
e: 1
/se
Estimation: Random Effect Model
Frequentist’s method When is assumed known
When is unknown: Restricted MLE (REML)
22 2
1 1ˆ ( ) ( ) / ( ) with ( ) 1 /( )
k k
MLE i i i i ii iW Y W W s
2 2 2 2 2 2
1 1
2 2
1 1
1
ˆˆFind REML of as the solution to ( )( ( ) ) / ( ) 1
ˆ ˆ ˆ ˆ = ( ) / ( ) with ( ) 1/( )
ˆ ~ ( , 1 / ( ))
k k
R i i R i ii i
k k
R i R i i i R i Ri i
k
R ii
kW Y s W
k
W Y W W s
N W
2
2 2 2
2 2
ˆ ˆ ˆFind empirical Bayes estimator of (1 ) where /( )
ˆ ˆ~ ( , (1 )) It ignores the uncertainly of the hyperparameter { , }
R R R Ri i R i i i i i R
R Ri i i i
B B Y B s s
N s B
Estimation: Random Effect Model (cont.)
Bayesian method Let
The posterior distribution is only analytically tractable for conjugate distributions. Otherwise, requires MCMC using BUGS or WinBUGS.
2 2
21 k
2 2 2 2
2B
~ (0, ) and ~ gamma( , ),
Then, the joint posterior distribution for ={ , , ... , , } is
( | , ) ( | , ) ( | , ) ( ) ( )
The posterior distribution of is
ˆ ( | , )
i i i ii
N a c d
V
p V Y s p y s p p p
E Y s
2i
2
, { ( ) }iP V d d d
Random Effect: DerSimonian-Laird Method
steroid.DSL <- meta.DSL(n.trt, n.ctrl, ev.trt,ev.ctrl,names=name, data=cochrane)summary(steroid.DSL)Random effects ( DerSimonian-Laird ) meta-analysisCall: meta.DSL(ntrt = n.trt, nctrl = n.ctrl, ptrt = ev.trt, pctrl = ev.ctrl, names = name, data = cochrane)------------------------------------ OR (lower 95% upper)Auckland 0.58 0.38 0.89Block 0.16 0.02 1.45Doran 0.25 0.07 0.81Gamsu 0.70 0.34 1.45Morrison 0.35 0.09 1.41Papageorgiou 0.14 0.02 1.16Tauesch 1.02 0.37 2.77------------------------------------SummaryOR= 0.53 95% CI ( 0.37,0.78 )Test for heterogeneity: X^2( 6 ) = 6.86 ( p-value 0.334 )Estimated random effects variance: 0.03
Random Effect: DerSimonian-Laird Method
StudyAucklandBlockDoranGamsuMorrisonPapageorgiouTauesch
Summary
Deaths/N(steroid)36 / 532
1 / 694 / 81
14 / 1313 / 671 / 718 / 56
Deaths/N(placebo)
60 / 5385 / 61
11 / 6320 / 137
7 / 597 / 75
10 / 71
OR0.580.160.250.700.350.141.02
0.53
0.1 0.5 1.0 1.5 2.0
Forest Plot
Random Effect: Bayesian Method
Data: list(lor=c(-5.47791E-01, -1.80359E+00, -1.40416E+00, -3.56675E-01, -1.05494E+00, -1.97490E+00, 1.65293E-02), sd=c(2.20347E-01, 1.11021E+00, 6.10841E-01, 3.72186E-01, 7.14875E-01, 1.08252E+00, 5.12081E-01), N=7)
Model: model{ for (j in 1 : N) { sinv[j] <- 1/(sd[j]*sd[j])
lor[j] ~ dnorm(beta[j], sinv[j]) beta[j] ~ dnorm(mu, tau) } mu ~ dnorm(0, 0.000001)
tau ~ dgamma(0.001,0.001) sigma <- 1/tau }
Initial Values: list(mu=0,tau=1)
>beta mean sd MC_error val2.5pc median val97.5pc start samplebeta[1] -0.5947 0.1851 0.00852 -0.943 -0.5929 -0.21830 5001 5000beta[2] -0.7186 0.3894 0.01536 -1.721 -0.6649 -0.12390 5001 5000beta[3] -0.7487 0.3418 0.01643 -1.632 -0.7003 -0.22140 5001 5000beta[4] -0.5680 0.2457 0.01045 -1.023 -0.5759 -0.02555 5001 5000beta[5] -0.6854 0.3278 0.01350 -1.468 -0.6507 -0.13080 5001 5000beta[6] -0.7396 0.3940 0.01698 -1.797 -0.6777 -0.17330 5001 5000beta[7] -0.5231 0.3002 0.01257 -1.018 -0.5512 0.20430 5001 5000> mu mean sd MC_error val2.5pc median val97.5pc start samplemu -0.6526 0.2448 0.01234 -1.174 -0.6392 -0.2289 5001 5000
> sigma mean sd MC_error val2.5pc median val97.5pc start samplesigma 0.1226 0.4387 0.01376 0.0007608 0.01902 0.8716 5001 5000
> exp(mu) mean sd MC_error val2.5pc median val97.5pcmu 0.5206902 1.277366 1.012416 0.3091280 0.5277144 0.795408
Random Effect: Bayesian Method
Summary of Meta-Analysis Result