Post on 24-Aug-2020
transcript
Alex Dmitrienko Eli Lilly and Company
Brian Wiens Myogen, Inc
Ajit Tamhane Northwestern University
Tree-structured gatekeeping procedures in clinical trials with multiple objectives
BASS XIII conference
[Slide 2]
Clinical trials with multiple objectivesType I error rate control
Gatekeeping proceduresSerial and parallel testing approaches
Tree-structured proceduresClinical trials with hierarchically ordered objectives
Generalization of gatekeeping methods
Extensions
Outline
[Slide 3]
Primary and secondary analysesProduct labels typically focus on primary findings
Secondary analyses (secondary endpoints or subgroup analyses) provide much useful information to prescribing physicians, patients, hospital administrators, etc
Multiple objectives
[Slide 4]
FDA guidanceClinical Studies Section of Labeling for Prescription Drug and Biological Products - Content and Format
“The clinical studies section should present those endpoints that establish the effectiveness of the drug... When it would be informative, the clinical studies section can also discuss other endpoints shown to be affected by the drug...”
Regulatory position
http://www.fda.gov/cber/gdlns/clinlab.htm
[Slide 5]
Selection of secondary analysesWhat secondary findings should be included in the product label?
Can pharmaceutical companies be accused of cherry-picking?
Multiplicity issuesShould the overall false-positive rate be controlled for primary and secondary analyses?
Secondary analyses
[Slide 6]
No error rate controlSmall number of secondary analyses
Generalized Type I error rateControl the probability of at least two or at least three incorrect conclusions
Type I error rateControl of familywise error rate in the strong sense (probability of at least one incorrect conclusion)
Gatekeeping strategies
Control of false-positive rate
[Slide 7]
Gatekeeping proceduresMultiple testing procedures that account for the hierarchical structure of multiple analyses
Serial gatekeeping methodsWestfall and Krishen (2001)
Parallel gatekeeping methodsDmitrienko, Offen and Westfall (2003)
General overviewDmitrienko et al (2005, Chapter 2)
Gatekeeping strategies
[Slide 8]
DesignTwo doses of adalimumab (Humira™) (L and H) versus placebo (P)
Keystone et al, Arthritis and Rheumatism, 2004
Three endpointsSigns and symptoms (ACR definition)
Structural progression (Sharp score)
Quality of life/disability (HAQ)
Serial gatekeeping strategy
Clinical trial exampleRheumatoid arthritis trial
[Slide 9]
Rheumatoid arthritis trialSerial gatekeeping strategy
Signs and symptoms
Global test
L vs P,H vs P
Structural progression
Global test
L vs P,H vs P
Quality of life
Global test
L vs P,H vs P
[Slide 10]
Rheumatoid arthritis trialSerial gatekeeping strategy
Signs and symptoms
p<0.001 p<0.001,p<0.001
Structural progression
p<0.001 p<0.001,p<0.001
Quality of life
p<0.01 p<0.01,p<0.01
Overall Type I error rate is 0.05
[Slide 11]
Improved testing strategy
Signs and symptoms
L vs P,H vs P
Structural progression
L vs P,H vs P
Quality of life
L vs P,H vs P
Improved serial gatekeping strategyEliminate global tests
Other ways to improveIdentify bottlenecks
What if the L-P test is not significant for structural progression?
[Slide 12]
Parallel gatekeeping strategy
Signs and symptomsL vs P
orH vs P
Structural progressionL vs P
orH vs P
Quality of lifeL vs P
orH vs P
Parallel gatekeeping strategyOnly one dose-placebo test needs to be significant to consider the next family of tests
Higher likelihood of making secondary claims
[Slide 13]
Parallel gatekeeping strategy
Signs and symptomsL vs P
orH vs P
Structural progressionL vs P
orH vs P
Quality of lifeL vs P
orH vs P
Parallel gatekeeping strategyNon-significant test
Significant test
Additional claimsL-P and H-P tests are significant for quality of life
[Slide 14]
Parallel gatekeeping strategyLogical restrictions
Signs and symptomsL vs P
orH vs P
Structural progressionL vs P
orH vs P
Quality of lifeL vs P
orH vs P
Logical restrictionsIs the L-P comparison meaningful for quality of life?
Secondary endpoints are restricted to doses at which the primary endpoint was significant (or all previous endpoints were significant)
[Slide 15]
Tree gatekeeping strategy
L vs P H vs P
L vs P H vs P
L vs P H vs P
Signs and symptoms
Structural progression
Quality of life
[Slide 16]
Tree gatekeeping strategyPruning irrelevant tests
L vs P H vs P
L vs P H vs P
H vs P
Signs and symptoms
Structural progression
Quality of life
Non-significant testSignificant test
[Slide 17]
Families of null hypothesesn null hypotheses grouped into m families, F
1,…,F
m, i.e., F
i={H
i1,…,H
i,ni}
Serial gatekeeping setH
ij, i>1, will be tested if and only if all
hypotheses in its serial gatekeeping set, Sij, are
rejected
Parallel gatekeeping setH
ij, i>1, will be tested if and only if one or more
one hypotheses in its parallel gatekeeping set, P
ij, are rejected
Tree gatekeeping proceduresNotation
[Slide 18]
Serial gatekeeping set
L vs P H vs P
L vs P H vs P
L vs P M vs P
Null hypothesis Hij
Serial gatekeeping set Sij
All hypotheses must be rejected in S
ij to test H
ij
[Slide 19]
Parallel gatekeeping set
L vs P H vs P
L vs P H vs P
L vs P S2M vs P
S2H vs P
Null hypothesis Hij
Parallel gatekeeping set Pij
At least one hypothesis must be rejected in P
ij to test H
[Slide 20]
Closed testing principleMarcus, Peritz and Gabriel (1976)
Simple tree gatekeeping frameworkTree gatekeeping procedures based on Bonferroni test
Tree gatekeeping procedures
[Slide 21]
Closed testing procedureNull hypotheses A, B and C
A∩B∩C
A∩B A∩C B∩C
A B C
Decision rule for A: All intersection hypotheses containing A must be rejected
[Slide 22]
Choice of individual tests
A∩B∩C
A∩B A∩C B∩C
A B C
Tests for individual intersection hypotheses must account for hierarchical structure of the problem
[Slide 23]
Choice of individual testsAccount for serial gatekeeping sets
A∩B A∩C B∩C
A B C
A∩B∩C
Are B or C in the serial gatekeeping set SA?
If yes, the test for A∩B∩C should not depend on A
Example: Reject A∩B∩C if pB≤α/2 or p
C≤α/2
This ensures that A cannot be rejected if one or more null hypotheses in S
A are accepted
[Slide 24]
Choice of individual testsAccount for parallel gatekeeping sets
A∩B A∩C B∩C
A B C
A∩B∩C
Are both B and C in the parallel gatekeeping set PA?
If yes, the test for A∩B∩C should not depend on A
Example: Reject A∩B∩C if pB≤α/2 or p
C≤α/2
This ensures that A cannot be rejected if all null hypotheses in P
A are accepted
[Slide 25]
Bonferroni tree gatekeeping procedure Accounts for hierarchical structure of the multiple testing problem
Control of familywise error rateControls the familywise error rate in the strong sense by the closed testing principle
ReferenceDmitrienko, Wiens, Tamhane, Wang (2006). To appear in Statistics in Medicine.
Bonferroni-based procedure
[Slide 26]
Rheumatoid arthritis trialHypothetical example
H11 H12
H21 H22
H31 H32
Signs and symptoms
Structural progression
Quality of life
L vs P H vs P
[Slide 27]
Null hypothesis Serial setH11 NAH12 NAH21 H11H22 H12H31 H11, H21H32 H12, H22
Rheumatoid arthritis trialSerial gatekeeping sets
Parallel gatekeeping sets are empty
[Slide 28]
Rheumatoid arthritis trialLogical restrictions
0.0180.036
0.0090.018
0.0380.076
0.0130.026
0.0430.076
0.0210.042
Signs and symptoms
Structural progression
Quality of life
L vs P H vs P
Raw p-values Multiplicity-adjusted p-values
[Slide 29]
Rheumatoid arthritis trialNo logical restrictions
0.0180.036
0.0090.018
0.0380.076
0.0130.036
0.0430.076
0.0210.076
Signs and symptoms
Structural progression
Quality of life
L vs P H vs P
Raw p-values Multiplicity-adjusted p-values
[Slide 30]
Null hypothesis Parallel setH11 NAH12 NAH21 H11, H12H22 H11, H12H31 H21, H22H32 H21, H22
Rheumatoid arthritis trialParallel gatekeeping sets
Serial gatekeeping sets are empty
[Slide 31]
Rheumatoid arthritis trialEqually important secondary endpoints
H11 H12
H21 H22
H31 H32
Signs and symptoms
Structural progression
Quality of life
L vs P H vs P
[Slide 32]
Null hypothesis Parallel setH11 NAH12 NAH21 H11H22 H12H31 H11H32 H12
Rheumatoid arthritis trialParallel gatekeeping sets
Serial gatekeeping sets are empty
[Slide 33]
Rheumatoid arthritis trialEqually important secondary endpoints
0.0180.036
0.0090.018
0.0380.076
0.0130.036
0.0430.076
0.0210.076
Signs and symptoms
Structural progression
Quality of life
L vs P H vs P
Raw p-values Multiplicity-adjusted p-values
[Slide 34]
Stepwise approachTests are carried out in a sequential manner (Holm or Hochberg tests)
Appealing in clinical trials
Stepwise tree gatekeeping procedureGeneral case of m endpoints (families)
Families F1,…,F
m-1 are tested using Bonferroni test
Holm test is carried out in Family Fm
LimitationIt is not known how to compute adjusted p-values
Stepwise procedure
[Slide 35]
Stepwise procedureStep 1: Signs and symptoms
α’=α=0.05
0.018 0.009
L vs P H vs P
α’/2=0.025 α’/2=0.025
Signs and symptoms
[Slide 36]
Stepwise procedureStep 2: Structural progression
α’=α=0.05
0.018 0.009
0.038 0.013
L vs P H vs P
α’/2=0.025 α’/2=0.025
α’/2=0.025 α’/2=0.025
α’=α=0.05
Sign and symptoms
Structural progression
[Slide 37]
Stepwise procedureStep 3: Quality of life
α’=α=0.05
0.018 0.009
0.038 0.013
0.021
L vs P H vs P
α’/2=0.025 α’/2=0.025
α’/2=0.025 α’/2=0.025
α’=0.025
α’=α=0.05
α’=α/2=0.025
Sign and symptoms
Structural progression
Quality of life
[Slide 38]
Focused on basic frameworkTree gatekeeping procedures based on Bonferroni test
Account for correlationCorrelation among multiple endpoints
Correlation among multiple dose-control comparisons
Account for correlation via resampling-based methods
Extensions
[Slide 39]
Tree gatekeeping proceduresEfficient way to account for hierarchically ordered multiple objectives in clinical trials
Extend serial and parallel gatekeeping methods
Software implementationSAS macro is available at biopharmnet.com/code
Closed testing principleControl the familywise error rate in the strong sense
Summary
[Slide 40]
Dmitrienko, Offen, Westfall. Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Statistics in Medicine. 2003; 22:2387-2400.Dmitrienko, Molenberghs, Chuang-Stein, Offen. Analysis of Clinical Trials Using SAS: A Practical Guide. SAS Press: Cary, NC, 2005.Dmitrienko, Wiens, Tamhane, Wang. Tree-structured gatekeeping tests in clinical trials with hierarchically ordered multiple objectives. Statistics in Medicine. 2006. To appear.Marcus, Peritz, Gabriel. On closed testing procedures with special reference to ordered analysis of variance. Biometrika. 1976; 63:655-660.Westfall, Krishen. Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference. 2001; 99:25-41.
References