Overview of ICH E9: Statistical Principles for Clinical Trials

Mario ChenFamily Health International

Biostatistics WorkshopIndia, March 2007

• Focus on statistical principles• Gives direction to researchers in design, conduct,

analysis, and evaluation of trials• Does not address use of specific statistical tests• Emphasis on later phase, confirmatory trials

• Target audience: individuals from a broad range of scientific disciplines• Statisticians, clinicians, pharmacologists,

epidemiologists

Scope and Direction

Scope and Direction• Trial Statistician:

• Responsible for all the statistical work associated with the trial

• Ensures statistical principles are appropriately applied

• Has the proper training and experience to implement the principles in this guidance

• Minimizing bias• Systematic tendency of any factors associated

with design, conduct, analysis and interpretation to lead to an estimate of treatment effect different from the true value.

• Maximizing precision• Obtaining small standard errors and narrow

confidence intervals.• Evaluating robustness

• Sensitivity of overall conclusions to various limitations of the data, assumptions, analysis procedures used.

Scope and Direction

Scope and Direction• Controlling the type I error

• Ensuring that the chance of declaring a treatment efficacious when it in fact does not work is low (e.g., α ≤ 0.05)

• “Multiplicity” refers to having more than one opportunity to detect a difference between drugs (e.g., interim analyses, multiple endpoints of interest)

Protocol and Analysis Plan• Principal features of statistical analysis should

be clearly specified in the protocol• Protocol (and amendments) should be approved by

trained statistician• A detailed Analysis Plan should be written

before data analysis begins

Types of Trials

• Confirmatory Trial• Exploratory Trial

Confirmatory Trial• An adequately controlled trial in which the

hypotheses are stated in advance and evaluated.

• Key hypothesis of interest • follows directly from the trial’s primary objective • is always pre-defined• is the hypothesis that is subsequently tested

when the trial is complete• Adherence to protocols and SOPs is

particularly important

Exploratory Trial• Clear and precise objectives, however, tests of

hypothesis may be data dependent

• Such trials cannot be the basis of the formal proof of efficacy

Population• Earlier phases may focus on a very narrow

subgroup• Confirmatory trials should more closely mirror

target population of the therapy under study• Issues of Generalizability

• Clear Inclusion/Exclusion criteria

Outcome Variables• Primary variable(s)

• directly related to the primary objective • preferable to specify only one• reliable and validated variable used in earlier

studies or in published literature• used when estimating the sample size

• Secondary variables• either supportive measurements related to the

primary objective, or • measurements of effects related to the

secondary objectives

Avoiding Bias by Design: Blinding

• Blinding limits the occurrence of conscious and unconscious bias arising from the influence one’s knowledge of treatment may have on• recruitment and allocation of subjects • their subsequent care• attitudes of subjects to the treatments • assessment of endpoints • handling of withdrawals • exclusion of data from analysis• choice of analysis methods

Avoiding Bias by Design: Randomization• introduces a deliberate element of chance

into the assignment of treatments• provides a sound statistical basis for the

comparison of treatment groups• tends to produce treatment groups with

distributions of prognostic factors (measured and unmeasured) are similar

Design Configuration

• Parallel group:• random assignment to A vs B

A

Population

B

Randomization

Design Configuration

• Crossover: • random assignment to AB or BA• subject serves as own control

A

Population

B A

Randomization

B

Wash out period

Design Configuration• Factorial Designs:

• interest in comparing A vs B and C vs D• randomly assign to A+C, A+D, B+C or B+D

A

Population

B

Randomization

C

D

C

D

[Other Study Designs]• Cohort Studies• Case-Control Studies• Descriptive Studies (e.g., surveys)

Type of Comparison• Trials to show “superiority”

• new treatment vs placebo, or new treatment vs active control

• test (and hopefully reject) the null hypothesis that there is no difference in outcomes between groups

• vs. the alternative hypothesis that there is a difference between the groups

• one-sided or two-sided

Type of Comparison• Trials to show “equivalence”

• new treatment vs. active control• test (and hopefully reject) the null hypothesis that

the new treatment performs differently than the active control by at least some small amount,

• vs. the alternative hypothesis that the difference between the groups is no greater than this small amount

• an amount that is sufficiently small that the treatments are considered equivalent for all practical purposes if the difference between the treatments is smaller than this amount (equivalence margin)

Type of Comparison• Trials to show “noninferiority”

• new treatment vs. active control• test (and hopefully reject) the null hypothesis

that the active control performs better than the new treatment by at least some small amount

• vs. the alternative hypothesis that the new treatment does not perform worse than the active control by more than this small amount

Type of comparison• Trials to show “equivalence”

• e.g., test null hypothesis that % cured with active control is ≥ 5% more than the % cured with thenew treatment and vice versa

• Trials to show “noninferiority”• e.g., test null hypothesis that % cured with active

control is ≥ 5% than the % cured with the new treatment

Sample Size• Determine based on:

• a primary endpoint • the null hypothesis • the test statistic (e.g., t-test, chi-square test,

logrank test)• the treatment difference to be detected (the

“alternative hypothesis”)• significant level (type I error) • desired power (type II error)• variability assumptions• the plan for handling treatment withdrawals and

protocol violations

Data Monitoring• Oversight of Trial Quality• Monitoring of Treatment Effects

Oversight of Trial Quality

• Checks perform in a blinded manner:• whether the protocol is being followed• the acceptability of data being accrued • the success of planned accrual targets • the appropriateness of the design assumptions• success in keeping patients in the trials

• Has no impact on type I error

Monitoring Treatment Effects: Interim Analysis• Usually for serious outcomes• Requires unblinded access to treatment group

summary data• Should only be done if included in the protocol• Goal, stop the trial early if:

• superiority of the new treatment is clear • future demonstration of a treatment effect is

unlikely • unacceptable adverse effects are apparent

Monitoring Treatment Effects: Interim Analysis• May requiere a IDMC, which should approve

interim plans

Study Team

IDMC Sponsor

IRB

PIStatistical Center

Monitoring Treatment Effects: Interim Analysis• Repeated testing of outcome data increases the

chance of a type I errorTest for difference in proportions failing in two groups without adjustment for multiple testing

Decision Rule: Reject null if |Z| ≥ 1.96Overall Type I

Error Rate

Single test at end of study -----> 0.05

Two tests, equally spaced -----> 0.08

Five tests, equally spaced -----> 0.14(Friedman, Furberg and DeMets, 1996)

Data Analysis• Include main features of analysis in protocol• For confirmatory trial, include statistical

methods to be used for the primary variable(s)

• For exploratory trials, include general principles and directions

• Additional ‘statistical analysis plan’• detailed procedures for primary and secondary

variables• do blind review of data, record date of breaking

blind

Analysis Sets

• Disposition of participants enrolled, summary of protocol violations

• Degree of compliance and missing data lead to different Analysis Sets:• Full Analysis Set• Per Protocol Set

• Rationale:• Minimize bias (Analysis Sets defined a priori)• Demonstrate lack of sensitivity

Full Analysis Set• ‘Full analysis set’ = the analysis set which is as

complete as possible and as close as possible to the intention-to-treat ideal of including all randomized subjects, it may exclude, for example:• Participants who failed to meet a major entry criteria• Participants who lack any data post randomization

Full Analysis Set• If specified in the plan, subjects who fail to

meet an entry criterion may be excluded without the possibility of introducing bias under the following circumstances:• the entry criterion was measured prior to

randomization• the detection of the relevant eligibility violations

can be made completely objectively • all subjects receive equal scrutiny for eligibility

violations • all detected violations of the particular entry

criterion are excluded

Per Protocol Set• ‘Per protocol set’ = subset of the

participants in the full analysis set who are more compliant with the protocol• complete a certain pre-specified minimal

exposure to the treatment regimen • have some minimum number of

measurements of the primary variable(s)• have no major protocol violations

• May give overly optimistic results in superiority trials

• May be the more conservative analysis set for equivalence or non-inferiority trials

Estimation and Confidence Intervals• Not just p-values, include confidence

intervals for estimated treatment effects• Prespecify any covariates to be controlled

for in primary or secondary analysis• To improve precision• To adjust for potential imbalances• To account for stratified designs• Never adjust for post-randomization variables

• Prespecify interactions and subgroups of interest if treatment effect is likely to vary by baseline factors (e.g., gender)

Evaluation of Safety• Choice of variables: Laboratory tests, vital

signs, adverse events• Safety Analysis Set: Usually those who

received at least one dose of the investigational drug

• Statistical Analysis

Reporting• Document deviations from analysis plan, when

and why they occurred• Account for all subjects who entered the study• Describe all reasons for exclusion from

analysis dataset and all protocol violations• Summarize measurements of all important

variables• Consider the effect of loss of subjects,

violations and missing data on analysis results• Describe participants lost, withdrawn, etc.