Why so Different Outcomes?

An overview of the different

aspects of clinical trialsClaudio Ceconi

ALL ABOUT CLINICAL TRIALS

Rome, 29th & 30th May 2015

An overview of the different

aspects of clinical trials

• Why do we need them?

• What is a clinical trial?

• Major aspects of clinical trials

• Study protocol

• Examples of ‘famous’ trials

Clinical Experience

Making the same mistakes with

increasing confidence over an

impressive number of years.

O’ Donnel: A skeptic medical dictionary

London: BMJ Books, 1997

Evidence-based medicine (EBM)

Applies the scientific method to medical practice.

According to the Centre for Evidence-Based Medicine:

"Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients."

Does accutane cause suicide?

Charles Bishop, 17, who crashed a small airplane into a Tampa building on 1/5/02 was prescribed accutane.

Accutane has been on the market since 1982 and has been used by hundreds of thousands of people. A half-million prescriptions were written in the United States in 2008.

While the FDA has about 140 reports of suicide after people took Accutane, the rate of suicide remains extremely low, making it difficult to pin down which cases might have occurred anyway and whether cases were caused by the medicine.

How to determine whether accutane cause suicide or not?

Data:

Signal + Noise

Data Analysis:

Extract the signal and filter out the noise

Statistics:

Find a needle in a haystack

Statistics and Medical Research

Premise: Advance in medical practice is based on researchResearch validity is based on scientific credibilityScientific Method:

QuestionDesignStudyDataAnalysisConclusion New Question

Good design + Good analysis Valid Inference

Statistics is needed in every step

Scientific Method

ObservationEvidence

Observation, Evidence

Hypothesis Generating

Experiments,Clinical Trials

Hypothesis Testing

Secondary Endpoints Subset Analyses

External Evidence Other Trials Mechanistic Studies

Clinical Investigations

INTERVENTIONAL(EXPERIMENTAL)

PLANNED OBSERVATIONAL STUDIES

NON RCTANALYTICDESCRIPTIVE *RCT

*RCT= Randomised controlled trial

14

Observational Study To test the association of a risk factor with an outcome Cohort study: start with a cohort and wait for

events to occurs Case-control study: compare individuals with a

disease to individuals who do not have the disease Cross-sectional study: prevalence of disease and

risk factor determined at same time

Descriptive aims: Prevalence of disease Time trends

Comparison of intervention in an observational trial Prone to bias and confounding

Bias and Confounding – the Enemies of Sound Research

Bias – systematic error that results in a conclusion which is not trueSelection bias: patients are not comparableInformation bias: the information is not comparable

Confounding - factors that affect the effect of the exposure on the outcomepositive confounding: the effect seems stronger negative confounding: the effect seems weaker

What is a clinical trial?

A clinical trial is a prospective study evaluating the effect and value of intervention(s) in human beings under pre-specified conditions.

A controlled clinical trial is a prospective study comparing the effect and value of intervention(s) against a control in human beings.

The clinical trial is the most definitive tool for evaluation of the applicability of clinical research

Greenhalgh, T. BMJ 1997;315:305-308

Why randomize?: Selection Bias

The major sources of bias in a Randomized Clinical Trial

19

Randomization

• To prevent selection bias

• To minimize differences between groups

• To minimize observer bias

Sdringola S, Eur Heart J. 2008

A 6 month RCT of high dose atorvastatin on myocardial perfusion abnormalities

by PET in coronary artery disease.

20

Example: Estrogen Replacement Therapy in post-menopausal women

• Important therapeutic question• Applies to ~30 million women in US• Prempro (estrogen/progestin combo)

may have been most prescribed drug in US• Potentially huge impact on public health• Complex: ERT effects multiple diseases

21

• Prospective cohort study, n = 48,470• 337,000 person years of follow-up

Risk of MajorEstrogen Use Coronary Disease* Relative Risk**Never Used 1.4 1.0Current user 0.6 0.56 (0.40-0.80)Former user 1.3 0.83 (0.65-1.05)* Events per 1000 women-years of follow-up** Relative Risk (95% CI) compared to never users

22

23

Estrogen Replacement Therapy (ERT)

Disease Effect on Risk*

Coronary heart disease Decrease by 40-80%Osteoporosis (hip fx) Decrease by 30-60%

Breast cancer Increase by 10-20%

Alzheimer’s Decrease by ?Endometrial cancer Increase by700%Pulmonary embolism &DVT Increase by 200 - 300%

* From observational (case-control and cohort) studies24

Women’s Health Initiative HRT study (7/10/02)

• Randomized trial (2)– 16,608 women with uterus (ERT + progestin vs. placebo)– ~11,000 women without uterus (ERT alone vs. placebo)

• Ages 50-79, mean age 64• Represent broad range of U.S. women• 40 clinical centers• Combination therapy arm stopped early (3 years)

– Mean 5.2 years of follow-up– Overall, health risks outweigh benefits– Significant increased risk for invasive breast cancer HRT users

25

0 1 2 3 4 5 6 7

Years

WHI E+P: Coronary Heart Disease

26

HERS/WHI Trials: Take Home Message

• Observational studies can be wrong– Cohort studies can be wrong– Meta-analysis of observational studies can be

wrong.

• What went wrong with observational studies of HRT?

27

Virtually all estrogen results werebased on observational data

• Women chose to take ERT• Are ERT users different from non-users?

– Age– Health status– More exercise– Health behaviors (see Dr.)– SES

• Try to adjust in analysis, but may not be possible• Confounders are impossible to fully identify

• Randomized trials alleviate these problems

28

How Can a Sound Study Design Prevent Confounding?

exposure

Adjustment for confounders in observational trial: avoids known confounders

Randomization in interventional trial: avoids all confounders

outcome

Rohit S Loomba et al. Circulation.2012; 126: A14459

confounder

The Association of Exposure, Confounder and Outcome

coffee

smoking and other confounder

cardiovascular mortality

?

Rohit S Loomba et al. Circulation.2012; 126: A14459

Schema generale dei RCTs

34

JAMA. 1998;280(7):605-613. doi:10.1001/jama.280.7.605

The Heart and Estrogen/progestin Replacement Study trial profile, showing numbers of participants from screening to closeout.

35

Study ProtocolAlways have a study protocol, it is like a schedule for the study

Contents Title and authors Rationale for study Research question, aims and subaims Study design Study location Study population with inclusion and exclusion criteria Definitions of exposure and outcome Statistical analysis Ethical considerations Data acquisition Study flow Logistics and resources

What is the question?

• Selection of the questions• Primary and secondary objectives• Interventions• Response variables• Surrogate endpoints, biomarkers

Fundamental Point

Each clinical trial must have a primary question.

The primary question, as well as any secondary or subsidiary questions, should be carefully selected, clearly defined, and stated in advance.

Primary Objective

Define one question the investigators are most interested in answering and is capable of being adequately answered.

Define the primary endpoint

Toxicity/Safety, efficacy (response/survival), QOL

Define the type of study:

Hypothesis testing or estimation,

Superiority or equivalence trials

The sample size is based on.

Secondary Objectives

Different endpointsSubgroup hypotheses Prospectively defined Based on reasonable expectations Limited in number

Hypothesis testing vs. hypothesis generatingHunting expedition vs. fishing expeditionMultiplicity Issues

Examples of post-hoc subset analyses

In the International Study of Infarct Survival-2 (ISIS-2

Examples of post-hoc subset analyses

In the International Study of Infarct Survival-2 (ISIS-2): the treatment effect seemed to differ by astrological sign More adverse effect of aspirin on mortality for

patients born under Gemini (5/21-6/21) or Libra (9/23-10/23) than others (P < 0.00001).

Response Variables

Dose limiting toxicities (DLT), complications

Response, incidence of a disease, total mortality, death from a specific cause

Overall survival, time to progression, time to cancer

Blood pressure, biomarkers, PSA, CD4 count

Quality of life

Cost and ease of administrating the intervention

In general, a single response variable should be identified to answer the primary question.

Surrogate Endpoints, Biomarkers

Primary endpoint may be expensive and take long time to observe.

Surrogate endpoints should be related to the primary endpoint by mechanism of action or in the common disease/mechanism pathway Factors associated with heart disease LDL-C for CAD Short-term response for long-term

survival

Validation of surrogate endpoints

Study Population

External validity Is the study population representative of

the source population Can results be translated to the general population of patients? OR

High-risk group to ensure a sufficient number of events OR

Specific subgroup e.g. women or elderly

Sample size Sufficient power required to minimize

error due to play of chance vs. restricted resources sample size calculation

Sample Size Calculation

Hazard ratio Mortality (%)

1.5 2.0 2.5 3.0

10 3077 1044 737 53020 1374 539 324 23930 809 302 182 14040 525 192 126 9350 354 129 80 63

n = number of total participants assuming a loss of-follow-up of 10%, alpha = 0.05, power 80% and based on the given hazard ratio between third and first tertile.

Internal and External Validity

49

Inclusion and Exclusion Criteria

Already selection of study site (e.g. tertiary centre) restricts patient selection!

Strictwell defined study population makes the effect more predictable (internal validity)safer due to exclusion of high-risk patientsdifficult to recruit patients increasing cost, time of recruitment and risk of the failure of the study

Broadincreases external validityfacilitates recruitment of patients

Relationship among target population, accessible population, and study samples

51

Greenhalgh, T. BMJ 1997;315:305-308

Outcomes AssesmentThe major sources of bias in a Randomized Clinical Trial

52

Greenhalgh, T. BMJ 1997;315:305-308

Outcomes AssesmentThe major sources of bias in a Randomized Clinical Trial

53

Greenhalgh, T. BMJ 1997;315:305-308

Outcomes AssesmentThe major sources of bias in a Randomized Clinical Trial

THE IMPORTANCE OF BEING BLIND!Information bias for subjective measurements

54

How One May End Up with Irrelevant Results?

Fishing expedition – search for a significant association in an existing database without a research question.

A question that is easy to answer but is not of interest for anyone

Before You Start the Adventure of a Clinical Trial

Challenge the ideaCheck your resourcesHave the adequate expertiseAlways have a study protocol

So, how many?