8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 1/28

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 2/28

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 3/28

HRT trial

Hormone replacement therapy

Multiple observational epidemiological study show benefit

Three clinical trials show evenharmful

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 4/28

Outcome Treatment 

Attributes/

confounders 

Randomization

blocks this

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 5/28

NASCIS trial

National Acute Spinal Cord Injury Studies

Methyprednisolone Rx in acute spinal cord injury

CAST trialCardiac arrhythmia suppression trial

Treat PVC or not?

CONVINCE trial

Controlled Onset Verapamil Investigation of Cardiovascular Endpoints

Stopped prematurely for commercial reason

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 6/28

Design

 Analysis

Presentation

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 7/28

Ethical issue

Phase of trials

Sampling issue

Randomization issue

Blinding/masking

Measurement issue

Power issue

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 8/28

 Adherence issue

Multiple comparison

Statistical error

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 9/28

Inappropriate or poor reporting of results

 Wrong or poor interpretation of results

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 10/28

Fail to recognize roles of Researcher vs.Physician

Risk/benefit equipoise

Employee/vulnerable population

Responsibility of investigators

Collaboration with pharmaceuticals

Fail to understand the related regulation

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 11/28

Fail to form DSMB

Early termination/extended recruitment

 What if Placebo not ethical?

Equivalence trial

 Active control

Effectiveness issue

CONVINCE trialNon-inferiority trial

CAST trial

 VAL/HCTZ (Valsartin/Hydrochlorthiazide) vs. Amlodipine

0% 10 %

Ho Ha

0% 20 %

Ho Ha

0% 15 %

Ho Ha

!"

33% reduction

50% reduction

67% reduction

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 12/28

 Alternative RCTType

Null Hypothesis Hypothesis

Traditional New = old New ≠ old

Equivalence New < old + δ New ≥ old + δ

Non-inferiority New ⊀ old New = old

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 13/28

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 14/28

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 15/28

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 16/28

Fail to report inclusion/exclusion criteria

Fail to obtain equipoise between:

Homogeneity

Generalizability

Modified Zelen’s design

Double randomized consent trial

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 17/28

Method of randomization not clearly stated

Simple, Restricted, or Adaptive, which & why?

How to have good randomization?

Independent group

Reproducible, unpredictable

 Adherence, blinding

 Verify, audit, periodical check

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 18/28

Hard to mask for device trial

Patient bias (placebo/sham effect)

Investigator bias

Evaluator bias

PROBE design (Prospective randomizedopen blinded end-point)

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 19/28

Instrument utilization

Even VAS not optimal for pain

Objective is better: heart rate

Suboptimal measurement

Surrogate outcome/exposure

CAST trial

Duration of follow up

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 20/28

Fail to estimation recruitment feasibility

Fail to report withdrawals

Fail to calculation for interim analysis

Fail to use efficient trial

Non-inferiority trial

CAST for one-sided issue

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 21/28

 As-treated analysis

Compare who actually received treatment vs. control

Ignore randomization

Per-protocol analysis

Compare who comply with assigned treatment

Compare compliers in treatment group with full control group

Intention-to-treat analysisStandard estimate

Ignore compliance just use randomization

Instrumental variable/complier average causal effect (IV/CACE)

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 22/28

Failure to include a multiple-comparisoncorrection

NASCIS

Inflation of Type I error

Post-hoc subgroup analysis

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 23/28

Typical errors with Student’s t-test

Failure to prove test assumptions

Use of an unpaired t-test for paired data or vice versa

Unequal sample sizes for paired t-test

Improper multiple pair-wise comparisons

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 24/28

Typical errors with Chi-square tests

No Yates-continuity correction reported if small numbers

Use of chi-square when expected numbersin a cell <5

No explicit statement of tested Null-Hypotheses

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 25/28

Report p-values only, no confidence intervals

CIs given for each group rather than forcontrasts

“p = NS”, “p <0.05” or other arbitrary thresholds instead of reporting exact p-values

Numerical information given to an unrealisticlevel of precisions

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 26/28

“Non significant” interpreted as “no effect”, or “no difference”

Drawing conclusions not supported by data

Significance claimed without data analysis or statistical testmentioned

Disregard for Type II error when reporting non-significantresults

Missing discussion of the problem of multiple significance testing if done

Failure to discuss sources of potential bias and confoundingfactors

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 27/28

 What we need is a brand new idea that has been thoroughly tested!

8/6/2019 Common Pitfalls in Clinical Trials_010511

http://slidepdf.com/reader/full/common-pitfalls-in-clinical-trials010511 28/28

Questions?