Unsafety: Making no mockery of honest ad-hockery

Janet WittesStatistics Collaborative

ASA/FDA 2005

Topic du jour

Sleight of tongue

Game: Remove safety and add virtue in two easy steps

D S M C_ _ __ _ _ _

Sleight of tongue

Game: Remove safety and add virtue in two easy steps

D S M CD M C_ _ _ _

Sleight of tongue

Game: Remove safety and add virtue in two easy steps

D S M CD M CI D M C

Efficacy

Stop for “overwhelming” efficacy =0.05; power = 90%; four looks Probability of stopping early>70% Early stop: estimate pulls toward

null

Our current approaches

Prespecify Classify precisely Give lots of data Rely on mechanism

Divide and (un)conquer (e.g. Neuropathy)

Event T C

Neuropathic pain 1 0

Neuropathy 1 0

Neuropathy NOS 5 2

Neuropathy peripheral 2 0

…

…

…

e.g. Neuropathy

Event T C

Anosmia

…..

Autonomic neuropathy 1 0

…

Cranial neuropathy 2 1

…

…

e.g. Neuropathy

Event T C

…

Parathesia 3 2

Parathesia NOS 4 0

Parathesia other 0 1

…

Peripheral motor neuropathy 6 0

Peripheral sensory neuropathy 3 2

Other examples

Heart failure• Separate near synonyms• Allocate to heart and lung

Other examples

Heart failure• Separate near synonyms• Allocate to heart and lung

Bleeding: distribute over body systems

Consider mechanism

If you don’t get the drug, you can’t react to it• Eschew ITT: safety population• Modified Daley’s Rule: censor early and

often• Don’t collect extraneous information

Appeal to statistical conservatism

Populations

ITT mITT Safety: one dose of study med ATP Etc.

e.g., Vioxx- short follow-up

Through 36 months

With denominators

Bresalier et al., NEJM, Feb 2005

The denominators

Mo Rx Po0 1287 129912 1129 119518 1057 115624 938 104230 896 100136 727 835

No data dredging

We test hypotheses Too many type 1 errors if we

dredge

What we typically present

Current behavior• Same tables for interim & final

analyses• Long complete listings

What we should do• Interim data are different from final• Presenting too much dulls the mind

Sentinel events

Single event (e.g., death in a vaccine trial)

Several events (e.g., 3 retinal thromboses)

Sentinel event rate (e.g., WHI)

Lachenbruch and Wittes, SIM, to appear

How to handle

1. Identify sentinel event2. Establish stat’l method for future

events Need reasonable power Type I error rate > 1-sided 0.025

Methods

Individual• Number of non-events until the k’th

eventNegative binomial SPRT

• Time to the next (or k’th) event (-dist’n) Rate: event rate in future patients

NormalPoisson

Pitfalls

Time is subtle Power is low Censoring is tricky

Pull-up

Safety hypothesis: • E.g., Cox-2 leads to 2 fold increase in

MI etc.• Design: stop early if you reject• Estimated relative risk must be pulled

toward 2 Insight from Joe Heyse

The option

Respect PI: Adjudicate adverse events

Precision: Reclassify, reorganize Mechanism: Be an empiricist Dredging: Use sentinel events