Bonferroni: Friend or Foe?Multiple Testing in Cardiovascular Medicine

Dhruv S. Kazi, MD, MScAHA Cardiovascular Outcomes Research FellowStanford Universitykazi@stanford.edu

“Off hand, I’d say you’re suffering from an arrow through your head, but just to play it safe, let’s get an echo.”

Deat

h fro

m C

ardi

ovas

cula

r Cau

ses,

No

nfat

al M

yoca

rdia

l Inf

arct

ion,

or S

troke

= 9 billion dollars

Yusuf, S, et al. N Engl J Med 2001;345:494-502

CURE

Liver

Clopidogrel

Cytochrome P450-dependent oxidation

Binds to P2Y12 Receptor on Platelets

Ticagrelor Binds to P2Y12 Receptor on Platelets

CYP2C19

Potential Strategies

Clopidogrel Ticagrelor

Which one would you want?

Cannon, CP, et al. Lancet 2010; 375: 283-93.

Primary Efficacy Endpoint in the PLATO-Invasive RCT

Cannon, CP, et al. Lancet 2010; 375: 283-93.

Methods

Cohort: 100,000 patients who present with ACS and undergo PCI, age at entry – 65 years

Analytic Horizon: Lifetime

Perspective: “Ideal Insurer”

Interventions– DAPT 12 months from last ACS or PCI, whichever is

later– Aspirin monotherapy for life thereafter

Possible Explanations?

Possible Explanations?

True Difference Chance Finding

Fraud?

The Multiple-Look Problem

Number of analyses

Cum

ulati

ve p

rob

of

a po

sitiv

e as

soci

ation

0 10 20 30 40 50 60 70 80 90 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

So how do you get around this?

Traditionally, “don’t run multiple subgroups” unless:- The analyses are pre-specified- The analyses are biologically plausible

And if you must, conduct rigorous statistical adjustment!

Bonferroni Adjustment

Conservative Assumes independence 1-(1-α)1/n ~ α/n

But does this make sense?BMJ. 1998 April 18; 316(7139): 1236–1238.

How Do We Proceed? (Do you still want the drug?) Multiple testing is problematic (even if pre-specified) The challenges of a priori hypotheses

Conclusions

Multiple testing is a complicated question: with real clinical consequences

Statistical adjustment is a necessary but imperfect solution

Trial and Error. Kaul S, et al. J Am Coll Cardiol 2010;55:415–27

Conclusion

The p value is no substitute for a brain.

Thank You!

kazi@stanford.edu