Applied Interpretation of Clinical Studies

Jim Hoehns, Pharm.D., BCPS, FCCP

Why is this Lecture Important?

To make informed drug therapy decisions we….– Need to understand size of treatment

effect– Need to relate drug information

accurately Patient and physician perception of

drug therapy effect influences subsequent behaviors and actions

Ann Intern Med 2007;146:848-56.

Let’s Talk About “Risk”…

Description Fatality risk per 100,000 person-

yearsAspirin prophylaxis in 50 y.o. men

10.4

Clozapine for schizophrenia 35Rofecoxib (Vioxx) for arthritis pain

76

Passenger in car 11Driving motorcycle 450Firefighter 10.6Semi-truck driver 44.8Rock climbing 36

Cohen JT. Health Aff 2007;25:636-46.

Outline

Absolute risk Absolute risk reduction (ARR) Relative risk (RR) Relative risk reduction (RRR) Odds ratio (OR) Hazard ratio (HR) Number needed to treat (NNT) Number need to treat to harm (NNT:H)

– aka NNH Confidence intervals (CI)

Taking the right “STEPS” when evaluating new information

S = SafetyT = Tolerability

“Pooled drop-out rates” E = Effectiveness -- Studies

showing that the new drug is better than your current choice

P = PriceS = Simplicity of use

Allen Shaughnessey, Pharm.D.

Point Estimation

Definition: A “point estimate” is a one-number summary of data– If you had just one number to

summarize the inference from your study…

Examples:– Safety and efficacy trials: response

rate, median survivals– Comparative trials: odds ratio,

hazard ratio

Elizabeth S. Garrett, Ph.D.

Which Looks Better?

Lipitor 80mg/d (vs. Lipitor 10mg/d) lowered the risk of MI and stroke by 22%

Lipitor 80mg/d (vs. Lipitor 10mg/d) lowered the risk of MI and stroke by 2.2%

Absolute Risk & Absolute Risk Reduction Risk is the probability or frequency of

an outcome Migraine medication (6 mon.)

– Placebo: 30% recurrence Control event rate (CER)

– Drug M: 5% recurrence Experimental event rate (EER)

Absolute risk reduction (ARR)– CER – EER = ARR– 30% - 5% = 25%

Absolute Risk Reduction Helps discriminate huge

treatment effects from small ones– Preserves information on the

baseline risk– Clinically meaningful information

Relative Risk and Relative Risk Reduction

Relative Risk - risk in treatment group relative to that in control group– Ratio of two incidence rates– EER/CER = RR– .05/.30 = 0.17– Appropriate for trials; not appropriate for case-

control studies Relative risk reduction

– Expression of reduction in relative risk– 1 – RR = RRR (1 - 0.17 = 0.83 or 83% RRR)– RRR does not tell about size of effect on an

absolute scale

Measures risk of developing condition over a specified time

Relative Risk

Relative Risk = Risk of the outcome if the risk factor is present

Risk of the outcome if the risk factor is absent

Example:Breast CA No Breast CA

Estrogen Tx:

No Estrogen Tx:

a = 80 b = 920

c = 10 d = 990

Relative Risk =

aa + b

cc + d

= 8

Odds Ratio

Retrospective study (classically) A method of representing probability Estimates the odds of having the RF if the

condition is present divided by the odds of having the RF if the condition is not present– OR > 1: increased risk of group 1 to 2– OR = 1: no difference in risk of group 1 compared

to group 2– OR < 1: lower risk “protective” in risk of group 1

compared to group 2

Odds Ratio

Odds Ratio = Odds of being on HRT if Breast CA present

Odds of being on HRT if Breast CA not present

Example:Breast CA No Breast CA

Estrogen Tx:

No Estrogen Tx:

a = 80 b = 920

c = 10 d = 990

OR = a / cb / d

= 8.6

Ross et al.

Case-control study in Los Angeles– 1897 cases with incident breast CA

1637 controls– matched on age, race-ethnicity,

neighborhood All patients: no hysterectomy Adjusted for breast CA risks

– e.g. age at menopause, age at menarche, family Hx, nulliparity, body weight, etc.

Ross et al. JNCI 2000;92:328-32.

Ross et al, Results

HRT Type Odds Ratio 95% C.I.

No HRT 1.0 Referent

Any HRT 1.10 (1.02 – 1.18)

ERT 1.06 (0.97 – 1.15)

CHRT 1.24 (1.07 – 1.45)

SEPRT 1.38 (1.13 – 1.68)

CCRT 1.09 (0.88 – 1.35)

Odds Ratio of Breast CA per 5 Years of Use

Relative Risk vs. Odds Ratio

With AV

fistula

W/O AV

fistula

throm.

With AV

fistula

W/O AV

fistula

throm.

Yes 59 122 181 Yes 59 244 303

No 48 190 238 No 48 380 428

OR = (59/48)/(122/190)=1.91RR = (59/181)/(48/238)=1.65

OR = (59/48)/(244/380)=1.91RR = (59/303)/(48/428)=1.77

Th

rom

bop

hili

a

Tripepi G. Kidney International 2007

Number Needed to Treat (NNT)

NNT is the number of patients needed to treat to prevent one event or outcome

reciprocal of the absolute risk reduction

E.g. 0.30 – 0.05 = 0.25 = ARR 1/ARR = 1/0.25 = 4 = NNT incorporates baseline risk w/o

treatment and risk reduction with treatment

clinically useful; tells how much effort required to prevent one event

Number Needed to Treat Be cognizant of the “time” factor The smaller the NNT, the more impressive

the result Patients may have a different baseline

risk than the “average” study patient Limitations

– Expressed as single number (point estimate) “True” NNT may be higher or lower 95% confidence intervals are useful

– NNT is 7 (95%CI 3 – 11)

– Need a binary outcome

Number Needed to Harm (NNH) or (NNT:H) Number needed to treat to cause

harm to one more patient 1/absolute risk increase Example

– Adverse event (>3X ULN AST or ALT) Lipitor 10mg QD: 0.2% Lipitor 80mg QD: 1.2% 1/0.01 = 100 NNH

Number Needed to Treat (NNT)Disorde

rIntervention

Events Being

Prevented

CER EER Follow up time

NNT

DBP 115-129

BP drugs

Death, stroke or MI

13% 1.4% 1.5 yrs 8

DBP 90-129

BP drugs

Death, stroke, or

MI

5.5% 4.7% 5.5 yrs 128

Sym carotid stenosi

s

CEA (vs.

medical

therapy)

Death or major stroke

18% 8% 2 yrs 10

Mild-mod Alz

Aricept v. PBO

No functional

decline

44% 59% 1 yr 7

Renal insuff/angiogra

m

PO mucomyst v. PBO

Contrast media

induced ↓ in renal function

12% 4% 48 hrs 12

McQuay, H. J. et. al. Ann Intern Med 1997;126:712-720

Putting it Together

Don’t Forget…..

Relative risk reduction is always larger, and “looks” better than absolute risk reduction

Confidence Intervals (CI) Provides a measure of precision (or

uncertainty) of an estimate (i.e.study results) for making inferences about the population of all such patients

95% CI– 95% of such intervals will contain the true

population value– Range of values within which we can be

95% sure that the true value lies The smaller the study (i.e. less

patients) the wider the confidence intervals

Confidence Intervals

CIs and significance tests are closely related mathematically

A “significant” P value of <0.05 will correspond to a 95% CI which excludes the value indicating no difference– 0 for the difference between 2

means or proportions– 1 for a relative risk or odds ratio

Which of the following lipid parameters were significantly changed in patients receiving the soy-containing diets?

Which quartile group is significantly different from the <30 group?

HRT and Breast Cancer

Hormone Relative Risk 95% C.I.

None 1.0 reference Conj. Estrogen 1.32 (1.14 - 1.54)

Other Estrogens 1.28 (0.97 - 1.71) E + P 1.41 (1.15 - 1.74)

Progestins Alone 2.24 (1.26 - 3.98)

Nurses Health Study, 1978 to 1992

NEJM 1995;332:1589-93.

Hazard Ratio

Compares the risk of event in two populations– A relative measure

Ratio of risk in group 1 to risk in group 2

Assumption: “proportional hazards”– “risk is constant over time”

Used to analyze time-to-event curves

WHI Study Summaries

Clinical Event WHI (E+P) WHI (E)

CHD events 1.29 (1.02-1.63)

0.91 (0.75-1.12)

Stroke 1.41 (1.07-1.85)

1.39 (1.10-1.77)

Pulmon. emb. 2.13 (1.39-3.25)

1.34 (0.87-2.06)

Breast CA 1.26 (1.00-1.59)

0.77 (0.59-1.01)

Colon CA 0.63 (0.43-0.92)

1.08 (0.75-1.55)

Hip fracture 0.66 (0.45-0.98)

0.61 (0.41-0.91)

Death 0.98 (0.82-1.18)

1.04 (0.88-1.22)

JAMA 2004;291:1769-71.

Hazard Ratio (95% CI)

Intensive Lipid Lowering – Acute Coronary Syndromes

PROVE IT-TIMI 22 Study Treatment: Pravachol 40mg QD (LDL goal

<100) vs. Lipitor 80mg QD (LDL goal~70)– Statin naïve (75%): baseline TChol ≤240– On statins (25%): baseline TChol ≤200

N=4,162 with ACS in past 10 days– Mean duration: 2 years

Primary outcome: death (any cause), MI, unstable angina, PTCI, or CABG

N Engl J Med 2004;350:1495-504.

PROVE IT - Results Mean LDL

– Pravachol 40mg: 106 (baseline) 95mg/dL– Lipitor 80mg: 106 (baseline) 62mg/dL

Primary outcome– Pravachol 40mg: 26.3%– Lipitor 80mg: 22.4%

Baseline LDL ≥125mg/dL: 34% RRR Baseline LDL <125m/dL: 7% RRR

Safety– >3XULN ALT: Pravachol 40mg: 1.1%, Lipitor 80mg

3.3%– DC med due to myalgias/CK: Pravachol 2.7%, Lipitor

3.3%– DC rate: Pravachol (33%), Lipitor (30.4%)

RRR 16%ARR 3.9%NNT 25.6

P=0.02

N Engl J Med 2004;350:1495-504.

Intensive Lipid Lowering in Patients with Stable Coronary Disease

TNT Study Treatment: Lipitor 10mg QD (LDL goal

<100) vs. Lipitor 80mg QD (LDL goal: 75 mg/dL)– 8 week run-in of Lipitor 10mg QD– Randomized if LDL <130

N=10,001, median duration: 4.9 yrs– History of MI, angina, and hx of

revascularization Primary endpoint: CHD death, MI,

resuscitation after cardiac arrest, or stroke

N Engl J Med 2005;352:1425-35.

TNT Study - Results Mean LDL

– Lipitor 10mg: 98mg/dL (baseline) → 101mg/dL– Lipitor 80mg: 97mg/dL (baseline) → 77mg/dL

Primary outcome– Lipitor 10mg: 10.9%– Lipitor 80mg: 8.7%

Mortality– All-cause: HR 1.01 (0.85-1.19)– Noncardiovascular:

Lipitor 10mg (2.5%), Lipitor 80mg (3.2%) HR: 1.25 (0.99-1.57) P=0.06

RRR 22%ARR 2.2%NNT 45

TNT – Cost Analysis

Drug cost– 10mg (1yr) $811; 10mg (4.9yrs)

$3,974– 80mg (1yr) $1,119; 80mg (4.9yrs)

$5,484 NNT = 45.4

– 45.4 X $5,484 = $248,980 Incremental cost over 10mg QD

– $248,980 - $180,420 = $68,560www.drugstore.com Aug 2005

Lipitor 80mg in PROVE-IT and TNT

PROVE-ITN=4162

TNTN=1000

1Significant decrease in primary outcome?

Yes Yes

NNT (primary outcome) 25.6 45.4

Patients with highest LDL at baseline observed greatest benefit?

Yes ?

>3XULN AST/ALT 3.3% 1.2%

Significant increase in myalgias? No No

Increase in noncardiovascular mortality?

No Yes

CLASS Study

Annualized incidence of upper GI ulcer complications– Celecoxib 0.76%– NSAIDs 1.45%– Relative risk 0.53 (0.26-1.11)– Absolute risk reduction 0.69%– NNT= 145

CLASS Study

Annualized incidence of upper GI ulcer complications plus symptomatic ulcers– Celecoxib 2.08%– NSAIDs 3.54%– Relative risk 0.59 (0.38-0.94)– Absolute risk reduction 1.46%– NNT= 68.5– Cost of preventing one event=

$49,715

Vertebral Fractures

Absolute or Relative Measure?Absolute RiskAbsolute Risk ReductionRelative RiskRelative Risk ReductionOdds ratioHazard ratioNumber need to treat or harm

Summary

Distinguish between absolute and relative (benefits or harms) drug effects– RRR looks “better” than ARR– NNT is a useful measure

Relative measures– Odds ratio, relative risk, hazard ratio

<1: “protective” effect; lower risk 1: no difference in risk >1: increased risk

For the 3 abstracts determine….. ARR NNT RR RRR

Pain 1997;70:193-201

NNT 5.0 (4.1-6.9)

NNT 3.1 (2.6-3.8)

AB

CD

Clinicallysignificant??

Study B (Hypertension) Fatal or nonfatal stroke

– Placebo (CER): 17.7%– Indapamide (EER) 12.4%– ARR: 17.7 – 12.4 = 5.3%

NNT = 1/0.053 = 18.9 or 19 (over 18 mon.)

– Relative risk: .124/.177 = 0.7 Reported HR: 0.7 (0.49 – 1.01)

– RRR: 1 – 0.7 = 0.3 or 30%

Study C (diabetes)

Nonfatal MI, nonfatal stroke, or CV death– Standard tx (CER): 7.2%– Intensive tx (EER): 6.9%– ARR: 7.2 – 6.9 = 0.3%

NNT: 1/0.003 = 333– Relative risk: 0.069/0.072 = 0.96– RRR: 1 – 0.96 = .04 or 4%– Statistically nonsignificant result

Study D (DVT prophylaxis) Any DVT, nonfatal PE, or death

– Enoxaparin (CER): 18.9%– Rivaroxaban (EER): 9.6%– ARR: 18.9 – 9.6 = 9.3%– NNT = 1/0.093 = 10.8 or 11 (over 17

days)– Relative risk: 0.096/0.189 = 0.51 or

51%– RRR: 1 – 0.51 = 0.49