+ All Categories
Home > Documents > Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis,...

Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis,...

Date post: 31-Mar-2015
Category:
Upload: alison-parnell
View: 213 times
Download: 0 times
Share this document with a friend
Popular Tags:
75
Absolute Risk Reduction, Number Needed to Treat, Back- of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael A. Kohn, MD, MPP Using Randomized Trials to Quantify Treatment Effects
Transcript
Page 1: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope

Cost Effectiveness Analysis, Testing Thresholds Revisited

26 October 2006

Michael A. Kohn, MD, MPP

Using Randomized Trials to Quantify Treatment Effects

Page 2: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Quantifying the Benefit of a Treatment: Take Home Points

• RCT Checklist: Importance of patient-oriented effect measures (POEMs), randomization, intention-to-treat analysis, good follow-up, blinding, and between-groups comparisons.

• The Relative Risk or Relative Risk Reduction associated with an intervention is of minimal use* without a baseline prevalence of bad outcomes.

• You need to have an absolute risk reduction to calculate number needed to treat. (NNT = 1/ARR)

• For undesired effects of treatment, calculate the absolute risk increase (ARI), and the number needed to harm (NNH = 1/ARI)

• Back-of-the-envelope CEA: Cost per bad outcome prevented = Treatment Cost x NNT

*Unless the RR is 1 and RRR is 0.

Page 3: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

RCT Checklist

Page 4: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

• Patient-Oriented Effect Measures (POEMs) vs. surrogate outcomes

• Randomization to address issues of confounding• Intention-to-treat analysis (once randomized always

analyzed)• Good follow-up to eliminate differential losses to follow-

up• Blinding of patients and clinicians to prevent differential

co-interventions• Blinding of outcome assessors to prevent bias• Between groups rather than within groups comparison• Compare entire randomization groups, not subgroups

*For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

RCT Checklist for Study Validity*

Page 5: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

RCTs of Orthopedic Treatments

1. Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514

2. Parker MJ, Khan RJ, Crawford J, Pryor GA. Hemiarthroplasty versus internal fixation for displaced intracapsular hip fractures in the elderly. A randomised trial of 455 patients. J Bone Joint Surg Br. Nov 2002;84(8):1150-1155.

Page 6: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Endpoints: Arthroscopy vs. immobilization for 1st shoulder

dislocation

Kirkley A, Griffin S, Richards C, Miniaci A, Mohtadi N. Prospective randomized clinical trial comparing the effectiveness of immediate arthroscopic stabilization versus immobilization and rehabilitation in first traumatic anterior dislocations of the shoulder. Arthroscopy. Jul-Aug 1999;15(5):507-514.

Page 7: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Outcomes Affected by Treatments*

• Dichotomous (e.g. recurrent dislocation)

• Continuous (e.g. WOSI**)

Endpoints

• Patient relevant (e.g., ability to return to sports)

• Surrogate (e.g., MRI findings)

* Example: Arthroscopy vs. conservative tx for 1st Anterior Shoulder Dislocation (Arthroscopy. 1999 Jul-Aug;15(5):507-14. )**Western Ontario Shoulder Disability Index

Page 8: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Outcomes Affected by Treatments

• Dichotomous (e.g. recurrent dislocation)

• Continuous (e.g. WOSI)

Endpoints

• Patient relevant (e.g., ability to return to sports)

• Surrogate (e.g., MRI findings)

Page 9: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Randomization, Intention-to-Treat Analysis, and Follow-up: Arthroplasty vs. Internal

Fixation?• Pt is a 81-year-old woman with a displaced, intracapsular

femoral neck fracture. • Pt’s son is a physician. He asks about hemiarthroplasty

vs. internal fixation.

Pubmed search Parker MJ, Khan RJ, Crawford J, Pryor GA. Hemiarthroplasty versus internal fixation for displaced intracapsular hip fractures in the elderly. A randomised trial of 455 patients. J Bone Joint Surg Br. Nov 2002;84(8):1150-1155.

Page 10: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.
Page 11: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.
Page 12: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.
Page 13: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Randomization, Intention-to-Treat Analysis, and Follow-up: Parker MH et al. Bone Joint

Surg Br. 84(8):1150-1155.

Randomized controlled trial of the effects of hemiarthroplasty vs. internal fixation on re-operation and other outcomes in > 70-year-old patients with displaced, intracapsular femoral neck fractures.

Page 14: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Randomization: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

Why do a randomized experiment?

Why not do an observational study comparing mortality, re-operation rates, etc. between hemiarthroplasty patients and internal fixation patients?

Page 15: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Intention-to-Treat: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

Some patients randomized to the hemiarthroplasty group ended up getting internal fixation.

Why not include these patients’ outcomes in the internal fixation group or at least exclude them from the hemiarthroplasty group?

Page 16: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Losses to Follow-Up: Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.*

If each treatment group had 20% loss to follow-up, there could still be bias.

What if those in the internal fixation group were lost to follow-up because they got better and those in the hemi-arthroplasty group were lost because they died?

*In fact, there were no losses to follow-up in this study.

Page 17: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Blinding

Blinding of Patients

• Addresses placebo effect

Blinding of Patients and Clinicians

• Eliminates differential co-interventions

Blinding of Outcome Assessment

• Eliminates biased outcome assessment

Page 18: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Blinding

Blinding less important when opportunity for cointerventions that affect outcomes is minimal, and outcome is not subjective.

• Arthroplasty vs Internal Fixation for hip fracture, with endpoints of mortality and re-operation: patients, clinicians, and outcome assessors not blinded.

• Arthroscopy vs. non-operative management of shoulder dislocation, with endpoints of re-dislocation, and WOSI*: patients not blinded, but clinicians and outcome assessors (therapists) were blinded.

*Western Ontario Shoulder Disability Index**Moseley JB, et al. A controlled trial of arthroscopic surgery for osteoarthritis of the knee. N Engl J Med 2002;347(2):81-88.

Page 19: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Between-groups Comparison

Nissen SE, Tsunoda T, Tuzcu EM, Schoenhagen P, Cooper CJ, Yasin M, et al. Effect of recombinant ApoA-I Milano on coronary atherosclerosis in patients with acute coronary syndromes: a randomized controlled trial. Jama 2003;290(17):2292-2300.

Page 20: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Sub-group Analysis: ISIS II*

30-day mortality

*Lancet 1988;2(8607):349-360.

Overall Geminis and Libras

Other signs

Aspirin 9.4% 11.1% 9.0%

Placebo 11.8% 10.2% 12.1%

Page 21: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

• Patient-Oriented Effect Measures (POEMs) vs. surrogate outcomes

• Randomization to address issues of confounding• Intention-to-treat analysis (once randomized always

analyzed)• Good follow-up to eliminate differential losses to follow-

up• Blinding of patients and clinicians to prevent differential

co-interventions• Blinding of outcome assessors to prevent bias• Between groups rather than within groups comparison• Compare entire randomization groups, not subgroups

*For checklist on study validity, see Chapter 1B1 “Therapy”, in Guyatt and Rennie (eds.), Users Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice; AMA Press; 2002. (Or try http://www.cche.net/usersguides/therapy.asp#Valid )

DONE: RCT Checklist for Study Validity*

Page 22: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Effect Size

(Dichotomous Outcomes)

RR

RRR

ARR

NNT

ARI

NNH

Page 23: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

This study was properly randomized but not blinded, used an intention-to-treat analysis, and had NO losses to follow-up.

Results follow…

Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

Page 24: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Reduced Re-operation

Re-operation No Re-operation Risk

Hemiarthroplasty 12 217 229 12/229 = 5.2%

Internal Fixation 90 136 226 90/226 = 39.8%

Relative Risk (RR): 5.2%/39.8% = 0.13

Relative Risk Reduction (RRR): 1 - RR = 0.87

Absolute Risk Reduction (ARR):39.8% - 5.2%

= 34.6%

Number Needed to Treat (NNT) 1/ARR = 3

(Need to treat 3 patients with hemirarthoplasty instead of internal fixation to prevent one patient requiring re-operation.)

Page 25: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Measures of Treatment Effect

RR= Relative Risk or Risk Ratio =

RR < 1 tx is associated with decreased risk, as is the usually the case for a primary endpoint. RR>1 means tx is associated with increased risk, as is usually the case for a side effect.

RRR = Relative Risk Reduction = 1-RR

)/(

)/(

dcc

baa

Bad Outcome

No Bad Outcome

Totals

Treatment a b a + b

Control c d c + d

Totals a + c b + d N = a + b + c + d

Page 26: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Beware of the Odds Ratio

RR= Relative Risk or Risk Ratio =

(a/b) (a/c)

OR = Odds Ratio = ------- = -------- = ad/bc

(c/d) (b/d)

)/(

)/(

dcc

baa

Bad Outcome

No Bad Outcome

Totals

Treatment a b a + b

Control c d c + d

Totals a + c b + d N = a + b + c + d

Page 27: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Measures of Treatment Effect

ARR = Absolute Risk Reduction = c/(c+d) - a/(a+b)

NNT = Number Needed to Treat (to prevent 1 bad outcome) = 1/ARR

Bad Outcome

No Bad Outcome

Totals

Treatment a b a + b

Control c d c + d

Totals a + c b + d N = a + b + c + d

Page 28: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

NNT Practice

In patients < 30 years old with first-time acute anterior shoulder dislocation, prompt arthroscopic surgery (vs. standard conservative therapy) reduces the 2-year re-dislocation rate by almost 33% in absolute terms (from about 50% to about 17%).*

How many first-time dislocation patients do we need to treat with arthroscopy to prevent one having re-dislocation at 2 years?

*Kirkley A, et al. Arthroscopy. Jul-Aug 1999;15(5):507-514. Numbers rounded for purposes of exposition.

Page 29: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

The relative risk (RR) or relative risk reduction (RRR = 1-RR) associated with a treatment is of minimal use without knowing the baseline level of risk*.

Problem with the Relative Risk

*The RR is not completely useless without the baseline risk. If RR=1, the tx is useless regardless of the baseline risk. If RR << 1, then the treatment is beneficial; if RR >> 1, the treatment is harmful. Also, if you already know the baseline risk in your own population, the RR may be all you need.

Page 30: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

The selective estrogen receptor modulator raloxifene (Evista®) at a dose of 60 mg /d for 3 years reduces vertebral fracture risk by 33% in women with osteoporosis.*

How many women with osteoporosis do we need to treat with raloxifene to prevent a vertebral fracture?

Problem with the Relative Risk

*JAMA. 1999 Aug 18;282(7):637-45. Numbers rounded for exposition.

Page 31: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

The selective estrogen receptor modulator raloxifene (Evista®) at a dose of 60 mg /d for 3 years reduces vertebral fracture risk by 33% in women with osteoporosis.

Baseline 3-year risk of vertebral fracture = 10%

How many women with osteoporosis do we need to treat with raloxifene to prevent a vertebral fracture?

Problem with the Relative RiskNeed Baseline Risk

Page 32: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Baseline 3-year risk of vertebral fracture = 10%RRR = 0.33ARR = 0.1 x 0.33 = .033NNT = 1/0.033 = 30Need to treat 30 osteoporotic women with

raloxifene for 3 years to prevent a vertebral fracture.

Problem with the Relative RiskNeed Baseline Risk

Page 33: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Effect of Flu Vaccination on All-Cause Mortality During the Flu Season

The study population included almost 300,000 subjects at least 65 years old, of whom about 58% were vaccinated. Among vaccinated and unvaccinated subjects, 1.2% and 2.0% respectively died during the flu season.

Nichol et al. N Engl J Med. Apr 3 2003;348(14):1322-1332. This was NOT an RCT, lots of confounding to address, but it’s related to the flu.

Pooled computerized data from 3 large managed care organizations.

Page 34: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

No Flu Shot?

How about Tamiflu?

Page 35: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Mechanism of Action of Neuraminidase Inhibitors

Moscona A. Neuraminidase inhibitors for influenza. N Engl J Med 2005;353(13):1363-73.

Page 36: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Flu Prophylaxis?• Pt is a 14-year-old girl with fever, myalgias, cough and

sore throat X 1 day• Should you rx prophylactic Tamiflu® for the pt’s

grandparents (in their 70s) who live in the same household and didn’t get the flu shot this year?

Pubmed search Welliver R et al. Effectiveness of Oseltamivir in Preventing Influenza in Household Contacts: A Randomized Controlled Trial. JAMA 2001; 285:748-754.

Page 37: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Prophylactic Oseltamivir: Index Case Flu+*

Household Contacts

Flu No Flu Risk

Oseltamivir 3 206 209 3/209 = 1.4%

Placebo 26 180 206 26/206 = 12.6%

29 386 415

Relative Risk (RR): 1.4%/12.6% = 0.11

Relative Risk Reduction (RRR): 1 - RR = 0.89

Absolute Risk Reduction (ARR): 12.6% - 1.4% = 11.2%

Number Needed to Treat (NNT) ????

*Welliver R et al. Effectiveness of Oseltamivir in Preventing Influenza in Household Contacts: A Randomized Controlled Trial. JAMA 2001; 285:748-754.

Page 38: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Prophylactic Oseltamivir: Index Case Flu+*

Household Contacts

Flu No Flu Risk

Oseltamivir 3 206 209 3/209 = 1.4%

Placebo 26 180 206 26/206 = 12.6%

29 386 415

Relative Risk (RR): 1.4%/12.6% = 0.11

Relative Risk Reduction (RRR): 1 - RR = 0.89

Absolute Risk Reduction (ARR): 12.6% - 1.4% = 11.2%

Number Needed to Treat (NNT) 1/ARR = 9

*Welliver R et al. Effectiveness of Oseltamivir in Preventing Influenza in Household Contacts: A Randomized Controlled Trial. JAMA 2001; 285:748-754.

Page 39: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Number Needed To HarmNausea No Nausea Risk

Oseltamivir 27 467 494 27/494 = 5.5%

Placebo 12 449 461 12/461 = 2.6%

Relative Risk (RR): 5.5%/2.6% = 2.1

Absolute Risk Increase (ARI): 5.5% - 2.6% = 2.9%

Number Needed to Harm (NNH): 1/ARI = 35

NNH is really number needed to treat to cause one undesired effect.

Page 40: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Number Needed to Harm

Not an apt term for number needed to treat to cause one bad outcome.

Would prefer NNTc (“Number Needed to Treat to cause”) vs. NNTp (“Number Needed to Treat to prevent”), but NNH is well established.

Page 41: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Ratio of Undesired to Desired Effects

“Harms” / Bad Outcome Prevented =

ARI/ARR =

NNT/NNH

Or

Bad Outcomes Prevented / Harm Caused =

ARR/ARI =

NNH/NNT

Page 42: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Number Needed to HarmTransfusion No Transfusion

Hemiarthroplasty 44 179 223 44/223 = 19.7%

Internal Fixation 4 219 223 4/223 = 1.8%

Relative Risk (RR): 19.7%/1.8% = 11.00

Absolute Risk Increase (ARI): 19.7% - 1.8% = 17.9%

Number Needed to Harm (NNH) ????

Page 43: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Number Needed to HarmTransfusion No Transfusion

Hemiarthroplasty 44 179 223 44/223 = 19.7%

Internal Fixation 4 219 223 4/223 = 1.8%

Relative Risk (RR): 19.7%/1.8% = 11.00

Absolute Risk Increase (ARI): 19.7% - 1.8% = 17.9%

Number Needed to Harm (NNH) 1/ARI = 6

(Need to treat 6 patients with hemirarthoplasty instead of internal fixation to cause one patient requiring transfusion.)

Page 44: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Ratio of Desired to Undesired Effects

Bad Outcomes Prevented / Harm Caused = ARR/ARI = NNH/NNT*Arthroplasty vs. Internal Fixation for Hip FxRisk Difference for re-operation:

∆ Risk Re-Op = 5.2% - 39.8% = -34.6%Risk Difference for transfusion:

∆ Risk Trx = 19.7% - 1.8% = +17.9% Re-operations prevented/Transfusion Caused: -

34.6/17.9 = -1.93 ≈ -2

*Easier here to divide ∆ re-operation by ∆ transfusion, rather than use NNH or NNT.

Page 45: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Ratio of Undesired to Desired Effects

Cases of Nausea / Flu Case Prevented =

2.9%/ 11% = 0.25

Or

Flu Cases Prevented / Nausea Caused =

11%/2.9% = 4

Page 46: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE CEA

Back-of-the-Envelope Cost Effectiveness Analysis

Page 47: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Back-of-the-Envelope Cost Effectiveness Analysis

How many patients do I need to treat (at the treatment cost) to prevent 1 bad outcome?

Number Needed to Treat (NNT) = 1/ARR

Cost of preventing one bad outcome =

NNT x Treatment Cost*

*This is just ∆$Cost /∆Risk .

Page 48: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE CEA: Oseltamivir

Index Case Flu +• NNT = 9 (Treat 9 household contacts,

prevent 1 flu case)

• NNT x Treatment Cost* = 9 x $35 = $315/flu case prevented

•Cost of Tamiflu 75 mg #10 = $59.99 www.drugstore.com 3/4/2004•$79.99 10/25/06

Page 49: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE CEA Example

Raloxifene vs. placebo in women with osteoporosis

Page 50: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Raloxifene vs. Placebo

Raloxifene (Evista®) 60 mg/d x 30 d = $87*

36 months of treatment = 36 x $87 = $3132

Need to treat 30 patients to prevent 1 fx

30 x $3132 ≈ $93,960 per vertebral fx prevented.

*Drugstore.com 10/25/06

Page 51: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE CEA Example

Letrozole (Femara®) to prevent breast cancer recurrence after 5 years of tamoxifen therapy.

Page 52: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Drug cuts risk of breast-cancer relapse

Findings so promising, study halted so scientists could release news

By Sabin RussellChronicle Medical Writer

Front Page, San Francisco Chronicle 10/10/03

RCT of Letrozole (Femara®), after tamoxifen, to prevent breast cancer recurrence

Page 53: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

RRR or ARR?

“The trial was interrupted almost 2½ years after it began. Researchers had scheduled a midpoint peak at the data, and found letrozole was apparently working far better than expected. The women who took it had 43 percent fewer recurrences of their breast cancer compared to those assigned in the study to take a placebo, or dummy pill.”

Page 54: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Femara Trial Results

Recurrence No Recurrence

Letrozole 75 2500 2575

Placebo 132 2450 2582

Risk(Letrozole) = 61/2575 = 2.9%Risk(Placebo) = 106/2582 = 5.1%RR = 2.9/5.1 = 0.57RRR = 1- 0.57 = 0.43

N Engl J Med. 2003 Nov 6;349(19):1793-802.

Page 55: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Femara Trial Results

ARR = 5.1% - 2.9% = 2.2%

NNT = 1/2.2 = 45

Treatment Cost = $252/month* x 12 months/year x 2.4 years = $7260

Femara Cost per Recurrence Prevented =

$7260 x 45 = $327,000

*2.5mg tablets are available from the International Pharmacy for $252/30 day supply (30 tablets) (Price on www.drugstore.com 10/25/06 = $240/30 pills.)

Page 56: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE CEA Examples

• Oseltamivir to prevent flu in household contacts of flu+ individuals:$315 per case of flu prevented

• Raloxifene to prevent vertebral fracture: $93,960/fracture prevented at 3 years.

• Letrozole after tamoxifen to prevent recurrent breast cancer: $327,000/recurrence prevented at 2.4 years

Page 57: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE vs. “Real” CEA

Estimates treatment costs per bad outcome prevented – including the bad outcome’s costs

Treatment Costs---------------------------------------------------Bad Outcome + Bad Outcome’s Costs

“Real” Cost Effectiveness Analysis:

Treatment Costs – Bad Outcome’s Costs-------------------------------------------------------

Bad Outcome

Page 58: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE vs. “Real” CEABack of the envelope:

Treatment Costs---------------------------------------------------Bad Outcome + Bad Outcome’s Costs

“Real”:

Treatment Costs – Bad Outcome’s Costs-------------------------------------------------------

Bad Outcome

Note that “real” analysis LOWERS the cost per bad outcome prevented and makes treatment look better.

Page 59: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Quantifying the Benefit* of a Treatment: Take Home Points

• The Relative Risk or Relative Risk Reduction associated with an intervention is of minimal use without a baseline prevalence of bad outcomes.

• You need to have an absolute risk reduction to calculate number needed to treat. (NNT = 1/ARR)

• For undesired effects of treatment, calculate the absolute risk increase (ARI), and the number needed to harm (NNH = 1/ARI)

• Back-of-the-envelope CEA: Cost per bad outcome prevented = Treatment Cost x NNT

*With regard to dichotomous outcomes

Page 60: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Flu Treatment?• Pt is a 14-year-old girl with fever, myalgias, cough and

sore throat X 1 day• Should you prescribe Tamiflu for the patient herself?

Pubmed search Treanor JJ, et al. Efficacy and safety of the oral neuraminidase inhibitor oseltamivir in treating acute influenza: a randomized controlled trial. US Oral Neuraminidase Study Group. Jama 2000;283(8):1016-24.

Page 61: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Continuous OutcomesFlu +

Median Illness DurationOseltamivir 70 hoursPlacebo 100 hoursDifference -30 hours

Treatment Cost = $60 (75 mg bid x 5 days)$60/30 hours = $2 in drug cost per hour of

symptoms x 24 hours/day ≈ $50/day of symptoms

Treanor JJ, et al. Jama 2000;283(8):1016-24.

Page 62: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Flu Prophylaxis?• Pt is a 14-year-old girl with fever, myalgias, cough and

sore throat X 1 day• Should you rx prophylactic Tamiflu® for the pt’s

grandparents (in their 70s) who live in the same household and didn’t get the flu shot this year?

What if the 14-year-old doesn’t have the flu?

The relative risk reduction is the same (89%), but the baseline risk for the grandparents is so low that prophylactic oseltamivir doesn’t do much. (ARR is negligible, NNT is enormous.)

Page 63: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Prophylactic oseltamivir works if the index case has the flu, but you don’t know whether she does. You know that 45% of similar patients have laboratory proven influenza.

Probability of Flu + = 45%

Page 64: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

NNT is calculated for patients with a particular condition “D”.

P = probability of that condition “D” in your patients

Your NNT* = NNT / P

Note that NNT* goes up as P = probability of condition “D” goes down.

Patient May Not Have The Condition That You’re Treating

*Assumes that treatment for patients without condition D has no value. (For D- patients, ARR = 0, NNT = ∞)

Page 65: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

NNT if index case flu + = 9.

Probability of flu = 0.45

NNT* = 9 / 0.45 = 20

Probability of Flu + = 45%

*Assumes that treatment when index case is flu - has no value (ARR = 0, NNT = ∞)

Page 66: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

No Adjustment of NNH for Disease Prevalence

Oseltamivir is no less likely to cause nausea in household contacts of Flu- patients than in those of Flu+ patients.

Prevalence of Flu+ = P = 45%ARI for nausea is still: 5.5% - 2.6% = 2.9%But, ARR is now:

0.45(12.6 %-1.4%) = 5%Cases Flu Prevented for each case of nausea

caused:5%/2.6% ≈ 2

Page 67: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

BOTE CEA: Oseltamivir

Index Case Flu +• NNT = 9 (Treat 9 household contacts, prevent 1

flu case)• NNT x Treatment Cost* = 9 x $35 = $315/flu

case prevented

45% Prob Flu+• NNT* = 9/0.45 = 20• NNT* x Treatment Cost = 20 x $35 = $700/flu

case prevented

* Cost of Tamiflu 75 mg #10 = $59.99 www.drugstore.com 3/4/2004

Page 68: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Treatment Threshold Probability from Threshold Cost Effectiveness Ratio

Assume it is worth $840 in oseltamivir to prevent one case of the flu. We can calculate the treatment threshold probability PTT:

NNT* x $35 = $840

NNT/ PTT = $840/$35 = 24

NNT/24 = PTT

9/24 = 0.375 = PTT

Page 69: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Treatment Threshold Probability

from B and C

B = 1/9 x $840 - $35 = $58.33

C = $35

B/C = 0.6 = Threshold Odds

B/(B+C) = 0.375 = Threshold Prob = PTT

Page 70: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

When does it make sense to test the index patient?

QuickVue Rapid Bedside Antigen Test*

Sensitivity = 75%

Specificity = 95%

*Poehling KA, et al. Bedside diagnosis of influenzavirus infections in hospitalized children. Pediatrics 2002;110:83-8.

Page 71: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Review: Testing Thresholds for an Imperfect but Costless Test (Chapter 3)

Probability of Disease

Treat none; no test

Treat based on test results

Treat all; no test

No Treat-Test Threshold

Test-Treat Threshold

Treatment Threshold

Page 72: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Review: Calculate Testing Thresholds

Treatment Threshold = PTT = 0.375

Treatment Threshold Odds = 0.375/0.625 = 0.6

LR(+) = {Sens/(1-Spec)} = 0.75/(1-0.95) = 0.75/0.05 = 15

LR(-) = {(1-Sens)/Spec} = (1-0.75)/0.95 = 0.25/0.95 = 0.26

No Treat-Test Threshold Odds = 0.6/15 = 0.04

No Treat-Test Threshold Probability = 0.04/(1+0.04) = 0.04

Test-Treat Threshold Odds = 0.6/0.26 = 2.28

Test-Treat Threshold Probability = 2.28/(1+2.28) = 0.70

Page 73: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Testing Thresholds

LR+ (15) LR- (.26) |----------------------->|<----------|

+-----------|------------------------|-----------|---|----------------+ | | | | | | | | | | .01 .02 | .1 .2 .3 | .5 | .8 .9 .95 0.04 0.375 0.7

PROBABILITY

Page 74: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Interpret the Results

• If the prior probability of influenza is <4 % (the lower testing threshold), then even if the rapid antigen test is positive, the post-test probability will still be below 37.5% (the treatment threshold), and you would not treat the patient’s household contacts.

• If the prior probability is >70% (the higher testing threshold), then even if the antigen test is negative, the post-test probability will be above 37.5%, and you would treat the household contacts anyway.

Page 75: Absolute Risk Reduction, Number Needed to Treat, Back-of-the-Envelope Cost Effectiveness Analysis, Testing Thresholds Revisited 26 October 2006 Michael.

Interpret the Results

• Between 4% and 70%, the test MAY be indicated, because it has at least the potential of affecting management. Note that so far, we have not considered costs or risks of the test (as opposed to those of the treatment)-- when these are factored in as well, the testing range will be narrower.


Recommended