Using Decision Analysis Modeling to Translate Epidemiologic Research into Health Policy
Elena Losina, PhDHIV Epidemiology and Outcomes Research Program
Massachusetts General HospitalBoston, MA
Supported by NIAID R37 AI42006, R01 AI058736
Outline•
Rationale for using decision analysis modeling in HIV•
Overview of CEPAC model•
Model validation and calibration•
Examples of analyses using CEPAC model–
Life expectancy projections–
Cost-effectiveness of specific treatment strategies–
Informing design of clinical trials•
Challenges–
Need to make decisions in the absence of data•
Policy decisions in developing countries
–
Finding the ‘right’
data–
Prospective collaboration vs. retrospective secondary data analysis
Question
• How can we use epidemiologic, clinical, and economic data to understand current and project future impact and value of strategies for HIV treatment?
What should
be
the standard of care?
Abidjan
Cape Town
Chennai
Vail
Ideal Means of Evaluation
• Evaluations of strategies related to timing and sequencing of antiretroviral agents would require large-scale randomized trials with HIV-related mortality as the primary outcome
Barriers to Design of Such Trials• Feasibility:–
Size
– Recruitment
– Cost
– Time horizon
– Concern about results being outdated
•
INITIO study of initial HIV therapy was designed, enrolled, and conducted over six years. By the time results were presented in 2005, none of the original regimens were considered standard because of new evidence regarding toxicity or potency.
Role of Decision Sciences
• Synthesize data from multiple sources
• Help decision-makers understand the likely impact of different strategies and set priorities for further data acquisition
• Offer a practical framework for managing uncertainty via sensitivity and “what-if”
analyses• Permit analyses that extrapolate beyond the limitations of time horizons, geographic settings, and target populations
Goal to Convince You …
1.
Decision science and cost-effectiveness analysis are critical for informing health policy
2.
Data collected within epidemiologic studies can provide tremendous insight into policy
•
Inputs into probabilistic decision analysis model•
Means of calibration and validation of such models
HIV Clinical Policy Questions•
U.S.–
When should we start ART? Will RCT provide the answers? When?–
How do we identify patients earlier? Is routine HIV testing cost-
effective?
–
What are lifetime costs of HIV-infected patients?–
Should we be doing genotypic resistance testing on all naïve patients before starting ART? At what prevalence of resistance?
–
What is the per-person and population level impact of late initiation and premature discontinuation?
•
International –
What will be the impact on life expectancy, cost and cost-
effectiveness of ART in South Africa? Côte d’Ivoire? India?
–
Are second-line regimens cost-effective?–
When to start?
Cost Analysis
•
HIV Cost and Services Utilization Study (HCSUS) US$18,300/patient/year 1998–
Bozette, N Engl J Med 2001•
Estimated HIV care cost is $618,900 per person/2004 US$ –
Schackman, Med Care 2006, CEPAC, based on HIVRN data•
$9.2 Billion for HIV/AIDS care in low-middle income countries –
Schwartlander, Science 2001
Cost-effectiveness Analysis
• Two different outcome measures– Cost ($)– Effectiveness: Years of life saved (YLS)
QALYs• Cost-effectiveness ratio:
Additional Cost:Additional QALYs
• The value of resources spent
Who Cares?
• All about limited resources
• Need and capability exceed resources–
Individual, state, national, global
• Challenges–
Medicaid
– HDAP
– South Africa Ministry of Health
– PEPFAR
Cost-effectiveness of Preventing AIDS Complications (CEPAC) Model
•
Simulation state transition model of HIV•
Compares clinical outcomes (survival), cost- effectiveness
•
CD4 (true, observed), HIV RNA (true, observed), adherence, resistance, HIV testing
•
Antiretroviral strategies•
Prophylaxis for multiple opportunistic infections
Supported by NIAID, NIMH, NIAAA, CDC
Overview of CEPAC Capabilities
Screening/Intake Screening/Intake ModuleModule
New HIV screening New HIV screening programprogram
Detection via background HIV
Screening
CEPAC-treatment module
Undiagnosed HIV-infected patient
Detection via development of
an OI
The CEPAC Treatment Module
• CD4 cell count (current and nadir)• Resistance assays• Antiretroviral therapy (success, failure, prior regimen failure)• HIV RNA (current and setpoint)• Opportunistic infections (current and prior events) and prophylaxis• Treatment-related toxicity
Acute Clinical
Event
Death
Chronic HIVInfection
Primary HIVInfection
Methods
•
1 million simulations run - Monte Carlo
– Each “patient”
drawn randomly from initial
distribution of age, sex, CD4, RNA–
Track clinical events (e.g. PCP or TB cases)
– Life expectancy, quality-adjusted life expectancy
– Lifetime direct medical costs, total costs
•
Payer or societal perspective ($/QALY)
0
50
100
150
200
250
300
350
400
Time (years)
CD
4 co
unt (
cells
/ul)
Vira
l loa
d (c
opie
s/m
l)
CD4HVL >100,000
30-001-100,000
10,001-30,000
3,001-10,000
501-3,000
21-500
1st line ART2nd-line ART
3rd/4th line ART
10 QALYs: 8.47Total costs: $203,380
5
A Patient “Trace”: US
Bacterial pneumonia
CMV
Death
Data Domains1)
Natural history
2)
Treatment efficacy/effectiveness
3)
Cost
4)
Quality of Life
Where do the data come from?
Natural History: Data Sources•
US:–
MACS–
WIHS–
HIVRN•
Other developed countries–
France•
Developing countries–
India–
Côte d’Ivoire–
South Africa–
OECS
Natural History: Data Elements•
Incidence of specific OIs by CD4 cell count strata
•
Mortality –
by CD4 cell strata–
Acute–
Attribution of history of OI to mortality–
Non-HIV related•
Cost –
Chronic care, by CD4 cell count strata–
Cost of acute care–
Last month of life
Treatment Efficacy/Effectiveness
•
Two-dimensional–
HIV RNA–
CD4•
Time frame–
Early efficacy–
Late failures•
Collaboration with ALLRT (ACTG)•
Independent ART effect•
RCT vs. Actual clinical care
Treatment Efficacy: Time Frame
• ‘Early failure’–
Estimated directly from reported RCTs
– HIV RNA suppression
– CD4 change
• ‘Late failure’–
Assumes exponential rates
– Estimated from two time points beyond ‘initial period’
Dealing with Uncertainty
• Sensitivity analysis–
One and two-way sensitivity analyses
– Wide range of plausible values
– Thresholds for policy conclusions
Model Validation and Calibration
• Validation–
Internal
– External
• Calibration–
Flexibility of the model
– Ability to ‘match’
outcomes from cohorts and
RCTs
Model Validation and Calibration
0 0.05 0.1 0.15 0.2 0.25 0.3 0.35
bcim
bcis
fngm
fngs
tb
mlr
iso
toxo
mac
othm
othsOther SevOther Mild
MACToxo
IsosporaMalaria
TBFungal SevFungal Mild
Bacterial SevBacterial Mild
Prevalence
Data
Model
Internal Validity: ANRS 059, Abidjan
Select Analyses Using the CEPAC US Model
0%10%20%30%40%50%60%70%80%90%
100%
0 5 10 15 20 25 30 35 40
Years from treatment start
% A
live
Untreated HIV/AIDS1989 - PCP1993 - PCP/MAC1996 - PCP/MAC + ART 11998 - PCP/MAC + ART 22000 - PCP/MAC + ART 32003 - PCP/MAC + ART 4
AIDS Survival by Era
Walensky, et al. JID 2007
Life Expectancy by CD4 at ART Initiation and Number of Regimens
0
5
10
15
20
25
30
Very Late
16 14 11
2118
13
2420
14
2621
15
Late
2823
15
Early
Life
Exp
ecta
ncy
(yea
rs)
3 2 1 5 4
# of Regimens
00.10.20.30.40.50.60.70.80.9
1
0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70
Life Expectancy for HIV- and HIV+
Populations in the US
Time (Years)
Surv
ival
Pro
babi
lity
00.10.20.30.40.50.60.70.80.9
1
0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70
Life Expectancy for HIV- and HIV+
Populations in the US
Time (Years)
Surv
ival
Pro
babi
lity
44
HIV negative
00.10.20.30.40.50.60.70.80.9
1
0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70
Life Expectancy for HIV- and HIV+
Populations in the US
Time (Years)
Surv
ival
Pro
babi
lity
44
HIV negative
28
Guideline-Concordant Care
44 –
28 = 16 YLL due to HIV
Years of Life Lost
00.10.20.30.40.50.60.70.80.9
1
0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70
Life Expectancy for HIV- and HIV+
Populations in the US
Time (Years)
Surv
ival
Pro
babi
lity
44
HIV negative
28
Guideline-Concordant Care
44 –
28 = 16 YLL due to HIV
Years of Life Lost
23Actual ART Use
44 –
23 = 21 YLL actual21 –
16 = 5 YLL due tosuboptimal care
Should We Genotype Test Naïve Patients?
CE
RA
tio($
/QA
LY)
0
10,000
20,000
30,000
40,000
50,000
60,000
70,000
0 5 10 15 20 25 30 35
% Resistant
Sax, et al. CID 2005
Expanded Testing for HIV: US
StrategyDiscounted
Lifetime Costs
Discounted Life
ExpectancyC-E ratio
(US$/YLS)
Background alone
Background & one-time EIA
Background & EIA every 5 years
Background & EIA every 3 years
Background & annual EIA
32,700
33,800
37,300
38,900
41,700
254
255
256
257
257
---
36,000
50,000
63,000
100,000
Paltiel et al. NEJM 2005
Cost-effectiveness Ratios for Screening Programs
C-E ratioScreening Program
($/QALY)*
Reference
HIV screening inpatients
$38,600 Walensky, Am J Med 2005
Breast cancer screeningAnnual mammogram, women 50–69 y/o
$57,500 Salzmann Ann Intern Med 1997
Colon cancer FOBT + SIG q5y, adults 50–85 y/o $57,700 Frazier JAMA 2000
HIV screening every 3 years High risk patients
$63,000 Paltiel, N Engl
J Med 2005
Diabetes Mellitus, Type 2 Fasting plasma glucose, adults >25 y/o $70,000 CDC C-E Study Grp. JAMA 1998
*all costs adjusted to 2001 US dollars
CEPAC-International: Informing Health Policy Decisions in
Developing Countries
Cost-Effectiveness of ART in Côte d’Ivoire: Single ART Regimen Available
Strategy Discounted Lifetime Costs
Discounted
Life
Expectancy
C-E ratio (US$/YLS)
No treatment 780 31.4 ---
T/S prophylaxis onlyT/S and ART* (start n=2, stop n=1)
8101,230
32.841.4
240590
T/S and ART* (start n=1, stop n=1) 1,720 50.7 620T/S and ART* (start n=1, stop n=3)T/S and ART (CD4-guided criteria)
2,1703,420
56.869.6
8901,180
*ART start/stop decisions based on clinical criteria (no CD4 available)
Goldie et al. NEJM 2006
One-way Sensitivity Analyses
$650 $1,050 $1,450 $1,850 $2,250 $2,650 $3,050 $3,450
OD treatment cost
ART efficacy
CD4 test cost
ART cost
Routine care cost
Cost-Effectiveness ($/Years of life saved)
(Cost x0.5 – 3)
(Cost x0.5 – 4)
(Cost x0.25 – 4)
(↓
20%, ↑
20%)
Base caseGDP
Côte d’Ivoire$708
3 x GDPCôte d’Ivoire
$2,124
$650 $1,050 $1,450 $1,850 $2,250 $2,650 $3,050 $3,450
(Cost x0.5 –
4)
$1,180Côte d’Ivoire
$708
3 x GDPCôte d’Ivoire
$2,124
(Cost x0.5 –
4)
(Cost x0.5 –
2)
(Cost x0.5 –
3)
ART Strategies in India
Strategy for ART Initiation
Mean Lifetime Mean
Survival (Months)
IncrementalC-E Ratio ($/YLS)
No treatment 680 35.2 ---
Costs (2000US$)
One line of ARTNNRTI, start ART < 200/μl 1,600 59.4 450NNRTI, start ART < 350/μl 1,950 66.7 580
NNRTI, start ART < 350/μl or severe OI 2,100 69.1 800Two lines of ART
NNRTI first, ART <350/μl or severe OI 5,470 93.0 1,680PI first, ART <350/μl or severe OI 6,510 96.2 3,920
Freedberg et al. CROI 2007
Informing the Design of Clinical Trials
New Motivation to Address “When to Start”
ART
• Increased stability in ART regimens
• ‘ART effect’
may be greater at higher CD4
• SMART study suggests that AIDS and non-AIDS related events were decreased on ART• Another reason to consider earlier therapy
To ‘Trial’ or Not to ‘Trial’
•
Will results will be obsolete by the time they are available?•
Long enrollment period: are there enough patients to study?•
Is it too expensive given the information to be learned?•
Where should it be done: developed vs. developing countries?
•
Is there enough data to inform the design of a trial?• US
• Developing countries
•
ACTG-CEPAC collaboration•
ACTG 5245/HPTN 052 --
A CEPAC-International Simulation
Question
• Can a simulation model help inform the question re earlier therapy, or the design of the RCT?
Selecting Trial Primary Endpoints: ACTG/CEPAC Collaboration
• Mortality
• Mortality or TB
• Mortality or AIDS-defining event
• Mortality or AIDS
• Mortality or any opportunistic disease
Reaching Mortality EndpointsYear Hazard Ratio
South AfricaHazard RatioCôte d’Ivoire
1 0.24 0.282 0.22 0.223 0.60 0.574 1.07 0.825 0.76 0.856 0.75 0.877 0.70 0.838 0.75 0.869 0.84 0.9410 0.90 0.93
South Africa
0.000.050.100.150.200.25
0 5 10Year
Ann
ual H
azar
d
Côte d’Ivoire
0.000.050.100.150.200.25
0 5 10Year
Ann
ual H
azar
d
Deferred
Immediate
Reaching Mortality or OI Endpoints
Year Hazard RatioSouth Africa
Hazard RatioCôte d’Ivoire
1 0.57 0.532 0.38 0.383 0.73 0.734 1.02 0.925 0.90 0.906 0.86 0.867 0.76 0.828 0.72 0.789 0.75 0.8310 0.81 0.82
South Africa
0.000.050.100.150.200.25
0 5 10Year
Ann
ual H
azar
d
Côte d’Ivoire
0.000.050.100.150.200.25
0 5 10Year
Ann
ual H
azar
d
Deferred
Immediate
Reaching Mortality or OI Endpoints, Omitting TB/Severe Bacterial Events:
South Africa
0.000.050.100.150.200.25
0 5 10Year
Ann
ual H
azar
d
DeferredImmediateDeferred, no TB or Bac SevImmediate, no TB or Bac Sev
Year Hazard RatioBase case
Hazard Ratio (-) TB and Bac
Sev
1 0.57 0.422 0.38 0.363 0.73 0.754 1.02 1.005 0.90 0.846 0.86 0.807 0.76 0.758 0.72 0.749 0.75 0.82
10 0.81 0.88
Main Findings
•
Hazard ratios change over time•
Hazard ratios for immediate therapy are further from 1.0 early and closer to 1.0 later
•
Time dependence of hazard ratios is partially due to:
–
The independent “ART effect”–
Differential incidence of ‘outcomes’
high CD4 cell strata: 200-350, 350-500, >500
•
Including non-CD4-dependent, “non-HIV” events
increases the needed sample size (not shown)
Lessons Learned• Optimal timing of when to start ART is of fundamental importance
• The answer is critical to individual survival, costs of care and global health policy
• The trial feasibility is a function of –
Definition of primary endpoint
– OI incidence in different regions of the world
– Diagnostic criteria
– Prevalence and incidence of HIV
Challenges
• Data are incomplete in many areas
• Models are approximations of reality
• Tension between model complexity and data availability
How to Overcome Challenges
• Prospective collaborations vs. retrospective secondary data analysis
• Participation in the design of study instruments
• Collaborative analyses–
ALLRT
– IeDEA
Conclusions
• Critical questions at the interface of HIV clinical care, epidemiology and policy in every country in the world
• Decision analysis is a method to integrate data from many sources to help inform optimal testing and treatment strategies for HIV
CEPAC Investigators: USMGH/BWHWendy Aaronson, MPHNomita Divi, MScMariam FofanaKenneth Freedberg, MD, MScHeather Hsu Elena Losina, PhDZhigang Lu, MDPaul Sax, MDCallie ScottStacie Waldman, MPHRochelle Walensky, MD, MPHBingxia
Wang, PhD
Harvard SPHSue Goldie, MD, MPHAlethea McCormack, PhDGeorge Seage, PhDMilton Weinstein, PhD
CornellBruce Schackman, PhD
YaleA. David Paltiel, PhD
Supported by NIAID, NIMH, CDC
Lindsey WolfHong Zhang, SM
IndiaN. Kumarasamy, MBBS, PhDTim Flanigan, MDKenneth Mayer, MDAnitha Cecelia, MS
Organization of Eastern Caribbean StatesKathleen Allen-Ferdinand, MDPaul Ricketts, MDHazel Williams-Roberts, MD
CEPAC Investigators: International
Côte d’IvoireXavier Anglaret, MD, PhDEugene Messou, MDCatherine Seyler, MD, PhDSiaka
Touré, MD, MPH South AfricaGlenda Gray, MDNeil Martinson, MBBCh, MPHJames McIntyre, MBChB, MRCOGLerato Mohapi, MBBCHRobin Wood, MDLille, FranceYazdan Yazdanpanah, MD, PhD
Supported by NIAID, NIMH, CDC