Eupha 2.lessons learned from published economic evaluations by_mathijsvandenberg

transcript

1 11 November 2010

Lessons learned from published economic evaluations in public health

Matthijs van den Berg

Johan Polder

Ardine de Wit

2 Lessons learned from published economic evaluations in public health | 11 November 2010

Content

1. Introduction

2. RIVM project

3. Cost-effectiveness of public health interventions

4. Critical issues in public health economic evaluations

Lessons learned from published economic evaluations in public health | 11 November 2010

Cost-effectiveness is quite the thing

● Science

– Exponential increase in cost-effectiveness studies

– Both absolute and relative

● Practice

● Policy

Exponential increase cost-effectiveness studies

1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005

Exponential increase proportion c-e studies

1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005

● Science

● Practice

– Dutch manual for evidence-based guideline development:

› Cost-effectiveness should be one of the ‘other considerations’that should be taken into account before coming to recommendations (CBO, 2007)

– Guidelines more frequently use cost-effectiveness considerations

› Cardiovascular risk management

● Policy

Dutch guideline ‘Cardiovascular risk management’

● Guideline:

– CVD 10 year mortality risk > 5%: lifestyle advice

– CVD 10 year mortality risk >10%: pharmacological treatment

● Recommendations are based on analyses of cost-effectiveness and budget impact

● The threshold for cost-effectiveness is set on € 20.000 per QALY gained, which corresponds with the value of ƒ 40.000 in the 1998-guideline, corrected for inflation

● Science

● Practice

● Policy

– Dutch policy document on public health: ‘Being healthy, and staying healthy; a vision of health and prevention’

– Health care insurance board: judgement framework for reimbursement decisions on the basic health care package

● One of the main principles in public health policy making

– “Forms of preventive intervention that do not work will be stopped,…

– and promising forms of intervention will be investigated to establish how (cost-)effective they are....

– Forms of intervention that are known to be (cost-)effective should, in principle, be funded through the appropriate insurance systems, subject to the usual budgetary constraints.”

● Dutch Health Insurance Board uses a judgement framework for which interventions to cover, consisting of 4 criteria:

– Necessity

– Effectiveness

– Cost-effectiveness

– Feasibility

● Health Council of the Netherlands uses frameworks for decisions on new vaccinations and screenings

– Cost-effectiveness is a main assessment criterion in both frameworks

Take-home message

● Cost-effectiveness is hot, especially in public health, but don’t take the result of a cost-effectiveness analysis (i.e. an ICER) for granted

● Read cost-effectiveness analyses of preventive interventions very carefully and ask yourself questions like:

– Are the assumptions on effectiveness well-founded and realistic?

– Which costs are included and which are ignored?

– What are critical choices in the model that was used?

RIVM project ‘Cost-effectiveness of prevention’

● Aim of the project

– To identify potentially cost-effective preventive interventions that have not yet been systematically implemented in the Netherlands.

– i.e. to find opportunities for public health policy

● Basic idea of the project:

– Systematic and frequent literature search

– Read and appraise the cost-effectiveness analyses

– Write a report or a paper

Number and type of prevention-CEAs

● In 2008 we found over 230 papers studying the cost-effectiveness of a preventive intervention or program

● About half of the included studies used QALY’s of DALY’s as outcome measure

What diseases do prevention-CEAs focus on?

– Infectious diseases: 32%

– Cancers: 21%

– Cardiovascular diseases: 10%

– Mental- and behaviour diseases: 7%

– Musculoskeletal system: 7%

– Rest 23%

● Vaccination gets disproportionally much attention in cost-effectiveness analyses relative to the burden of infectious diseases

● Mental- and behavioural diseases cause large burden of disease, while they were the subject of only 7% of included studies

How cost-effective were the studied interventions?

>80% lower than 50,000/(QA)LY

Median: €12,000/(QA)LY

● What do you conclude from this chart?

How cost-effective were the studied interventions?

● Almost all preventive interventions have favourable cost-effectiveness ratios

– Is prevention always cost-effective?

– Is there a publication bias?

– Do researchers work up to favourable ICERs?

Some critical comments

● Based on observations in economic evaluations of public health interventions

● Three categories

– Costs (numerator of the ICER)

– Effects (denominator of the ICER)

– Models

● Which costs are included?

– Health care perspective (only health care costs and health benefits) or societal perspective (all costs and all benefits)

– The societal perspective is preferred

– Public health measures have impact outside the healthcare sector

– Most CEA’s on prevention use the health care perspective, ignoring very relevant cost categories

Effects

● Is the intervention scenario realistic?

– What is the reach of the intervention? What participation rate is assumed?

– Case: colorectal cancer screening using colonoscopy

•Unpleasant bowel

preparation

•Uncomfortable

proceduce

•Risk of bowel

perforation

Participation rate

● Is the intervention scenario realistic?

– What is the reach of the intervention? What participation rate is assumed?

– Case: colorectal cancer screening using colonoscopy

– How many people would participate in such a screening?

Participation rate

● Systematic review of cost-effectiveness analyses of colonoscopy screening

– Model assumptions on uptake: ~85%

– Real life data for participation: ~30%

● Conclusion:

– In many CEAs model assumptions on screening uptake were more positive than real life data suggest

Adherence

● Systematic review of medication CEAs found that many studies used adherence rates from clinical trials which notoriously overestimate adherence found in the general population

● However in a small subset of studies that varied adherence in sensitivity analyses, the impact on the cost-effectiveness ratio was substantial

● With policymakers increasingly turning to these studies for guidance, failure to account for nonadherence may lead to suboptimal resource allocation strategies

Uncertainty in effectiveness

● “The aim of this paper was to determine the cost-effectiveness of a Dutch school-based smoking education program.

● The incremental cost-effectiveness ratio of the school program was estimated at €19,900 per quality adjusted life year gained.

● Main problem in estimating the cost-effectiveness was the lack of proper effectiveness data on daily smokers among adolescents.”

Long term effects

● “The range of effectiveness of the vaccine was considered between30% en 90% with a positive baseline at 80%.”

● “The costs of the vaccine was assumed to be $450 for three doses and ranged from $300 to $2000.”

● “Vaccination was found to be potentially cost-effective with an ICER of -$2384 relative to standard treatment.”

› However, such a vaccine is not available at the moment

› This is a purely hypothetical excercise

Models

●Time horizon

–Many modelling studies on the cost-effectiveness of preventive interventions have very long time horizons (E.g. 80 yrs, 100 yrs, lifetime)

–Necessary to capture all health benefits (e.g. stop smoking interventions and the prevention of lung cancer)

–Shorter time horizons would make the ICERs of many preventive interventions very unfavourable

Models

●Discount rates

– Most international CEAs use same discount rate for both costs and effects (e.g. 4% or even 6%)

– Dutch guidelines prescribe different discount rates for costs and effects (4% and 1.5%)

– This has a large influence on the ICER, making prevention a lot more cost-effective

●Case: cost-effectiveness of HPV vaccination

– Dozens of recently published CEAs

– Including several Dutch model studies

– Most of these conclude favourable cost-effectiveness

– However, de Kok et al., 2009 does not

De Kok et al.: “In conclusion, many uncertainties still exist about the effects of HPV vaccination on HPV-related diseases. Our cost-effectiveness analysis shows that in the Netherlands, a country with low cervical cancer incidence and mortality, HPV vaccination is not cost-effective […].”

Coupé et al.: “In a sensitivity analysis, de Kok et al. reported an ICER of € 19 700 per QALY at discount rates of 4% for costs and 1.5% for effects. The difference in reported ICERs betweenthe two studies can thus be largely explained by the discount rates that were used.

Postma: “I conclude that the conclusion by de Kok et al. is misleading, should be re-visited and would probably better be formulated as “In the Netherlands, HPV vaccination is likely to be cost-effective if compared with screening alone”, […]”

●Moreover

– Assumptions concerning participation rate in HPV vaccination were about 85%

– While the real participation rate in the Netherlands appeared to be about 50%

– In what way would that influence the cost-effectiveness ratio?

Another example: rotavirus vaccination

● “The intervention had a gross incremental cost-effectiveness ratio of AUD$ 3.70 (95% uncertainty interval (UI) $2.40, $7.70) per DALY.”

● “When the present value of potential savings in future health-care costs was considered, the intervention was ‘dominant’ […].”

● “Restricting televised advertisements targeting children couldpotentially be one of the most cost-effective population-basedobesity prevention interventions available to governmentstoday.The ICER was most sensitive to effectiveness assumptions.”

● “Limited evidence of the effectiveness of this intervention was a keyconcern”

● “The most relevant study to assess the effectiveness of a reductionin advertising was a randomized controlled trial that compared the impact of exposure to different food advertisements on food and beverage selections in 288 5- to 8-year old children on holidaycamp for a period of 2 weeks in Quebec in 1982.

● The children were exposed to sweet commercials, no commercials, fruit commercials or nutritional public service announcements. Children who viewed sweet commercials chose (and ate) significantly more sweets over fruit as snacks compared withchildren in the other three groups (75 versus 67%, 64 and 65%, respectively).”

● “Whether te reduction in BMI would be maintained over the lifetimeof the child is unknown and difficult to predict.”

Conclusions

●Don’t take the result of a cost-effectiveness analysis (i.e. an ICER) for granted

●Please read cost-effectiveness analyses of preventive interventions very carefully

– Are the effect assumptions realistic?

– Are all relevant costs included?

– What are critical model choices?

Eupha 2.lessons learned from published economic evaluations by_mathijsvandenberg

Documents