Date post: | 31-Mar-2015 |
Category: |
Documents |
Upload: | raymond-leavey |
View: | 219 times |
Download: | 2 times |
Evidence-Based Public Health: A Course in Chronic Disease
Prevention
MODULE 7: Economic Evaluation
Kathy Gillespie
March 2013
“An economist is a man who states the obvious in terms of the incomprehensible.”
Alfred A. Knopf
2
Learning Objectives
Be able to answer the following questions:
1. Why is economic evaluation needed?
2. What is economic evaluation? Are there different types?
3. When is an economic evaluation necessary or useful?
4
Learning Objectives
More questions…
4. How is an economic evaluation done?
5. Who can help me with an economic evaluation?
6. Where can I find information about economic evaluations?
5
Where This Module Fits
6
The green and blue sections, primarily
Next Module: Logic Model
EE helps with costs and objectives
7
Why?
Why EE?
• Evidence-based practice requires knowing – Does it work?– At what cost?
• EE starts after efficacy has been determined.
Want to promote interventions that work at a reasonable cost, i.e. that provide “good value for the money”
9
Examples
• Increasing Physical Activity: Evidence shows that the following will work (The Community Guide)– Community wide campaigns– School-based physical education– Street-scale urban design and land use
policies and practices– Social support interventions in community
settings
• Which to invest in? or a mix?10
Examples, cont.
• Budget cuts: A department’s budget must be cut by 5%. How to do this so as to minimize the effect on the public’s health?
• Grant awards: A number of applications have been received for a limited pool of grant dollars. How to select the winners?
11
Need to Weigh Costs and Benefits
• A scale can compare apples and oranges because both are measured by weight
• EE provides the scale – an agreed upon way to measure health benefits and costs
12
What?
What Is EE?
• EE examines the costs and benefits of alternative programs to inform decisions about the allocation of scarce resources using an established set of economic tools.
• There are four types of EE.
14
Types of Economic Evaluation• Cost-benefit (CBA)• Cost-utility (CUA)• Cost-effectiveness (CEA• Cost-minimization
The number of projects that can be compared declines as we move down the list.
15
EE and Business
• EE methods are closely related to several business methods– Return on Investment, or ROI– Make or buy decision– A business plan
• Investment decision making
16
It Helps to Have a Map
• What is the intervention?
• Who is the relevant group?
• How long should the group be followed?
• What can happen at each time period?
• DECISION ANALYSIS can provide the map.
17
Example Decision Tree
18Dowding D and Thompson C, Using decision analysis to integrate evidenceinto decision making, Evid Based Nursing, 2009 12: 102-104.
Components of Economic Evaluation in Public Health
19
Public Health
Program
EconomicInputs
HealthOutputs
CostsDirect CostsIndirect CostsAverted Treatment Costs
BenefitsYOLSQALYSDollars
Always compare a new program to some alternative.
Components of Economic Evaluation in Public Health
Costs A
New Program A Benefits A
Choice
Costs B Program B•Comparison program•May be new or old•Could be ‘doing nothing’
20
Benefits B
YOLSQALYSDollars
DirectIndirectAverted Treatment
Costs
The Product of an EE
21
Incremental Costs
EE ratio =
IncrementalBenefits
The EE Ratio
• The EE ratio is often called the Incremental Cost Effectiveness Ratio, or
ICER• Emphasizes that we are comparing 2 or
more interventions
22
Dimensions of Economic Evaluation
23
Type Inputs Outputs
CMA
ROI
CBA
CEA
CUA
Dollars
Dollars
Dollars
Dollars
Dollars
Natural Units Organization
Dollars Organization
Dollars Society
Natural Units Society
Utilities Society
Perspective
Every EE is Conducted from a Particular Perspective
• Perspective (or viewpoint) determines whose interests are paramount in the evaluation
• Perspective determines what costs and consequences are considered
• Perspective determines how the results are interpreted
24
Determining the Perspective• Single provider’s practice• FQHC• Healthcare system• Insurer – public (Medicaid) or private
(BCBS)• State agency• Federal program (CDC, Medicare)• Society
25
Example: Increasing Physical Activity
• Community wide campaigns
• Suppose a campaign would cost $150,000 and yield an improvement of 1,000 quality-adjusted life years
• ICER = $150/QALY
• Social support interventions in community settings
• Suppose this would cost $200,000 and yield 2,000 quality-adjusted life years
• ICER = $100/QALY
26
Some Assumptions of Economics
• Resources are scarce.• Human wants are unlimited, and more is
(almost) always preferred to less.• If there is perfect competition, the market
will do an efficient job of allocating scarce resources to maximize profits and satisfaction. Note: society may not consider this outcome equitable.
27
Some Assumptions of Economics
• If the market works well, then there are only 2 parties to each trade – a buyer and a seller – and they bear all the costs and reap all the benefits of the trade.
28
When?
When Is EE Needed?
• EE needed if there is market failure (inefficiency).– Perfect competition not present.– Could occur naturally or the market could be
‘broken’.
• EE helpful if there are concerns about inequity.
30
Inefficiency Reasons for EE • When the market fails, or is broken.
– Complex decisions with high stakes.– Little or no price information available.– High information costs to find out about the
product.• Markets involving externalities or public
goods.– Externalities are spillover effects from trades.– Public goods are non-excludable.
31
When is EE needed? Really?
• Practically speaking, may be mandated or requested by funding agency.
• Useful when comparing options internally.• Useful when justifying programs externally.
32
Inequity Reasons for EE
• Social or government decision-making processes.– If the distribution of goods and services that
would emerge from the market is considered inequitable, then society can legislate, regulate, or tax/subsidize.
– EE measures the current or desired outcomes.
33
Public Health Requires EE
• Provides services when the market fails– WIC program, health clinics
• Complex decisions, high stakes - health and life
• Provides services with external benefits– Immunization programs, sanitation,
inspections– Tobacco control programs
34
Public Health Requires EE
• Provides public goods and services– Health monitoring and assessment– Social marketing and public information
campaigns
• Access to health and equity are often concerns
35
How?
The Product of an EE
37
Incremental Costs
EE ratio =
IncrementalBenefits
Quadrant IV Quadrant I
Quadrant IIQuadrant III
Saves money,Improves health
Costs money,Improves health
Costs money,Worsens health
Saves money,Worsens health
Aggregate Costs
Aggregate Health Benefits
Possible Outcomes of an EE
38
Measuring Costs
• All economic evaluation methods require listing and measurement of costs.
• One tricky point.– Mathematically, a benefit can be a negative
cost, and a cost can be a negative benefit.
39
Negative Costs and Benefits
• Whether some items are negative costs or benefits depends on the method.
• Makes a difference because we are constructing a ratio.
• General idea: all items falling on the health system are costs, positive or negative.
40
Types of Costs
• Direct (or program) costs • Indirect costs
– Time and travel costs to participants– Averted productivity losses (a negative cost)?– Cost of treatment during gained life
expectancy ?• Averted treatment costs (a negative cost)
41
Direct Costs
All costs of the intervention• Labor• Supplies• Rent• Utilities• Costs of treating side effects of the
intervention, if any
42
Indirect Costs
Also called spillover costs.• Time and travel costs to participants.• Costs to parties outside controlling program or
agency.• Averted productivity losses (a negative cost)?• Cost of treatment during gained life expectancy?
43
Averted Productivity Losses
• The present value of future wages gained.– The intervention increases the worker’s
longevity and/or reduces disability.
• Used in cost-benefit analyses.• Not included in cost-utility analyses.
– Double counting of the benefit.
44
Costs of Treatment During Gained Life Expectancy
• If the intervention saves a life, that person will die of something else later on. What are his/her expected total medical expenditures in the added life years? A controversial cost.
• Pro: these are expenses that would be avoided in the absence of the intervention, and therefore should be counted.
• Con: health expenses should not be singled out for inclusion.
45
Averted Treatment Costs
• The averted costs to society of treating persons for the disease are subtracted from total costs.
• Included in the cost side because they impact the health care budget.– A negative cost, not a benefit.
46
Measuring Benefits
• Benefits can be measured in many ways• Different units of measurement
– Dollars– Years of life saved– Quality adjusted years of life gained– Specific health outcome
• Different time frames for the outcome– Intermediate outcome– Final health outcomes
47
Measuring Benefits
Prevention poses special problems:• Benefits may be:
– small and cumulative.– complex and interrelated.
• Are difficult to measure or validate without large samples of panel data observed over long time periods.
• Consequently, often focus on intermediate outcomes.
48
Measuring Benefits
Prevention poses special problems:• Aim is often to change behavior among
more people.– Increases costs in the short term.
• Community versus individual approach.– Economies of scale possible.
• Provide walking trails; tax gasoline.
– Individual interventions often more costly.
49
How to Put it Together?
• Have costs and benefits, how to put them together?
• Depends on the type of EE.
50
DefinitionCost-minimization Analysis
51
• Populations served must be identical
• Outputs for all alternatives must be identical
• Timeframes must be the same• Search for alternative that yields
the lowest total cost
CMA Example
52
• Deliver dental sealants to school children• Choose staffing that minimizes program
cost per studentCost per Student
0
5
10
15
20
25
30
1 2 3 4
Supervision Level
Co
st
No supervision General supervisionIndirect supervisionDirect supervision
Scherrer, CR, et al, Public Health Sealant Delivery Programs, Medical Decision Making, Nov-Dec 2007
DefinitionReturn on Investment (ROI) or Business Case
• Answers question: Will an activity pay for itself?
• Analysis of net discounted cash flow to entity paying for an intervention• Typically short-term (1-5 years)• Perspective is key to determining costs
and consequences to include in analysis
53
Why develop a business case?
• A positive business case increases the probability that a prevention intervention will be sustained
• ROI analyses are most easily understood by administrators and budget officers
54
Brownson CA and Kilpatrick KE, Building a Business Case for Diabetes Self Management: A Handbook for Program Managers, Robert Wood Johnson Foundation, 2008 (available at www.diabetesinitiative.org/resources)
Return on Investment Example
55
• CareOregon (Portland): initiated case management for high-risk Medicaid enrollees with multiple comorbidities
• Study design: before-after design compared to baseline
• Investment costs: $526,290• Discounted savings: $6,423,776• Return on investment: 12.21: 1
Greene SB, et al, Searching for a business case for quality in Medicaid managed care, Health Care Management Review, 2008, 33(4), 350-360.
DefinitionCost-benefit Analysis
• Cost-benefit analysis values both the costs and benefits of a program, project, or treatment in monetary terms.
• Result of analysis– The net benefit of the project (e.g. benefits
minus costs, or $12,000) OR– The ratio of costs divided by benefits (e.g. ½).
56
Cost Benefit Example• Intervention: Neighborhood-based
program to prevent teen pregnancy
• Program costs: $9,386 per participant per year
• Effects: reduced teen pregnancy from 94/1000 to 40/1000
• Cost: $26,142 per birth averted
• Saved: $81,256 society costs/birth averted
57
Rosenthal MS, et al., Economic Evaluation of a Comprehensive Teenage Pregnancy Prevention Program, Am J Prev Med 2009;37(6s1)
Example ICBR
58
ICB ratio = $26,142 = .322
$81,256
Smaller ratio is better
Cost-benefit Analysis
• Gold Standard for EE.• Allows for economic comparison of widely
disparate publicly funded programs in such areas as health, education, and the environment.
• Problem: valuing a life in monetary terms.
59
DefinitionCost-effectiveness AnalysisCost-effectiveness analysis measures the
benefits of a program in naturally occurring health units, such as lives saved.
Example of a study result: $10,000 per life saved.
60
CEA Example• Intervention: Smoking cessation
program in the workplace
• Effect measured: Number of people who quit smoking
• ICER = $596 cost per additional quitter
• This cost was less than “high intensity” interventions by clinicians
61
Tanaka H et al. Effectiveness of low-intensity intra-workplace intervention on smoking cessation in Japanese employees: a three year interventionTrial, J. Occup Health 2006; 48(3):175-82.
Cost-effectiveness Analysis
• CEA formerly most common form of EE conducted in health arena.
• Limited in its ability to report outcomes.– Often, there are multiple outcomes.
• Limited in its ability to compare interventions.
62
Three Projects Example
• A nursing program for newborns and their parents costs $50,000 per year and serves 50 infants with high-risk conditions
• A screening program that visits local malls and community centers costs $15,000 and provides information to about 5,000 persons
• A vaccination program costs $100,000 and provides vaccinations to 20,000 area children
63
Three Projects Example and CEA
• Nurses for infants
• Cost-effectiveness ratio is $50,000/50 infants or $1,000 per infant
• Screening program
• Cost-effectiveness ratio is $15,000/5000 or $3 per attendee
64
• Vaccination program
• Cost-effectiveness ratio is $100,000/20,000 or $5 per child
DefinitionCost-utility Analysis
Cost-utility analysis compares the costs of different programs, projects, or treatments with their outcomes measured in “utility based units” which are related to a person’s health related quality of life.
Example of a study result: $10,000 per quality-adjusted life year, or $10,000/QALY.
65
DefinitionCost-utility Analysis• Becoming the most common form of
analysis.• Widely used in Britain and Canada.• Allows comparison of many projects with
health-related outcomes.• Often called cost-effectiveness analysis;
closely related.
66
CUA Formula
(direct costs+ indirect costs- averted treatment costs)
67
Quality adjustedlife years
Output of CUA is ‘cost per QALY’
=
Cost - utility ratio
Cost-utility Outcomes
• Outputs are measured in terms of a preference-based outcome measure.– Quality Adjusted Life Years (QALYS): the
number of years at full health that would be valued equivalently to a given number of years of life experienced with a disease or disability.
– Other measures are available, this is the most common.
68
Why Measure Quality of Life?
• Health care outcomes are multidimensional.- Length of life, or mortality.
- Quality of life, or morbidity.
• Allows for more than one disease or health problem to be compared.
• Considers the individual’s preference for health outcomes.
69
How QALYS Are Measured
• One year of life in excellent health is assigned a value of 1; death is given a value of 0.
• A health state is described and its utility or quality elicited.
• The value assigned to quality of life is referred to as health utility.
70
0 1.5
Diabetes (Diet & Exercise)
Diabetes (Oral Agent)
Diabetes with Neuropathy
Diabetes with Neuro and High BP
Diabetes with Neuro, High BP, and Stroke
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
0.69
0.67
0.60
0.59
0.52
An example of estimated health utility values for diabetes
Coffey et al. 2002 Diabetes Care 71
QALYs Gained from an intervention
death without the program
death with the program
without program
with program
He
alt
h r
ela
ted
qu
alit
y o
f lif
e
optimal health
Duration (years)0
1
72
Problems With QALYs
• Subjective and difficult to measure.
• Whose QALYs should count? A representative sample of the population or the affected group?
• QALY scales will differ depending upon factors such as age, gender
73
Stop Smoking Example
• After considering your community and its public health problems, your organization has identified smoking as a problem you can address.
• Two interventions are being considered; only one can be done.
74
Stop Smoking Example Using CUA
• Multicomponent interventions that include patient telephone support
• Suppose this would cost $150,000 and yield 1,000 quality-adjusted life years
• Healthcare provider reminder system
• Suppose this would cost $200,000 and yield 2,000 quality-adjusted life years
75
ICER for B vs. A = $50,000/1000 QALYs or $50/QALY
Cost Utility Example• Intervention: Diabetes self-management
programs in primary care settings
• Program costs: $866 per participant per year
• Effects: 87.5% benefited, A1c -.5%, total cholesterol. -10%
• ICER: $39,563/QALY saved
76
Brownson CA, et al., Cost-effectiveness of Diabetes Self-managementPrograms in Community Primary Care Settings, Diabetes Ed, v. 35, no.5, 2009
How Do I Know If It’s Cost-Effective?
• Are we almost there yet?
• Once I have the ratio, how do I know if it’s too high, too low, or just right?
77
Is It Worth It?
• Results can be used internally or externally– To rank programs internally– To argue for external support
• Intermediate results, such as productivity gains, can be highlighted for some stakeholders
78
Using Results Internally
• For internal use– Rank options from lowest to highest ratio.– Start spending on lowest ratio, move on until
the money is exhausted– What have we spent before?
79
Using Results Externally
• The ICER is compared to a threshold value
• Suggested U.S. threshold is $50,000 to $100,000 per QALY at minimum
80
Exercise
81
Two Other Important Features
• Before study is complete, should consider– Discounting– Sensitivity analysis
82
Discounting
• Time value of money.– A dollar in the future will be worth less than a
dollar in the present.• Needed to compare present value and future
value of benefits from project.• Recommend a discount rate of 3-5%.• Discount rate chosen can affect results.
83
Discounting in Practice
• Suppose you want to find the present value of $100 received in 10 years. Several options for finding this.
• Tables in finance and accounting books.• Excel or other spreadsheet programs.• The internet: Google “present value
calculator” and several pop up.
84
Sensitivity Analysis
• EE is based on estimates and assumptions – want to vary them and see how robust the results are.
• Variables to test in sensitivity analysis should include the “top 3” or “top 5” “wobbliest” assumptions.
• Analysis should be redone varying the assumptions.
85
EE Results Reporting
• The ICER(s) should be reported for the intervention(s) studied for the base case.
• ICERs may be reported for subgroups of the population.
• ICERs should be reported for different assumptions (sensitivity analysis).
86
How is This Done in Practice?
• Back of the envelope EE– Draw a decision tree, follow the costs and
benefits, form the ratio– No economist required
• Primary data collection– Often alongside an intervention– Collect cost and benefit data– Economist works with team
87
How is This Done in Practice?
• Synthetic EE– Follow a hypothetical cohort of persons– Use the literature to obtain cost and benefit
estimates– Rely on modeling and computer analysis– Economist leads the team
88
Who?
Who Does EE?
• Some economists are trained to do EE.• Most EEs require a multi-disciplinary team.• Few public health agencies have the
resources to have a staff economist – so think collaboratively – local colleges.
90
Where?
Web Resources
• http://www.thecommunityguide.org/econ/default.htm The economics section of The Community Guide, it includes systematic reviews of EEs for recommended activities.
• http://www.tufts-nemc.org/cearegistry/ The Cost Effectiveness Analysis Registry at Tufts – New England Medical Center. Includes EE results and QALY estimates for numerous conditions.
92
Web Resources
• http://www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME The Cochrane library, a reliable source of information on the effects of interventions in health care. Economic evaluations are available at
• http://www.mrw.interscience.wiley.com/cochrane/cochrane_cleed_articles_fs.html or by clicking from the web site above.
93
Web Resources
• http://www.york.ac.uk/inst/crd/ The Centre for Reviews and Dissemination at the University of York. Includes several searchable databases; of particular interest is the NHS Economic Evaluation Database (NHS EED), which provides article summaries, similar to the Community Guide.
• http://www.nice.org.uk The National Institute for Clinical Excellence. A more general database (the prior website is reachable through this site, for example) it includes RSS feed capabilities.
94
NHS – EED
• Provides summaries of EE articles– Follows a standard format– Summary is about 2 pages or less– Can then click on full report, which may be
several pages
• Easily searchable– Good accessible instructions for searching
95
Sample NHS EED Results
Topic Hits
Smoking cessation 146
Diabetes prevention 279
Obesity 208
Obesity prevention 40
Physical activity 106
“Physical activity” 44
96
Reference Books
• Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL. Methods for the economic evaluation of health care programmes. Third edition. Oxford: Oxford University Press; 2005.
• Muennig P (Contributing Editor, Kahn K). Designing and Conducting Cost-Effectiveness Analysis in Health and Medicine. San Francisco: Jossey-Bass, 2002. (2nd edition forthcoming).
97
Search Tips
• General strategy: search for the intervention of interest AND economic evaluation
• Often easier to run 2 searches – 1 for intervention and 1 for economic evaluation, and then combine them
98
Search Tips
• If you want any type of economic evaluation, search for ‘cost-benefit’ OR ‘cost effectiveness’ OR ‘cost utility’– “cost-effective” is a buzzword that will net a lot
of articles!• Articles in intervention journals will be
more verbose on the EE and terse on the intervention; articles in economic journals will be the reverse.
99
Economic Journals With an EE Focus
• Health Economics• Journal of Health Economics• Cost Effectiveness and Resource
Allocation• Health Technology Assessment• Applied Health Economics and Health
Policy• Value in Health
100
Summary
• Economic evaluation is the comparison of costs and benefits to determine the most efficient allocation of scarce resources.
• Economic evaluations can use existing or new information and can provide a reliable tool for decision making among public health professionals and policy makers.
• Though relatively sophisticated, the underlying logic and structure of an economic evaluation can be understood.
101
Summary
• Several challenges (e.g., inconsistent quality, methodological issues, difficulties in implementation) should be kept in mind when considering the use of economic evaluations.
• Economic evaluation will be increasingly used, especially in times of limited public health resources, and practitioners must be able to understand them so that they can argue for setting appropriate public health priorities.
102
“It is our choices … that show what we truly are, far more than our abilities.”
J.K. Rowling, Harry Potter and The Chamber of Secrets, 1999
103