Date post: | 02-Jan-2016 |
Category: |
Documents |
Upload: | laurel-morales |
View: | 34 times |
Download: | 0 times |
Contribution Analysis: An introduction
Anita MorrisonScottish Government
National Performance Framework
Evaluation in Scottish Government
• A need to revisit our practice in context of performance management and outcomes-based approach – shift balance from small scale, process evaluation
• Building a more ‘strategic’ approach to evaluation planning and delivery - multiple policies / interventions to deliver outcomes
• Ensure the right kind of evidence is available – answering
questions of effectiveness, contribution, for Spending Review 09 and beyond
Key analytical questions
• What progress are we making towards the Purpose, Purpose targets and National Outcomes?
• What contribution have our policies and programmes made?
• How cost-effective are our policies and programmes?
• What are the lessons for future policy intervention /changes in resource allocation?
Contribution Analysis
Contribution analysis
• Holy grail of attribution…“Builds a reasonably credible case about the difference the program is making.” (John Mayne)
• Good contribution analysis is, in effect, theory-based evaluation• Use of logic models and results chains strengthens programme
planning and design• Clearly sets out assumptions and external factors• Also strengthens approach to research / analytical planning
Developing in a couple of ways in Scotland: contribution of organisations; contribution of policies and programmes
Alcohol – Partner Contributions to Outcomes
Inputs
Activities
Outputs
Reach
Short-term outcomes
Intermediate outcomes
High level outcomes
NHS
Brief advice
Preventive services
Hazardous and harmful
drinkers
Service uptake &
engagement
Police
Enforcement of drink driving laws
Random breath testing
Drivers
Increased detection rate
Local authorities
Enforcement of planning
controls & licensing laws
Enforcement actions
Licensed trade
Compliance with laws
BehaviourReduced alcohol consumption levelsLess drunkenness; less drink-driving
Improved mental wellbeingReduced inequalities in healthy life expectancy
Reduced inequalities in alcohol-related deaths and hospital admissions
EnvironmentsPhysical: Reduced exposure to alcohol-related hazardsEconomic: Reduced availability/affordability of alcohol
Social: Drunkenness less attractive; sensible drinking the norm
SG, UK govts, EU
Industry regulation
Taxation,displays, promotions,
advertising
Alcohol industry
Increased priceReduced incentives
Scottish Govt
Media campaigns
Sensible drinking
messages
General public - targeted
Understanding risks, attitudes
to drinking
Vol orgs
Detox, Intensive support
Addiction services
Adults with alcohol
problems
Increased sobriety &
stability
A Basic Results Chain
7. End results 7. What is our impact on ‘ends’?
6. Practice and behavior change 6. Do we influence [behavioural] change?
5. Knowledge, attitude, skill and / or aspirations changes
5. What do people learn? Do we address their needs?
4. Reactions 4. Are clients satisfied? How do people learn about us?
3. Engagement / involvement 3. Who do we reach? Who uses / participates?
2. Activities and outputs 2. What do we offer? How do we deliver?
1. Inputs 1. How much does our program cost? ($, HR etc)
Program (Results) Chain of Events(Theory of Action) Key Questions
Source: Adapted from Claude Bennett 1979. Taken from Michael Quinn Patton, Utilization-Focused Evaluation: The New Century Text, Thousand Oaks, California, 1997, p 235.
Indirect Influence
Direct Influence
Control
WHY?
WHAT?
WHO?
HOW?
Assembling and critiquing the story
Multiple sources of evidence to accommodate; can include:
• Population based statistics• Survey data• Needs assessments• Process and outcome evaluations• Economic analysis and evaluation• Systematic reviews and other evidence synthesis• International policy and analytical work
Baseline and forward look – data /evidence gaps
How do you actually do this in practice?
• To date – few ‘live’ examples of contribution analysis applied in the UK – Health Scotland led on this
• Scottish Government early work: outcome 15, GES population target, interest on outcome 8, outcome 12
• Sessions to develop and discuss logic model (mix of analysts and policy leads)
• Baseline ‘narrative’ and performance story so far
• Analytical planning – what data and analysis will we need in the future?
Some tentative conclusions
• Outcomes focus is an opportunity to undertake longer-term, more strategic analytical work
• Needs input of policy, practice and analytical professionals
• Contribution analysis not a panacea – but a tool to help think and discuss policy and service development differently
• Does help you to plan any research and analysis required in assessing progress
Over to you….
• Are you using this, or components of this, already?
• Can this be applied to outcomes / indicators in Communities?
• Or to larger programmes you are evaluating?