Contribution analysis

Anita Morrison and Jackie Horne, Office of the Chief Researcher,

Scottish Government

Contribution analysis, results chains, logic modelling….

It’s all about outcomes

• Clarify the language – activities, inputs, outputs and outcomes!

• Examine how we can map interventions to outcomes (both singly and multiply)

• Examine how we can assess progress and performance towards outcomes

• Get working on some useful stuff!

National Performance Framework

Contribution analysis

• Holy grail of attribution…

“Builds a reasonably credible case about the difference the program is making.” (John Mayne)

• Good contribution analysis is, in effect, theory-based evaluation• Use of logic models and results chains strengthens programme

planning , design and review• Clearly sets out assumptions and external factors• Also strengthens approach to research / analytical planning

The Six Steps…..

1: Set out the attribution problem to be addressed2: Develop a theory of change and ‘model’ or ‘map’ it (including

external factors and underlying assumptions)3: Assemble the contribution story 4: Assess the performance story5: Seek out additional evidence6: Revise and strengthen the contribution story

Understanding what we mean by outcomes

Not straightforward…..

Distinguishing between inputs, activities,outputs and outcomes…………

Understanding the relationship between policies and outcomes –

single intervention

A simple results chain – example - exercise classes for over 65s and HLE

Inputs

Activities (policies)

Outputs

Reach

Short term outcomes

Intermediate outcomes

High level outcomes

Over 65s, especially those not doing enough exercise

Staff, equipment, leisure centres, etc.

Invest in exercise facilities and classes for over 65s

More accessible exercise classes and facilities for over 65s.

Increased levels of physical activity in over 65s

Lower blood pressure, improved mobility among over 65s

Increase in healthy life expectancy

Results chain-Twilight basketball funded through Cashback

National Outcome

We live our lives safe from crime, disorder and danger (NO9)

Intermediate Outcomes

•Less anti-social behaviour•Less time spent bored and on streets•Constructive use of leisure time

Reach 16-21 year olds in areas hardest hit by crime

Output Coaching, games, leagues, role models, education during time-out

Activities Twilight basketball (funded through Cashback)

Inputs Staff, funding, etc.

Results chain-Twilight basketball funded through Cashback

National Outcome

Our young people are successful learners, confident individuals, effective contributors and responsible citizens (NO 4)

Intermediate Outcomes

•Aspirations and confidence increase•Young people recognise that they have more opportunities•Young people learn a new skill

Reach 16-21 year olds in areas hardest hit by crime

Output Coaching, games, leagues, role models, education during time-out

Activities Twilight basketball (funded through Cashback)

Inputs Staff, funding, etc.

National outcome We live our lives safe from crime, disorder and danger (NO 9)

Our young people are successful learners, confident individuals, effective contributors and responsible citizens. (NO 4)

Intermediate outcomes

→Less antisocial behaviour.

→ Less time spent bored and on streets.

→ Constructive use of leisure time.

→ Aspirations and confidence increase.

→ Young people recognise that they have more opportunities.

→ Young people learn a new skill.

Reach 16 – 21 year old in areas hardest hit by crime.

Output Coaching, games, leagues, role models, education during time out.

Activities Twilight basket ball (funded through Cashback)

Inputs

A simple results chain

National outcome/purpose target

Intermediate outcomes

→

→

→

→

→

→

Reach

Output

Activities

Inputs

Multiple policies - logic models and

contribution analysis

• Move beyond results chains to consider inter-linkages between policies and outcomes

• Consider strength (and weaknesses) of the contribution of different policies to outcomes

• Assess and measure progress towards purpose targets and national outcomes

• Inform cost-effectiveness and future resource allocation

• Highlight strengths and weaknesses in evidence base

Alcohol – Partner Contributions to Outcomes

Inputs

Activities

Outputs

Reach

Short-term outcomes

Intermediate outcomes

High level outcomes

NHS

Brief advice

Preventive services

Hazardous and harmful

drinkers

Service uptake &

engagement

Police

Enforcement of drink driving laws

Random breath testing

Drivers

Increased detection rate

Local authorities

Enforcement of planning

controls & licensing laws

Enforcement actions

Licensed trade

Compliance with laws

BehaviourReduced alcohol consumption levelsLess drunkenness; less drink-driving

Improved mental wellbeingReduced inequalities in healthy life expectancy

Reduced inequalities in alcohol-related deaths and hospital admissions

EnvironmentsPhysical: Reduced exposure to alcohol-related hazardsEconomic: Reduced availability/affordability of alcohol

Social: Drunkenness less attractive; sensible drinking the norm

SG, UK govts, EU

Industry regulation

Taxation,displays, promotions,

advertising

Alcohol industry

Increased priceReduced incentives

Scottish Govt

Media campaigns

Sensible drinking

messages

General public - targeted

Understanding risks, attitudes

to drinking

Vol orgs

Detox, Intensive support

Addiction services

Adults with alcohol

problems

Increased sobriety &

stability

The actual and potential uses of this work? Is this the same for you?

• Generating a shared understanding of an outcome; and what government interventions might be best to deliver it

• Organisational capacity building in ‘outcomes working’

• Systematically assessing our performance /progress

• Reporting to our Board on progress against outcomes

• A basis for business planning and business improvement

• Possibly contributing to outcomes budgeting processes

Producing a logic model…

Contact details

Anita Morrison

anita.morrison@scotland.gsi.gov.uk

0131 244 6530

Jackie Horne

jackie.horne@scotland.gsi.gov.uk

0131 244 4816