SOCIAL IMPACT MEASUREMENT

Telling a story of what you do and why

it matters

Count or proportion of social change

occurring

Using data to make decisions

Demonstrating change is caused by

an intervention

Performance management:

continuous evaluation and improvement

(Tris Lumley, NPC, April 2014)

PEOPLEConstituent Voice

ORGSManagerial / administrative data

FIELDImpact evidence

What are your priorities / needs / desires?

Survey Panel

What do we know?

Field-level evidence /

‘what works’

What should wedo?

Programmedesign

How are wedoing? What should wechange? Performance

management

How can we serve you better?

How are wedoing compared to others? Shared

Measurement / Benchmarks

DataLabs

How should wecollaborate? Field-level

collaborationPooled funding

Recommendedpractices

What should ourpriorities be?

How are wedoing collectively? What do we need to change?What have we learned that will improve services? What works for whom, in what circumstances?

ImpactEvaluation

How can we serve you better next time? Constituent

voice / Feedback

How did we do?Where did we fail?How do we share what we’ve learned?

Summative evaluation

BEFORE DURING AFTER

Field-level Priorities

Constituent voice /

Feedback

1

222

2

22 2

siaassociation.org

Nonprofit, NGO or community organisation but not charity; 19%

Charity; 17%

Consultancy; 15% Public sector; 14%

Social enterprise; 12%

Private sector; 10%

Academic institution; 10%

Network, association or membership organisation; 3%International mining company ; 0%

0

1

2

3

4

5

Socialenterprise

Private sector Nonprofit, NGOor communityorganisation

but not charity

Public Consultancy Charity Academicinstitution

Network,association ormembershiporganisation

Sliding scale where 5 is most expert

I do it for my program or organisation I do it for other programs or organisations

Responses to the question: I can identify measures of social impact relating to a program or organisation

6962

2925

18 16 1510

5

n=109

2

2

2

2

2

3

3

3

3

3

3

4

5

6

6

7

7

7

7

8

14

47

Results Framework

de Bono's Thinking tools

External evaluation

Outcome Mapping

Administrative data

Validated scales

Focus Groups

HACT Social Value Tool

Warwick Edinburgh Scale

Outcomes-based evaluation

London Benchmarking Group2

Cost-Benefit Analysis

Results Based Accountability

Before and after comparison

Logic Model

Data analysis

Bespoke methods/tools

Theory of Change

Outcomes Star

Interviews

Surveys

SROI

#opendata

3

(John Burrett, haikuanalytics , 2014)

(John Burrett, haikuanalytics , 2014)

(John Burrett, haikuanalytics , 2014)

4

-84%

-71%

-48%

-45%

-42%

-26%

-25%

-24%

-15%

-12%

5%

13%

24%

32%

38%

53%

57%

57%

72%

77%

Health records sold to privae healthcare companies

Online retailers to target advertisements

Technology companies monitoring searches for flu epidemic

Health records shared with private healthcare companies

Energy companies to predict energy needs

GP health records shared with researchers

Email and internet search traffic monitoring to identify terrorists

Pharmaceutical companies share with academic researchers

Tax and benefit records to catch fraud

Hospitals and GPs relating to care

Should happenShould not happen

(Royal Statistical Society UK, 2014)

Delivering services to people that need them without a performance management system is like driving a car with no instruments on the dashboard. With the windscreen obscured by ice. You know where you want to go, but you have no idea if you’re getting there.

Service delivery

Data

Daily data entry onto a shared IT

system

Analysis

Analyst support to collate and analyse

data in order to produce data dashboards.

Review meetings

Regular meetings to discuss dashboards, resource allocation

and service improvement.

1. Consent for data

collection and use

2. Constructing a

shared case

management IT

system

3. Building capacity of all

service delivery workers

to collect and use data

5. Feedback

loops and

reporting

6. Adjusting

services in

response

4. Performance

analysts

(adapted from Social Finance)

(http://dcpni.org/)

5

Pay for performance

Pay for success

Performance-based contracting

Payment by Results

Outcomes-based contracting

• UK Department of Work and Pensions Innovation Fund: disadvantaged young people

• UK It’s All About Me: adoption• Manchester: children in care• Granite School District: early

childhood education• Saskatchewan – single mothers

• Peterborough: ex-offenders• Massachusetts: high risk young men• New York State: employment for ex-

offenders• New South Wales Benevolent Society

Social Benefit Bond: families with children in care

Comparing intervention

participants to a control group

ATTRIBUTION

COHORT

Count or proportion of social

change among participants

CONTRIBUTION

INDIVIDUAL

lan

gu

ag

e u

sed

(Ministry of Justice UK, Justice Data Lab)

No63

68%

Yes30

32%

Statistically Significant?

increased5

17%

reduced25

83%

Effect on reoffending?

(Ministry of Justice UK, Justice Data Lab Oct 14)

0

500

1,000

1,500

2,000

2,500

3,000

3,500

4,000

4,500

-40 -30 -20 -10 0 10 20

Treatmen

t size

Effect on one year proven re-offending rate (ppts) [all statistically significant requests published to date]

(Ministry of Justice UK, Justice Data Lab Oct 14)

0.01 0.1 1 10 100reduces crime increases crime

(Petrosino (2013) Scared Straight Update)

6

other areascampbellcollaboration.org

healthcochrane.org

7

8

Partners Funders

9

10

outcomesstar.org.uk/

emmatomkinson.com

@emma_tomkinson