Post on 16-Dec-2015
transcript
Multi-strand initiatives: using theory of change evaluations
Alan Dyson: D.A.Dyson@manchester.ac.ukKirstin Kerr: Kirstin.Kerr@manchester.ac.uk
Karen Laing: k.j.c.laing@newcastle.ac.ukLiz Todd: liz.todd@newcastle.ac.uk
Introduction
• Aims• The team – Alan Dyson, Karen Laing, Liz Todd• Plan:
Introduction to theory of change evaluationTheory of change in actionUsing evidenceFinal thoughts
Dyson & Kerr
Cummings, Dyson, Muijs, Papps, Pearson, Raffo, Tiplady & Todd
Cummings Dyson & Todd
Full service extended schools national evaluation
Colleen Cummings, Alan Dyson, Ivy Papps, Daniel Muijs, Diana Pearson, Carlo Ruffo, Lucy Tiplady, Liz Todd: Newcastle University, University of Manchester, Tecis Ltdhttp://www.ncl.ac.uk/cflat/ESnetwork.htm
Theory of Change
A systematic and cumulative study of the links between activities, outcomes and context of the initiative
Fullbright-Anderson, Kubisch and Connell, 1998: 16
5
Theories of Change
• Popularized in 1990s to capture complex
initiatives
• Outcomes-based
• Causal model
• Articulate underlying assumptions
What is involved in theory of change….
• Qualitative and quantitative data collected over time
• Identify early, intermediate and long-term outcomes
• Theorise retrospectively and prospectively
The starting situation
What it’s like now – and why
Strands of action
What we are going to do about it
Intended outputs
What we will need to do differently
Steps of change
How things will change (for beneficiaries)
Intended outcomes
How the starting situation will change for CYP
How a Theory of Change approach can help
• Working together• It is dialogical, involving conversation and
negotiation• Enables continuous feedback• Can feed into project planning• Provides guidance about data collection
methods and self evaluation
Asking the right questions
• What is the situation you face?• What needs to change?• How will these changes be made?• What actions will you take?• What effect will those actions have? On
whom?• How will you know if change is happening?• What will happen for person A, person B etc
Lack of jobs / unemployment
Inability to connect to local service jobs
Reluctance to commute (locally/into Newcastle)
Resentment – feeling that something has been taken away from the community (not always sure what)
Lost generation of grandparents / parents
Loss of area esteem – some fracturing of community (e.g. parts of the area ‘in work’ resented)
‘Unhealthy place’ physically & mentally
Presenting problems:
Youth unemployment Youth disorder Poor mental and physical health – unfit families Children starting KS3 below level 3 Poor early language development ‘Troubled’ families High benefit claimant rates – disability and JSA
Underlying problems:
In some cases:
Parents have low expectations of their own children
Families don’t know how to be good parents or find it too hard
Families abdicate responsibility to services or refuse to engage with services
The hard to reach remain hard to reach
Children have aspirations but not the opportunities / capacities / support to act on these
Community not supportive of aspirations
Opportunities that are available are not recognised and acted on
A Children’s Community will seek to achieve the following outcomes for the area:
Reduction in presenting problems (no noticeable difference to other areas of N. Tyneside)
The area has the same opportunities and support structures as more advantaged areas. It provides the same ‘offer’ in terms of life chances (i.e. employment/ parental support/community support).
The community supports the realisation of aspirations. It establishes & sustains another way of behaving – i.e.:
-the community becomes self-empowered
-parents expect better for their children and act on this (services stop substituting for parents)
-the community is ‘fit for life’ mentally & physically
-there is a positive community identity linked to employment
-community members can connect to ‘wider world’ contexts and opportunities
There is upward mobility, both in and out of the area
The area attracts and sustains employment
Ship Building (Lost) 100% employment (Lost) Traditional gender roles (Maintained) Self-contained community (Maintained) Sufficient community fabric (Fractured)
Past context:
Recent context:
Resulting in:
Pre
sen
tin
g a
s:
The Children’s Community
Suggested core strands of action for the Children’s Community:
1. Getting things right early – e.g. speech, language communication & parenting. Aim to set children on positive pathway rather than addressing deficits later
2. Being fit for life – e.g. making sure children are healthy (physically and mentally) and are able to engage with opportunities
3. Realising aspirations & behaving in new ways – e.g. empowering the community to support children to achieve their aspirations and to do things they have not done before – making the community positive for residents and attractive to others (inc. employers).
The perpetuating situation (c.2003)
Getting things right early actions:
Transition support
Support for parenting
High quality pre-school provision
Higher attainment in primary
schools
Higher attainment throughout schooling
Access better post-school
opportunities (including jobs)
Parents with higher aspirations for their children
Higher aspirations
Good schools capitalise on better
prior attainment
Better opportunities
available in area
Better parenting in the home
Children with capacity to take advantage of
schooling (confidence & resilience)
Children doing well through transition
Getting things right early
Mentors:
access children in primary schools identify children who may have
problems later but who do not have additional provision because they are coping in primary school (Jan-July)
provide nurturing relationships and establish friendship groups
liaise with parents offer summer school in secondary
school identify issues secondary school
needs to be alerted to
Children learn to trust mentors who become ‘the trusted adult’
Children become familiar with secondary school & become more confident about transition
Parents ‘buy into’ the programme & encourage their children to participate
Secondary school prepares appropriate action Children’s confidence grows
In secondary school (autumn term), mentors:
maintain contact with children (eg through breakfast clubs)
monitor children in school to identify emerging problems
equip children with coping skills (eg anti-bullying strategies)
pass information onto guidance team (& Relate counsellor in Churchill)
wean children off support so they can cope alone
Children develop coping strategies
Secondary school implement effective support strategies
Children cope with secondary school without support
Children cope with transition with support
Transition mentors
Guidance team implement strategies
Drawing conclusions
• Formative:– ‘Initiative X is triggering changes a.b.c….which are
likely/unlikely to lead to outcomes 1, 2, 3….’
• Summative:– ‘Initiative X triggered changes a.b.c….leading to outcomes 1,
2, 3….’ – ‘Initiative X also triggered changes l, m, n….which may yet
lead to outcomes 1, 2, 3….’
• But: • ‘Initiative X# may not trigger changes a.b.c….or lead to
outcomes 1, 2, 3….’