Theory of Change Introductory training Resource pack Sniffer / Adaptation Scotland Aug/Sept 2015 Praxis is the process by which a theory, lesson, or skill is enacted, embodied, or realised. "Praxis" may also refer to the act of engaging, applying, exercising, realizing, or practicing ideas Dr Sean DOOLAN, BSc, MBA Praxis & Research in Sustainability, Climate & Development Praxis = process by which a theory, lesson, or skill is enacted, embodied, or realised Praxis = the act of engaging, applying, exercising, realizing, or practicing ideas 1
Transcript
Theory of Change
Introductory training
Resource pack
Sniffer / Adaptation Scotland
Aug/Sept 2015
Praxis is the process by which a theory, lesson, or skill is enacted, embodied, or realised. "Praxis" may also refer to the act of engaging, applying, exercising, realizing, or practicing ideas
Dr Sean DOOLAN, BSc, MBA
Praxis & Research in
Sustainability, Climate & Development
Praxis = process by which a theory, lesson, or skill is enacted, embodied, or realised
Praxis = the act of engaging, applying, exercising, realizing, or practicing ideas1
• Describe what ToC involves & rationale for use
• Identify how & when ToC should be developed
• How ToC informs & complements other planning & M&E tools
• Increase confidence in working through steps & processes in developing ToC
Learning objectives / 1
2
• Explore ways to facilitate development of ToC within Sniffer and/or with partners
• Develop action plan for applying learning, i.e. use a ToC in the REALITY of change
Learning objectives / 2
3
We need to get better at understanding
& influencing change
Theory of change
Should make SENSE
Diagram should be easily read
Language & graphic needs to
resonate with stakeholders
Need to own & use
Need to continue using ToC, i.e.
sustainable as part of organisational culture
4
What is a Theory of Change? / 1
• Ongoing PROCESS of reflection to EXPLORE change & HOW it happens
• WHAT that means for, and part Sniffer plays, in a particular context, sector and/or group of stakeholders
• PRODUCT on understanding of causal pathways
5
What is a Theory of Change? / 2
• Considers project within wider analysis of how
change happens
• Makes all parties explain understanding of change –
but also challenges to explore understanding further
• Often presented as a diagram with an
accompanying narrative summary
Focus is on what you think will CHANGE
NOT on what you plan to DO6
Definition of ToC
• Many approaches - process & product
• A process of desired change by making explicit the way we think about a current situation or problem, its underlying causes, the long-term change we seek, and what needs to happen in order for that change to come about
• A product that contains a set of hypotheses, outcomes, assumptions & indicators that make up causal pathways of change needed to bring about a desired long-term goal
7
Origins of Theory of Change
Complex evaluations
Questioning & assumptions behind development thinking
Freire - ‘theory in use’Aspen Institute – focus on communitiesDevelopment agencies
Theories of social change
Lack of clear frameworks to assess programmes
Lack of connection between desired outcomes & activities
Dissatisfaction with logframes
More complex, systemic, network approaches
Increased demand to show impact & VfM
Current interest
Need for focus in programmes
The “missing middle”
8
ToC vs logframe / logic models / results chains
Logframe
1. Specific: Based on specific
pathways or interventions the
project will address;
2. Linear & structured
3. Describes outputs, outcomes,
indicators
4. Used for M&E, accountability to
funders (& beneficiaries …)
Theory of Change
1. Broad: Shows all domains &
pathways that may reach a
goal, including those the project
will not directly address
2. Non-linear & adaptive
3. Describes conditions & rationale
/ reasons for linkages along
causal pathway
4. Used for understanding “the big
picture”
Both
May include risks & assumptions,
outcomes, indicators9
ToC - Four interconnecting elements
1 How change happens
2 Change pathway & linked
domains of change
3 Impact assessment framework
4 Reflection &
adaptation / refinement
of ToC
10
Element 1: How change happens
Big Picture thinking
• “How change happens” in relation to issues &
problems that Sniffer / Adaptation Scotland or project seeks to address
• Thinking goes beyond own intervention &
considers all aspects of addressing identified issues
• Consider above and below the “threshold of
accountability”
11
Element 2: Sniffer / project Change Pathway
• This relates directly to your understanding of how
change happens (first component)
• It describes in detail your unique ways of
understanding & addressing these issues, including:
– WHO you work with
– HOW you work with them
– To achieve or INFLUENCE what changes
– The ASSUMPTIONS you have made in designing this
pathway
12
Review: Domains of change
• Goal typically has 2-5 domains of change
• ToC consists of domains of change linked
together in a hypothesis to achieve desired
goal – the change pathway
Do not limit the domains of change only to
areas that you intend to address …
13
Map a Pathway of Change
• Numerous outcomes (sub-purposes, achievements, pre-conditions, etc.) are desirable & will contribute to long-term goal
• However, ALL outcomes on pathway of change should be REQUIRED to attain the identified domain of change
1. Prioritise outcomes (and subsequently actions linked to each outcome)
2. Weed out incremental outcomes that may be desired but are unnecessary to achieve the intended goal
14
Element 3: Impact assessment
framework
Informed by both earlier components
Provides a robust way of understanding &
reporting on Sniffer’s / project’s contribution to
change (attribution)
Enables you to test & adapt your ToC
15
Element 4: Reflection & refinement of ToC
Evaluations & impact assessments will provide
information to reflect on your ToC:
• Work with right people?
• In right way?
• To what extent were assumptions valid?
• Did you negotiate barriers & facilitating factors
effectively?
• To what extent did you achieve or influence &
changes you planned?
• What does this tell you about how change
happens & your change pathway?
16
Elements of a ToC
– A problem statement
– A desired long-term goal
– Domains of change (key leverage points; purpose)
– Pathways of change, which include breakthroughs & incremental outcomes
– Assumptions & risks
– Interventions for each incremental outcome
– Indicators for each incremental outcome
– Defined stakeholders & roles
– Diagram & narrative summary 17
18
Problem statement
Desired long-term goal from outputs & outcomes
Domain of change
Domain of change
Domain of change
Stakeholders
Assumptions Pathways of Change
Breakthrough
Breakthrough
Output
Outcome
Output Output
Output
Outcome
Output Output
Outcome
Outcome
Outcome
Outcome
Theory of Change conceptual model
Causal analysis: Identify underlying
causes (leverage points)
Look for
• Direct causes – specific conditions contributing directly to problem
• Secondary causes – attitudes & behavioursthat contribute to specific conditions
Inventory of causes that could be contributing to a problem helps to identify causal streams
• Household level – Household & individual constraints (e.g., behaviours, knowledge & skill levels, attitudes, access to assets) that limit opportunities to achieve positive outcomes
• Community level – Weak community cohesion & lack of shared values that hinder delivery & maintenance of social & economic infrastructure & assets
• External level – Constraints that are external to the community, such as government policies, delivery of services, access to assets, market & social forces beyond the community’s control
20
Constructing a ToC
Data collection & analysis (identify problems)
Draft problem statement
Causal analysis Identify key underlying causes at the household, community, or
external level; Look for direct, secondary, tertiary causes
Problem tree
Solution tree
Pathways of change: Check theoretical assumptions between solutions
Prioritize domains of change that project will address Clearly outline who will address other necessary Domains
Identify interventions for each outcome, assumptions related to interventions, & related risks
Identify indicators for each outcome.
PLAUSIBLE !
FEASIBLE !
TESTABLE !
21
Comparative advantage
• Identify strengths of Sniffer, Adaptation Scotland, partners, communities, wider stakeholders etc.
• Think outside the box – resources available to you, people, projects, locations where you have experience …
• Acknowledge & note weaknesses & gaps
22
Partnerships
• Remember that Sniffer doesn’t have to do everything!
• Formal & informal partners fill in the gaps & support areas where weakness may exist
• Think outside the box – Go beyond usual suspects - don’t just restrict to
• A communication tool to capture complexity of initiative
• A clear & testable set of hypotheses about how change will occur
• A visual representation of expected change & how it will occur
• An agreement among stakeholders about what defines success & what it takes to achieve it
• A blueprint for evaluation with measurable indicators of success identified
34
How ToC is being used
35
• Works best if there is an opportune moment (resources
& need to reflect):
– In preparation for new strategic plan
– Applying for new grant, developing a new project
– To inform an evaluation or impact assessment
• Few organisations undertake the whole process in one
go (if ever!)
When to develop ToC?
36
Conducting a ToC process
37
Caution – mind the gap
If you skip the PROCESS,
the ToC becomes a paper exercise.
It can be meaningless …
38
• No single or “right” answer…
• Can be 1-5 pages in length
• Short narrative & diagram
• All sorts of shapes & illustrations
• Need to be able to illustrate – your organisational (or project) pathway to change,
– the links & assumptions you are making in choosing this path
Examples
STAR-Ghana, DFID adaptation workstreams, CARE Adaptation Learning Programme (ALP), IDRC Climate Change Adaptation in Africa (CCAA), Building Resilience and Adaptation to Climate Extremes and Disasters Programme (BRACED), IDRC organisational capacity development
What does the ToC beast look like?
39
IDRC – organisational capacity development
40
Moser, S. C. and J. A. Ekstrom. (2010)A framework to diagnose barriers to climate change adaptation.PNAS, 107 (51): 22026-22031, DOI:10.1073/pnas.1007887107
Scope & scale of adaptation
41
BRACED
42
43
44
45
46
ToC narrative
47
Scope of CARE Adaptation Learning Programme, ALP
48
CARE Adaptation Learning Programme, ALP
49
CARE ALP
50
Voice mechanisms
Channels – CC Adaptation in Africa
Capacity-building model
Adaptive capacity & benchmarking
Writing & media support
Outcome mapping
Communications & story-telling
Influencing model
Mapping institutional landscape
Political economy – creating space
Governance levels & space Policy windows
51
CCAA channels & means
52
CCAA
53
Mechanisms for voice &
accountability
World Bank (2003) Making Services Work for the Poor. World Development Report 2004
54
Accountability mechanisms & relations
Accountability and Voice for Service Delivery at the Local Level. TheIDLgroup & UNDP (2008) 55
Diverse voices
Carl Jackson
Westhill Knowledge & IDS
Mobilizing Knowledge for Development Programme www.ids.ac.uk/knowledge-services 56
Challenge – a shift in ways of working
Construction of new relationships between communities, service providers and the State
Working both sides of the equation – to focus on participation, responsiveness and accountability
• From ‘coping’ to ‘voice’
• From ‘shouting’ to ‘counting’
• From ‘reaction’ to ‘informed action’
• From ‘sporadic action’ to ‘organised action’
• From ‘confrontation’ to ‘win-win situation’
57
Lesson-learning from community-based & social approaches, knowledge-brokering
• Share your own experiences of considering or developing & using Theories of Change
• Discuss
– advantages ToC might bring
– challenges/questions & doubts about developing & using ToC
See handouts for more information
61
Advantages: what partners say
62
Some challenges
Balancing learning & accountability
Adapting process to context & content
Continuing & learning & reflection
Facilitation, not remote inputs by committee
Reconciling ToC & pathways for change with other organisationalprocesses
Keeping it simple but valid
Common buzzwords & initiatives
Separated by a common language – different understandings of terms
Understanding where stakeholders are coming from – experience & perspectives
Whose reality counts?
63
How change happens
64
Element 1: How change happens
Big Picture thinking
• “How change happens” in relation to issues that
Sniffer or project seeks to address
• Go beyond your own intervention & consider all
aspects of addressing identified issues
65
Vision of
success
Key success factor
Key successfactor
Key success factor
Key success factor
Key success factor
66
Different ways of understanding HOW change happens
Ensure wide consultation & buy-in
Commission a research paper
Problem tree analysis
Vision of success exercise & critical success factors
Success stories & analysis of why they were successful(Appreciative Inquiry, Most Significant Change)
More academic approach
More participatory approach
67
Task in group / 1
1. Agree a “vision of success” in relation to selected project
– If all issues & problems & their underlying causes were successfully addressed, what would this look like?
– Be quite specific – write a sentence of no more than 25 words which outlines what changes you expect to see for whom
68
2 Understand & agree key success factors to ensure
that vision becomes a reality
– List up to five key success factors
– For each success factor, be specific about
what should be in place & who should be
involved
Note: Think beyond Sniffer strategy – think PEST
politics, economics, social relations, technology etc
Task in group / 2
69
Developing an organisational or
project Change Pathway
70
Element 2: Your organisational / project
Change Pathway
• Relates directly to your understanding of how
change happens (first component)
• Describes in detail your ways of understanding &
addressing these issues, including:
– WHO you work with
– HOW you work with them
– To achieve or INFLUENCE what changes
– The ASSUMPTIONS that you have made in
designing this pathway
71
Developing a Change Pathway - Step 1
Review & exercise
– Which areas of change can Sniffer or project influence?
• Directly?
• Indirectly? (other factors/ organisations will also influence these changes)
– Which areas of change lie beyond the scope of Sniffer or project?
• Threshold of accountability
72
Developing Change Pathway - Step 2
– Who do you plan to work with (target groups)?
– How do you plan to work with each group?
– What short & medium term changes (for these
different groups) do you hope to achieve or influence?
– How do all of these factors link together? (What
leads to what? Who influences whom?)
– What are your assumptions in choosing this
particular pathway (why this one rather than
another)?
73
Which domains of change to address? Criteria
• Domain significantly contributes to achieving
the desired goal
• High synergy with other domains
• Addressing the domain & anticipated outcomes
will maximise Sniffer’s comparative advantage
• Potential for partnering
• Responds to key stakeholder or funder
interests & opportunity for resources
74
Use post-its for all statements – move them about
1. Clarify & write up overall goal (place on far right)
2. Identify & write up your high level strategies (what you do with whom) – one per post it. Place in a line on the left side
3. Create “ SO THAT” chains, i.e. by doing X we will achieve Y change in short-term, which will lead to Z change in long-term. Write these up (IF-THEN logic)
4. Link strategies with short-term outcomes & goals (place post-its on chart & link with arrows)
5. Test the logic - does it work?
6. Articulate your assumptions
Suggested method for brainstorming ToC
75
Assessing impact
76
Element 3: Impact assessment
framework
Informed by both earlier components
Robust means of understanding & reporting
on Sniffer’s / projects’s contribution to change
Enables you to test & adapt your ToC
77
Impact assessment - definition
“The systematic analysis of significant
and/or lasting change – positive or
negative, intended or not – in & lives of
target groups, brought about by a given
action or a series of actions”
It explores & answers & most important
question of all:
So what ACTUALLY changed?
78
Why assess impact?
• To understand the implications of work
• To be accountable to stakeholders
• To support institutional learning & decision
making & to improve future work
• To contribute to policy development & effective
advocacy / influencing
• To help demonstrate organisational / project
performance & delivery – evidence & VfM!
VfM = Value for Money
79
Impact assessment framework
• Focus on articulated changes at all levels
• Consider & report on FIVE key questions:
– WHAT has changed (positive/
negative/intended or not)?
– For WHOM?
– How SIGNIFICANT is this?
– What, if anything, did our
organisation/programme CONTRIBUTE?
80
Contribution analysis to impact
81
Impact assessment framework
www.thinknpc.org
82
The differences - in brief
83
Domains of Change Framework for Empowerment & rights
1Changes in & ability
of marginalisedwomen & men to
access & enjoy their full human rights
5Changes in individual
self confidence & self efficacy
4 Changes in ability
and collaboration of marginalised groups
in fighting inequality & claiming rights
3 Changes in civil
society organisations’ capacity to support marginalized groups to claim their rights
2Changes in
Governments & non state actors[1]
willingness & ability to deliver on their responsibilities for most marginalised
people
Outcomes
Policies & laws supporting rights of marginalised people are developed strengthened & implemented
Government & non state actors allocate adequate resources & ensure availability & access of services for all marginalised people
Governments & non state actors operate in equitable & transparent ways
Effective & accessible mechanisms to seek redress for marginalised people are in place
Outcomes
CSOs facilitate space for communities & marginalised groups to influence decision makers
CSOs challenge duty bearers & those
discriminating against rights holders
CSOs & communities monitor duty
bearers & hold them to account
Communities are aware of & rights & implications for their roles & responsibilities
CSOs advocate for social change
Communities promote inclusive values
Access to available goods & services is equitable
[1] Non state actors includes & private/ corporate sector & international organisations
Outcomes
Marginalized women men, girls & boys have improved self image
Marginalized women men, girls & boys feel capable of & can influence decisions that affect their lives
Marginalized women men, girls & boys have adequate life skills, including literacy & numeracy
Marginalized women, men, girls & are secure & safe
Outcomes
Marginalised groups better understand their rights & responsibilities
Marginalised groups coordinate and/or play a role in bringing about change
Marginalised groups influence decision makers in issues that concern their lives
Marginalised groups call on existing legislation & judicial services & systems to support their rights & hold duty bearers to account claims
Marginalised groups actively participate in democratic spaces
84
85
86
87
ToC - Four interconnecting elements
1 How change happens
2 Change pathway & linked
domains of change
3 Impact assessment framework
4 Reflection &
adaptation / refinement
of ToC
88
What is a ToC?
Change?
89
Reflection & questions
Think about the different elements &
processes
• What did you find most interesting/useful about
each element?
• Any questions? Areas where you would like
further discussion or clarification?
90
Impact assessment: approaches
Three approaches
– Post-programme: Testing logic of log frame
(impact “evaluation”)
– Participatory ToC approach used to design
monitor & assess efforts
– Research: looking back sometime later &
assessing changes & their relation to project
efforts
91
Impact assessment: strategies
Four strategies
• Build into existing M&E
• Tracer & tracker studies
• Ensure key moments of critical reflection
• Commission retrospective study
92
Examples of impact assessment frameworks in climate change
93
94
What’s involved in developing/
facilitating a ToC process?
• Starting point & initial preparation of partners
• Starting from the end-point vision
• Securing buy in & ownership
• Strong workshop process & facilitation
• Commitment to follow up
• Ensure results are communicated & used effectively
95
Constructing a ToC
Data collection & analysis (identify problems) Draft problem statement Causal analysis
Identify key underlying causes at the household, community, or external level; Look for direct, secondary, tertiary causes
Problem tree Solution tree Pathways of change: Check theoretical assumptions
between solutions Prioritize domains of change that project will address
Clearly outline who will address other necessary Domains
Identify interventions for each outcome, assumptions related to interventions, & related risks
Identify indicators for each outcome.
PLAUSIBLE !
FEASIBLE !
TESTABLE !
96
Thoughts
• Theories not Theory
• Embrace uncertainty
• Beware magic bullets
• Assume you will be wrong, at least to start
• Getting the right PROCESS and PLAYERS matters more than ‘the Perfect Plan’
97
International Climate Fund
transformational criteria / 1
• Scale: National, sectoral or economy-wide programmes including institutional and policy reform so that reaches critical deployment mass
• Replicable: Projects that others can copy, leading to larger scale or faster roll-out, for example, key policy changes
• Innovative: Piloting new ways of achieving objectives that could lead to wider and sustained change. These projects are often high risk but with corresponding high potential returns
• Leverage: Projects that leverage others to help increase the impact beyond the project itself should increase the likelihood of this being transformational, by unlocking potential for scaling-up and replication.
98
ICF transformational criteria / 2
• Sustainable: Programmes that are sustainable are more likely to have an impact after they have ended. However, not all piloting and innovation programmes will be sustainable, as there is an element of experimentation and risk involved
• Political will and local ownership: Working with national stakeholders, including the powerful, who want to deliver change consistent with their own political economy will be more effective
•
• Increased capacity and capability to act: Strengthening local capacity supports continued action on climate change and lays the conditions for transformational change
• Evidence of effectiveness is credible and shared widely.Others are unlikely to follow unless they are confident of the case for change. This argues for substantial and quality M&E of key projects, presenting failure alongside success.
99
Case study task: Plan a ToC approach to planning and / or evaluation
• Work in buzz pairs or group
• Develop an outline plan for developing or facilitating a ToC approach, in Sniffer or with your partners
• Discuss & answer questions set
• Make notes to report back on key points
100
Setting the context / 1
• At what level will you be working? – Sniffer? Adaptation Scotland? – Projects(s) within Sniffer? – Partner organisation(s) to common goal? – Other?
• Why planning to introduce/ develop a ToC ?
• Which elements will you prioritise?
• Why now?
101
Setting the context / 2
• What (if any) elements of Theory of Change already exist within Sniffer, partners or project?
