www.ciat.cgiar.org Science to cultivate change
Outcomes thinking in CIAT and CRPs
March 2013CIAT
Presented by Sophie Alvarez
CIAT is committed to emphasizing the achievement of development outcomes. A results-based program management in the CGIAR shifts the focus from outputs or products into how these products will
be used, and by whom, to ultimately contribute to CGIAR’s overall goals of
poverty and hunger alleviation.
Achievement of Development Outcomes
This has implications in the way WE:
• Plan projects and initiatives,
• Implement them, especially regarding clarity on who we have to work with and who we
work for, and strategies to do it,
• Learn & document and report, and
• Measure performance.
Achievement of Development Outcomes
ActivitiesResearch Outputs
Research Outcomes
Development Outcomes
Impact
Implementers Next users End users
Generic Impact Pathway and Actors in Research for Agricultural Development
Impact Pathways describe results chains, showing the linkages between the sequence of results in getting to impact, but do not explain the ‘theory’ of why the intervention is expected to work i.e. the links in the pathway.
What are Impact Pathways and Theories of Change?
A theory of change (ToC) complements impact pathways by describing the causal linkages through which it is expected that an intervention will bring about the desired results. Put simply, a ToC is a causal model of how the intervention worked or is expected to work.
ActivitiesResearch Outputs
Research Outcomes
Development Outcomes
Impact
Set of activities
WHY Output 2 WHY Outcome 2 WHY Outcome
4 WHY Impact
Example of an Impact Pathway
Protocols for HR cassava
development
Evidence of value in use
of HR cassava
Tech transfer from other
crop systems
Evidence of value in use
Train host scientist to
support home
institute
Needs analysis
Manuals and "cookbooks"
USED BY…
Other CGIAR scientists
NARS 1
NARS 2
Results in…
NARS breeders using HR cassava breeding
materials
KAS and Practice changes
New varieties released by govt.
NARS breeders know how to select HR cassava in field
Other CG scientists doing needs analysis with women
farmers
Development Outcomes
Increased productivity for the beneficiaries
Changes in social arrangement, such as in
gender norms and power structures
Activities
Implementers
Research(includes participatory
and action)
Engagement (communicate and/or influence, co-learning, demonstrating good
practices, and networking)
Capacity Building (for key target groups,
partners, centers, CRPs)
Activities
Implementers
Us and partners:CENTERSCRPsNGOs, NARS, Private ….
Working in partnerships and networks
Research Outputs
Implementers Next users
New information and understanding
(CGIAR research outputs)
Research information (knowledge and basic understanding)
New understanding for putting research into action
New technologies and practices
Research Outputs
Implementers Next users
Capacity development events
On the job training and activities
Professional development courses and curricula
Research Outputs
Implementers Next users
Engagement events and networks
Communication campaigns
Innovative platforms
Research Outcomes
Development Outcomes
ImplementersNext and end users
Capacity change (changes in knowledge, attitudes, and skills)
Key practice/ behavioral changes, practice change*
Direct benefits to beneficiaries
Changes in the enabling environment
2 Major types of outcomes:
that underpin or support…..
Key capacity changes (knowledge, attitudes and skills)
Research Outcomes
Understand the implications of evidence-based research results and knowledge
Have the ability to apply the results and knowledge into policy
Have improved capacities to negotiate science-based results into policy agendas
Are aware of and convinced by results relevant to their policy area and understand policy implications
Changes in capacity of •Next users•End users
•Other CG scientists and CRPs
For example, policy makers in area x:
Key capacity changes (knowledge, attitudes and skills)
Research Outcomes
Describe things such as:
– Understand, accept, – Are prepared to use, know how to do…– Have improved capacities to participate, to negotiate…– Are aware of, see added value of…– Are motivated, “believe”, are convinced…– Have an improved attitude towards learning about…
and
Have a defined next user group, such as NARS researchers or Policy Makers, or ourselves
Key practice/ behavioral changes
Land use planners using GIS maps and decision support models
NARS breeders using CGIAR-developed maize breeding materials, and
varieties releasedSmallholder farmers adopt improved
maize varieties
X# Farmers in the (-) region adopt and adapt improved nutrient management
practices for higher yields and improved soil quality
Research Outcomes
Describe actions such as:– Using, making improved use of…
• And we need to know (at least descriptively) how and if they are using it now
– Adapting... Adopting• And we need to know what we will consider significant and
sustainable adoption
And also
Have a defined next user group, such as NARS researchers or Policy Makers
Key practice/ behavioral changes
Research Outcomes
Direct benefits to beneficiaries
Development Outcomes
Increased productivity for the beneficiaries
Reduced degradation of natural resources, better functioning or conservation of ecosystems and landscapes
Improved distribution of income, food security and nutrition benefits to the targeted poor and women
Direct benefits to beneficiaries
Development Outcomes
Increased income for smallholder farmers from adopting improved varieties
Reduced deforestation; increased carbon sequestration (an ISPC IDO)
Increased consumption of biofortified foods (an ISPC IDO)
Reduced erosion, better water quality, and flood control
Changes in the enabling environment
Development Outcomes
New or better functioning institutions both formal and informal:
new or improved formal governance arrangements, such as management systems, networks, planning bodies, government institutions, innovation platforms, etc.new or improved private sector institutions, markets and practices changes in social arrangement, such as in gender norms and power structureschanged patterns of interactions within and between organisations and social unitschanges in global public goods and the R4D paradigm related to agricultural research
New policies and policy instruments mechanisms such as:
new regional and national policies, regulations and administrative rules with a realistic implementation plan
Changes in the enabling environment
Development Outcomes
Functioning seed markets (an ISPC IDO)
Increased market
opportunities for poor from a
specific value chain (an ISPC
IDO)
Increased access to assets for
women in country X
Policies controlling illegal logging adopted (an ISPC RO)
National policies that support pro-poor
value chain development and
credit schemes for women (an ISPC RO)
End Users
Individuals and households
(farmers, fishers, women)
End users/ Next usersBeneficiaries/ Intermediaries
Indirect boundary partners/ Primary boundary partners
Next users
Research development partners (also implementers)Groups which will use our research outputs
Examples:•Civil Society groups•Communities and regions of communities•Advanced research institutions•CGIAR researchers from different centers and disciplines•NARES individuals/groups (local researchers, officials)•Private sector groups (seed producers, microfinance agencies, etc.)•Regional/national policy makers•Local, national and international NGOs
ActivitiesResearch Outputs
Research Outcomes
Development Outcomes
Impact
Implementers Next users End users
Generic Impact Pathway and Actors
Incl
ud
es ID
OS
IDOs can relate to either or both types of Development Outcomes
What are Intermediate Development Outcomes (IDOs)?
IDOs can relate to either or both types of Development Outcomes
What are Intermediate Development Outcomes (IDOs)?
What are Intermediate Development Outcomes (IDOs)?
Are intended to affect positively the welfare of the targeted population or environment
Are the result of multiple activities by diverse actors both inside and outside the CGIAR
Can cut across individual impact pathways, e.g., an IDO on enabling environment could apply to several impact pathways
Capacity and behavioral changes are NOT development outcomes, therefore NOT IDOs- (for the most part)
What are Intermediate Development Outcomes (IDOs)?
There must be a link between the IDOs and the activities undertaken - are attributable to outputs in that the contribution made can be assessed
Represent major milestones in the individual CRPs’ impact pathways towards the SLOs
Occur as a result of adopting improved varieties or better farm management approaches and relate to those directly targeted by the specific activities.
What are Intermediate Development Outcomes (IDOs)?
CRP IDO
5-7 years
≧10s of thousands to 100s of thousands
Specific contributing outcome
CRP Strategic Goal= impact
10-15
≧ millions
“Standalone” result: aggregation of
outcomes
Timeframe
Scale (numbers impacted)
Type
What are CRP impacts (CRP Strategic Goals)?
CRP IDO
5-7 years
≧10s of thousands to 100s of thousands
Specific contributing outcome
CRP Strategic Goal= impact
10-15
≧ millions
“Standalone” result: aggregation
of outcomes
Timeframe
Scale (numbers impacted)
Type
What are CRP impacts (CRP Strategic Goals) and SLOs?
CRP impacts are aggregates across the CRP and reflect the interactions among several factors.
CRP impacts can be thought of as specific instances of System Level Outcomes (SLOs)
Are a purpose unto themselves, not a step to achieving something else. In that sense, increased stability-connectivity- or redistribution (of power, of benefits) are not a good CRP impact or SLO.
High-level SLOs are aspirational and the theoretical impact pathways to them from research are very long.
Theory of change
A theory of change (ToC) complements impact pathways by describing the causal linkages through which it is expected that an intervention will bring about the desired results. Put simply, a ToC is a causal model of how the intervention worked or is expected to work.
Useful to distinguish:
Types of strategies being used. These could be around the various types of engagement or capacity development efforts undertaken across projects and the specific results sought through research activities.
By target groups. Focusing on activities aimed at the different target groups (such as research partners, households, women). For each a ToCcould be readily developed and used as the basis for the impact storyassociated with each group.
Theory of change
Are time dependent. Reflect understanding and knowledge up to that point
Have different purposes. develop a common understanding , and to communicate with others, on how and why an intervention works, as a basis for a monitoring system or impact evaluation, for making causal claims
Are based on prior research and stakeholder views. ToC are based on a combination of prior social science research, experience and on stakeholder views
Need to recognize uncertainties. In fact, ToC need to be thought of in probabilistic terms: that the ToC model is likely sufficient
Are usually developed in a participatory manner. Usually the process involves some or all of the various stakeholders
Can be either ex ante or ex post. Can be envisaged both before the intervention has been implemented and also after the intervention has been in place for some time.
CRP monitoring will be tracking progress to the direct benefit IDOs through, for example, adoption studies and surveys of changes in income levels.
The pathway from research activities and outputs to research outcomes will be where most of a CRP’s monitoring effort would be focused.
Measuring- M&E Ex- post socio- econ impact assessments and impact evaluations, done some time after the intervention has been put in place and often once the project is completed.
Knowing the role played by an outcome in getting to impacts is important in assessing which of many possible outcomes are most useful to track
Measuring- M&E and contribution claims
Some sorts of contribution claims we will be able to make:
Some early changes (effects of the CRP activities, such as aspects of capacity development—the relevant aspects of changes in knowledge, attitudes, skills, and opportunities for the targeted groups, incremental effect on community innovation rate, self-reliance and their natural resource base for a number of years) have been realized in line with the ToC.
The upstream elements of the ToC are robust, i.e. are supported by empirical evidence. Therefore, contributions to the upstream outcomes and impacts are likely to occur.
What are we doing now?
Each CRP setting out its results strategy framework showing its:
strategic goals, main (few) impact pathways and their outcomes, the general assumptions and linkages among the impact pathways
and the CGIAR system-level outcomes, and CRP IDOs.
… accompanied by narratives describing:
the development challenges being addressed, why are you involved in each area? What is the CRP comparative advantage?
the rationale for the impact pathways used and the major assumptions associated with these pathways (external events and conditions needed for the impact pathway to be successful)
What IDOs and impacts do we expect to accomplish with each IP?
1. Developing CRP Results Strategy Frameworks
What are we doing now?
2. Grouping CRP Outcomes
3. Identifying IDOs
4. Developing ToC for the CRP Results Strategies (impact pathways and ToCs)
5. ???
Lots of clarification still needed…Some are simply LANGUAGE:
End users/ Next users, Beneficiaries/ Intermediaries, Indirect boundary partners/ Primary boundary partners, Key target groups, Significant others
Behavioral- practice changes = ISPC research outcomes/ immediate outcomes? / intermediate outcomes?
Others are not:Commodity vs other more systemic CRPs- are IP and ToC, etc. just as good for both?
Greater participation of women in decision-making in a targeted domain - what is this? An IDO? A practice outcome? A CRP outcome? An impact?
How to measure aspects of capacity development (the relevant aspects of changes in knowledge, attitudes, skills, innovative capacity and opportunities) —as “step one” in achieving outcomes?
What is the role of baselines?
Thanks
Materials adapted from:
Discussions and materials in the CGIAR CRP IDO discussion group PIPA and OMCRPs (especially RTB, CCAFS, AAS) planning materials Conversations in the KS and CS and ISS – DAPA teams
Some participant questions • Probability of success- may make us work towards “safer” outcomes- low risk, low
pay-off
• System seems to make CRPs working together harder- collaboration vs. competition.
• Does the consortium have a TOC?
• CRP level- vs institutional level– what can we do internally (at CIAT)?
• We are having this presentation because this is part if our core business. This is a first awareness- raising, next we will:– Share this PPT with the regions and CIAT
– ISPC meetings to happen in Cali, further clarification
– Sit down with CRP focal points and discuss what with ISPC
• Follow up meeting
– Other meeting (general) –specific to each CRP/ knowledge base
– In theory we can understand: but how to manage + make operational- we have to pull this together.
• Do not call a new paradigm- we have always worked towards outcomes and impact
• How do we go about fitting something into the CRP? How to feedback into a CRP?