www.ciat.cgiar.org Science to cultivate change

Outcomes thinking in CIAT and CRPs

March 2013CIAT

Presented by Sophie Alvarez

CIAT is committed to emphasizing the achievement of development outcomes. A results-based program management in the CGIAR shifts the focus from outputs or products into how these products will

be used, and by whom, to ultimately contribute to CGIAR’s overall goals of

poverty and hunger alleviation.

Achievement of Development Outcomes

This has implications in the way WE:

• Plan projects and initiatives,

• Implement them, especially regarding clarity on who we have to work with and who we

work for, and strategies to do it,

• Learn & document and report, and

• Measure performance.

Achievement of Development Outcomes

ActivitiesResearch Outputs

Research Outcomes

Development Outcomes

Impact

Implementers Next users End users

Generic Impact Pathway and Actors in Research for Agricultural Development

Impact Pathways describe results chains, showing the linkages between the sequence of results in getting to impact, but do not explain the ‘theory’ of why the intervention is expected to work i.e. the links in the pathway.

What are Impact Pathways and Theories of Change?

A theory of change (ToC) complements impact pathways by describing the causal linkages through which it is expected that an intervention will bring about the desired results. Put simply, a ToC is a causal model of how the intervention worked or is expected to work.

ActivitiesResearch Outputs

Research Outcomes

Development Outcomes

Impact

Set of activities

WHY Output 2 WHY Outcome 2 WHY Outcome

4 WHY Impact

Example of an Impact Pathway

Protocols for HR cassava

development

Evidence of value in use

of HR cassava

Tech transfer from other

crop systems

Evidence of value in use

Train host scientist to

support home

institute

Needs analysis

Manuals and "cookbooks"

USED BY…

Other CGIAR scientists

NARS 1

NARS 2

Results in…

NARS breeders using HR cassava breeding

materials

KAS and Practice changes

New varieties released by govt.

NARS breeders know how to select HR cassava in field

Other CG scientists doing needs analysis with women

farmers

Development Outcomes

Increased productivity for the beneficiaries

Changes in social arrangement, such as in

gender norms and power structures

Activities

Implementers

Research(includes participatory

and action)

Engagement (communicate and/or influence, co-learning, demonstrating good

practices, and networking)

Capacity Building (for key target groups,

partners, centers, CRPs)

Activities

Implementers

Us and partners:CENTERSCRPsNGOs, NARS, Private ….

Working in partnerships and networks

Research Outputs

Implementers Next users

New information and understanding

(CGIAR research outputs)

Research information (knowledge and basic understanding)

New understanding for putting research into action

New technologies and practices

Research Outputs

Implementers Next users

Capacity development events

On the job training and activities

Professional development courses and curricula

Research Outputs

Implementers Next users

Engagement events and networks

Communication campaigns

Innovative platforms

Research Outcomes

Development Outcomes

ImplementersNext and end users

Capacity change (changes in knowledge, attitudes, and skills)

Key practice/ behavioral changes, practice change*

Direct benefits to beneficiaries

Changes in the enabling environment

2 Major types of outcomes:

that underpin or support…..

Key capacity changes (knowledge, attitudes and skills)

Research Outcomes

Understand the implications of evidence-based research results and knowledge

Have the ability to apply the results and knowledge into policy

Have improved capacities to negotiate science-based results into policy agendas

Are aware of and convinced by results relevant to their policy area and understand policy implications

Changes in capacity of •Next users•End users

•Other CG scientists and CRPs

For example, policy makers in area x:

Key capacity changes (knowledge, attitudes and skills)

Research Outcomes

Describe things such as:

– Understand, accept, – Are prepared to use, know how to do…– Have improved capacities to participate, to negotiate…– Are aware of, see added value of…– Are motivated, “believe”, are convinced…– Have an improved attitude towards learning about…

and

Have a defined next user group, such as NARS researchers or Policy Makers, or ourselves

Key practice/ behavioral changes

Land use planners using GIS maps and decision support models

NARS breeders using CGIAR-developed maize breeding materials, and

varieties releasedSmallholder farmers adopt improved

maize varieties

X# Farmers in the (-) region adopt and adapt improved nutrient management

practices for higher yields and improved soil quality

Research Outcomes

Describe actions such as:– Using, making improved use of…

• And we need to know (at least descriptively) how and if they are using it now

– Adapting... Adopting• And we need to know what we will consider significant and

sustainable adoption

And also

Have a defined next user group, such as NARS researchers or Policy Makers

Key practice/ behavioral changes

Research Outcomes

Direct benefits to beneficiaries

Development Outcomes

Increased productivity for the beneficiaries

Reduced degradation of natural resources, better functioning or conservation of ecosystems and landscapes

Improved distribution of income, food security and nutrition benefits to the targeted poor and women

Direct benefits to beneficiaries

Development Outcomes

Increased income for smallholder farmers from adopting improved varieties

Reduced deforestation; increased carbon sequestration (an ISPC IDO)

Increased consumption of biofortified foods (an ISPC IDO)

Reduced erosion, better water quality, and flood control

Changes in the enabling environment

Development Outcomes

New or better functioning institutions both formal and informal:

new or improved formal governance arrangements, such as management systems, networks, planning bodies, government institutions, innovation platforms, etc.new or improved private sector institutions, markets and practices changes in social arrangement, such as in gender norms and power structureschanged patterns of interactions within and between organisations and social unitschanges in global public goods and the R4D paradigm related to agricultural research

New policies and policy instruments mechanisms such as:

new regional and national policies, regulations and administrative rules with a realistic implementation plan

Changes in the enabling environment

Development Outcomes

Functioning seed markets (an ISPC IDO)

Increased market

opportunities for poor from a

specific value chain (an ISPC

IDO)

Increased access to assets for

women in country X

Policies controlling illegal logging adopted (an ISPC RO)

National policies that support pro-poor

value chain development and

credit schemes for women (an ISPC RO)

End Users

Individuals and households

(farmers, fishers, women)

End users/ Next usersBeneficiaries/ Intermediaries

Indirect boundary partners/ Primary boundary partners

Next users

Research development partners (also implementers)Groups which will use our research outputs

Examples:•Civil Society groups•Communities and regions of communities•Advanced research institutions•CGIAR researchers from different centers and disciplines•NARES individuals/groups (local researchers, officials)•Private sector groups (seed producers, microfinance agencies, etc.)•Regional/national policy makers•Local, national and international NGOs

ActivitiesResearch Outputs

Research Outcomes

Development Outcomes

Impact

Implementers Next users End users

Generic Impact Pathway and Actors

Incl

ud

es ID

OS

IDOs can relate to either or both types of Development Outcomes

What are Intermediate Development Outcomes (IDOs)?

IDOs can relate to either or both types of Development Outcomes

What are Intermediate Development Outcomes (IDOs)?

What are Intermediate Development Outcomes (IDOs)?

Are intended to affect positively the welfare of the targeted population or environment

Are the result of multiple activities by diverse actors both inside and outside the CGIAR

Can cut across individual impact pathways, e.g., an IDO on enabling environment could apply to several impact pathways

Capacity and behavioral changes are NOT development outcomes, therefore NOT IDOs- (for the most part)

What are Intermediate Development Outcomes (IDOs)?

There must be a link between the IDOs and the activities undertaken - are attributable to outputs in that the contribution made can be assessed

Represent major milestones in the individual CRPs’ impact pathways towards the SLOs

Occur as a result of adopting improved varieties or better farm management approaches and relate to those directly targeted by the specific activities.

What are Intermediate Development Outcomes (IDOs)?

CRP IDO

5-7 years

≧10s of thousands to 100s of thousands

Specific contributing outcome

CRP Strategic Goal= impact

10-15

≧ millions

“Standalone” result: aggregation of

outcomes

Timeframe

Scale (numbers impacted)

Type

What are CRP impacts (CRP Strategic Goals)?

CRP IDO

5-7 years

≧10s of thousands to 100s of thousands

Specific contributing outcome

CRP Strategic Goal= impact

10-15

≧ millions

“Standalone” result: aggregation

of outcomes

Timeframe

Scale (numbers impacted)

Type

What are CRP impacts (CRP Strategic Goals) and SLOs?

CRP impacts are aggregates across the CRP and reflect the interactions among several factors.

CRP impacts can be thought of as specific instances of System Level Outcomes (SLOs)

Are a purpose unto themselves, not a step to achieving something else. In that sense, increased stability-connectivity- or redistribution (of power, of benefits) are not a good CRP impact or SLO.

High-level SLOs are aspirational and the theoretical impact pathways to them from research are very long.

Theory of change

A theory of change (ToC) complements impact pathways by describing the causal linkages through which it is expected that an intervention will bring about the desired results. Put simply, a ToC is a causal model of how the intervention worked or is expected to work.

Useful to distinguish:

Types of strategies being used. These could be around the various types of engagement or capacity development efforts undertaken across projects and the specific results sought through research activities.

By target groups. Focusing on activities aimed at the different target groups (such as research partners, households, women). For each a ToCcould be readily developed and used as the basis for the impact storyassociated with each group.

Theory of change

Are time dependent. Reflect understanding and knowledge up to that point

Have different purposes. develop a common understanding , and to communicate with others, on how and why an intervention works, as a basis for a monitoring system or impact evaluation, for making causal claims

Are based on prior research and stakeholder views. ToC are based on a combination of prior social science research, experience and on stakeholder views

Need to recognize uncertainties. In fact, ToC need to be thought of in probabilistic terms: that the ToC model is likely sufficient

Are usually developed in a participatory manner. Usually the process involves some or all of the various stakeholders

Can be either ex ante or ex post. Can be envisaged both before the intervention has been implemented and also after the intervention has been in place for some time.

CRP monitoring will be tracking progress to the direct benefit IDOs through, for example, adoption studies and surveys of changes in income levels.

The pathway from research activities and outputs to research outcomes will be where most of a CRP’s monitoring effort would be focused.

Measuring- M&E Ex- post socio- econ impact assessments and impact evaluations, done some time after the intervention has been put in place and often once the project is completed.

Knowing the role played by an outcome in getting to impacts is important in assessing which of many possible outcomes are most useful to track

Measuring- M&E and contribution claims

Some sorts of contribution claims we will be able to make:

Some early changes (effects of the CRP activities, such as aspects of capacity development—the relevant aspects of changes in knowledge, attitudes, skills, and opportunities for the targeted groups, incremental effect on community innovation rate, self-reliance and their natural resource base for a number of years) have been realized in line with the ToC.

The upstream elements of the ToC are robust, i.e. are supported by empirical evidence. Therefore, contributions to the upstream outcomes and impacts are likely to occur.

What are we doing now?

Each CRP setting out its results strategy framework showing its:

strategic goals, main (few) impact pathways and their outcomes, the general assumptions and linkages among the impact pathways

and the CGIAR system-level outcomes, and CRP IDOs.

… accompanied by narratives describing:

the development challenges being addressed, why are you involved in each area? What is the CRP comparative advantage?

the rationale for the impact pathways used and the major assumptions associated with these pathways (external events and conditions needed for the impact pathway to be successful)

What IDOs and impacts do we expect to accomplish with each IP?

1. Developing CRP Results Strategy Frameworks

What are we doing now?

2. Grouping CRP Outcomes

3. Identifying IDOs

4. Developing ToC for the CRP Results Strategies (impact pathways and ToCs)

5. ???

Lots of clarification still needed…Some are simply LANGUAGE:

End users/ Next users, Beneficiaries/ Intermediaries, Indirect boundary partners/ Primary boundary partners, Key target groups, Significant others

Behavioral- practice changes = ISPC research outcomes/ immediate outcomes? / intermediate outcomes?

Others are not:Commodity vs other more systemic CRPs- are IP and ToC, etc. just as good for both?

Greater participation of women in decision-making in a targeted domain - what is this? An IDO? A practice outcome? A CRP outcome? An impact?

How to measure aspects of capacity development (the relevant aspects of changes in knowledge, attitudes, skills, innovative capacity and opportunities) —as “step one” in achieving outcomes?

What is the role of baselines?

Thanks

Materials adapted from:

Discussions and materials in the CGIAR CRP IDO discussion group PIPA and OMCRPs (especially RTB, CCAFS, AAS) planning materials Conversations in the KS and CS and ISS – DAPA teams

Some participant questions • Probability of success- may make us work towards “safer” outcomes- low risk, low

pay-off

• System seems to make CRPs working together harder- collaboration vs. competition.

• Does the consortium have a TOC?

• CRP level- vs institutional level– what can we do internally (at CIAT)?

• We are having this presentation because this is part if our core business. This is a first awareness- raising, next we will:– Share this PPT with the regions and CIAT

– ISPC meetings to happen in Cali, further clarification

– Sit down with CRP focal points and discuss what with ISPC

• Follow up meeting

– Other meeting (general) –specific to each CRP/ knowledge base

– In theory we can understand: but how to manage + make operational- we have to pull this together.

• Do not call a new paradigm- we have always worked towards outcomes and impact

• How do we go about fitting something into the CRP? How to feedback into a CRP?