Network of Networks on Impact Evaluation
Impact evaluation design for: PADYP BeninJocelyne Delarue - AFD
NONIE design clinic1 Cairo April, 2009
1Original format and presentation developed by the World Bank SIEF
Network of Networks on Impact Evaluation
Section A: Presentation of project
Network of Networks on Impact Evaluation
Project background & context• Objective : Develop agricultural productivity through farming system
management• Second phase of the project : scaling up
– Expansion from 3 000 to 18 000 members – Expansion in new communes, new zones and new farming systems
(12000 cotton farmers, 3000 non cotton farmers)• Activities (2009 – 2013) :
– training of 75 new advisers and 525 key farmers (KF)– Every adviser supports 60 members and 140 more through KF (20
m/KF)– Prepare sustainability (participation of farmers to service payment)
• Counterfactual situation : classical extension services recently extended by Benin government
• The project is financed by AFD (10 M. €), cotton cooperatives (3.4 M.€) and Benin State (1.4 M.€)
Network of Networks on Impact Evaluation
Support centers
Advisers Key farmers
Literacy courses
Farming system management
support
Changes in management decisions at farm level and family level
Experience sharing and field visits
Group debates
Activity planning
Inputs utilization
more rationale
Better yields
Costs reduction
Sales at best
prices
Better income Food security
New techniques
New crops & activities
Stock management
Cash Management
Reduction of hunger period
Labour utilization
more rationale
Reduction of debts
RESULTS CHAIN
Xxx Impact indicator
Network of Networks on Impact Evaluation
Key research questions• Is the farming system management (FSM) service more
efficient than top-down extension service to enhance agricultural productivity ? (cf. the causal chain)
• Who are the members ?• Are there spill over effects on non members ? (networks
effects)• What are the effects at a more aggregated level (village,
value chain)?• Operational questions
– How to succeed the scaling up– How to involve durably the farmers cooperatives in
this service provision
Network of Networks on Impact Evaluation
Outcome indicators• Number of farms which use farming system management
tools/methods– Activity planning– Stock management– Cash management– Credit management
• Number of farms who apply new cropping techniques• Number of farms who adopt new crops and activities
Impact Indicators• Yields, Costs, Productivity for each crop• Prices obtained for crops sold• Food security in the family• Income
Network of Networks on Impact Evaluation
Identification strategy/method
• Use the expansion of the program (pipeline method ? Instrumental variable ?)
• Comparison with a counterfactual situation : top-down extension services, currently extended also
• Combination of qualitative and quantitative methods
Network of Networks on Impact Evaluation
Sample and data (1)Value Chain Departements Nb communes
Cotton Alibori 6
Cotton Borgou 6
Cotton Atacora 3
Cotton Donga 1
Mixed farming Collines 3
Mixed farming Zou 3
Mixed farming Plateau 3
Mixed farming Ouémé 3
Mixed farming Couffo 2
Pineapple Atlantique 5
Vegetables Atlantique 1
Vegetables Ouémé 1
Vegetables Mono 3
Vegetables Littoral 1
Total 12 40
Network of Networks on Impact Evaluation
Sample and data (2)
2008 2009 2010 2011 2012 2013 Total
Advisers 0 35 25 15 0 0 75
Total 0 35 60 75 75 75 75
Key Farmers
0 245 175 105 0 0 525
Total 0 245 420 525 525 525 525
Members 0 10000 5000 3000 0 0 18000
Total 0 10000 15000 18000 0 0 18000
Network of Networks on Impact Evaluation
Sample and data (3)Monitoring system of the project (activities, number of advisers, key farmers, members, method use…)
Members’ farming systems caracteristics and economic results
Secondary data : census, market prices, …
Probably : Available data on crop productivity without farming system management
Network of Networks on Impact Evaluation
Time frame/work plan• Impact evaluation : 2009 - 2012
Network of Networks on Impact Evaluation
Sources and finances• AFD
• CIRAD
Network of Networks on Impact Evaluation
Section B: Impact evaluation of project
Network of Networks on Impact Evaluation
Evaluation framework and context• [Insert text on framework and context
here]
Network of Networks on Impact Evaluation
Evaluation questions• Farmers
• Villages – spillover effects
• Qualitative : pathways
Network of Networks on Impact Evaluation
Evaluation audience and policy it is trying to influence (if appl.)
• [Insert text on audience and policy here]
Network of Networks on Impact Evaluation
Evaluation design • Intensity of the treatment
• Full baseline full final, and follow ups
• Contamination of gates project (productivity can lower prices)
• Looking demand side on markets
Network of Networks on Impact Evaluation
Challenges and risks• Key unit of analysis : the community – need a
large number of communities• Stratify farmers• Rate of adhesion : if too weak, pb of sample• Pipeline : how much the people at the end of the
pipeline can be good counterfactual – key criteria of how pipeline is implemented (equitable, transparent, accountable) on community
• Pipeline on HH : the first to register will tend to be future key farmers, more dynamic…
Network of Networks on Impact Evaluation
Methodology• Pipeline ? Quantifiable criteria for HH
selection
Network of Networks on Impact Evaluation
Evaluation implementation• Use agricultural census
Network of Networks on Impact Evaluation
Management• [Insert text on evaluation management
here]
Network of Networks on Impact Evaluation
Expected results• [Insert text on expected evaluation results
here]
Network of Networks on Impact Evaluation
Expected conclusions• [Insert text on expected evaluation
conclusions here]
