Post on 19-Mar-2016
description
transcript
Monitoring and Evaluating the Food Security and Nutrition Effects of
Agricultural ProjectsAnna Herforth - June 13, 2013 - LCIRAH
Nutrition Indicators in Agriculture Survey:
Preliminary results
Anna Herforth and Terri BallardFAO consultants
Funded by the EU-FAO Improved Global Governance for Hunger Reduction Programme (2012-2015)
Nutrition indicators in Agriculture projects - Survey
• Aim: to understand how agriculture projects will measure impact on nutrition: which indicators are being used, how, and why
• Why?– Researchers may be able to connect if desired;
informal technical support possible– Discussions about the evidence base can be informed
about the current generation of new evidence• How:– Follow up on DFID-funded LCIRAH mapping study
• 50% of studies identified were applicable• Excluded secondary data analysis, unspecified research, and
research with no agricultural intervention
Survey questions based on theory• Key principles:– Incorporate explicit nutrition objectives and indicators
into their design– Maintain or improve the natural resource base – Empower women – Facilitate production diversification, and increase
production of nutritious foods – Expand markets and market access for vulnerable
groups– Incorporate nutrition promotion and education to
enhance the impact of production and income– Collaborate and coordinate with other sectors
See: http://unscn.org/files/Agriculture-Nutrition-CoP/Agriculture-Nutrition_Key_recommendations.pdf
Preliminary Results: Response
• 77 project PIs contacted
• 68 responded (88%)– 3 of these excluded– 2 did not complete82% with complete
data
Preliminary Results: IndicatorsType of measure % measuring NotesFood consumption or diet almost all Many measuring HDDS, WDDS, and
IDDS for kids; intake of specific foods
of these, specific varieties? half (of above) Biofortified varieties; natural varieties
Food security most HFIAS, HHS, seasonality, coping strategies
Knowledge or behaviors most Specific to project
Women’s empowerment or labor
half Indicators somewhat unclear; a couple using/testing WEAI
Economic outcomes many of these, 2/3 disaggregating by gender
Nutritional status many stunting, underweight, BMI, anemia, indicators of VAS
Link with water, health, or sanitation
many
Natural resource management some Few indictors described; e.g. use of soil and water conservation practices
Preliminary Results: Design
• Majority are measuring in a comparison population
• Majority are collecting qualitative data• Sample sizes range from 120 to 4000 (one
9000)
Early conclusions• Focus is on nutrition impact among producers• Total shift from previous generation of research
regarding measurement of diet quality• Newly developed indicators get used• High number of studies measure nutritional status,
but available sample sizes suggest low power• Seems to be attention to program impact pathways– Indicators chosen mostly because important to project
goals, or evaluates a specific aspect of project– Interest in support for: “Adapting indicators to fit your
particular study activities and aims”
What these results represent
• This sample describes projects that set out to affect nutrition
• It does not describe larger agriculture programs or investments where nutrition is not necessarily the primary goal
Monitoring and Evaluating the Food Security and Nutrition Effects of
Agricultural ProjectsF. James Levinson and Anna Herforth
Ag2Nut Community of Practicehttp://knowledge-gateway.org/ag2nut
Purpose
Monitoring and evaluation of ag projects seeking to address food security and nutrition (in addition to their generally primary production objectives) would be highly desirable:
• there is, to date, so little empirical data documenting successes and failures; and
• possible adverse effects need to be identified and addressed rapidly within programs
Constraints
• Impact on food security on nutrition not necessarily first priority of agricultural managers
• Not familiar with how to measure• M&E in general is often not very strong– FAO/World Bank study (2010) of M&E in agriculture
projects in Asia, Africa and Latin America
Common shortcomings of Ag M&E• M&E is often perceived as an externally imposed
obligation with findings seldom integrated into management and action systems
• Ag managers complain of unmanageable data collection and reporting demands
• What M&E is carried out gives primary attention to physical achievements to the neglect of project outcomes
• Inadequate institutional capacity consistently limit M&E
(All of these, of course, are problems common to M&E in development projects more generally.)
A role for external Ag-Nut M&E teams?• Roles– identify sensible indicators to measure nutrition-
relevant impact based on the type of activities in the program
– carry out the key M&E necessary for tracking progress
– feed back to the program management (monitoring)
– Understand reasons for impact or lack thereof (eval)
– could support nutrition-sensitive program design or adjustment
Sentinel SitesGeographically representative sentinel sites within
the overall project area are one approach to M&E in ag projects with Ag2Nut interests
Baseline data would be followed by the collection of quantitative and qualitative data at 6 month intervals from these sites, and from comparable sites in non-project areas.
Examples of data• Extent to which households have been
reached/affected by the project• Relevant outputs specific to project (e.g. certain
crops/foods consumed)• Household food insecurity levels• Dietary quality• Women’s empowerment• If it makes sense, nutritional status among young
children and women
All complemented by qualitative data to better understand the dynamics of project effects
Identifying harmful food security or nutrition effects
• Employment levels have remained static or deteriorated;
• Small producers have been excluded;• Household food insecurity has deteriorated (overall or
seasonally);• Intra-household equity of income has declined;• The labor burden of women has increased;• Debt burden has increased;• In irrigation/water use projects, changes in water-
borne diseases;• In livestock projects, changes in zoonotic disease• Harm to natural resources (particularly water, soil,
biodiversity)
Data of particular interest to project managers
Among possible indicators:• Access, use and satisfaction with services
provided under the project• Changes in farmer assets• Percentage of households considering
themselves better off now than 12 months ago
• Percentage of the labor force underemployed or unemployed
Conditions
Such sentinel site data collection is likely to be useful if:•Good quality data can be sensibly aggregated and presented to project management in timely fashion– Challenge to collect data that are meaningful enough
to be useful, and brief enough to be usable•Information indicating harmful effects or shortcomings in project implementation – will be addressed by project management
How would it be supported?
Adequate staff and funding:•There is a need to identify Ag-Nut M&E teams capable of participating actively in an initial stream of nutrition-sensitive agriculture projects. •External funds probably necessary
In Sum
Through the creative use of separately managed sentinel site-based M&E, it should be possible to:
• Generate cooperative efforts• Generate much needed data• Document successes in nutrition-sensitive
agriculture• Develop prototypes, training modules and TA
mechanisms for subsequent use
Building the evidence base
• Need more examples of successful programs– Not just nutrition outcomes, but win-wins with
other goals• Ultimately, what is it that we want to scale
up?– Probably not individual programs– Rather, principles that explain how individual
programs have positive impact– Evaluations need to offer generalizable lessons
learned
Discussion
Ag2Nut Community of Practice http://knowledge-gateway.org/ag2nut