Ann. N.Y. Acad. Sci. ISSN 0077-8923 ANNALS OF THE NEW YORK ACADEMY OF SCIENCES Issue: Paths of Convergence for Agriculture, Health, and Wealth Agriculture, nutrition, and health in global development: typology and metrics for integrated interventions and research William A. Masters, 1 Patrick Webb, 1 Jeffrey K. Griffiths, 1,2 and Richard J. Deckelbaum 3 1 Friedman School of Nutrition Science and Policy, Tufts University, Boston, Massachusetts. 2 Department of Public Health, School of Medicine, Tufts University, Boston, Massachusetts. 3 Mailman School of Public Health, Columbia University, New York, New York Address for correspondence:William A. Masters, Friedman School of Nutrition Science and Policy, Tufts University, 150 Harrison Ave., Boston, MA 02111. [email protected]Despite rhetoric arguing that enhanced agriculture leads to improved nutrition and health, there is scant empirical evidence about potential synergies across sectors or about the mix of actions that best supports all three sectors. The geographic scale and socioeconomic nature of these interventions require integration of previously separate research methods. This paper proposes a typology of interventions and a metric of integration among them to help researchers build on each other’s results, facilitating integration in methods to inform the design of multisector interventions. The typology recognizes the importance of regional effect modifiers that are not themselves subject to randomized assignment, and trade-offs in how policies and programs are implemented, evaluated, and scaled. Using this typology could facilitate methodological pluralism, helping researchers in one field use knowledge generated elsewhere, each using the most appropriate method for their situation. Keywords: integration; agriculture; health; nutrition; research; development Introduction Agriculture, nutrition, and health are interrelated through biological and behavioral pathways that make these sectors distinctively tied to ecologi- cal and socioeconomic conditions. Place-based fac- tors constrain and influence peoples’ nutrition and health outcomes at every scale and in each context. For example, rainfall, temperature, and soil nutri- ents interact with crop genetics to influence plant growth and food quality as well as farm incomes, which in turn influence food purchase and health- seeking behaviors, all of which combine to influence individuals’ nutrition and health outcomes. Ecolog- ical conditions also influence the growth and repro- duction of pathogens, parasites, and disease vectors of all kinds, weighing heavily on the effectiveness of interventions aimed at meeting development goals. Since the 2007/2008 food-price crisis, increasing attention to agriculture and nutrition as a factor in health and development has led donors and governments to pursue more integrated national plans, with a view to capitalizing on potential synergies between sectors in each location. a For example, countries as diverse as Nepal, Haiti, and Kenya all promote intersectoral coordination and interministerial collaboration to achieve common a This wave of attention to integration is, in some ways, a return to efforts of the 1970s articulated by Winikoff: “Is there any way in which programs can be effective if they focus on any single problem without being com- bined in a so-called ‘integrated attack’?” 1 Some of the most prominent interventions of that era were integrated rural development projects, 2 inspired in part by the systems- modeling approach popularized by Limits to Growth. 3 What has changed since then is a sharp reduction in the extent of global poverty, 4 and three decades of accumu- lated capacity to achieve and evaluate success in eradicat- ing the remaining pockets of geographically concentrated extreme deprivation, combined with breakthroughs in re- search capacity ranging from spatial geocoding to genome sequencing. doi: 10.1111/nyas.12352 1 Ann. N.Y. Acad. Sci. xxxx (2014) 1–12 C 2014 New York Academy of Sciences.
Transcript
Ann. N.Y. Acad. Sci. ISSN 0077-8923
ANNALS OF THE NEW YORK ACADEMY OF SCIENCESIssue: Paths of Convergence for Agriculture, Health, and Wealth
Agriculture, nutrition, and health in global development:typology and metrics for integrated interventions andresearch
William A. Masters,1 Patrick Webb,1 Jeffrey K. Griffiths,1,2 and Richard J. Deckelbaum3
1Friedman School of Nutrition Science and Policy, Tufts University, Boston, Massachusetts. 2Department of Public Health,School of Medicine, Tufts University, Boston, Massachusetts. 3Mailman School of Public Health, Columbia University, NewYork, New York
Address for correspondence: William A. Masters, Friedman School of Nutrition Science and Policy, Tufts University, 150Harrison Ave., Boston, MA 02111. [email protected]
Despite rhetoric arguing that enhanced agriculture leads to improved nutrition and health, there is scant empiricalevidence about potential synergies across sectors or about the mix of actions that best supports all three sectors. Thegeographic scale and socioeconomic nature of these interventions require integration of previously separate researchmethods. This paper proposes a typology of interventions and a metric of integration among them to help researchersbuild on each other’s results, facilitating integration in methods to inform the design of multisector interventions.The typology recognizes the importance of regional effect modifiers that are not themselves subject to randomizedassignment, and trade-offs in how policies and programs are implemented, evaluated, and scaled. Using this typologycould facilitate methodological pluralism, helping researchers in one field use knowledge generated elsewhere, eachusing the most appropriate method for their situation.
Keywords: integration; agriculture; health; nutrition; research; development
Introduction
Agriculture, nutrition, and health are interrelatedthrough biological and behavioral pathways thatmake these sectors distinctively tied to ecologi-cal and socioeconomic conditions. Place-based fac-tors constrain and influence peoples’ nutrition andhealth outcomes at every scale and in each context.For example, rainfall, temperature, and soil nutri-ents interact with crop genetics to influence plantgrowth and food quality as well as farm incomes,which in turn influence food purchase and health-seeking behaviors, all of which combine to influenceindividuals’ nutrition and health outcomes. Ecolog-ical conditions also influence the growth and repro-duction of pathogens, parasites, and disease vectorsof all kinds, weighing heavily on the effectiveness ofinterventions aimed at meeting development goals.
Since the 2007/2008 food-price crisis, increasingattention to agriculture and nutrition as a factorin health and development has led donors andgovernments to pursue more integrated national
plans, with a view to capitalizing on potentialsynergies between sectors in each location.a Forexample, countries as diverse as Nepal, Haiti, andKenya all promote intersectoral coordination andinterministerial collaboration to achieve common
aThis wave of attention to integration is, in some ways,a return to efforts of the 1970s articulated by Winikoff:“Is there any way in which programs can be effectiveif they focus on any single problem without being com-bined in a so-called ‘integrated attack’?”1 Some of the mostprominent interventions of that era were integrated ruraldevelopment projects,2 inspired in part by the systems-modeling approach popularized by Limits to Growth.3
What has changed since then is a sharp reduction in theextent of global poverty,4 and three decades of accumu-lated capacity to achieve and evaluate success in eradicat-ing the remaining pockets of geographically concentratedextreme deprivation, combined with breakthroughs in re-search capacity ranging from spatial geocoding to genomesequencing.
Integrating agriculture, nutrition, and health Masters et al.
goals around enhanced nutrition, health, and foodsecurity.5–7 Indeed, the call for integrated actionrepresents a new global agenda, as highlighted bythe L’Aquila Joint Statement on Global Food Secu-rity, which argued that “food security, nutrition andsustainable agriculture must remain a priority issueon the political agenda, to be addressed through across-cutting . . . approach.”b,8
The potential gains from integrated interventionscall for enhanced research methods that explicitlyaccount for biological and ecological relationshipsat their natural scale, to reveal regional-level ef-fects that may be quite different from the sum ofindividual-level changes potentially observed in alimited-scale randomized control trial. However, theincreasing focus of policies and programs on inter-sectoral integration to solve location-specific prob-lems poses deep challenges for researchers, regard-ing how the value added from integration is mostappropriately measured. The challenge is illustratedby controversy and confusion regarding what con-stitutes appropriate evidence in relation to multi-sectoral programming at the village level9 or in eco-nomic development more generally.10,11 Programsthat seek to link food, water, health, and nutritionat a regional scale are already being implementedaround the globe, but few have been rigorously ana-lyzed, and most remain focused on measuring siloedoutcomes within sectors and then aggregating theseas if their impact equaled the sum of their parts.To measure synergies, we need a research agendathat explicitly addresses intersectoral linkages andthe cost-effectiveness, replicability, and scalabilityof integrated processes.
This paper describes plausible causal pathwaysfrom agriculture, nutrition, and health interven-tions to measurable outcomes at their natural scaleand provides a new typology of interventions and
bThe L’Aquila Joint Statement was endorsed by the G8countries plus Algeria, Angola, Australia, Brazil, Den-mark, Egypt, Ethiopia, India, Indonesia, Libya (Presi-dency of the African Union), Mexico, the Netherlands,Nigeria, the People’s Republic of China, the Republic ofKorea, Senegal, Spain, South Africa, Turkey, the Commis-sion of the African Union, the FAO, the IEA, the IFAD,the ILO, the IMF, the OECD, the Secretary General’s UNHigh-Level Task Force on the Global Food Security Crisis,the WFP, the World Bank, and the WTO.
outcomes to help researchers estimate the effective-ness of combined program elements. By naturalscale, we mean the smallest demographic or geo-graphic unit over which the intervention can occurand potentially be replicated or randomized. Thenatural scale for medical intervention is typicallythe individual, whereas for agriculture it may bean ecosystem or marketplace. Various aspects of anintervention may have different scales and units ofobservation. The simplest such problems can read-ily be handled with clustered research designs, butcombining interventions at different scales greatlyincreases the problem’s complexity.
Although our typology of interventions and out-comes could be applied to any number of inte-grated sectors, we focus here on just the agriculture–nutrition–health nexus that has received heightenedattention recently in the context of successive worldfood-price crises, new global commitments to scal-ing up nutrition actions, and concerns over the eco-logical stability of natural resources that underpinglobal food production and health outcomes. Theadoption of this kind of integration typology wouldallow researchers to describe policy or program in-terventions and choose research methods to eval-uate their success in ways that reveal system-levelinteractions among sectors and across regions thatare typically ignored by traditional evaluations fo-cused on individual changes in individual subjects.
Interactions across scales and sectors:a typology of interventions and outcomes
Table 1 provides a typology of program interven-tions and outcomes designed to reveal interactionsbetween agriculture, nutrition, and health at var-ious scales. The rows show specific interventionsand desired outcomes in the agricultural, nutrition,and health sectors, as conventionally defined. Theouter columns show place-specific attributes of re-gions, which serve as effect modifiers for person-specific interventions and outcomes shown in theinner columns. The potential causal mechanismsinvolved connect everything with everything else,both horizontally between interventions and resultsat various scales, and vertically among these sec-tors as well as others such as education, finance,or infrastructure. As shown in the bottom row ofTable 1, the research techniques needed to identifythese causal relationships depend crucially on thegeographic scale of analysis.
• Local normsregarding diet,infant feeding,hygiene, andsanitation
Health interventions andoutcomes
• Local water,sanitation, andhygiene
• Local deworming,vaccination, andvector control
• Local health systemservices forall-causeprevention andtreatment
• Access to orprovision ofhealthcareproducts, services,and information
• Lower morbidityand mortality
• Enhanced humanproductivity
• Improved maternaland child health
• Local exposure todisease
• Local supply ofhealthcare services
Integrated research methods Impacts and causal mechanisms can beidentified using both natural and controlledexperiments varying exposure acrossindividuals
Impacts of region-wide interventions can be identified only by inference from observational data,using knowledge of individual-level causal mechanisms
The middle columns of Table 1 list interventionsand outcomes that can be identified at the levelof individuals and households. The causal relation-ships involved have long been subject to both ob-servational and experimental study, primarily acrossrows within the sectoral boxes. Most often, a specificintervention is associated with a specific outcome.
Ideal experimental designs using fully double-blindassignment, preregistration of research protocols,and other criteria may not be feasible for manyinterventions, but researchers are increasingly ableto use some degree of randomization from bothcontrolled and natural experiments to identifymechanisms and magnitudes of causal linkages. In
Integrating agriculture, nutrition, and health Masters et al.
each case, the unit of observation for these impacts isan individual person, exposed to a particular treat-ment or intervention at a particular time and place.
The outer columns of Table 1 show regional ef-fect modifiers. These attributes of a particular placeare themselves subject to change, influenced byregional-scale policies and programs, but by defini-tion they are conditioning factors that do not varyamong the population of intended beneficiaries orobserved subjects affected by the specific interven-tions listed in the middle columns of this typology.Such places can be defined at any scale up to andincluding the world as a whole. The causal effect ofinterventions in the middle columns can be identi-fied through randomization over a variety of scales,for example, in studies that randomize over clustersor exploit natural experiments at the level of a dis-trict or state, but the impacts of many importantinterventions in the outer columns of Table 1 mustbe inferred from observational data and knowledgeabout causal mechanisms in the inner columns.
The typology of interventions and their effectsreveals how increasingly integrated agriculture–nutrition–health policies and programs can andshould be subject to increasingly diverse researchmethods. Causal mechanisms within and acrossrows in the inner columns can be identified bycomplex research designs designed to capture in-teraction effects, and variation in the conditioningfactors around any given study can then be ad-dressed by conducting multiple studies in diverselocations. A meta-analysis of those results can thenbe used to identify which causal relationships are ro-bust to variation in these potential effect modifiers.But by definition there is no random assignmentof the regional-scale phenomena, and their role inany given causal pathways can be understood onlythrough inference from observational data, inter-preted using knowledge of underlying causal mech-anisms.
For example, the experiment by Miguel and Kre-mer, which studied the introduction of dewormingmedication in Kenyan schools, can be credited withsharp increases in funding for these interventions inrecent years.12,13 Their results were regional in na-ture, in the sense that much of deworming’s impactoccurred not only to the individual child who re-ceived medication, but through ecological effects toother children. Relatively few studies are capable ofdetecting such effects: for deworming, only five of
the 42 trials used in the most recent “Cochrane Re-view” were clustered to detect such externalities,14
and the updated series in the Lancet on maternaland child undernutrition dropped deworming as anevidence-based proven intervention for nutrition.15
Results also vary by the degree to which children areexposed to helminthic infections in the first place.The Miguel and Kremer intervention was conductedin a region of high and rising infection rates, andnone of the other studies were conducted acrosssufficiently diverse regions to identify interactioneffects between treatment and exposure. Observa-tional studies are needed to capture interaction withnonrandom prior conditions. For the effect of par-asitic diseases on regional development, Bleakley16
showed how differences in initial disease prevalenceaffected the impact of hookworm eradication in theUnited States, and McMillan et al.17 documentedhow onchocerciasis control in Burkina Faso changedagricultural and other institutions in river valleyswhere the disease had been most widespread.
Cross-sector interactions can originate in any sec-tor. For example, the top left corner of Table 1 showsregional interventions in agriculture, including thecreation of better, more locally adapted crop andlivestock techniques, market arrangements, and en-vironmental innovations. Such interventions canmake a region more prosperous and hence more ableto afford and implement all of the other interven-tions. Similar region-specific programs exist in thenutrition and health sectors, as shown in the secondand third rows of Table 1. Region-wide improve-ments affect the availability of any given product orservice in a given locality, and also change individu-als’ access to that product or service. The middle twocolumns of Table 1 show how interventions that canbe provided to individuals translate into outcomesthat can be measured at the individual level, firstin agriculture such as crop or livestock productiv-ity or farm income, diets, and consumption of foodand specific nutrients, then for nutritional outcomesincluding height and weight anthropometrics, andbiomarkers in blood, urine, and tissue samples thatare known to correlate with immunity to disease,cognition, and other aspects of maternal and childhealth.
The final column of Table 1 shows results de-fined at a regional scale, such as the wage rate andemployment opportunities that are available toworkers, food prices, and other market outcomes.
Masters et al. Integrating agriculture, nutrition, and health
This kind of outcome arises from the interactionamong people, and is shared among the people ina given region. For health, the comparable regionalphenomena include disease transmission, which isa function of place, population density, and popula-tion (herd) immunity. The analysis of anthropome-try, biomarkers, and disease prevalence rates can ofcourse be analyzed at the regional level, for example,through documented cases in a given population,but the underlying data are individual in nature.
The typology presented in Table 1 offers a partialcatalog of policy, program, and intervention po-tentials at various scales, linked to expected resultsthat researchers might measure. Causal pathwaysbetween interventions and results could run in alldirections. For example, agricultural interventions,which decrease food and water contamination (on-field aflatoxin control, penning livestock away fromhuman water sources, controlling schistosomiasisin irrigation programs) can be posited to causallyinfluence both nutrition and health. The typologyreveals the inherent need for methodological plu-ralism when addressing increasingly integrated re-search questions: first to take account of linkagesbetween agriculture, nutrition, and health withinindividual households, but especially to considerfeedback effects at the ecological or regional scalethat might drive population-wide impacts.
Impact pathways: metrics of integrationbetween agriculture, nutrition, and health
Researchers can draw on the typology of Table 1using a variety of logical frameworks regarding thepossible pathways between interventions and out-comes. Studies addressing agriculture–nutrition–health linkages typically present their causal frame-work in a flow chart such as Figure 2, which isbased on a widely used logical framework developedby the International Food Policy Research Institute(IFPRI).18 In this diagram, causality flows from leftto right, as assets and employment generate incomethat is spent on food and care practices that entermaternal and child nutrition and health. The pos-sibility of reverse causality is shown in arrows fromright to left around the outside of the flow chart.Such flow charts are composed of elements such asthose in our Table 1, with each box representingsomething that is potentially measurable at one oranother geographic scale, and the arrows represent-
ing directions of causality between the measurableelements.
The causal framework illustrated here augmentsthe IFPRI framework with potential confoundersfor any observed relationship: for example, in re-cent years developing countries have experienceddiverse and rapid changes in their labor–land ratios,age structure, and gender balance, posing variousdemographic challenges, even as climate change anddisease exposure also vary widely. These trends anddifferences are not just random noise, but system-atic confounders that threaten the external validityof any localized study, and may mask the effect ofa multilocational program or trial. For any givenset of interventions in the typology of Table 1, acausal framework, such as Figure 1, permits pro-gram design and evaluation to take account of po-tential synergies and interaction effects in ways thatare potentially measurable.
Graphical frameworks such as Figure 1 can beexpanded into infinitely complicated spaghetti di-agrams, or simplified into highly reductionist lin-ear models. Whatever their degree of granularity,however, they allow us to define and measure pro-gram integration in a novel way. Using Figure 2 orany other logical framework for program impact,we propose to define metrics of integration for op-erational research in two distinct dimensions: thebreadth of integration across various arrows andboxes in the causal framework, and the depth towhich those program elements are delivered to-gether in any given intervention on the ground. Theproduct of these two dimensions could then be usedto measure the overall extent of integration.
Conceptually, a metric for the breadth of inte-gration across causal pathways could start at zero(for a highly specialized, single intervention such asvitamin A supplementation that is not integratedwith anything else) and go to infinity (for a uni-versal intervention that does everything in the mostdetailed imaginable causal framework). In practice,a useful metric might use just three categories, withan intervention that spans one or two elements ina given framework being classified as having someintegration (a score of 1), an intervention that spansthree to five of the framework’s elements being clas-sified as having more integration (a score of 2), andone that spans six or more elements having an evenwider breadth of integration (a score of 3). Sucha measure could take values of 1, 2, or 3. What is
Integrating agriculture, nutrition, and health Masters et al.
Figure 1. Example of a causal framework with confounders used for metrics of integration. Causal framework in square boxesand shadowed text is that of Ref. 18, augmented by the authors with social and ecological confounders in dashed ovals and italictext.
actually being more or less integrated would dependon the causal framework being used to identify path-ways, but a larger number would always correspondto a greater breadth of integration.
Similarly, a metric for the depth of integrationin program implementation could be scored usingthree categories. The simplest form of integratedprogramming is colocation, in which separate pro-grams are placed adjacent to each other in a givenvillage or region. A deeper form involves organi-zational partnerships and mergers, in which staffmembers from different organizations are asked towork together in each location. The most extremeform of integration is cross-training and resourcesharing, in which individuals in a multifunctionalorganization are asked to provide multiple synergis-tic services. These could be scored as 1 (colocation),2 (partnerships or mergers between organizations),or 3 (cross-training of multifunctional individuals).Again, what is being integrated would vary by pro-gram, but the metric is designed so that a larger
number would always correspond to a greater depthof integration.
Clearly, a greater extent of integration inbreadth or depth could improve a project’s cost-effectiveness, if integration allows the organizationto take advantage of synergies between programelements. But greater integration could also re-duce cost-effectiveness if it prevents the organiza-tion and its staff from specializing in what they dobest. In any given setting, there might be an in-verted U-shaped relationship between a project’scost-effectiveness and its breadth or depth of inte-gration, with the most effective level of integrationbeing somewhere between pursuing each elementin separate silo projects and pursuing every elementsimultaneously in undifferentiated all-purpose pro-grams. The measures of integration that we proposeoffer the simplest possible scale with which to iden-tify such a relationship, if it exists, with three lev-els of integration from some (1) to more (2) andmost (3).
Masters et al. Integrating agriculture, nutrition, and health
Table 2. Breadth and depth of integration in a sample of 17 Non-governmental Organization (NGO) programs
Breadth of program (number of elements)
1–2 3–5 6+Depth of integration between elements
1. Colocation of program elements 2
2. Partnership among organizations 1 1 8
3. Cross-training and multifunctionality 1 1 3
Source: Data shown are the number of programs in each breadth and depth category, from authors’ classification of17 integrated agriculture-nutrition projects implemented since 2008 by Catholic Relief Services (CRS), from internalCRS file data.
An opportunity to apply this metric of integra-tion depth and breadth arose in 2013 for an internalreview of agriculture–nutrition integration inprojects implemented by Catholic Relief Services(CRS). That review covered all CRS English-language proposals involving agriculture and nu-trition funded between 2008 and 2013 in whichthere was explicit use of the terms “integrated” or“integration” or other unambiguous linkage, suchas interventions to control aflatoxin in food crops.The result was a sample of 17 projects implementedacross Africa, Asia, and Latin America.
To measure the integration in this sample ofprojects, we began by constructing a causal frame-work similar to Figure 1, in which the cells weredefined as all of the distinct program elements actu-ally used by CRS in its work on agriculture and nu-trition, and the arrows were causal pathways show-ing how those program elements affect developmentoutcomes. Two consultants then scored the propos-als in terms of breadth (number of different programelements in each proposal) and depth (whether twoor more elements were proposed to be colocated,delivered by different organizations in partnershipfor the project, or delivered through cross-trainingof personnel). Each proposal was scored separatelyby the consultants, who generally reported identi-cal scores but in a few cases identified ambiguitiesthat required e-mail correspondence with CRS fieldstaff to clarify the degree of integration linked to ourframework.
Results of this pilot test are summarized inTable 2, showing that 13 of the 17 projects in-cluded six or more program elements, and thateight of them did so through partnerships amongorganizations (subgrantees). Five projects usedcross-training and multifunctionality, mostly for
projects with more than six program elements. Fromour initial observation, it appears that CRS’s inte-grated agriculture–nutrition projects typically aimfor relatively large numbers of program elements(more than six), with an intermediate depth of in-tegration between them (using partnerships amongspecialized organizations, rather than colocation orcross-training). These classifications were based ontextual analysis of project proposals, rather thanfield observations, and serve primarily to demon-strate the feasibility of measuring integration in thisway. Next steps might include describing trendsand patterns of integration across multiple agen-cies and then testing hypotheses such as whetherthe more integrated programs with more diverse el-ements are actually more cost-effective than morespecialized programs and whether partnerships areactually more cost-effective than colocation orcross-training of multifunctional service deliverypersonnel.
The pilot data in Table 2 show that classifyingprojects using this metric is feasible and could use-fully allow researchers to estimate how the degreeof integration among interventions is associatedwith expected impacts. In so doing, a variety ofresearch designs are likely to be useful, as detailedlater.
Trade-offs in research: integration indelivery science, impact evaluation, andstudy design
To identify causal pathways and estimate effect sizesfor more or less integrated programs and policies,researchers can choose among a complex set ofmethodologies. Choices among methods always in-volve trade-offs, as illustrated in Figure 2. This di-agram illustrates a sequence of three-dimensional
Integrating agriculture, nutrition, and health Masters et al.
Figure 2. The many dimensions of integrated program delivery, impact evaluation, and research relevance.
spaces in which any intervention and researchproject might fall. In each three-dimensional space,the more desirable direction is upward along thevertical axis, to the right along the horizontal axis,and diagonally along the third axis showing depth,so that the most desirable situations are at the peakpoint shown at the center of each cube. The threecubes illustrate dimensions that characterize differ-ent aspects of integration in research: first, deliveryscience (how programs are implemented); second,evaluation and measurement of impact (whetherthe program met its objectives); and finally, the rel-evance for future policies of a given research design(whether the research itself is influential). Successin each of these can lead to a virtuous circle of im-provement. Conversely, weakness in any dimensionwill limit what can be learned.
Delivery science and program implementationThe first set of trade-offs in Figure 2 illustrates thedelivery science aspect of integrated programs. De-livery science focuses on overcoming implementa-tion constraints and promoting cost-effectivenessat scale. This emerging field addresses the “how” aswell as the “what” and “where” of a given interven-tion. Figure 2 shows the degree to which programdelivery moves from a total lack of effectivenessalong each axis toward the highest possible degreeof success.
In Figure 2A, the vertical axis illustrates the ap-propriateness of an intervention’s design. This mightbe measured as the degree to which iron folate sup-plements are effectively targeted to individuals atrisk of anemia, for example, by taking account ofwhether malaria prevention and control measuresare in place where malaria risk is endemic. In thiscase, appropriateness relies on researchers’ a prioriknowledge that iron folate supplementation mightbe desirable only under certain conditions.19 Sim-ilarly, appropriate design may focus on promotingexclusive breastfeeding and birth spacing in popula-tions where age of first pregnancy is low and fertilityrates are high. Resolving anemia and reducing ma-ternal pregnancy risks will have effects on nutritionof the mother, but also on the next generation ofchildren to be born, which has long-term relevanceto productivity (including in agriculture).
The horizontal axis of Figure 2A illustrates fidelityof implementation, which might be measured as thedegree to which an intervention actually reaches theintended beneficiaries as designed, which might de-pend, for example, on how program staff are com-pensated and the incentives they have to screen ben-eficiaries according to defined protocols. Those twodimensions are the typical focus of research on in-tervention effectiveness, typically in controlled trialswhere design and delivery are closely monitored fora specific sample of individuals.
Masters et al. Integrating agriculture, nutrition, and health
The third dimension, along the depth axis, showsthe program’s magnitude of funding and effort rel-ative to the geographic region where results are ex-pected. This dimension is especially important be-cause of the nonlinearities and threshold effects thatcharacterize bioecological systems, for example, indisease-control vaccination programs that require ahigh level of coverage to achieve herd immunity.
Taken together, these three dimensions of deliv-ery science are critical to any understanding of howto maximize the benefits of complex programming.Design and implementation strategies are of courseimportant, but analysis of fidelity, coverage, and in-tensity of application of programming is essential tothe identification of pathways toward impact.
Program evaluation and measurement ofimpactGiven the quality of program implementationalong the three dimensions of Figure 2A, theevaluation methods that a researcher might use canbe described along the three dimensions of Figure2B. Here, the vertical axis shows the sensitivity ofmetrics relative to expected results. For example,measurement of wasting prevalence, defined asthe fraction of children below a given level ofweight-for-height, would miss relevant changes inweight-for-height among children at other levels ofbody weight that might matter for health.20 Mea-sures that capture both under- and overnutritionmay be particularly valuable given the increasingprevalence of both forms of malnutrition. Similarly,growing more food can result in less impact onnutrition where that food is contaminated withmycotoxins that potentially impair immunity ofthe consumer to diseases. The horizontal axisshows the appropriateness of study design, withrespect to whether observed changes were actuallycaused by the intervention or might be due to otherfactors. The value of data depends not only onmeasurement accuracy, but also on how treatmentswere randomized and other aspects of data quality.
In Figure 2B, the third dimension on the depthaxis shows the intervention’s magnitude of impact.That dimension is closely related to the magnitudeof effort in Figure 2A, but it also depends on the pro-gram’s design and fidelity of implementation. Thenet result is crucial for program evaluation becausethe results of low-magnitude programs are difficultto measure with statistical significance. An interven-
tion’s magnitude is defined relative to the adminis-trative or ecological region of interest, which is sub-ject to changes other than the program intervention.Distinguishing signal from noise depends in part onsample size. In clinical settings, the expected mag-nitude of impact relative to variance associated withother factors informs study design and sample sizethrough power calculations, but in field research thesample size is often dictated by the intervention it-self, for example, through the number of distinctoperations that can be randomized.
Research design and policy relevanceFigure 2C illustrates the quality of study design,in terms of the validity and relevance of itsresearch findings. Here, the vertical axis showsthe internal validity of causal attribution withinthe observed population, while the horizontalaxis shows its external validity for other peopleto whom the intervention might apply elsewhere.Internal validity is typically highest in double-blind,placebo-controlled randomized trials, but the de-gree of internal validity depends on how the specificprotocol is implemented. For example, if behavioralresponses are involved, to avoid the Hawthorneeffect it may be preferable to conduct trials in wayssuch that subjects do not even know they are in atrial.
The horizontal axis of Figure 2C represents ex-ternal validity, namely whether the effect observedin a trial can be generalized to other populations.Research with high internal validity might not begeneralizable, perhaps because the interventionwould be implemented differently, or becausesubjects are not representative of the generalpopulation. Returning to the previous example ofiron folate supplementation to combat anemia, aclinical study of its effectiveness among childrenin a malaria-free area may produce a highly validattribution of causality for that population, eventhough that study may have very little externalvalidity for anemic children in other locations. Eval-uations of integrated agriculture–nutrition–healthstudies are particularly vulnerable to low externalvalidity due to the location-specific, agroecologicalconstraints they address. This puts a premium ondrawing appropriately sized representative samplesfor data collection, which in turn makes it difficultto implement the randomization strategies neededfor internal validity.
Integrating agriculture, nutrition, and health Masters et al.
In Figure 2C, the third dimension of depth showsthe study’s policy relevance. This attribute of studydesign is different from either internal or exter-nal validity. The relevance of a study depends onwhether the metrics and methods are salient to adecision maker’s needs. Relevance may thereforeinvolve the timing of a study, for example, to informdecisions during the period when decision makersare responsive to research results. Relevance mayalso involve metrics, for example, to inform deci-sions that are bound by law or other constraintsto target a specific kind of change such as the Mil-lenium Development Goals (MDGs). However, rel-evance can also color other aspects of study design.Studies may have high validity but low relevanceto policy decisions. Researchers often face trade-offs between evaluations tailored to specific decisionmakers or to address policy questions and those de-signed to produce results of the highest internal orexternal validity.
Taken together, the three panels of Figure 2 de-scribe how integrated research can drive betterproject implementation and the best possible out-comes in Figure 2A, the highest measured resultsin Figure 2B, and the greatest policy relevance inFigure 2. Not all research methods will succeed inall dimensions, but research projects that take all ofthese dimensions into account are more likely to besuccessful.
Currently, such research activities are all too rare.A gap analysis undertaken by Hawkes et al. reviewedongoing and planned research projects around theworld that focused explicitly on understanding thelinkages between agriculture and nutrition.21 Theauthors identified 151 research undertakings, morethan half of which targeted Sub-Saharan Africa. Ofthe 151 studies characterized, 92 reported a partic-ular focus on nutritional impacts for women andchildren. Despite this explicit focus on (1) inter-actions among sectors and (2) nutrition outcomesfor key target groups, the vast majority of the re-search so identified approached their subject matterthrough one of the three cubes presented in Fig-ure 2, while none at all attempted to link data gath-ering and analysis across all three research domains.As Hawkes et al. stated, “no single project com-pletes the research chain fully.” While some studiescome close, the main links missing in the researchchains identified by that gap analysis included pay-ing attention to value chains (beyond production
of commodities and their consumption), and to lo-cal and regional food environments (determined bymarket access, retail price dynamics, and local in-tegration with national and international systems),as well as appropriate measures of actual nutritionimpacts using valid methodologies. Such gaps areseen as preventing a more complete understandingof the full pathways of change, and as such, a betterunderstanding that “the most appropriate methodsand metrics is [sic] needed to be able to translateresearch into practice, and plan and design futureresearch.”
Conclusions: toward more effectiveintegrated research methods
Policies and programs designed to overcomelocation-specific biological and ecological con-straints in agriculture, nutrition, and health havedistinctive features that call for a new languageof intervention design, delivery, and evaluation. Ithas been argued that such language is essential toprogress in attaining global targets for health andnutrition.c This paper offers a typology of interven-tions and outcomes, with corresponding metrics ofintegration among them, to help guide evaluationapproaches that can help researchers capture keyfeatures of program integration in ways that betterinform policymaking and program design. Thesefeatures cannot all be tested empirically, especiallynot in all combinations, but this typology allows usto think about integration in a way that could betested in practice.
A first dimension of special concern for integratedagriculture–nutrition–health programs is their ge-ographic scale. For ecological as well as adminis-trative reasons, programs and evaluations are oftenscale-dependent, with interventions and expectedresults that are defined over a geographic regionrather than an individual person or household. Thetypology proposed here therefore begins with a cat-alog of the interventions and results that are defined
c Shekar et al. state that the progress toward improvedintegrated programming aimed at enhancing nutritionremains constrained by the “absence of explicit trainingin these competencies in leading universities, absence ofresearch on the delivery sciences including operational re-search, lack of or poor-quality assessments, scarce fund-ing for delivery research, and the reluctance of journals topublish it.”22
Masters et al. Integrating agriculture, nutrition, and health
at the geographic as opposed to the individual level,and then provides three-dimensional criteria alongwhich a given program intervention and evaluationmethod could be situated. These dimensions iden-tify the relevance of delivery science for character-izing how an intervention is implemented, beyondthe characteristics of what, where, and for whomthe program operates. The typology then identifiesspecific needs of integrated programs in terms ofevaluation methods, and the specific issues in studyvalidity and policy relevance that arise for these pro-grams. Metrics of integration, in turn, capture boththe breadth of integration across diverse programelements, and the depth with which those elementsare integrated in a given program on the ground. Thevalue of such metrics can be tested retrospectivelyand prospectively in relation to selected groupsof integrated interventions at various scales ofoperations.
The typology, metrics, and trade-offs describedhere can help researchers design more effective stud-ies by matching the most appropriate methods tospecific interventions and effects. No single researchmethod can answer all questions. Each study addsto previous work by extending it in particular di-rections, looking under the lamppost illuminatedby a particular approach and set of circumstances.The challenge posed by the need for greater integra-tion in research methods will not be met by a neworthodoxy in research methods, but by methodolog-ical pluralism. In so doing, the challenge of integra-tion is not only to match problems with appropriatemethods, but more importantly to bring a widerrange of previous research using other methods tobear on each new study. For example, credible ob-servational studies of regional differences must takeaccount of what is known about individual-level re-sponses from experimental studies, and vice versa: acontrolled experiment must attend to both externaland internal validity, by appropriate choice of loca-tion and other circumstances. Researchers will alsoneed increasing sophistication about the trade-offsillustrated in Figure 2, seeking the highest possi-ble impact in terms of program delivery, evaluationdesign, and policy relevance.
As the pursuit of global development targets un-der the MDGs reaches the 2015 deadline, new quan-tified, time-bound goals for international develop-ment are needed. These will be defined in terms ofmetrics that have major implications for program
design and evaluation. Researchers will need to de-sign studies that capture the impact of cross-sectoralprograms on the new targets. These are likely to gobeyond the siloed, sector-specific goals of the pastand offer greater understanding of how people in-teract with their resources in specific locations. Theconcepts presented here are designed to be of broadapplicability, permitting program designers and re-searchers to identify specific characteristics likely toproduce the highest possible measurable impactsfrom the interventions with which they work.
Acknowledgments
The authors thank the United States Agency for In-ternational Development for making this paper pos-sible, through the USAID Feed the Future Innova-tion Labs for Collaborative Research on Nutrition inAfrica and Asia. We are also grateful to Valerie Rhoeat Catholic Relief Services for permission to use in-ternal program review results conducted under herleadership by Mackenzie Sehlke, Nadira Saleh, andWilliam Masters, using the methodology describedin this paper.
Conflicts of interest
The authors declare no conflicts of interest.
References
1. Winikoff, B. 1978. Nutrition, population, and health: someimplications for policy. Science 200: 895–902.
2. Ruttan, V. 1975. Integrated rural development programs: askeptical perspective. Int. Dev. Rev. 17: 9–16.
3. Meadows, D.H., D.L. Meadows, J. Randers & W.W. Behrens.1972. The Limits to Growth. New York: Universe Books.
4. Lam, D. 2011. How the world survived the population bomb:lessons from 50 years of extraordinary demographic history.Demography 48: 1231–1262.
5. Government of Haiti. 2012. Politique nationale de nutrition.Port au Prince, Haiti: Ministere de la sante publique et de lapopulation.
6. Government of Nepal. 2012. Multi-Sectoral Nutrition Planfor Accelerating the Reduction of Maternal and Child Under-Nutrition in Nepal. Kathmandu, Nepal: National PlanningCommission.
7. Government of Timor-Leste. 2012. National Nutrition Strat-egy 2012–2017. Dili, Timor-Leste: Ministry of Health. Draftas of September 11, 2012.
8. G8 (Group of Eight Forum Countries). 2009. “L’Aquila JointStatement on Global Food Security. Accessed January 5,2014. http://feedthefuture.gov/sites/default/files/resource/files/afsi_jointstatement_2009.pdf.
9. Pronyk, P. 2012. Errors in a paper on the Millennium Villagesproject—Editors’ reply. Lancet 379: 1946–1947.
Integrating agriculture, nutrition, and health Masters et al.
10. Banerjee, A. & E. Duflo. 2011. Poor Economics: A RadicalRethinking of the Way to Fight Global Poverty. New York, NY:Public Affairs.
11. Deaton, A. 2010. Instruments, randomization, and learningabout development. J. Econ. Lit. 48: 424–455.
12. Miguel, E. & M. Kremer. 2004. Worms: identifying impactson education and health in the presence of treatment exter-nalities. Econometrica 72: 159–217.
14. Taylor-Robinson, D.C., N. Maayan, K. Soares-Weiser,et al. 2012. Deworming drugs for soil-transmitted intesti-nal worms in children: effects on nutritional indicators,haemoglobin and school performance. Cochrane DatabaseSyst. Rev. 11, Art. No. CD000371.
15. Bhutta, Z., J. Das, A. Rizvi, et al.; for Lancet Maternal andChild Nutrition & Interventions Review Groups. 2013. Evi-dence based interventions for improving maternal and childnutrition: what can be done and at what cost? Lancet. 382:452–477.
16. Bleakley, H. 2007. Disease and development: evidence from
hookworm eradication in the American South. Q. J. Econ.122: 73–117.
17. McMillan, M.S., W.A. Masters & H. Kazianga. 2013. Dis-ease Control, Demographic Change and Institutional De-velopment in Africa. National Bureau of Economic Re-search Working Paper No. 19245. Cambridge, MA: NBER(http://www.nber.org/papers/w19245).
18. Gillespie, S., J. Harris & S. Kadiyala. 2012. The Agriculture-Nutrition Disconnect in India: What Do We Know? FPRI Dis-cussion Paper 01187. Washington DC: IFPRI.
19. WHO. 2012. Guideline: Daily Iron and Folic Acid Supplemen-tation in Pregnant Women. Geneva: World Health Organiza-tion.
20. Bhagowalia, P., S.E. Chen & W.A. Masters. 2011. Effects anddeterminants of mild underweight among preschool chil-dren across countries and over time. Econ. Hum. Biol. 9:66–77.
21. Hawkes, C., R. Turner, J. Waage, et al. 2012. Current andplanned research on agriculture for improved nutrition: amapping and a gap analysis. A report for DFID. August 21,2012.
22. Shekar, M. et al. 2008. Delivery sciences in nutrition. Lancet371: 1751.
