Quantitative Methods

Quantitative Methods• Attempts to generalize across some

“population”• Techniques for testing strength of

relationship, probability that it is “true”• Tradeoffs between sample size (which

helps in generalizing, testing), depth, skills, and cost of study

Sampling• Essential for statistical testing, generalizing• Guard against biases• Need to identify “population”• Sampling methods

– Select randomly– Stratify by hypothesized key variables (H/M/T in irrigation)– Multi-stage: e.g. select communities then households– NOT “convenience sample”!!

• These principles are also useful for qualitative studies!! (How representative are your findings?)

What Level of Analysis?• Potential Levels of Interest

– Intra-household (do outcomes differ for different members of the family)

– Household– Particular group(s) within the community?– Community-level outcomes– Groups that function above level of

community?

What Level Data Collection?

• Even if level of analysis is household or intra-household, often very important to collect some data at the community level:– Agro-ecological Data– Distance to Market– Community Infrastructure– Social “Cohesion”– Cooperative Capacity

Alternatively, Even if Community, Might Want Household-Level Data

• Heterogeneity within the community: distribution of wealth (land or livestock holdings), how many with higher education, proportion of different ethnic groups etc.– Potential Difficulty: If you are collecting

household and community-level data, some people are tempted to use “household averages” within a community, but… very few studies include enough households per community so that the sampled households are representative; averages of sampled households can be very misleading.

Data Collection Methods• Individual or household survey• Key informant interview

– NGOs, officials, local leaders, poor women, etc.

• Focus groups

Consider what you can get from each– Private or public knowledge– Norm or exceptions

Mapping Data to MethodsData Key

informantFocus group

Secondary

HH Survey

Water scarcity

Castes Landhold Income WUA particip

WUA structure

Complications• Usually, we have “abstract” concepts

(e.g. social cohesiveness) that require operationalization – How will it be measured, concretely?

• Even if variable at first seems easy… may not be! e.g. – size of landholdings in areas without

titles– crop yield and how much of crops sold – often different and/or non uniform

measures

What to Collect• Preparation is key…Preparation is key… Think through

variables you want to explain AND variables you think are explanatory

• Draw out diagram—what is likely to cause or influence other variables?

• Distinguish between:– Exogenous: externally generated, not

influenced by other variables in the set– Endogenous: those that are influenced

by other variables that you are studying

Exogenous and Endogenous

Resource scarcity

Group size

Castes

Market development

Collective action

Education

User group

Resource outcome

Exogenous and Endogenous• Can’t use an endogenous variable as an

independent variable in a regression– e.g. user association in regression on collective

action• Option 1: Instrumental variables

– Find a proxy for the endogenous variable, that is not influenced by other variables in the system.

• Option 2: Multi-stage analysis– User association as function of exogenous variables– CA (work days) as function of predicted value of

user association & other exogenous variables (not same set of independent variables)

What to Collect: Household Decisions

• Lists are excellent!Endogenous: HH participation in Planting Trees on Common LandMeasure: Number of Hours Contributed (Actual)Exogenous: Opportunity Cost of Adult LaborMeasure(s): Wage rates for Adult Males and Females (Actual), or

Dependency Ratio (Proxy, Adult Females), orTotal Number of Adults (Proxy, Family Labor,

which assumes that labor markets completely

missing)Education (Proxy, Household Data by Adult)

What to Collect, Household Instruments

• After going through the list, go back through the exogenous variables, and critically assess: Is it really exogenous?

• If you have reason to believe that it is not, then must come up with potential variables to use as instruments

• Example: Endogenous: Household Participation in planting trees on common lands/pasturesExogenous: Number of livestock (often used as a measure of “wealth”). Is it exogenous? Probably not. Why? Decision on participation and livestock likely to be a joint – same variables will affect both decisions.Potential Instruments: Stock (number or value) of consumer durables; whether household is from traditionally “livestock-owning” ethnic group, total number of livestock held at formation of household.

What to Collect, CommunityEndogenous: Maintenance of Community IrrigationMeasure(s): Engineer/Other Expert Assessment – Quality Ranking (Actual, outside expert), orNumber of Hours and/or Money Contributed (Actual,Survey of Person In-Charge of Maintenance, perhaps corroborated with household surveys?), orMeasure of number of months functioned in last 12 months (Actual, Survey of Person In-Charge, or Focus Groups? What if information differs across sources?)Exogenous 1: Benefits of MaintenanceMeasures(s): Might assume benefits same across all communities (water for humans and/or animals, the same), so no variable, orAlternative water sources (own tubewells? % of community members, either from group interviews or key informant interview); price of purchased water (either from group interviews, or from shop that sells locally, etc.)

What to CollectEndogenous: Maintenance of Community BoreholesExogenous 2: Heterogeneity within the communityMeasure(s)Ethnic Differentiation (diversity index, or number of groups) Actual, Key Informant and/or Focus GroupsWealth Differentiation (diversity in landholdings, % communitymembers with vehicles or other expensive consumer durables) Actual, Key Informant and/or Focus Groups, possible corroboration with household level data (if enough household data))Difference in Education (proportion of households with at least one member with primary, secondary schooling; Focus Groups but still difficult to get if large community

Complications Specific to Property Rights

• Property Rights…– To a specific resource, or specific land/water

area?– Of a specific person over certain resources

• If so, are the rights complete (to all resources, without condition?) and/or unconditional (always have rights, or does strength of right depend on other factors (e.g. relative rainfall)

• If incomplete or conditional, can characterize in any way that is “quantifiable” or easily comparable across individuals, across communities and regions?

Complications Specific to Collective Action• Measuring “Capacity” to Undertake Collective

Action at the group or community level: Potential Measures: Trust, Social Cohesion,

Agency, History of Success and/or Failure– How do you measure “trust” amongst members and

get an indicator of trust at the level required (the group or the community). Some have tried the following:Question at Household Level: Do you trust immediate family members to watch your children? family friends? Neighbors in the village?Potential Problems: If not enough households sampled, then may not be representative; what if a household has no small children?

Complications Specific to Collective Action

Potential Measures: Trust, Social Cohesion, Agency, History of Success and/or Failure– How do you get a quantifiable, comparable (across

communities) measure social cohesion? Social differentiation often used, but there can be tensions even in communities where members all of same ethnic group and speak same language

– Agency (A. Krishna): even when there is the “will” there is often not the “way”. How do you measure the capacity to transform “social capital” into collective action?•Possibilities: Characteristics of Leadership and/or Entreprenuership.

Complications Specific to Collective Action

Potential Measures: Trust, Social Cohesion, Agency, History of Success and/or Failure

– How do you get a quantifiable, comparable (across communities) measure of the historical experience with collective action, particularly in large, heterogeneous communities?Potential Difficulties with simply “counting” successes and failures:1. Greater capacity may well lead a group to try more things, and more difficult things – so number of failed (or less successful) activities should be greater! Also, implies endogeneity!

2. In practice, people prefer to speak of successes, so data collected likely to be biased

Recall Data• Subjective: Successes and failures; the

human tendency:•The past was always better, generically, or•Difficulty discussing negatives/failures

• Objective: Historical recall data on assets (livestock), rainfall, climate, other shocks

•Nearly impossible to get good data for more than 5 years, even 5 years is very difficult (e.g. livestock holdings),

•Rainfall, as with positive and negative subjective events, people tend to forget how “bad” it was in past. Borana “forgot” 1991-92 drought, which was quite devastating.

•Shocks (pest infestation, human disease, drought): Worst shocks will be those most recently experienced, irrespective of whether more likely to happen again.

• Possible Solution: Wherever possible, get secondary corroborating data, or do very extensive job of triangulating information from multiple sources

• Preferences and Subjective Assessments:– Need to make sure that everyone has the

same “reference point”; e.g.•Those in communities with generally bad soil might rank their own plot as favorable… relative to the generally poor soils in the area; if ask someone in different area, might rank quality of his/her plot as “average”, when in fact is much better because soils generally more favorable

•Same with questions of leadership, for instance. If community used to fairly poor leadership, current leader may be relatively good, but in fact may still be worse than in other community where leadership ranked as poor – since that community may have a different experience and expectations

• Open-Ended, Semi-Structured Interviews: – Enumerator Bias:

•In Burkina Faso, information on waterpoint management perfectly correlated with 1 of 3 enumerators. Enumerator had previous water project implementation experience; his 18 communities reflected his bias in how should be, so no information on how is

•In India, information on individuals’ risk preferences were nearly completely correlated with the enumerator…

Final ThoughtsSome things simply not amenable to

quantification:• Processes (how decisions are made, enforced)• Variables with many “dimensions”; e.g. certain

tenure situations where many multiple, overlapping claims over same resource

• Many subjective assessments – this does NOT mean are not important, it means difficult to construct questions that allow comparable information to be generated across households in many different regions