SAMPLING AND NON-SAMPLING ERRORSISSUES TO CONSIDER
BISHWA NATH TIWARI
UNDP-APRC
BANGKOK
14 MARCH 2013
First Regional Training Assessing Costs and Benefits of Adaptation: Methods and Data
Email: [email protected]
1
CONTENT OF PRESENTATION
Household Survey design and errors
•Steps for conducting survey•Errors
Sampling and non-sampling errors
Suggestion to minimize those errors – Issues to consider
2
NEED FOR DATA
• Data is critical for effective planning, design & implementation of adaptation projects
• BUT there is lack of disaggregated data by:
• Region, districts, settlements (informal), villages• Sex and age groups• Caste & ethnic groups• Economic groups• Vulnerable/disadvantaged groups
• Lack of updated data, and regular availability of data
• Lack of quality data (with less errors)
3
SOURCES OF DATA
Some international sources of data: Internal Energy Agency, Global Footprint Network, Centre for International Earth Science Information Network, Centre for Research on the Epidemiology of Disasters, World Bank World Development Indicators
Generally, secondary data
lack disaggregation
CC is global externality but its impacts are local, therefore need for
hh level data
4
STEPS FOR CONDUCTING A HOUSEHOLD SURVEY
1. Define universe –households where the adaptation project is to be implemented
2. Determine sample size
3. Decide sampling technique
4. Prepare sampling frame and sample households
5. Develop questionnaire taking into account the variables affecting the adaptation capacity and pretest the questionnaire
6. Organize field survey – selection and training of enumerators, organization in groups with division of responsibility including supervisory responsibility
7. Check data in the field & minimize non-response
5
Gay (2005) conducted a survey in coastal town of Dumangas in the Philippines to know if farmers are willing to pay to reduce their vulnerability, and if so, how much do they value a planned adaptation program to climate change.
Using CVM, a WTP survey was conducted among 450 hhs; 391 (87%) were willing to pay for planned adaptation program, 59 were not.
The mean and median WTP for planned adaptation program were PHP34.37 and PHP23.96 per month, respectively.
Factors influencing farmers’ WTP for planned adaptation program were education, farm experience, farm size, knowledge about climate change, land tenure, access to credit, and household income.
Provision of alternative livelihoods and training that are less affected by climate change were the most important planned adaptation projects for farmers
Relocation for people affected by sea level rise was the least preferred option
Source: Defiesta, Gay (2010). Social Vulnerability and Willingness to Pay for Adaptation to Climate Change of Farmers in Dumangas, Iloilo .A Dissertation submitted to SEARCA college, the Philippines.
EXAMPLE OF HH SURVEY: DUMANGAS FARMERS ARE WILLING TO PAY FOR ADAPTATION PROGRAM
6
EXAMPLE OF HH SURVEY: SOUTH AMERICAN FARMERS ADAPT TO CLIMATE CHANGE BY CHANGING CROPS
Estimating a logit model across 949 farmers in 7 countries, Seo and Mendelsohn (2008) found that both temperature and precipitation affect the crops that South American farmers choose. Farmers choose fruits and vegetables in warmer locations and wheat and potatoes in cooler locations. …. Global warming will cause South American farmers to switch away from maize, wheat, and potatoes towards squash, fruits and vegetables.
Predictions of the impact of climate change on net revenue reflect not only changes in yields per crop but also crop switching.
The paper use data collected in 7 South American Countries. The Household surveys asked detailed questions on farming activities during the one year period of July 2003 to June 2004.
Seo, S. Niggol and Robert Mendelsohn (2006). An analysis of crop choice: Adapting to climate change in South American farms. E C O L O G I C A L E C O N O M I C S 6 7 ( 2 0 0 8 ) 1 0 9 – 1 1 6
7
ERROR OR BIAS
Error due to sampling design and sampling technique
All other error over and above the sampling error
8
HOW TO MINIMIZE SAMPLING ERRORS – POINTS TO CONSIDER Sampling error depends on:
• Sample size
• Sampling technique
• Heterogeneity of universe
Suggestions to minimize sampling errors
• Use of appropriate probability sampling technique
• Minimize stages of sampling – design effect
• Determine sample size taking into account the variance of the main indicators/attributes of the universe
• Prepare updated sampling frame so as to minimize the non-response 10
SOURCES OF NON-SAMPLING ERROR
Non-sampling error
11
SOURCES OF NON- SAMPLING ERRORS – EXAMPLES
Specification error: Unless all agricultural crops are included in a survey it is difficult to enumerate income of farmers from agriculture
Strategic Bias: While piloting questionnaire of NMIS cycle 3, it was found that some respondents were reporting false info about sources of drinking water with a strategy that they will get tube well.
Error from enumerator due to lack of training: (i) Use of lead question; (ii) Lack of probing
12
HOW NON-SAMPLING ERRORS CAN OCCUR- EXAMPLES
Using data from a survey of more than 9,000 farmers across 11 African countries, Kurukulasuriya et. al (2006) estimates how farm net revenues are affected by climate change compared with current mean temperature. Revenues fall with warming for dry land crops and livestock, whereas revenues rise for irrigated crops. At first, warming has little net aggregate effect as the gains for irrigated crops offset the losses for dry land crops and livestock. ….The final effects depend on changes in precipitation, because revenues from all farm types increase with precipitation.
Because irrigated farms are less sensitive to climate, where water is available, irrigation is a practical adaptation to climate change in Africa.
Strategic bias (eg, reporting less revenue) from the farmers with
irrigated farms can change the conclusion
of the study!
A specification error of dry land or irrigated
crops can change the result of the study!
Source: Kurukulasuriya et. al (2006). Will African Agriculture Survive Climate Change? The World Bank Economic Review.
13
HOW TO MINIMIZE NON-SAMPLING ERRORS – POINTS TO CONSIDER Use of structured questionnaire and FGDs – skip pattern, codes Use of survey manual with standard definition and specification Pre-test of questionnaire Use of mix of instruments – structured and unstructured
questionnaire/FGD A thorough training of enumerators with background knowledge on
the subject Good rapport building with respondents No lead question; no guess Use of probing questions – who, what, where, why and how Request for time if the questionnaire is not completed in one sitting Interview place – calm and quiet environment Improved coding of questionnaire Programming for data entry and double data entry to minimize
possible entry errors Data cleaning and editing 14
THANK YOU
15
