Date post: | 19-Jan-2016 |
Category: |
Documents |
Upload: | whitney-swan |
View: | 213 times |
Download: | 0 times |
1
Fieldwork logistics and data quality control procedures
Kathleen BeegleWorkshop 17, Session 2Designing and Implementing Household SurveysMarch 31, 2009
2
What makes a good survey? Relevance
Answer the policy questions of import Quality
Methodology Accuracy and reliability Adherence to international statistical guidelines
Timeliness Punctuality Production time Comparability over time
Sustainability Public dissemination Documentation
3
Overview Overview of the main actors Quick description of broad activities of an
LSMS Options for organization of field work &
quality control aspects
4
Who does what Core team
Principal Investigators who understand the purpose of the survey
Local team Will implement the survey and process data For LSMS is usually the national statistics office For smaller/specialized surveys, usually local firms Need to identify who in the local team is the project
manager Technical assistance
Short-term consultants for specialized expertise such as sample design and data entry system development
Long-term assistance: possible project head if lacking in local team
5
Activities Broad activities involved with an LSMS
Management/logistics Questionnaire development Sampling Field staff: recruitment, training, payments Field work Data management Data analysis and final documentation
Some activities are sequential (training after questionnaire finalization), but lots of overlap in timing (sampling & questionnaire development; logistics is on-going).
6
Activity groups 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
1 Management and logistics2 Questionnaire development3 Sampling4 Staffing and training5 Data management6 Field work7 Data analysis and documentation
Design
Data collection
Analysis
Management: Prepare a timeline…
7
….and make it detailed, with deadlines!
4 Staffing and training 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
4.01 Select and train field test staff4.02 Prepare supervision procedures & manuals4.03 Prepare interviewer manual4.04 Select interviewers4.05 Train interviewers
1 Management and logistics 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
1.01 Agreements and institutional environment1.02 Appoint core staff1.03 Core staff logistics1.04 Acquire anthropometric equipment1.05 Acquire computers1.06 Acquire survey materials1.07 Mobility startegy and vehicle acquisition1.08 Publicity and hh motivation strategy
8
Questionnaire development
Pilot/pre-test questionnaires in various settings as relevant (urban, rural, etc)
Careful review of translation (if any). Back translation is ideal.
Finalize questionnaire before any training. Prepare Manuals with specific instructions for
enumerators and supervisors, other staff, to accompany the questionnaire. This is also part of the final documentation to assist data users.
9
Field staff
Recruit & train more enumerators than you need Enables a selection process during training –
competition among candidates Provides reserves in case field staff leave early
Training in two parts (if large field staff) Training of Trainers (TOT): train field supervisors Training of enumerators: main training with active
participation of supervisors as facilitators
10
Field staff Training for an LSMS – a long, multi-topic
questionnaire – 3-4 weeks. Includes: in-class study of questionnaires in-class mock interviews/exercises tests field practice and feedback more if the survey will use special instruments
(anthropometric, GPS, etc.) Common training for all team members if feasible
Payments Usually not paid per questionnaire, instead paid daily rate Bonuses paid at end of field work can reduce staff attrition
during field work (important for longer surveys) Performance bonuses (if they can be objectively determined)
11
Field work There are many options for field work structure.
Your selection will depend on the specifics of the survey, timeline, and the budget: Mobile teams with integrated DE. Improves quality of data
because teams get immediate feedback on inconsistencies in the questionnaire.
Mobile teams without DE. DE in office. Enumerator in the village. DE in office. Difficult to supervise
well! May not be cost effective (with exception of surveys over several moths with ~20 HHs per village).
Local office from which teams operate. Feasible for a survey focused in one region/small geographic area.
No excuse not to do concurrent DE! Don’t wait until end of field work to start entering questionnaires. This is often the reason for lags in data availability.
Field work: Composition of a mobile field team
Supervisor Interviewers Anthropo-
metrist
Data entry
operator.maybe.
13
14
15
Other factors of survey quality Management Team for the survey
Composed of Project Manager Fieldwork Manager Data Manger
Dedicated to the project in all three phases Design and Preparation Implementation Dataset documentation and initial tabulations
16
Other factors of survey quality Actions of the field team supervisors
Visual scrutiny of completed questionnaires Less critical if DE is integrated with field work.
Computers do it better Visual observation (direct observation) of interviews
Some of this is needed, but not too much Check-up visits
Critical, and can only be implemented by human supervisors
Need to be random
Supervisors must be supervised too!!
17
Other factors of survey quality Review data files as they come in – done by
Core staff (i.e. in Stata, look for inconsistencies across modules of the questionnaire which is normally not done in DE programs, compare with baseline data if a panel, etc…)
Send Addendum notes/instructions to field staff as issues arise (for things missing from the Manuals)
Keep detailed documentation, needed for analysis and relevant for panel surveys
18
Aspects for evaluation work If the evaluation entails panel data (baseline and follow-up
surveys), plan surveys accordingly. At baseline, think about how to minimize attrition and match
people correctly get adequate address/map/phone information to locate
households again Record complete names of all household members on
questionnaire and enter in DE Other possibilities: GPS, photos of respondents
At follow-up pre-print some baseline data (Roster: names, age, sex, roster
IDs) and also baseline information relevant to the evaluation for measuring changes (assets?)
collect some minimal information on respondents/households who drop out or move (in case you are unable to find them for a re-interview). For example: did they move, when, why…