1

AADAPT Workshop South AsiaGoa, December 17-21, 2009

Survey work Maria Isabel Beltran

2

Type of data

For evaluation purposes: Administrative data Surveys our focus, we can complement

with other sources of information▪ Household▪ Plot▪ Associations▪ Community

Census and other country surveys

3

Data collection: Who does it?

Who collects the data? 2 main cases: The ministry▪ Hiring of enumerators? Who are they going to be?▪ People inside the project have incentives to present

a better or worse picture for their areas▪ A lot of effort to follow the process

An agency (statistical office or private firm)▪ OK, this is the type of work they do, but STILL A LOT

OF EFFORT is needed to ensure quality (TORs, sample, questionnaire, training, supervision)

4

Data collection

Questionnaire design

Training

Pilot test (and re-training)

Field work

Supervision

Data entry & data cleaning

5

Data collection: Questionnaire

Who defines it? YOU (the IE team, not the firm)

Purpose of survey? Define: respondents, indicators,

level, modules. Time & quantity trade off

Internal consistency

Omission of key issues & skip patterns

Clear and explicit questions for all circumstances

Avoid open questions (pre-code) / recall period

Respondent burden, sensitive issues last

Data collection: Training & Pilot Test

Often underestimated part of the process.

Training reduce variability in data collection

Pilot ensures the questionnaire is collecting all

information needed to answer questions, all correct

information, flows and logic of the questionnaire.

Test the instruments cover all conceivable

situations

Involve the enumerators in the project the

importance of the data collected.

Training…

8

Data collection: Field Work

Almost always, it is better if organized in groups of enumerators (2-3)

Time Vs. quality Have a clear field work plan and division of

responsibilities among the group Daily targets Gambia: Enumerator 1 Enumerator 2 Enumerator 3

Talks to head teacher Children, math test

Classroom observation

Head teacher question.

Children, reading test Teacher tests

Oral tests Oral tests Oral tests

9

Data collection: Supervision

Supervision protocol, 1 per 2 teams?

Have a supervision strategy: 10% of

the sample, 100% ? Only non valid

responses?

Use an independent firm or team;

that has received the training

Supervise the supervisors

10

Data collection: Data Entry and Clean-up No need to wait for data collection to finish to

start data entry. Make corrections while the data is still being collected. (Missing values, inaccuracies)

Integrated concurrent data entry Vs. Concurrent Centralized data entry Vs. Computer assisted interviews

Data entry: ONE TIME NOT ENOUGH double entry at the same time, one after the other, one with supervision, … etc

If not planned… data cleaning = long & frustrating

Data is lost, quality decreases (decisions not documented)

11

Data collection: example from India

Integrate the data collection and data entry. Timely data Feedback on field work on real time Early detection of errors (like lack of uniform

criteria)

The Medical Advice, Quality and Absenteeism in Rural India project of the Center for Policy Research, New Delhi

The Medical Advice, Quality and Absenteeism in Rural India

3 separate firms: data collection, supervision, data entry

Define all possible error per questions and program them

type0: No

errortype1:

ID errortype2:

Formatting error

type3: entered

skip code but not skipped

type4: skipped but no

skip code

type5: cross check error

type6: header ID does

not match page 1

type7: blank

instead of -99

type8: one digit

instead of two

Total

s1q1 855 0 0 0 0 0 0 0 0 0s1q10 838 0 17 0 0 0 0 0 0 17s1q11 831 0 24 0 0 0 0 0 0 24s1q12 828 0 27 0 0 0 0 0 0 27s1q13 855 0 0 0 0 0 0 0 0 0s1q14 852 3 0 0 0 0 0 0 0 3s1q15 854 0 1 0 0 0 0 0 0 1

The Medical Advice, Quality and Absenteeism in Rural India

14

Useful Data

Relevant data

Reliable data

Data that is ready when needed… ON TIME, to

answer operational and policy questions.

Need to have staff dedicated to the project in all

phases (design, preparation, implementation, dataset

documentation & validation) field coordinator.

15

Thank You