Post on 11-May-2015
description
transcript
CARRY OUT SURVEY
Why should I do a survey & why should I use the LQAS method?
Where should I conduct my survey?
PRE-SURVEY
Who should I interview?
What questions do I ask and how do I ask them?
Uses of surveys
Random Sampling
Using LQAS sampling for surveys
Using LQAS for baseline surveys
Interview locations
Selecting households
Selecting respondents
Field practice for numbering and selecting households
Developing and reviewing the survey questionnaires
Interviewing skills
Field practice for interviewing
Planning for the data collection/survey
What do I do with the information I have collected?
Field work debriefing
Tabulating results
Analyzing results
POST-SURVEY
Standard Survey Chronology (with reference to Lot Quality Assurance Sampling--
LQAS)
A
B C
DE
Key Concepts: Program Area and Supervision Areas
Together, A, B, C, D, and E represent the Program Area
A, B, C, D, and E represent 5 Supervision Areas.
Why Survey/Why LQAS
Key Concept: Coverage
An important use of surveys is to measure coverage.
What is coverage?
COVERAGE is the percentage of people in any given area (a program area or supervision area) who
a) Know a key piece of information
b) Practice a recommended behavior or
c) Receive a particular service.
Why Survey/Why LQAS
Key Concept: Coverage
Knowing coverage enables us to...
Plan by allowing us to choose priorities.
To focus our efforts on improving those knowledge and practices that have low coverage.
Over time, repeated measures of coverage show us if our efforts are leading to improvements in coverage.
Additionally, knowing the coverage is especially poor in one or more supervision areas helps us choose priorities. We can decide to focus our efforts in those supervision areas with poor coverage.
Why Survey/Why LQAS
What Surveys Can Show You
Surveys can help you identify the level of coverage of the program area as a whole, AND if there are:
large differences in coverage regarding knowledge and practices among supervision areas
little difference in coverage regarding knowledge and practices among supervision areas
Why Survey/Why LQAS
Scenario One
Indicator: Percent of women (15-49) who know 2 or more ways to prevent HIV transmission.
Scenario Two Scenario Three
A = 30
B = 40C = 80
D = 75E = 20
A = 85
B = 80C = 90
D = 85E = 80
A = 25
B = 20C = 30
D = 25E = 20
Why Survey/Why LQAS
Uses of Surveys
Identify knowledge and practices with:
1. Large differences in coverage among supervision areas (SAs).
Identify the low-coverage SAs to be able to:•learn causes of low coverage.•focus our efforts and resources on these SAs.•improve coverage of the whole program area by improving coverage in these SAs.
Identify high-coverage SAs to be able to:•study and learn what is working well.•identify things that can be applied to other SAs.
2. Little difference in coverage among SAs.
If coverage is generally high, shift resources to improve other knowledge and practices.
If coverage is generally low:•learn causes of low coverage.•identify/study other program areas to learn what is working well.•identify things that can be applied in your own program area.
Why Survey/Why LQAS
Key Concept: Random Sampling
Sampling allows you to use the “few” to describe the “whole” (to generalize from the few to the whole)
AND
Random sampling is a critical way to improve your ability to generalize in this way (it improves “external validity”)
Why Survey/Why LQAS
Key Concept: Lot Quality Assurance Sampling
A = ?
B = 80C = ?
D = 75E = 45
Indicator: Percent of women (15-49) who know 2 or more ways to prevent HIV transmission.
A special type of random sampling that allows us to use small samples to distinguish between supervision areas in relation to coverage--to see if certain areas have much higher or lower coverage than others.
LQAS allows us to make comparisons between supervision areas AND estimate overall program coverage.
Why Survey/Why LQAS
Watch!
LQAS Sampling ResultsIndicator: Percent of women (15-49) who know at least 2 ways to
prevent HIV transmission.
Total black checkers in the bag
Total black and red checkers in the bag
= =
Verify coveragein the bag for
SA A
Total black checkers in the bag
Total black and red checkers in the bag
= = Verify coveragein the bag for
SA C
Why Survey/Why LQAS
Supervision Areas: Program Area
SA “A” SA “C”
Sample Sample Sample Sample
1 6 1 6
2 7 2 7
3 8 3 8
4 9 4 9
5 10 5 10
%
%
100
100
Why Survey/Why LQAS
“Limits” of LQAS
Let’s say we want ALL SAs to achieve the result that at least 50% of all women 15-49 know at least 2 ways to prevent the transmission of HIV.
If we take a sample of 19, what is the probability of misclassifying an SA as having achieved the target (using a decision rule of 7) or of NOT having achieved the target for various TRUE population rates?
True Population Proportion who Know 2 Ways to Prevent HIV Transmission in the SA
Probability of classifying the SA as having achieved
the target of 50%(based on n=19 with
decision rule of 7 or more who know 2 ways)
Probability of classifying the SA as not having
achieved the target of 50%(based on n=19 with
decision rule of 7 or more who know 2 ways)
15% 2% 98%
20% 7% 93%
25% 18% 82%
30% 33% 67%
35% 52% 48%
40% 69% 31%
45% 83% 17%
50% 92% 8%
55% 97% 3%
60% 99% 1%
70% 100% 0%
What a Random Sample of 19 Can Tell Us
• Good for deciding what are the higher-performing supervision areas to learn from
• Good for deciding what are the lower-performing supervision areas
• Good for differentiating knowledge/practices that have high coverage from those of low coverage
• Good for setting priorities among supervision areas with large differences in coverage
• Good for setting priorities among knowledge/practices within an SA (if one intervention area is high but another is low, we would concentrate on the low-coverage intervention)
Why Survey/Why LQAS
What a Random Sample of 19 Cannot Tell Us
• Not good for calculating exact coverage in an SA (but can be used to calculate coverage for an entire program)
• Not good for setting priorities among supervision areas that have little difference in coverage among them
Why Survey/Why LQAS
So…Why Use a Random Sample of 19?
• A sample of 19 provides an acceptable level of error for making management decisions; at least 92% of the time it correctly identifies SAs that have reached their coverage target.
• Samples larger than 19 have practically the same statistical precision as 19. They do not result in better information, and they cost more.
Why Survey/Why LQAS
One More Thing… LQAS in Baseline Surveys
Generally we think of using LQAS for ongoing monitoring to assess whether we have evidence that we are meeting pre-set targets. However, it can be used at baseline to assess whether certain SAs appear to be “lagging” behind the program area average coverage.
Supervision Areas A, B, C, D and E
Indicator: Women who know 2 or
more ways to prevent HIV transmission
Number Correc
t
Average Coverage Estimate
65.3%
Equal to or Above Average Coverage?
SA A 12 Yes
SA B 9 No
SA C 16 Decision Rule11
Yes
SA D 11 Yes
SA E 14 Yes
Five SAs & One Indicator
1. Add number correct in all SAs: 12+9+16+11+14=62
2. Add all sample sizes 19*5=95
3. Average Coverage Estimate 62/95=65.3% round up to nearest 5%=70%
4. Decision rule for sample of 19 and 70% is 11
n=Average Coverage (for baselines) or Coverage Target (for monitoring/evaluation)
10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95
19 -- -- 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Indicator Cor-rect out of 19
Pgm.Cove-rage Est.
Rule
Equal or
Above
Women “using” condoms w/ sex
7 45% 6 Yes
Men using condoms w/ sex
4 20% 1 Yes
Women know how HIV transmitted
4 45% 6 No
Men know how HIV transmitted
13 65% 10 Yes
Women who know where to get test
6 30% 3 Yes
One SA & Five Indicators
CARRY OUT SURVEY
Why should I do a survey & why should I use the LQAS method?
Where should I conduct my survey?
PRE-SURVEY
Who should I interview?
What questions do I ask and how do I ask them?
Uses of surveys
Random Sampling
Using LQAS sampling for surveys
Using LQAS for baseline surveys
Interview locations
Selecting households
Selecting respondents
Field practice for numbering and selecting households
Developing and reviewing the survey questionnaires
Interviewing skills
Field practice for interviewing
Planning for the data collection/survey
What do I do with the information I have collected?
Field work debriefing
Tabulating results
Analyzing results
POST-SURVEY
Standard Survey Chronology (with reference to Lot Quality Assurance Sampling--
LQAS)
Where and With Whom?
Where to Conduct the Survey and With Whom
These questions are context specific. The question “with whom” is determined by program participants whom you want to survey. And the question of “where” to find them will depend on the spread of your program and whether you have lists of participants from which you can draw a sample.
In the example we have been using, the “who” of our program is women 15-49 whom we want to help to prevent getting HIV. Still knowing the “who” is not the end of question because we still need to figure out how to take a random sample of them.
In most cases program participants are scattered in various communities or locales and thus we need a sampling method that gives everyone an equal chance of being selected.
Broadly speaking I have seen two scenarios in the field:
1. A situation in which all program participants are “registered” on a list that is updated regularly. This is common in programs like microfinance where “clients” are tracked.
2. A situation in which program participants are not registered--not known by name--and scattered around a variety of communities.
Let’s look at these two scenarios and examine how to randomly sample
Where and With Whom?
A
B C
DE
Programs in Which Participants are “Listed”
Let’s go back to our 5 SAs and assume that, in this case, the 5 are “bank” branches of a microfinance institution. In each branch (each SA) lists of clients are maintained.
If we are doing an education program with members and want to learn if 50% in branch “A” know 2 ways to prevent HIV transmission we can take a sample of 19 from this branch.
To do this we start by assembling a numbered list (in a completely random order--NOT by community for example) of all client names.
Next we use some means (random number table or other) to select 19 numbers at random. The numbers must be between 1 and the highest number on the list.
We then decide when and how we will interview each client whose name corresponds with the number chosen.
Programs in which Participants are not “Listed”
(but for which we know all the communities in which they live and have a general idea of population size in each
community)
This case is common in community-based programs in which “all” members of a group (women 15-49) in a certain geographic area are intended beneficiaries
A
B C
DE
If we are doing an education program with women 15-49 in an entire program area and want to assess whether 50% of them in SA A (a geographic area) know at least two ways to prevent HIV transmission the approach is a bit more challenging and usually requires “multi-stage”, “systematic” sampling. Here are the general steps (to be repeated in each SA)
Step 1. List communities and total population.Step 2. Calculate the cumulative population.Step 3. Calculate the sampling interval.Step 4. Choose a random number.Step 5. Beginning with the random number, use the sampling interval to identify communities for the 19 sets of interviews.
This ONLY gives us the communities in which we need to sample! We need another step to identify the actual women we will interview (that is why we call it “multi-stage”)
Where and With Whom?
Where and With Whom?
Name of the Community
Population
Pagal 152
Santai 381
Ishri 115
Garafa 97
Nevi 253
Masrag 126
Farry 188
Jilwa 216
Guimbe 554
Kilkil 92
Total 2174
Step 1. List communities and total population.
Cumulative Population
152
533
648
745
998
1124
1312
1528
2082
2174
Step 2. Calculate
the cumulative population.
Step 3. Calculate the sampling interval=Total Cumulative Population/Sample Size (2714/19=114.42)
Step 4. Choose a random number between 1 and the interval -
Step 5. Beginning with the random number, use the sampling interval to identify communities for the 19 sets of interviews. X is the first then
X+114.42, etc.
Interview Location Number
77
191, 305, 420
534
649
763, 877, 992
1106
1221
1335, 1450
1564, 1678, 1793, 1907, 2022
2136
Supervision Area A
Interview Location
Where and With Whom?
Choosing the “Final” RespondentWhen the program participants are not listed (second scenario) but
after having selected the communities.
Let’s continue with the scenario with which we have been working. We want to interview women 15-49 on communities where our HIV/AIDS programming has gone on to see if women know 2 ways to prevent the transmission of HIV.
Once we have our communities selected (previous step) we need a way to identify women once we get to the community. To do this we still have two steps:
1. Choosing a place (household) in which to start the survey
2. Choosing the respondent
Let’s look at a couple of different scenarios for choosing the household and then examine how to choose a woman. Let’s say we are interviewing in Guimbe (from the previous step). We need to interview 5 women there.
Where and With Whom?
Choosing the First HouseholdThough Guimbe is a big community, let’s look at several scenarios about how to select a starting household in various size communities. You
will also use this approach to select subsequent households in settings, like Guimbe, where you are to conduct more than one
interview.(I will provide a reference about how to do this in urban settings.)IF: THEN:
A complete household list is available (tax list, census, map)
• Assign a number to each house• Choose a random number between 1 and the highest numbered
house• Start with that house
The community size is about 30 households or less (not the case for Guimbe)
• Make a household list or map with the location of each household
• Assign a number to each house• Choose a random number between 1 and the highest numbered
house• Start with that house
The community size is more than about 30 households (Guimbe would be this case)
• Subdivide the community into 2-5 sections with about the same number of households in each section.
• Select one section at random• If that subdivision has more houses than you can easily
count, subdivide it into 2-5 sections and select a section at random
• Make a household list or map with the location of each household
• Assign a number to each house• Choose a random number between 1 and the highest numbered
house• Start with that house
Where and With Whom?
Choosing the “Final” RespondentSo, now you have selected the household where to start. Use this table
to decide how to proceed…
IF: THEN:The type of respondent you are looking for is at the household you selected
Interview that person if she consents (see the bottom of the table for what to do next)
The type of respondent you are looking for does NOT live in the household you selected
Go the next nearest household from the FRONT ENTRANCE to the household you are at and check for an appropriate respondent at this household. Continue this process until you find the type of respondent you are looking for.
If two households are equally near, then choose the one with the closest door or flip a coin.
The type of respondent you are looking lives in the household BUT is away/absent at the time of the interview AND is more than 30 minutes away
The type of respondent you are looking for lives at the household BUT is absent BUT is within 30 minutes of where you are
Go find the respondent with the help of someone who knows where she is. IF you cannot find the person in the next 30 minutes THEN…
Go the next nearest household from the FRONT ENTRANCE to the household you are at and check for an appropriate respondent at this household. Continue this process until you find the type of respondent you are looking for. If two households are equally near, then choose the one with the closest door or flip a coin.In Guimbe you need to conduct five interviews so once you have completed the first interview
go back to the steps for selecting the first household and begin again following the approach that is appropriate to the size of the community. You will repeat this each time you must select a new respondent.