Sampling ProceduresSampling Procedures

January 24 & 26, 2011January 24 & 26, 2011

ObjectivesObjectives

By the end of this meeting, participants By the end of this meeting, participants should be able to:should be able to:

1)1) Distinguish the techniques behind Distinguish the techniques behind and advantages of various kinds of and advantages of various kinds of sampling procedures.sampling procedures.

2)2) Evaluate a survey sample based on Evaluate a survey sample based on common problems that should be common problems that should be avoided.avoided.

3)3) Calculate and interpret the margin of Calculate and interpret the margin of error for survey findings.error for survey findings.

SamplingSampling

Suppose that you wanted to know how the Suppose that you wanted to know how the American people felt about the war in American people felt about the war in Afghanistan. What would you do?Afghanistan. What would you do?

a)a) Obviously asking the entire public the Obviously asking the entire public the question would be prohibitively expensive. question would be prohibitively expensive.

b)b) Fortunately, you would not have to. A Fortunately, you would not have to. A carefully created and implemented survey carefully created and implemented survey of a few thousand people would give you a of a few thousand people would give you a good reflection of the views of the entire good reflection of the views of the entire country. country.

Sampling MethodSampling Method

a)a) First thing a researcher needs to do First thing a researcher needs to do is decide what is the is decide what is the population of population of interestinterest. .

b)b) From there a researcher needs to From there a researcher needs to decide what is the decide what is the sampling framesampling frame..

Types of SampleTypes of Sample

There are two broad types of sampleThere are two broad types of sample

a)a) Non-probability sampleNon-probability sample

b)b) Probability sampleProbability sample

Types of Non-probability Types of Non-probability SamplesSamples

a)a) Typical sample- choosing people that Typical sample- choosing people that seem typical based on various seem typical based on various demographic factorsdemographic factors

b)b) Purposive sample- choosing people Purposive sample- choosing people deliberatively possibly based on the deliberatively possibly based on the advice of others advice of others

c)c) Volunteer subjects- surveying people Volunteer subjects- surveying people that volunteer (American Idol, various that volunteer (American Idol, various insta-polls)insta-polls)

Types of Non-probability Types of Non-probability SamplesSamples

d)d) Haphazard sampling- choosing those Haphazard sampling- choosing those people that are easiest to contact (people that are easiest to contact (Literary Literary Digest 1936 pollDigest 1936 poll, class polls, street corner , class polls, street corner polls)polls)

e)e) Quota Sampling- choosing people that Quota Sampling- choosing people that perfectly reflect the population of study perfectly reflect the population of study (minorities, women, homosexuals, etc. )(minorities, women, homosexuals, etc. )

f)f) Snowball sampling- interviewing a random Snowball sampling- interviewing a random sample and asking them to identify people sample and asking them to identify people that they know that fit certain criteria. that they know that fit certain criteria. Generally used for rare populationsGenerally used for rare populations

Probability SamplingProbability Sampling

a)a) Despite their widespread usage non-Despite their widespread usage non-probability samples are almost probability samples are almost universally inferior to probability universally inferior to probability samples due to the introduction of samples due to the introduction of biasbias

b)b) Probability samplingProbability sampling means that means that each person in the population has a each person in the population has a known probability of being chosen known probability of being chosen (although not necessarily equal)(although not necessarily equal)

Types of Probability SamplesTypes of Probability Samples

a)a) Simple random sample- using a list of Simple random sample- using a list of the entire population, a random the entire population, a random group is chosengroup is chosen

b)b) Systematic selection procedure- Systematic selection procedure- using a list of the entire population, a using a list of the entire population, a random number is selected and that random number is selected and that many units are skipped between many units are skipped between interviewsinterviews• Need to avoid periodicity Need to avoid periodicity • Hard to generate listsHard to generate lists

Types of Probability SamplesTypes of Probability Samples

c)c) Stratified sample- divide the Stratified sample- divide the population in pieces and randomly population in pieces and randomly sample within those pieces (regions, sample within those pieces (regions, counties, dorms, etc.; this is the counties, dorms, etc.; this is the method used for election exit polls)method used for election exit polls)

d)d) Cluster sample- divide the population Cluster sample- divide the population into clusters (think neighborhoods) into clusters (think neighborhoods) and conduct several interviews in and conduct several interviews in each cluster. This introduces bias but each cluster. This introduces bias but can significantly reduce cost.can significantly reduce cost.

Types of Probability SamplesTypes of Probability Samples

e)e) Multistage area sample- a portion of the Multistage area sample- a portion of the geographic area is sampled, followed by a geographic area is sampled, followed by a sampling of areas within the selected areas. sampling of areas within the selected areas. The areas will be weighted based on their The areas will be weighted based on their population. population.

f)f) Hybrid sampling- a combination of any of Hybrid sampling- a combination of any of the previous sampling methods. Such types the previous sampling methods. Such types include Multiple Frame Designs, repeated include Multiple Frame Designs, repeated attempts to sample the same population attempts to sample the same population and Parallel Samples, comparing a baseline and Parallel Samples, comparing a baseline sample to another sample. sample to another sample.

Telephone SamplesTelephone Samples

a)a) Telephone surveys are one of the most Telephone surveys are one of the most common types of surveyscommon types of surveys

b)b) They can be potentially difficult because They can be potentially difficult because not all people are listed and telephone not all people are listed and telephone books are frequently out of date. books are frequently out of date.

c)c) Some researchers use a method called Some researchers use a method called add a digit dialing. In this method, add a digit dialing. In this method, numbers are chosen at random from the numbers are chosen at random from the directory and a digit is added to that directory and a digit is added to that numbernumber

Telephone SamplesTelephone Samples

d)d) Another method is random digit dialing. A Another method is random digit dialing. A computer will generate a telephone computer will generate a telephone number at random (perhaps an area code, number at random (perhaps an area code, prefix or just a suffix). This method needs prefix or just a suffix). This method needs roughly 5 numbers for each number in the roughly 5 numbers for each number in the sample due to the high rate of failuresample due to the high rate of failure

e)e) Telephone samples suffer from multiple Telephone samples suffer from multiple phone lines in each home, refusal, phone lines in each home, refusal, difficulty reaching any one, etc. difficulty reaching any one, etc. • Bias?Bias?• Alternative methods?Alternative methods?

Telephone SamplesTelephone Samples

f)f) One issue that faces telephone One issue that faces telephone surveys is which person to speak to surveys is which person to speak to within the household. One of the within the household. One of the most common methods used in the most common methods used in the next birthday method.next birthday method.

Problems in SamplingProblems in Sampling

Generally random sampling will give a sample Generally random sampling will give a sample that reflects the broader population. There are that reflects the broader population. There are still potential problems that need to be still potential problems that need to be considered.considered.

a)a) Noncoverage error- parts of the population Noncoverage error- parts of the population may not be covered in the sample (for may not be covered in the sample (for example, people without phones, the infirmed, example, people without phones, the infirmed, etc. )etc. )

b)b) Sampling the wrong population- the precise Sampling the wrong population- the precise population needs to be sampled not just a part population needs to be sampled not just a part of itof it

Problems in SamplingProblems in Sampling

c)c) Response rate- lower response rates Response rate- lower response rates are not necessarily a problem unless are not necessarily a problem unless those that refuse have similar those that refuse have similar demographic factors. demographic factors.

d)d) Sampling error- this is the error Sampling error- this is the error inherent in using a sample to inherent in using a sample to generalize to a broader populationgeneralize to a broader population

Sampling ErrorSampling Error

We need to consider sampling error:We need to consider sampling error:a)a) The error that arises from trying to represent a The error that arises from trying to represent a

population with a sample.population with a sample.b)b) Sampling error does not include other sorts of Sampling error does not include other sorts of

error that can result from surveys.error that can result from surveys.We often speak of a 95% confidence interval.We often speak of a 95% confidence interval.a)a) If repeated samples were taken, 95% of the If repeated samples were taken, 95% of the

samples would contain results within the margin samples would contain results within the margin of error.of error.

b)b) ““A statistician would say that we are taking a 5% A statistician would say that we are taking a 5% chance of drawing a faulty conclusion…chance of drawing a faulty conclusion…”” (WKB, (WKB, 68).68).

Margin of ErrorMargin of Error

Margin of error (forumula on board)Margin of error (forumula on board)a) Where, t =1.96 for large samplesa) Where, t =1.96 for large samplesb) The value 1.96 comes from our b) The value 1.96 comes from our

understanding of the distribution of understanding of the distribution of possible values.possible values.

c) f is the sampling fraction (or the fraction of c) f is the sampling fraction (or the fraction of the population that is being sampled), (1-the population that is being sampled), (1-f)f)1/21/2 is ignored is ignoreda) when sampling with replacementa) when sampling with replacementb) or when the population is very largeb) or when the population is very large

Margin of ErrorMargin of Error

d) p is the sample proportion (for example, d) p is the sample proportion (for example, the proportion approving of President the proportion approving of President Obama).Obama).• p and (1-p) may be written in either proportion p and (1-p) may be written in either proportion

(0 to 1) or percentage (0 to 100) terms(0 to 1) or percentage (0 to 100) terms• Decide which based on what unit you want the Decide which based on what unit you want the

margin of error to be expressed in. margin of error to be expressed in.

Margin of Error: ExampleMargin of Error: Example

a)a) Suppose there are 1,872 political science majors on campus. Suppose there are 1,872 political science majors on campus. We randomly select 250 and ask them whether they watched We randomly select 250 and ask them whether they watched TV last night. 66 percent of the sample respond TV last night. 66 percent of the sample respond ““yes.yes.””

b)b) How confident can we be that this percentage reflects our How confident can we be that this percentage reflects our underlying population of interest (political science majors)?underlying population of interest (political science majors)?

c)c) Recall: margin of error formulaRecall: margin of error formulad)d) p = .66 and (1-p)= .34; for percentages: p=66 and (1-p)=34p = .66 and (1-p)= .34; for percentages: p=66 and (1-p)=34e)e) n = 250 and f = 250/1872n = 250 and f = 250/1872f)f) Margin of error=±1.96Margin of error=±1.96[([(66×34)/249]66×34)/249]1/21/2×[1−(250/1872)]×[1−(250/1872)]1/21/2= =

5.5%5.5%g)g) We are 95 percent confident that 66 ± 5.5% of social science We are 95 percent confident that 66 ± 5.5% of social science

majors watched TV last night.majors watched TV last night.h)h) Or, we can say, we are 95% confident that the true Or, we can say, we are 95% confident that the true

population percentage is in between 60.5% and 71.5%population percentage is in between 60.5% and 71.5%

For January 31For January 31

a)a) Read WKB chapter 4.Read WKB chapter 4.b)b) Answer the following questions:Answer the following questions:

a)a) Imagine we conducted a survey of 350 UGA Imagine we conducted a survey of 350 UGA students and found that 93% thought the students and found that 93% thought the Bulldogs would win the SEC championship. Bulldogs would win the SEC championship. What is the margin of error (at 95%)? Based on What is the margin of error (at 95%)? Based on that margin of error, we can be 95% sure that that margin of error, we can be 95% sure that championship predictions lie in what range?championship predictions lie in what range?

b)b) Imagine we conducted a survey of GA voters Imagine we conducted a survey of GA voters and found that Obama had approval of 55%. and found that Obama had approval of 55%. 750 people were surveyed, what is the margin 750 people were surveyed, what is the margin of error (at 95%)? Based on that margin of of error (at 95%)? Based on that margin of error, we can be 95% sure that Obama’s error, we can be 95% sure that Obama’s support in GA lies in what range?support in GA lies in what range?