JAMM 444: Public Opinion
Survey methodology
Comparing survey methods
Planning your surveys
Tuesday, Sept. 19
Schedule update Survey methods Sources of error Meet in small groups: team project
Schedule update
TODAY: Chapter 3, Survey methods Thursday: Chapter 9, Using public
opinion data Sept. 26: Barbara Foltz & Stephanie
Kane, UI Social Science Research Unit Sept. 28: Valerie Steffen, president,
Strategic Intelligence (Boise)
Survey methodology
Why do a survey? Measure opinion of a population too large
to contact individually Save time, save money
“Surveys can be used in a scientific way to realize the great benefits of interviewing a representative sample instead of the whole population.” Salant & Dillman, 1994
Surveys work, if...
Sample is large enough to yield desired precision.
Everyone in population has equal chance of being selected.
Questions enable respondents to give accurate answers.
Sampled respondents have similar characteristics to non-respondents.
5 key elements of a survey
1. Population
2. Type of sample
3. Method(s) for gathering data
4. Questionnaire
5. Analyses and inferences
5 steps for a successful survey
1. Be specific about what new information you need and why.
2. Understand, avoid 4 kinds of error.
3. Choose the best survey method.
4. Write good questions.
5. Test questionnaire before beginning real survey.
Choosing a sample
Identify population of interest: University of Idaho students Moscow residents Idaho voters ‘The American public’
Choosing a sample
Non-probability sampling 1936, Literary Digest
Probability sampling Simple random sampling (rarely used) Systematic random samples (every 30th
name in student directory) Stratified sample: divided into groups (strata) Cluster sampling (spread over a wide area)
Cluster sampling
Used for many national surveys: National Election Survey (NES) General Social Survey (GSS) Gallup Organization
350 geographical segments Sample to reflect U.S. as a whole
(geographic area, size of community) Random telephone numbers in each area
How big is a typical sample?
UI or Moscow: 200 to 500 Idaho: 500 to 800 National: 800 to 1,200
Four sources of error
1. Coverage error: Not allowing every person in population equal chance of being sampled.
2. Sampling error: Only some members of study population are asked to respond. (Range of possible results)
Four sources of error, cont.
3. Measurement error: inaccurate answers due to improper question wording
4. Non-response: Some people in sample don’t respond; different from respondents
Sampling error
The probable difference in results between interviewing everyone in the population vs. a scientific sample taken from the population.
Expressed as “plus or minus X percentage points”
http://www.ncpp.org/qajsa.htm#11
Sampling error
See explanation, note, p. 79, PO Rules of thumb:
The closer the margin, the larger the sample that is needed to reduce error.
For very large populations, a relatively small sample can produce reasonably accurate results. (See overhead)
How large a sample do you need?
The answer depends on: How much sampling error you can
tolerate The size of your population, if the
population is small How varied the population is The smallest sub-group that you are
likely to analyze
Survey methods
1. Face to face 2. Telephone 3. Mail 4. Internet What are advantages & disadvantages
of each method? Consider: cost, time, response rate, amount of information
For Thursday
Read: PO9: Public Opinion & Policy Making
Small Groups
Team 1: UI issue or controversy Team 2: Moscow or Latah County issue or
controversy Team 3: Political knowledge and/or civic
engagement by UI students Team 4: Idaho state candidates or U.S.
House of Representatives races Team 5: Idaho state ballot issues
Small Groups
1. Meet in groups for at least 15 minutes
2. Refine your topic
3. Brainstorm target population, survey methodology, timetable
4. Choose time and place to meet again before Sept. 26
Small Groups
ONE-PAGE MEMO due Sept. 26: Choose a name for your group (e.g.,
Bird’s Eye Consulting) List names of all participants. Rationale for survey (why?) & target
population (who?) Topics or candidates (what? who?) Methods (how?)