The survey

Broad-based information

on a population

“Getting the lay of the land”

Surveys

• A social science survey is a research method where a number of people answer a fixed set of questions concerning their knowledge, attitudes and behaviors

• The people are usually chosen carefully to represent a larger population

• Question topics and wording are crucial to the success of the method

Uses of surveys

• Surveys are mainly used to:• Gain an understanding of the characteristics of

populations • Public opinion• Media use• Population demographics, etc.

• Develop and test theories concerning relationships among variables in large populations

Self-report measures

• Depending on respondents to provide information about themselves has strengths and weaknesses as a data-gathering method

• Respondents may be the only source for much of the information you want to know• Knowledge, experiences, etc.

• A number of biases occur in people’s provision of information about themselves

Census v. sample survey

• When you conduct a census you measure every member of a population

• When you conduct a sample survey you measure a subset of the population• Sample surveys are used to estimate what a

census would have found

Census

• U.S. census• Course evaluations

• US News survey of US colleges

The sample survey• Most telecommunications surveys are

sample surveys because populations researchers study are usually very large • a census would be very expensive and

inefficient

Polls

• Polls usually are short surveys looking to determine public opinion on a topic of current interest• Usually are not used to develop more advanced

theory• Often look to identify opinions of subgroups

(demographic clusters)

Sample surveys in telecommunications:

• Nielsen ratings and surveys• Arbitron ratings and surveys• Gallup polls• Simmons Experian• gfkMRI

• Pew Research Center surveys

Survey validity

How do you collect the data?

• Personal interviews• Phone interviews• Mail interviews• Computer-mediated interviews

Why choose one over the other?

• Cost• Response rate• Respondent need for guidance• Anonymity/confidentiality• Speed• Control over data collection

Personal interviews

Personal interviews• An interviewer asks the respondent a

number of questions face-to-face

Source: www.lynnefeatherstone.org/gallery.htm

Advantages of the personal interview

• Interviewer can monitor respondent’s answers • Interviewer can react to nonverbal cues• Survey can include visual stimuli

• Movie posters, videos, etc. • Interviewer can probe for deeper answers• Response rates are high

• High level of control over the interview situation• Respondent identity• Interviewer can prevent input from other people in

household, noise and other distractions• May be the only way to reach certain

populations• Homeless• LGBT• Undocumented aliens

Disadvantages of personal interviews

• Expensive• Slow• Supervision of interviewing staff is difficult• Significant potential for interviewer bias

Telephone interviews• An interviewer asks questions of the

respondent over the phone• Very common method

• Computer-automated dialing• Computer-aided interviewing

• Large facilities with multiple interviewing stations

Advantages of telephone interviewing

• Moderate cost• Can be carried out quickly• Supervising interviewers is relatively easy• Interviewers can help respondents with their

questions and concerns• High response rate

• Callbacks are relatively easy• Personal touch (human voice)

Disadvantages of telephone interviews

• Less control over the interview situation• Cannot use visuals• No face contact• Respondents get bored quickly

• Probes, depth limited

Disadvantages of phone interviews

• Response rate is lower than with personal interviews• Unlisted numbers• Wrong numbers (turnover is rapid)

• Class attempt to use UK phonebook was a disaster

• Cell phones• Refusals• Not-at-homes (answering machines)

Questionnaires v. interviews

• Questionnaires are presented to the respondents, who fill them out themselves• Distribution can happen in a wide variety of

ways• Product warranties• Restaurants• Doctors’ offices• Magazines• Blogs

Mail-distributed questionnaires

• A questionnaire is sent through the mail, self-administered

• Used for radio/tv diaries, disks with ads on them, product warranty cards, political polls by representatives

Advantages of mail questionnaires

• Low cost• Wide sample possible• No field staff to manage• Confidentiality• No interviewer bias• Respondent is not rushed, can answer

questions at her leisure • Can include limited graphics

Disadvantages of mail questionnaires

• Low response rates• May be biased in favor of those interested in

topic• Respondents must interpret questions without

help available• Complicated questions cannot be asked (nor

can extensive probes be used)• No ability to be certain the respondent is who

he says he is

Disadvantages of mail questionnaires

• Respondents must be literate in the language on the survey • U.S. has a high adult illiteracy rate• English may not be the respondent’s first

language

• Slow response• May take weeks or even months

Nielsen’s diary

• http://diary.tvratings.com/

Other distribution systems for paper questionnaires

• Handed out at worksites, doctor’s offices, etc.

• Group administration• Targeted audiences (not random)• Efficient• High response rate

Internet-based questionnaire distribution

• The survey is distributed in computer file form, either to list of people via e-mail or administered to those who visit a website

• Popularity rapidly rising• May be too popular—people are simply ignoring

Advantages of Internet-distributed questionnaires

• Very inexpensive• Data can automatically be included into the

database without inputting• Skip patterns can be programmed in• Data can be collected quickly• Audiovisual materials can accompany the

questionnaire

Disadvantages of computer-mediated questionnaires

• Sample bias• Many people have limited Internet access

• Upscale homes, workplaces• Internet access is not the same as use

• Little help is available for respondent• Little control

• (skipped questions, ‘help’ from others during response, etc.)

Disadvantages of computer-mediated questionnaires

• Respondent self-selection • Low response rate• Multiple response

• Groups may want to influence survey outcome, though this is relatively rare

Personal Telephone Mail Computer

Cost

Response rate

Control

Flexibility

Speed

Help

Confidentiality

Rapport

Sample bias