SURVEYS

PR1600: Introduction to Research Methods in PIR

Problems with surveys

Short video from ‘Yes, Prime Minister’

Identify examples of bad practice in conducting surveys. What are the different types of error?

Case 1: EU membership & UK

You are a political advisor to the Prime Minister, who asks you to find out about public opinion on the UK’s membership to the EU.

You need to conduct a survey. What question will you ask? To whom? How will you draw your sample?

Case 2: Islamophobia

You want to investigate Islamophobia in the UK.

You need to conduct a survey. What question will you ask? To whom?

DISCUSSION

The American Jewish community examined two different surveys on American public opinion about holocaust denial, which produced different results.

Look at the question wording and discuss which question you think is most reliable (and why).

For which question do you think holocaust denial was highest? Why?

Roper organization

1. The term Holocaust usually refers to the killing of millions of Jews in Nazi death camps during World War II. Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?

Response categories:Possible:Impossible :Don’t Know:

Bruskin organization

2. The Holocaust is the term used to describe the extermination of the Jews in World War Two. Some people have said it is possible that it never actually happened. In your own mind, are you certain that the Nazi extermination of the Jews happened, or does it seem possible to you that it never happened?

Response categories:• Certain it happened:• Possible it never happened:• Don’t Know:

SURVEY RESULTS

Results of Roper survey:Possible: 22 %Impossible : 65 %Don’t Know: 13 %

Findings of Bruskin survey: Certain it happened: 93 % Possible it never happened: 3.7 % Don’t Know: 3.3 %

Publishing survey results

You are the co-editor of a well-known American newspaper. You were sent the results of a survey about Obama’s presidential performance.

Before you publish the survey findings in your newspaper, what would you want to know about how the survey was conducted?

Which is/are more important?

1. Who did the survey?2. Who paid for the survey and why was it done?3. How many people were interviewed for the

survey?4. How were those people chosen?5. How many people answered the question?6. What area (nation, state or region) were the

people chosen from?7. Who should have been surveyed and was

not? 8. When was the survey done?9. What questions were asked? 10. In what order were the questions asked?

Avoid these questions in surveys!

• Leading questions

• Double-barrelled questions

• Questions with complicated words

• Questions that lead to ‘recall problems’

Find examples for each problematic question.

Next week: descriptive statistics