Chapter 14Questionnaire Development

Winston Jackson and Norine Verberg

Methods: Doing Social Research, 4e

14-2 © 2007 Pearson Education Canada

Questionnaire Design

Questionnaires have many applications and are widely used

The textbook covers: General guidelines Types of questions illustrated Steps and rules for questionnaire design Rules for ordering, formatting, and presenting

14-3 © 2007 Pearson Education Canada

General Guidelines

Consult the respondent Able to express opinion freely, opinions valued

Keep it short Ask only pertinent questions

Achieve precise measurement Collect data in its rawest form (income in

dollars, precise occupation, age to the year, not age category)

highest level of measurement possible

14-4 © 2007 Pearson Education Canada

Types of Questions Illustrated

1. Pre-coded, single-choice questions

2. Open-ended questions

3. Presence-absence questions

4. Rank-ordering questions

5. Likert-type questions

6. Index development

14-5 © 2007 Pearson Education Canada

Types: Pre-Coded, Single-Choice

Pre-coded, single-choice questions ask respondents to indicate which one category applies Answers are pre-coded for easy data entry

If not all options can be listed, include a category entitled “Other” with a space to indicate what the “Other” category implies “Please specify” ______________

Categories should be mutually-exclusive i.e., no categories overlap with one another

14-6 © 2007 Pearson Education Canada

Pre-Coded, Single-Choice: Example

4. What year are you in? Freshman 1 Sophomore 2 Junior 3 Senior 4

See Box 14.1 (p. 383) for more examples

14-7 © 2007 Pearson Education Canada

Types: Open-Ended Questions

Open-ended questions do not have pre-set answers. Excellent way to explore new areas

Use open-ended questions when: Too many response categories (year of birth) You don’t wish to impose categories on

respondents “Really” consulting respondents Qualitative – source of quotations Determining appropriate categorization You want a change in pace for respondents

14-8 © 2007 Pearson Education Canada

Open-Ended: Example

21. What is (or was) your father’s occupation (e.g., supervisor, railway machine shop … supervises work of about 25 people)? Job ________________________________________Brief Job Description _____________________________________________________________________

23. What is the one thing you would like to see changed at the university? ________________________________________________________________________________________

More examples in Box 14.3 on page 385

14-9 © 2007 Pearson Education Canada

Open-Ended: Caution

Minimize the number of open-ended questions because they: Are time-consuming to code May generate responses that are inconsistent Are more likely to be left blank

May decrease response rate Takes more time to complete the questionnaire Use opinion-seeking questions sparingly

Place open-ended questions at 2/3 mark

14-10 © 2007 Pearson Education Canada

Types: Presence-Absence Questions

Presence-absence questions ask respondents to check off which items in a list do or do not apply to them

Less commonly used than other types of questions

14-11 © 2007 Pearson Education Canada

Presence-Absence: Example

23. Have you ever had contact with a physically handicapped person in any of these groups? (Circle to indicate “yes” or “no” for each group)

Yes No Community ----------- 1 0

Family ----------------- 1 0

Relatives -------------- 1 0

Elementary school -- 1 0

High school ----------- 1 0

University class ------ 1 0

As a co-worker ------- 1 0

14-12 © 2007 Pearson Education Canada

Types: Rank-Ordering Questions

Rank-order questions ask respondents to indicate an ordering of response items, usually from most preferred to least preferred

Must be done with great care Ask for only three most important items Must make instructions explicit

These types of questions should be avoided or minimized because they take time

14-13 © 2007 Pearson Education Canada

Rank-Ordering Questions: Example

31. Rank-order the three most important characteristics you want in the job you make your life’s work. (Place a 1 beside the most important one; a 2 beside the second most important one; and a 3 beside the next most important one.)

High salary.…………. ____Security.…………….. ____Continued interest….. ____Power……………….. ____Prestige……………… ____Excitement………….. ____

14-14 © 2007 Pearson Education Canada

Types: Likert-Type Questions

Likert-type questions ask respondents to indicate how much they agree or disagree with a statement

Response options originally included: strongly disagree, disagree, are undecided or neutral, agree, strongly agree Today often used with numbered response

options (see example, next slide) Used in the development of indexes (see

Chapter 13)

14-15 © 2007 Pearson Education Canada

Likert-Type Questions: Example

In the following, circle a number to indicate the extent to which you agree or disagree with each statement:

52. I believe capital punishment represents the most effective deterrent to murder. Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

53. I believe a murderer can be rehabilitated to become a resonsible, functioning member of society. Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

54. I believe a life sentence is a satisfactory penalty for murder. Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

14-16 © 2007 Pearson Education Canada

Tips for Likert-Style Questions

1. Avoid the word “and” Makes the questions multidimensional

2. Always place “strongly agree” on right side, with 9 indicating strong agreement Response set, a situation in which the respondent

answers similarly to all answers, is best avoided by wording some statements positively, others negatively

3. Avoid negatives that can confuse respondents Use direct negative statements

14-17 © 2007 Pearson Education Canada

Tips for Likert-Style Questions (cont’d)4. Vary “strength of wording” to produce

variation in response1. The nursing care I received was good.

2. The nursing care I received was perfect in every instance

5. Before the first Likert-type item, provide a brief explanation of how respondents are to to indicate their answers

14-18 © 2007 Pearson Education Canada

Steps in Questionnaire Development

1. Make a list of variables. Usually includes:1. Background variables2. Dependent variable(s)3. Independent variables4. Others: intervening, antecedent, spurious

2. Anticipate how data will be analyzed Procedures depend on level of measurement

3. Write the proposed questions on index cards Facilitates editing and re-arranging order

4. Double check: do you have all the variables?

14-19 © 2007 Pearson Education Canada

Steps (cont’d)

5. Review the conceptual definitions of variables Done in anticipation of step 6 - developing

the wording Example: how to measure socioeconomic

status? As reflecting various levels of respect and

prestige: use Pineo and Porter’s scale of occupational prestige to measure SES

As reflecting variation in access to scarce resources: use total income to measure SES

14-20 © 2007 Pearson Education Canada

Steps (cont’d)

6. Develop wordings for questions After considering the conceptual definition,

also: Use simple words: Grade 7 or 8 level Avoid “and”: make sure only one question Vary wording to produce variation in replies Avoid complexity – make it simple Use existing wording if comparative study Take the edge off sensitive questions Be precise, highly specific when choosing

wordings

14-21 © 2007 Pearson Education Canada

Steps (cont’d)

7. Pre-test the questionnaire Start by filling out the survey yourself, then ask

individuals to do so Ask for feedback on the questionnaire Is it too long? Any problem with wording?

8. Pilot study Send questionnaire to a small sample of

respondents Use data to determine which items will be used for

indexes, modify unclear questions, create pre-coded, single-choice questions based on responses to open-ended questions

14-22 © 2007 Pearson Education Canada

Ordering the Questions

Use a brief statement to introduce the survey to the respondent – re: legitimacy of study Say who is conducting/sponsoring the study Assure confidentiality and anonymity

Ease them into it Place easy to answer/salient questions at the

beginning Place key and repeated questions at 1/3 point Group questions by type

E.g., put Likert-type questions together

14-23 © 2007 Pearson Education Canada

Formatting the Questions

Begin conditioning respondents E.g., have consistent formatting so respondents can get

used to how to answer (format types shown in next two slides)

Anticipate computer data entry Pre-code questions before data collection

Vary placement of response categories Response options are appealing and easy to understand

Clearly indicate branching (shown on slide 14-26) Clear respondents away from irrelevant questions

Make it easy to fill out; will increase response rate

14-24 © 2007 Pearson Education Canada

Four Format Styles Illustrated1. Babbie Format Style for Fixed-Choice Responses:

23. Have you ever smoked marijuana?

[ ] Yes

[ ] No

2. Dillman Format Style for Fixed-Choice Responses:

Q-22 Your Sex (Circle number of your answer)

1 MALE

2 FEMALE

Sources: Earl Babbie (1992) The Practice of Social Research, 6th ed. Belmont, CA: Wadsworth Publishing

Company. p. 155.Don A. Dillman (1978). Mail and Telephone Surveys: the Total Design Method. New York: John Wiley

and Sons. p. 134.

14-25 © 2007 Pearson Education Canada

Four Format Styles Illustrated3. Krahn Format Style for Fixed-Choice Responses:

54. In the past year, has any member of your immediate family (not counting yourself) been unemployed (out of work and not looking for work)?

No 1Yes 2

4. Jackson Format Style for Fixed-Choice Responses:

4. What year are you in?Freshman ------------- 1 Sophomore ----------- 2 Junior ------------------ 3 Senior ----------------- 4

Sources:Harvey Krahn (1991). “Sociological Methods of Research.” In Lorne Tepperman and R. Jack

Richardson, eds., The Social World: An Introduction to Sociology. Toronto: McGraw-Hill Ryerson. p. 51.

Winston Jackson (1995). Methods: Doing Social Research. Toronto: Prentice Hall. p. 388.

14-26 © 2007 Pearson Education Canada

Branching Question Illustrated

14-27 © 2007 Pearson Education Canada

Presentation of Questionnaire

Have a distinctive look E.g., coloured paper, graphics for questions is

appealing Can increase the appeal of doing the

questionnaire Don’t squeeze too much onto one page

Crowding questions on a page can make the questionnaire appear too long