Chapter FourteenQuestionnaire Development

Winston Jackson and Norine Verberg

Methods Doing Social Research, 4e

2 © 2007 Pearson Education Canada

Questionnaire Design

Questionnaires have many applications and are widely used

The textbook covers: General Guidelines Types of Questions Illustrated Steps and Rules for Questionnaire Design Rules for Ordering, Formatting, & Presenting

3 © 2007 Pearson Education Canada

General Guidelines

Consult the respondent able to express opinion freely, opinions valued

Keep it short only ask pertinent questions

Precise measurement collect data in its rawest form (income in

dollars, precise occupation, age to the year, not age category)

Highest level of measurement possible

4 © 2007 Pearson Education Canada

Types of Questions Illustrated

1. Pre-coded single-choice questions

2. Open-ended questions

3. Presence-absence questions

4. Rank-ordering questions

5. Likert-type questions

6. Index development

5 © 2007 Pearson Education Canada

Types: Pre-coded Single-choice

Pre-coded single-choice questions ask R to indicate which one category applies answers are pre-coded for easy data entry

If not all options can be listed, include a category entitled “Other” with a space to indicate what the “Other” category implies “please specify” ______________

Categories should be mutually-exclusive i.e., no categories overlap with one another

6 © 2007 Pearson Education Canada

Pre-coded Single-choice: Example

4. What year are you in? Freshman‑‑‑‑‑‑‑‑‑‑‑1 Sophomore‑‑‑‑‑‑‑‑‑‑------------------------------2 Junior‑‑‑‑‑‑‑‑‑‑‑‑‑---------------------------------‑3 Senior‑‑‑‑‑‑‑‑‑‑‑‑---------------------------------‑‑4

See Box 14.1 (p., 383) for more examples

7 © 2007 Pearson Education Canada

Types: Open-ended Questions

Open-ended questions do not have pre-set answers. Excellent way to explore new areas

Use open-ended questions when... too many response categories (year of birth) you don’t wish to impose categories on

respondent “really” consulting respondent qualitative--source of quotations determining appropriate categorization change-in-pace for respondent

8 © 2007 Pearson Education Canada

Open-ended: Example

21. What is (or was) your father’s occupation (e.g., supervisor, railway machine shop, supervises work of about 25 people)?

Job _____________________________________

Brief Job Description _______________________

________________________________________

23. What is the one thing you would like to see changed at the university? _________________________

_______________________________________

More examples in Box 14.3 on page 385

9 © 2007 Pearson Education Canada

Open-ended: Caution

Minimize the number of open-ended questions because: they are time-consuming to code may generate responses that are inconsistent more likely to be left blank

May increase response rate takes more time to complete the questionnaire use opinion-seeking questions sparingly

Place open-ended questions at 2/3 mark

10 © 2007 Pearson Education Canada

Types: Presence-absence Questions

Presence-absence questions ask R to check off which items in a list do or do not apply to them

Less commonly used than other types of questions

11 © 2007 Pearson Education Canada

Presence-absence: Example

23. Have you ever had contact with a physically handicapped person in any of these group? (Circle to indicate yes or no for each group)

Yes No Community ----------- 1 0 Family ----------------- 1 0 Relatives -------------- 1 0 Elementary school -- 1 0 High school ----------- 1 0 University class ------ 1 0 As a co-worker ------- 1 0

12 © 2007 Pearson Education Canada

Types: Rank-ordering questions

Rank-order questions ask R to indicate an ordering of response items, usually from most preferred to least preferred

Must be done with great care Ask for only three most important items Must make instructions explicit

These types of questions should be avoided or minimized because they take time

13 © 2007 Pearson Education Canada

Rank-ordering questions: example

31. Rank-order the three most important characteristics you want in the job you make your life’s work. Place a 1 beside the most important one; a 2 beside the second most important one; and a 3 beside the next most important one.

high salary.………….____security.……………..____continued interest….____power……………….____prestige……………..____excitement………….____

14 © 2007 Pearson Education Canada

Types: Likert-type questions

Likert-type questions ask R to indicate how much they agree or disagree with a statement

Response options originally included: strongly disagree, disagree, are undecided or neutral, agree, strongly agree Today often used with numbered response

options (see example, next slide) Used in the development of indexes (see

Chapter 13)

15 © 2007 Pearson Education Canada

Likert-type Questions: Example

In the following, circle a number to indicate the extent to which you agree or disagree with each statement:

52. I believe capital punishment represents the most effective deterrent to murder.Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

53. I believe a murderer can be rehabilitated to become a resonsible, functioning member of society.Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

54. I believe a life sentence is a satisfactory penalty for murder.Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

16 © 2007 Pearson Education Canada

Tips for Likert Style Questions

1. Avoid the word “and” Makes the questions multidimensional

2. Always place “strongly agree” on right side, with 9 indicating strong agreement Response set, a situation in which the

respondent answers similarly to all answers, is best avoided by wording some statements positively, others negatively

3. Avoid negatives that can confuse R Use direct negative statements

17 © 2007 Pearson Education Canada

Tips: Likert-style Questions

4. Vary “strength of wording” to produce variation in response

1. The nursing care I received was good.

2. The nursing care I received was perfect in every instance

5. Before the first Likert-type item, provide a brief explanation of how respondents are to to indicate their answers

18 © 2007 Pearson Education Canada

Steps in Questionnaire Development

1. Make a list variables. Usually includes:1. Background variables2. Dependent variable(s)3. Independent variables4. Others: intervening, antecedent, spurious

2. Anticipate how data will be analyzed Procedures depend on level of measurement

3. Write proposed questions on index cards Facilitates editing and re-arranging order

4. Double check: do you have all variables?

19 © 2007 Pearson Education Canada

Steps (cont’d)

5. Review the conceptual definitions of variables Done in anticipation of step 6 - developing

the wording Example: how to measure socioeconomic

status? As reflecting various levels of respect and

prestige: use Pineo and Porter’s scale of occupational prestige to measure SES

As reflecting variation in access to scarce resources: use total income to measure SES

20 © 2007 Pearson Education Canada

Steps (cont’d)

6. Developing wordings After considering the conceptual def’n, also: Use simple words: grade 7 or 8 level Avoiding “and”: make sure only one question Vary wording to produce variation in replies Avoid complexity - make it simple Use existing wording if comparative study Take edge off sensitive questions Be precise, highly specific when choosing

wordings

21 © 2007 Pearson Education Canada

Steps (cont’d)

7. Pre-test questionnaire Start by filling out the survey yourself, then ask

individuals to do so Ask for feedback on the questionnaire Is it too long? Any problem with wording?

8. Pilot study Send questionnaire to a small sample of respondents Use data to determine which items will be used for

indexes, modify unclear questions, create pre-coded single choice questions based on responses to open-ended questions

22 © 2007 Pearson Education Canada

Ordering the Questions

Use a brief statement to introduce the survey to the respondent - re: legitimacy of study say who is conducting/sponsoring the study assure confidentiality and anonymity

Ease them into it place easy to answer/salient questions at the

beginning Place key and repeated questions at 1/3 point Group questions by type

e.g., put Likert-type questions together

23 © 2007 Pearson Education Canada

Formatting the Questions

Begin conditioning respondents e.g., have consistent formatting so R can get used to

how to answer (format types shown in next slide) Anticipate computer data entry

Pre-code questions before data collection Vary placement of response categories

Response options are appealing & easy to understand Clearly indicate branching (shown slide after next)

Clear respondent away from irrelevant questions Make it easy to fill out; Will increase response rate

24 © 2007 Pearson Education Canada

Four Format Styles Illustrated

FPO Box 14.9 Four Format Styles Illustrated, page 400

25 © 2007 Pearson Education Canada

Branching Question Illustrated

FPO Box 14.10 Branching Question Illustrated, page 400

26 © 2007 Pearson Education Canada

Presentation of Questionnaire

Have a distinctive look e.g., coloured paper, graphics for questions is

appealing Can increase the appeal of doing the

questionnaire Don’t squeeze too much on to one page

Crowding questions on a page can make the questionnaire appear too long