8/8/2019 Questionnaire Design Fall 2004
1/34
Questionnaire designQuestionnaire design
A survey is only as good as theA survey is only as good as the
questions it asksquestions it asks
8/8/2019 Questionnaire Design Fall 2004
2/34
Emotional IntelligenceEmotional Intelligence
http://quiz.ivillage.co.uk/uk_work/testhttp://quiz.ivillage.co.uk/uk_work/tests/eqtest.htms/eqtest.htm
8/8/2019 Questionnaire Design Fall 2004
3/34
What should you ask?What should you ask?
The questions asked are a function ofThe questions asked are a function ofprevious decisionsprevious decisions
The questions asked are a function ofThe questions asked are a function offuture decisions (such as statisticalfuture decisions (such as statisticalanalysis)analysis)
8/8/2019 Questionnaire Design Fall 2004
4/34
Key criteriaKey criteria
Questionnaire relevancyQuestionnaire relevancy
No unnecessary information is collectedNo unnecessary information is collectedand only information needed to solve theand only information needed to solve theproblem is obtained. Be specific aboutproblem is obtained. Be specific aboutyour data needs; tie each question to anyour data needs; tie each question to anobjectiveobjective
Questionnaire accuracyQuestionnaire accuracy
Information is both reliable and validInformation is both reliable and valid
8/8/2019 Questionnaire Design Fall 2004
5/34
Phrasing QuestionsPhrasing Questions
Open ended response versus fixedOpen ended response versus fixedalternative questionsalternative questions
?? Decision criteria: type of research;Decision criteria: type of research;
time; method of delivery; budget;time; method of delivery; budget;concerns regarding researcher biasconcerns regarding researcher bias
8/8/2019 Questionnaire Design Fall 2004
6/34
AvoidAvoid
Leading questionsLeading questions
Overly complex questionsOverly complex questions
Use of jargonUse of jargon
Loaded questions (can use a counterbiasingLoaded questions (can use a counterbiasingstatement)statement)
AmbiguityAmbiguity Double barreled questionsDouble barreled questions
Making assumptionsMaking assumptions
8/8/2019 Questionnaire Design Fall 2004
7/34
Order?Order?
Order bias results from an alternativeOrder bias results from an alternativeanswers position in a set of answersanswers position in a set of answers
or from the sequencing of questionsor from the sequencing of questions Funneling technique: general to specificFunneling technique: general to specific
helps understand the frame of referencehelps understand the frame of referencefirstfirst
Anchoring effect: the first conceptAnchoring effect: the first conceptmeasured tends to become ameasured tends to become acomparison point from whichcomparison point from whichsubsequent evaluations are madesubsequent evaluations are made
8/8/2019 Questionnaire Design Fall 2004
8/34
Ranking, sorting, rating or choice?Ranking, sorting, rating or choice?
How many categories or responseHow many categories or responsepositions?positions?
Balanced or unbalanced?Balanced or unbalanced?
Forced choice or nonforced choice?Forced choice or nonforced choice?
Single measure or index?Single measure or index?
DecisionsDecisions
8/8/2019 Questionnaire Design Fall 2004
9/34
Types of questionsTypes of questions
8/8/2019 Questionnaire Design Fall 2004
10/34
Single dichotomy or dichotomous-alternativequestions
Are you currently registered in a course at theUniversity of Lethbridge?
Yes____ No____
Respondent chooses one of two alternatives(yes/no; male/female)
What scale would this data create?
Types of fixed alternative questions
8/8/2019 Questionnaire Design Fall 2004
11/34
MultiMulti--choice alternativechoice alternative
Respondent chooses from severalRespondent chooses from severalalternativesalternatives
Many typesMany types
Types of fixed alternativeTypes of fixed alternative
questionsquestions
8/8/2019 Questionnaire Design Fall 2004
12/34
Determinant choiceDeterminant choice Choose only one from several possible responsesChoose only one from several possible responses
Which faculty are you currently registered in atWhich faculty are you currently registered in atthe University ofLethbridge?the University ofLethbridge?
Management ___Management ___
Education ____Education ____
Arts/Science____Arts/Science____
Health sciences____Health sciences____Combined degree____Combined degree____
What type of scale would these data create?What type of scale would these data create?
MultiMulti--choice alternativechoice alternative
qu
estions
qu
estions
8/8/2019 Questionnaire Design Fall 2004
13/34
Frequency determinationFrequency determination
Asks for an answer about frequencyAsks for an answer about frequencyof occurrenceof occurrence
In a typical week, how often do youIn a typical week, how often do you
purchase chocolate chip cookies?purchase chocolate chip cookies?__never__never
__ once__ once
__ 2 or more times__ 2 or more timesWhat type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
14/34
Check listCheck list
Provide multiple answers to a single questionProvide multiple answers to a single question Should be mutually exclusive and exhaustiveShould be mutually exclusive and exhaustive
What brands of chocolate chip cookies have you, toWhat brands of chocolate chip cookies have you, tothe best of your memory, purchased in the pastthe best of your memory, purchased in the past
month (check all that apply?)month (check all that apply?)__ Dare__ Dare
__ Chips Ahoy__ Chips Ahoy
__ Presidents Choice Decadent etc. etc.__ Presidents Choice Decadent etc. etc.
What type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
15/34
Attitude rating scalesAttitude rating scales
Attitude:Attitude:
An enduring disposition to consistentlyAn enduring disposition to consistentlyrespond to various aspect of therespond to various aspect of the
world, including persons, events andworld, including persons, events andobjectsobjects
Typically seen as having threeTypically seen as having three
components:components:CognitiveCognitive
AffectiveAffective
BehaviouralBehavioural
8/8/2019 Questionnaire Design Fall 2004
16/34
AffectiveAffective
The feelings or emotions towardThe feelings or emotions toward
an objectan object
8/8/2019 Questionnaire Design Fall 2004
17/34
CognitiveCognitive
Knowledge and beliefsKnowledge and beliefs
8/8/2019 Questionnaire Design Fall 2004
18/34
BehavioralBehavioral
Predisposition to actionPredisposition to action
IntentionsIntentions
Behavioral expectationsBehavioral expectations
8/8/2019 Questionnaire Design Fall 2004
19/34
Attitude Scales: Scaling Defined
The term scaling refers to procedures for
attempting to determine quantitative measures of
subjective and sometimes abstract concepts. It isdefined as a procedure for the assignment of
numbers to a property of objects in order to
impart some of the characteristics of numbers to
the properties in question.
8/8/2019 Questionnaire Design Fall 2004
20/34
Unidimensional
Scaling
Multidimensional
Scaling
Procedures
designed to
measure only one
attribute of arespondent or
object
Procedures
designed to measure
several dimensions
of a respondent or
object
8/8/2019 Questionnaire Design Fall 2004
21/34
AttitudemeasuringAttitudemeasuring
processprocess RankingRanking
RatingRating
SortingSorting
ChoiceChoice
8/8/2019 Questionnaire Design Fall 2004
22/34
Types of attitude scalesTypes of attitude scales
Simple attitude scalesSimple attitude scales
Most basic formMost basic form respondent responds to arespondent responds to a
single questionsingle question Do not allow for fine distinctions orDo not allow for fine distinctions or
placement on continuaplacement on continua
You are at a company party and are feelingYou are at a company party and are feelingnervous, but you are obligated to be there. Donervous, but you are obligated to be there. Doyou:you:
__ find someone you know to buddy up with__ find someone you know to buddy up with
__ take it as an opportunity to meet new people__ take it as an opportunity to meet new people
What type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
23/34
Category scalesCategory scales
More sensitive; provides more informationMore sensitive; provides more information Overall, how satisfied are you with the high speedOverall, how satisfied are you with the high speed
performance of your Mercedes:performance of your Mercedes:
__ very satisfied__ very satisfied
__ somewhat satisfied__ somewhat satisfied__ neither satisfied nor dissatisfied__ neither satisfied nor dissatisfied
__ somewhat dissatisfied__ somewhat dissatisfied
__ very dissatisfied__ very dissatisfied
If you could choose, how long would each term be?If you could choose, how long would each term be? ___26 weeks __ 13 weeks __ 6 weeks ___4 ___26 weeks __ 13 weeks __ 6 weeks ___4
weeksweeks
What type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
24/34
Summated rating scalesSummated rating scales the Likertthe Likert
scalescale Respondents indicate their attitudes byRespondents indicate their attitudes by
checking how strongly they agree orchecking how strongly they agree ordisagree with statementsdisagree with statements
Chocolate chip cookies are my preferredChocolate chip cookies are my preferredvariety of cookievariety of cookie
Strongly disagree Disagree UncertainStrongly disagree Disagree Uncertain AgreeAgree Strongly AgreeStrongly Agree(1) (2) (3) (4) (5)(1) (2) (3) (4) (5)
What type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
25/34
Semantic Differential Rating scaleSemantic Differential Rating scale
An attitude measure consisting of a seriesAn attitude measure consisting of a seriesof sevenof seven--point bipolar rating scalespoint bipolar rating scalesallowing response to a conceptallowing response to a concept
Think of your favorite type of cookie. Rate it on eachThink of your favorite type of cookie. Rate it on each
of the following continua:of the following continua:
HardHard------------------------------------------------------------------------------------------------------------SoftSoft
Lots of chipsLots of chips------------------------------------------------------------------------------FewerFewer
chipschipsCrispyCrispy------------------------------------------------------------------------------------------------------
chewychewy
What type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
26/34
Numerical Rating scaleNumerical Rating scale
Similar to a semantic differential except that itSimilar to a semantic differential except that ituses numbers as response options to identifyuses numbers as response options to identifyresponse positions instead of verbal descriptionsresponse positions instead of verbal descriptions
Think of your favorite type of cookie. Rate it on each of theThink of your favorite type of cookie. Rate it on each of thefollowing continua:following continua:
HardHard------------------------------------------------------------------------------------------------------------------------------------------------SoftSoft
88 77 66 55 44 33 22 11
This scale is called an 8 point numerical scale, why?This scale is called an 8 point numerical scale, why?
What type of scale would these data create?What type of scale would these data create?
8/8/2019 Questionnaire Design Fall 2004
27/34
Constant Sum ScalesConstant Sum Scales
Attributes based on their importance to theAttributes based on their importance to theperson. Respondents are asked to divide aperson. Respondents are asked to divide aconstant sum to indicate the relative importanceconstant sum to indicate the relative importanceof attributesof attributes
Example: Suppose the photocopy budget perExample: Suppose the photocopy budget perprofessor was $100 per month. How much shouldprofessor was $100 per month. How much shouldbe allocated to the following. Divide the $100be allocated to the following. Divide the $100according to your preference:according to your preference:
____ photocopying for student needs;____ photocopying for student needs;
____ photocopying for research needs;____ photocopying for research needs;____ photocopying for committee needs.____ photocopying for committee needs.========
$100 TOTAL$100 TOTAL
8/8/2019 Questionnaire Design Fall 2004
28/34
Stapel ScalesStapel Scales
An attitude measure that places a single adjectiveAn attitude measure that places a single adjectivein the center of an evenin the center of an even--number range ofnumber range ofnumerical valuesnumerical values
Example:Example:
Research MethodologyResearch Methodology+3+3
+2+2
+1+1
ExcitingExciting--11
--22
--33
8/8/2019 Questionnaire Design Fall 2004
29/34
Graphic Rating ScalesGraphic Rating Scales
An attitude measure consisting of a graphicAn attitude measure consisting of a graphiccontinuum that allows respondents to ratecontinuum that allows respondents to ratean object by choosing any point on thean object by choosing any point on thecontinuumcontinuum
8/8/2019 Questionnaire Design Fall 2004
30/34
RankRank--Order ScalesOrder Scales
Scales in which the respondent comparesScales in which the respondent comparesone item with another or a group of itemsone item with another or a group of itemsagainst each other and ranks them.against each other and ranks them.
Example: handoutExample: handout
8/8/2019 Questionnaire Design Fall 2004
31/34
Most important skillsMost important skills Adaptability to changeAdaptability to change
Problem identificationProblem identification
Listening skillsListening skills Written communicationWritten communication
LeadershipLeadership
Informal Oral communicationInformal Oral communication
Analytical thinking/problem solvingAnalytical thinking/problem solving Time managementTime management
Coping with stress/job pressuresCoping with stress/job pressures
Interpersonal relationsInterpersonal relations
Formal oral presentationsFormal oral presentations
8/8/2019 Questionnaire Design Fall 2004
32/34
Most important skillsMost important skillsAdaptability to changeAdaptability to change
Problem identificationProblem identification
Listening skillsListening skillsWritten communicationWritten communication
LeadershipLeadership
Informal Oral communicationInformal Oral communication
Analytical thinking/problem solvingAnalytical thinking/problem solving
Time managementTime management
Coping with stress/job pressuresCoping with stress/job pressures
Interpersonal relationsInterpersonal relations
Formal oral presentationsFormal oral presentations
8 9
6 6
1 12 4
4 2
3 3
5 5
7 10
11 7
9 8
10 11
8/8/2019 Questionnaire Design Fall 2004
33/34
Paired Comparison ScalesPaired Comparison Scales
Respondent is presented with two objectsRespondent is presented with two objectsand is asked to pick the preferred.and is asked to pick the preferred.
Example: Which type of cookie do you preferExample: Which type of cookie do you prefer
__ chocolate chip__ chocolate chip
__ oatmeal__ oatmeal
__ I do not have a preference between these two__ I do not have a preference between these two
8/8/2019 Questionnaire Design Fall 2004
34/34
SortingSorting
Respondent indicates their attitudes orRespondent indicates their attitudes orbeliefs by arranging items.beliefs by arranging items.
Example: Please sort the following cards withExample: Please sort the following cards with
pictures of cookies into the following categoriespictures of cookies into the following categories
LikeLike
DislikeDislike
Neither like nor dislikeNeither like nor dislike