Questionnaire Design Fall 2004

8/8/2019 Questionnaire Design Fall 2004

1/34

Questionnaire designQuestionnaire design

A survey is only as good as theA survey is only as good as the

questions it asksquestions it asks


2/34

Emotional IntelligenceEmotional Intelligence

http://quiz.ivillage.co.uk/uk_work/testhttp://quiz.ivillage.co.uk/uk_work/tests/eqtest.htms/eqtest.htm


3/34

What should you ask?What should you ask?

The questions asked are a function ofThe questions asked are a function ofprevious decisionsprevious decisions

The questions asked are a function ofThe questions asked are a function offuture decisions (such as statisticalfuture decisions (such as statisticalanalysis)analysis)


4/34

Key criteriaKey criteria

Questionnaire relevancyQuestionnaire relevancy

No unnecessary information is collectedNo unnecessary information is collectedand only information needed to solve theand only information needed to solve theproblem is obtained. Be specific aboutproblem is obtained. Be specific aboutyour data needs; tie each question to anyour data needs; tie each question to anobjectiveobjective

Questionnaire accuracyQuestionnaire accuracy

Information is both reliable and validInformation is both reliable and valid


5/34

Phrasing QuestionsPhrasing Questions

Open ended response versus fixedOpen ended response versus fixedalternative questionsalternative questions

?? Decision criteria: type of research;Decision criteria: type of research;

time; method of delivery; budget;time; method of delivery; budget;concerns regarding researcher biasconcerns regarding researcher bias


6/34

AvoidAvoid

Leading questionsLeading questions

Overly complex questionsOverly complex questions

Use of jargonUse of jargon

Loaded questions (can use a counterbiasingLoaded questions (can use a counterbiasingstatement)statement)

AmbiguityAmbiguity Double barreled questionsDouble barreled questions

Making assumptionsMaking assumptions


7/34

Order?Order?

Order bias results from an alternativeOrder bias results from an alternativeanswers position in a set of answersanswers position in a set of answers

or from the sequencing of questionsor from the sequencing of questions Funneling technique: general to specificFunneling technique: general to specific

helps understand the frame of referencehelps understand the frame of referencefirstfirst

Anchoring effect: the first conceptAnchoring effect: the first conceptmeasured tends to become ameasured tends to become acomparison point from whichcomparison point from whichsubsequent evaluations are madesubsequent evaluations are made


8/34

Ranking, sorting, rating or choice?Ranking, sorting, rating or choice?

How many categories or responseHow many categories or responsepositions?positions?

Balanced or unbalanced?Balanced or unbalanced?

Forced choice or nonforced choice?Forced choice or nonforced choice?

Single measure or index?Single measure or index?

DecisionsDecisions


9/34

Types of questionsTypes of questions


10/34

Single dichotomy or dichotomous-alternativequestions

Are you currently registered in a course at theUniversity of Lethbridge?

Yes____ No____

Respondent chooses one of two alternatives(yes/no; male/female)

What scale would this data create?

Types of fixed alternative questions


11/34

MultiMulti--choice alternativechoice alternative

Respondent chooses from severalRespondent chooses from severalalternativesalternatives

Many typesMany types

Types of fixed alternativeTypes of fixed alternative

questionsquestions


12/34

Determinant choiceDeterminant choice Choose only one from several possible responsesChoose only one from several possible responses

Which faculty are you currently registered in atWhich faculty are you currently registered in atthe University ofLethbridge?the University ofLethbridge?

Management ___Management ___

Education ____Education ____

Arts/Science____Arts/Science____

Health sciences____Health sciences____Combined degree____Combined degree____

What type of scale would these data create?What type of scale would these data create?

MultiMulti--choice alternativechoice alternative

qu

estions

qu

estions


13/34

Frequency determinationFrequency determination

Asks for an answer about frequencyAsks for an answer about frequencyof occurrenceof occurrence

In a typical week, how often do youIn a typical week, how often do you

purchase chocolate chip cookies?purchase chocolate chip cookies?__never__never

__ once__ once

__ 2 or more times__ 2 or more timesWhat type of scale would these data create?What type of scale would these data create?


14/34

Check listCheck list

Provide multiple answers to a single questionProvide multiple answers to a single question Should be mutually exclusive and exhaustiveShould be mutually exclusive and exhaustive

What brands of chocolate chip cookies have you, toWhat brands of chocolate chip cookies have you, tothe best of your memory, purchased in the pastthe best of your memory, purchased in the past

month (check all that apply?)month (check all that apply?)__ Dare__ Dare

__ Chips Ahoy__ Chips Ahoy

__ Presidents Choice Decadent etc. etc.__ Presidents Choice Decadent etc. etc.



15/34

Attitude rating scalesAttitude rating scales

Attitude:Attitude:

An enduring disposition to consistentlyAn enduring disposition to consistentlyrespond to various aspect of therespond to various aspect of the

world, including persons, events andworld, including persons, events andobjectsobjects

Typically seen as having threeTypically seen as having three

components:components:CognitiveCognitive

AffectiveAffective

BehaviouralBehavioural


16/34

AffectiveAffective

The feelings or emotions towardThe feelings or emotions toward

an objectan object


17/34

CognitiveCognitive

Knowledge and beliefsKnowledge and beliefs


18/34

BehavioralBehavioral

Predisposition to actionPredisposition to action

IntentionsIntentions

Behavioral expectationsBehavioral expectations


19/34

Attitude Scales: Scaling Defined

The term scaling refers to procedures for

attempting to determine quantitative measures of

subjective and sometimes abstract concepts. It isdefined as a procedure for the assignment of

numbers to a property of objects in order to

impart some of the characteristics of numbers to

the properties in question.


20/34

Unidimensional

Scaling

Multidimensional

Scaling

Procedures

designed to

measure only one

attribute of arespondent or

object

Procedures

designed to measure

several dimensions

of a respondent or

object


21/34

AttitudemeasuringAttitudemeasuring

processprocess RankingRanking

RatingRating

SortingSorting

ChoiceChoice


22/34

Types of attitude scalesTypes of attitude scales

Simple attitude scalesSimple attitude scales

Most basic formMost basic form respondent responds to arespondent responds to a

single questionsingle question Do not allow for fine distinctions orDo not allow for fine distinctions or

placement on continuaplacement on continua

You are at a company party and are feelingYou are at a company party and are feelingnervous, but you are obligated to be there. Donervous, but you are obligated to be there. Doyou:you:

__ find someone you know to buddy up with__ find someone you know to buddy up with

__ take it as an opportunity to meet new people__ take it as an opportunity to meet new people



23/34

Category scalesCategory scales

More sensitive; provides more informationMore sensitive; provides more information Overall, how satisfied are you with the high speedOverall, how satisfied are you with the high speed

performance of your Mercedes:performance of your Mercedes:

__ very satisfied__ very satisfied

__ somewhat satisfied__ somewhat satisfied__ neither satisfied nor dissatisfied__ neither satisfied nor dissatisfied

__ somewhat dissatisfied__ somewhat dissatisfied

__ very dissatisfied__ very dissatisfied

If you could choose, how long would each term be?If you could choose, how long would each term be? ___26 weeks __ 13 weeks __ 6 weeks ___4 ___26 weeks __ 13 weeks __ 6 weeks ___4

weeksweeks



24/34

Summated rating scalesSummated rating scales the Likertthe Likert

scalescale Respondents indicate their attitudes byRespondents indicate their attitudes by

checking how strongly they agree orchecking how strongly they agree ordisagree with statementsdisagree with statements

Chocolate chip cookies are my preferredChocolate chip cookies are my preferredvariety of cookievariety of cookie

Strongly disagree Disagree UncertainStrongly disagree Disagree Uncertain AgreeAgree Strongly AgreeStrongly Agree(1) (2) (3) (4) (5)(1) (2) (3) (4) (5)



25/34

Semantic Differential Rating scaleSemantic Differential Rating scale

An attitude measure consisting of a seriesAn attitude measure consisting of a seriesof sevenof seven--point bipolar rating scalespoint bipolar rating scalesallowing response to a conceptallowing response to a concept

Think of your favorite type of cookie. Rate it on eachThink of your favorite type of cookie. Rate it on each

of the following continua:of the following continua:

HardHard------------------------------------------------------------------------------------------------------------SoftSoft

Lots of chipsLots of chips------------------------------------------------------------------------------FewerFewer

chipschipsCrispyCrispy------------------------------------------------------------------------------------------------------

chewychewy



26/34

Numerical Rating scaleNumerical Rating scale

Similar to a semantic differential except that itSimilar to a semantic differential except that ituses numbers as response options to identifyuses numbers as response options to identifyresponse positions instead of verbal descriptionsresponse positions instead of verbal descriptions

Think of your favorite type of cookie. Rate it on each of theThink of your favorite type of cookie. Rate it on each of thefollowing continua:following continua:

HardHard------------------------------------------------------------------------------------------------------------------------------------------------SoftSoft

88 77 66 55 44 33 22 11

This scale is called an 8 point numerical scale, why?This scale is called an 8 point numerical scale, why?



27/34

Constant Sum ScalesConstant Sum Scales

Attributes based on their importance to theAttributes based on their importance to theperson. Respondents are asked to divide aperson. Respondents are asked to divide aconstant sum to indicate the relative importanceconstant sum to indicate the relative importanceof attributesof attributes

Example: Suppose the photocopy budget perExample: Suppose the photocopy budget perprofessor was $100 per month. How much shouldprofessor was $100 per month. How much shouldbe allocated to the following. Divide the $100be allocated to the following. Divide the $100according to your preference:according to your preference:

____ photocopying for student needs;____ photocopying for student needs;

____ photocopying for research needs;____ photocopying for research needs;____ photocopying for committee needs.____ photocopying for committee needs.========

$100 TOTAL$100 TOTAL


28/34

Stapel ScalesStapel Scales

An attitude measure that places a single adjectiveAn attitude measure that places a single adjectivein the center of an evenin the center of an even--number range ofnumber range ofnumerical valuesnumerical values

Example:Example:

Research MethodologyResearch Methodology+3+3

+2+2

+1+1

ExcitingExciting--11

--22

--33


29/34

Graphic Rating ScalesGraphic Rating Scales

An attitude measure consisting of a graphicAn attitude measure consisting of a graphiccontinuum that allows respondents to ratecontinuum that allows respondents to ratean object by choosing any point on thean object by choosing any point on thecontinuumcontinuum


30/34

RankRank--Order ScalesOrder Scales

Scales in which the respondent comparesScales in which the respondent comparesone item with another or a group of itemsone item with another or a group of itemsagainst each other and ranks them.against each other and ranks them.

Example: handoutExample: handout


31/34

Most important skillsMost important skills Adaptability to changeAdaptability to change

Problem identificationProblem identification

Listening skillsListening skills Written communicationWritten communication

LeadershipLeadership

Informal Oral communicationInformal Oral communication

Analytical thinking/problem solvingAnalytical thinking/problem solving Time managementTime management

Coping with stress/job pressuresCoping with stress/job pressures

Interpersonal relationsInterpersonal relations

Formal oral presentationsFormal oral presentations


32/34

Most important skillsMost important skillsAdaptability to changeAdaptability to change

Problem identificationProblem identification

Listening skillsListening skillsWritten communicationWritten communication

LeadershipLeadership

Informal Oral communicationInformal Oral communication

Analytical thinking/problem solvingAnalytical thinking/problem solving

Time managementTime management

Coping with stress/job pressuresCoping with stress/job pressures

Interpersonal relationsInterpersonal relations

Formal oral presentationsFormal oral presentations

8 9

6 6

1 12 4

4 2

3 3

5 5

7 10

11 7

9 8

10 11


33/34

Paired Comparison ScalesPaired Comparison Scales

Respondent is presented with two objectsRespondent is presented with two objectsand is asked to pick the preferred.and is asked to pick the preferred.

Example: Which type of cookie do you preferExample: Which type of cookie do you prefer

__ chocolate chip__ chocolate chip

__ oatmeal__ oatmeal

__ I do not have a preference between these two__ I do not have a preference between these two


34/34

SortingSorting

Respondent indicates their attitudes orRespondent indicates their attitudes orbeliefs by arranging items.beliefs by arranging items.

Example: Please sort the following cards withExample: Please sort the following cards with

pictures of cookies into the following categoriespictures of cookies into the following categories

LikeLike

DislikeDislike

Neither like nor dislikeNeither like nor dislike

Date post:	10-Apr-2018
Category:	Documents
Upload:	lam-to
View:	219 times
Download:	0 times

Questionnaire Design Fall 2004

Documents