Date post: | 05-Jul-2018 |
Category: |
Documents |
Upload: | sajal-chakarvarty |
View: | 215 times |
Download: | 0 times |
of 58
8/16/2019 Question Design (1)
1/58
WHAT IS A QUESTIONNAIRE?
An instrument (form) tocollect answers to questions
collect factual data
gathers information or measures A series of written questions/items in a fixed,
rational order
8/16/2019 Question Design (1)
2/58
WHY USING A QUESTIONNAIRE?
A well designed questionnaire:
Givesaccurate andrelevant information to your
research question
Minimizes potential sources of biasWill more likely be completed
⇒ As simple and focused as possible
8/16/2019 Question Design (1)
3/58
Statistical Terms
• Population: Discrete group whose members can be defined by a set
of at least one shared characteristic
• Sample: A selection of that target population (can also be a
subsample)
• Random Sample: Sample in which every member of a given
population has an equal chance of being selected
• Margin of Sampling Error (MOSE): Every random sample has a
margin of error (statistical price you pay for not interviewingeveryone)
• Response Rate: ate at which people agreed to participate in
survey! "f those who refuse are different from those who agree#
results may be s$ewed (higher response rate % less chance for s$ew)
8/16/2019 Question Design (1)
4/58
Statistical Terms
• Population: Discrete group whose members can be defined by a set
of at least one shared characteristic
• Sample: A selection of that target population (can also be a
subsample)
• Random Sample: Sample in which every member of a given
population has an equal chance of being selected
• Margin of Sampling Error (MOSE): Every random sample has a
margin of error (statistical price you pay for not interviewingeveryone)
• Response Rate: ate at which people agreed to participate in
survey! "f those who refuse are different from those who agree#
results may be s$ewed (higher response rate % less chance for s$ew)
8/16/2019 Question Design (1)
5/58
ADVANTAGES OF QUESTIONNAIRES
Can reach a large number of people
relatively easily and economically
Provide quantifiable answers
Relatively easy to analyse
8/16/2019 Question Design (1)
6/58
DISADVANTAGES OF QUESTIONNAIRES
Provides only limited insight into
problemLimited response allowed by questions
Maybe not the right questions are asked
Varying response
Misunderstanding/misinterpretationNeed to get it right first timeHard to chase after missing data
8/16/2019 Question Design (1)
7/58
TYPES OF QUESTIONNAIRE
Face to face
Telephone
By mail
E-mail/InternetSelf-administrated
Interviewer
-administrated
8/16/2019 Question Design (1)
8/58
SELF-ADMINISTERED QUESTIONNAIRE
Advantages:Cheap and easy to administer
Preserves confidentiality
Completed at respondent's convenienceNo influence by interviewer
8/16/2019 Question Design (1)
9/58
SELF-ADMINISTERED QUESTIONNAIRE
Disadvantages:Low response rate
Questions can be misunderstood
No control by interviewerTime and resouces loss
8/16/2019 Question Design (1)
10/58
INTERVIEW-ADMINISTERED
QUESTIONNAIRE
Advantages:Participation by illiterate people
Clarification of ambiguity
Quick answers
8/16/2019 Question Design (1)
11/58
INTERVIEW-ADMINISTERED
QUESTIONNAIRE
Disadvantages:Interviewer bias
Needs more resources
Only short questionnaires possibleEspecially on telephone
Difficult for sensitive issues
8/16/2019 Question Design (1)
12/58
8/16/2019 Question Design (1)
13/58
Basic Survey Process
Steps:
1 !eciding on a mec"anism
# !ra$ing a sample
% !esigning t"e &uestionnaire
&! 'ielding a survey and collecting the data
! Analysis
! eporting
8/16/2019 Question Design (1)
14/58
STAGES IN DESIGNING A QUESTIONNAIRE (1)
Planning the study:Decide on goalsIdentify risk factors for getting bitten by dogs
Know the subjectLiterature, experts on dog bites
Formulate a hypothesisPostmen more likely to get bitten by dogs than thenormal population
Define information needed to test hypothesisOccupation, owning dog, outdoor activities, attitudetowards dogs
8/16/2019 Question Design (1)
15/58
STAGES IN DESIGNING A QUESTIONNAIRE
(2)
Determine study population:
Know the respondentsOccupation
Special sensitivitiesEducation
Ethnic
Language
⇒Questionnaire needs to be adapted to yourpopulation, not the opposite!
8/16/2019 Question Design (1)
16/58
STAGES IN DESIGNING A
QUESTIONNAIRE (3)
Design questions:
Content of the questions
Format of the questions
Presentation and layout
Coding schedule (if appropriate)
Pilot and refine questionnaire
8/16/2019 Question Design (1)
17/58
WHAT DO YOU PREFER?
8/16/2019 Question Design (1)
18/58
WHAT DO YOU PREFER?
8/16/2019 Question Design (1)
19/58
WHAT MAKES A WELL DESIGNED
QUESTIONNAIRE?
Good appearance
→ easy on the eye
Short and simple
Relevant and logical
⇒ High response rate
⇒ Easy data summarisation and analysis
8/16/2019 Question Design (1)
20/58
QUESTION ORDER
Decide onorder of items/questions Easy → difficult
General→ particular
Factual → abstract
Where to placesensitive questions?
Be aware of ordering effects!
8/16/2019 Question Design (1)
21/58
QUESTION ORDER
Questions should be ordered so as to seem
logical to the respondent
First questions should be relevant and easy
Questions are effectively ordered from mostsalient to least salient
Demographic questions should not be
covered at the beginning
Potentially objectionable questions areplaced near the end
8/16/2019 Question Design (1)
22/58
QUESTION ORDER
Group questions by topic/ response options
Starting questionsSimple
With closed format
Relevant to main subject
Non-offending
Neither demographic nor personal questionsDon’t put most important item last
8/16/2019 Question Design (1)
23/58
QUESTIONNAIRE INTRODUCTION
Survey/interview introductionWho you are/ what is your ILP/thesis topic
Why you are investigating
Where you obtained the respondent’s nameHow and where you can be contacted
Guarantee of confidentiality
Length of interview (be honest)
⇒
Usefulness of study should be clearto all respondents
8/16/2019 Question Design (1)
24/58
CONTENT OF QUESTIONS
Clearfocus on research question
Avoid sidetracking
Avoid unnecessary information
Demographic informationContact information (if non-anonymised)
8/16/2019 Question Design (1)
25/58
DO:
Use simple wording
Be brief
Be specific
8/16/2019 Question Design (1)
26/58
DO NOT:
Be vague
Be condescending or talk down to
respondent
Use biased wordingUse abbreviations or scientific jargon
Use objectionable questions
Be redundant
8/16/2019 Question Design (1)
27/58
FORMAT OF QUESTIONS
Adjust toresponding audience
Professionals vs. public
Keep sentences simple and short
Define key wordsRemember option “don’t know”
8/16/2019 Question Design (1)
28/58
FORMAT OF QUESTIONS
Ask for one information at a time
Do you own a dog or have frequent contacts with dogs?
Yes
No
Use mutually exclusive and exhaustive answeroptions
Vertical order of answer options
8/16/2019 Question Design (1)
29/58
BE ACCURATE
Do you often touch dogs?
Yes
No
vs.How often did you touch a dog during thepast 3 months?
Once
Twice Three times or more Not at all Don´t now
8/16/2019 Question Design (1)
30/58
BE APPROPRIATE
!re you a drun?
Yes
No
vs.
How often have you consumedalcoholic "everages during the past #months?
Daily $%# times&wee
Once a wee
'ess than once a wee
Don´t now
8/16/2019 Question Design (1)
31/58
BE OBJECTIVE
Did you drin the strange "rownish drin in(rague?
YesNo
Vs.
)hich "everage did you consume?
)ater
*eer )ine +arad, None of them Don´t now
8/16/2019 Question Design (1)
32/58
BE SIMPLE
Did you smoe not less than a meanamountof - cigarettes&$ days from ./// onwards?
Yes
No
vs.
Did you smoe an average of $ pac ofcigarettes&wee for the last 0 years?
Yes No
Don´t now
8/16/2019 Question Design (1)
33/58
BIAS
Bias = systematic differences in the
measurement of a response
8/16/2019 Question Design (1)
34/58
INFORMATION BIAS
Recall biasCases more likely to remember than controls
Observer bias Different interviewer – different interpretations
Different interpretation of similar questions
Reduce by structured questionnaire
8/16/2019 Question Design (1)
35/58
NON-RESPONSE BIAS
Those who respond are different from those who
do notTelephone interviews: more females, elderly
ReduceEnsure high response rate
Random choice of interview partners
Correct during analysis (eg age, sex)
8/16/2019 Question Design (1)
36/58
FORMAT OF QUESTIONS
Two main question formats
Closed format→ forced choice
Yes
!lways
No 1ometimesDon2t now Never
Open format→ free text
)hat is your most distressing symptom?(lease descri"e444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
8/16/2019 Question Design (1)
37/58
OPEN OR CLOSED?
Closed
Advantages:Simple and quick
Reduces discrimination against less literateEasy to code, record, analyze
Easy to compare
Easy to report results
8/16/2019 Question Design (1)
38/58
CLOSED QUESTIONS
Disadvantages: Restricted number of possible answers
Loss of information
Possible compromise: Insert field „others“
8/16/2019 Question Design (1)
39/58
OPEN QUESTIONS
Advantages:Not directive
Allowsexploration of issues
to generate hypothesisqualitative research, focus groups, trawling questionnaires
Used even if no comprehensive range of alternative choices
Good for exploring knowledge and attitudes
Detailed and unexpected answers possible
8/16/2019 Question Design (1)
40/58
OPEN QUESTIONS
Disadvantages:Interviewer bias
Time-consuming
Coding problemsDifficult to analyze!
Difficult to compare groups
8/16/2019 Question Design (1)
41/58
CLOSED QUESTIONS
1. Straightforward response
)hat is your age in years? 444 years
How long have you owned a dog? 444
years )hat is your se5 6gender7?
8ale
9emale
Did you stay in Hotel : on $3&-&;0? YesNo
Don2t now
8/16/2019 Question Design (1)
42/58
CLOSED QUESTIONS
2. Checklist
)hich of the following outdooractivitiesdid you do last wee?
8/16/2019 Question Design (1)
43/58
CLOSED QUESTIONS
3. Rating scale
Did you do use sunscreen during the followingoutdoor activities during the past si5 months?
!lways 1ometimes 1eldomly Never
8/16/2019 Question Design (1)
44/58
CLOSED QUESTIONS
4. Rating scaleNumerical
How useful would you thin that informationonthe ris of "iting from stray dogs would "e?(please circle) . $ 3 > 0 # -
Not at all useful ery useful
!nalogueHow much is your pain severe 6put the ticon the line7
;.;
8/16/2019 Question Design (1)
45/58
CLOSED QUESTIONS
5. Scales for measuring attitude (Lickert)
1tray dogs carry a higher ris of ra"ies
No@ A strongly disagreeNo@ A disagree quite a lot
No@ A disagree Bust a little
A2m not sure a"out this
Yes@ A agree Bust a little Yes@ A agree quite a lot
Yes@ A strongly agree
8/16/2019 Question Design (1)
46/58
PROBLEMS AND PITFALLS
Avoid questions that asktwo things at once -
you won’t know which ‘bit’ people are answering:
Have you ever had stomach ache anddiarrhoea?
Ambiguity.....
Do you go to the woods a lot?
8/16/2019 Question Design (1)
47/58
PROBLEMS AND PITFALLS
Avoid jargon/abbreviations/slang
How often do you get up at night to (C?6pass urine7
1hould ADCs "e treated in the community?
Avoidnot mutually exclusiveoptions )hat age are you?
.#%$;
$;%$0
$0%3;
30%>;
8/16/2019 Question Design (1)
48/58
PROBLEMS AND PITFALLS
Avoidleading questions
Do you thin that the food in the hotelmade you sic?
Did the hotel sta seem unhygenic to you?
Do you agree that the hospital sta wereclose to e5haustion?
Avoid making questionnairetoo long
Typographical / spelling errors
8/16/2019 Question Design (1)
49/58
PILOTING AND EVALUATION
Pilot with asimilar group of people to your
intended subjects
Highlights problems before startingEffects of alternative wording
Overall impression on respondents and interviewers
Final polishing after several amendments
8/16/2019 Question Design (1)
50/58
PRESENTATION AND LAYOUT
Clear consistentlayout Adequate space to answer
Large font size Appropriate page breaks
Avoidexperimental layouts
fancy logos
printed on recycled paper/is an equal opportunity employer etc
S O O
8/16/2019 Question Design (1)
51/58
PRESENTATION AND LAYOUT
Usingcolour or printing questionnaire on coloured
paper may help
Usefilter questions, if necessary
Giveclear instructions about how to answer the
questions
8/16/2019 Question Design (1)
52/58
WAYS TO INCREASE PERCEIVED
REWARD
Show positive regard
Say thank you Ask them for “advice”Give social validationGive a tangible reward
Make the questionnaire interestingState an upcoming deadline
8/16/2019 Question Design (1)
53/58
WAYS TO DECREASE PERCEIVED
COST
Avoid subordinating language
Do not embarrass the respondentDo not inconvenience the respondentMake questions appear short and easy Avoid asking personal information (if it must
be asked, use ‘soft’ wording and inform aboutconfidentiality)Keep subsequent requests similar
8/16/2019 Question Design (1)
54/58
CREATE A NAVIGATIONAL PATH
Instructions should be placed right where
they are needed
Matrices are confusing: it is best to order
questions from top to bottom, and left to
right
Uselarger font to attract attentionUse color shading to attract attention or
show groupingsUse spacing and similarities to show
groupings
8/16/2019 Question Design (1)
55/58
WAYS TO ESTABLISHTRUST
Provide a token of appreciation
Provide a sense of legitimate authorityMake completing the questionnaire seem
important
Remind respondent of previous relationship
with sponsor (if applicable)
8/16/2019 Question Design (1)
56/58
SUMMARY
A well designed questionnaire:
Will give appropriate data which allow toanswer
your research question
Will minimise potential sources ofbias, thusincreasing the validity of the questionnaire
Will much more likely becompleted
8/16/2019 Question Design (1)
57/58
FINALLY, KEEP YOURQUESTIONNAIRESHORT
AND THE QUESTIONSSIMPLE,FOCUSED AND
APPROPRIATE
8/16/2019 Question Design (1)
58/58