Date post: | 03-Jan-2016 |
Category: |
Documents |
Upload: | frank-dawson |
View: | 216 times |
Download: | 0 times |
Surveys
Outline
1. Definition
2. When and why to use surveys
3. How to create a survey
Surveys
Surveys - Definition
• A survey is a procedure for systematically collecting information about attitudes, preferences, knowledge, or behavior by asking people questions.
Surveys
When to Use Surveys
• When you want information about a population, but you cannot measure the whole population.
• It is usually the case that you cannot measure whole population
• Doing so is unnecessary
Surveys
Survey research - Definition
• Survey research has 3 major characteristics:
• Asking people questions using a formal procedure
Surveys
Survey research - Definition
• Survey research has 3 major characteristics:
• Using a quantitative method that requires standardized information as input
Surveys
Survey research - Definition
• Survey research has 3 major characteristics:
• Generalizing results from sample to population it was drawn from
Surveys
When to Use Survey Research
• Two distinct forms of survey research:
• Exploratory
• Used in new research areas where little theory has developed
• “descriptive”
Surveys
When to Use Survey Research
• Two distinct forms of survey research:
• Exploratory• Explanatory
• Looking for evidence of cause-effect relations among variables
• “If C causes D, then C and D should be correlated.”
Surveys
Advantages of survey approach
• Surveys are:
• Cheap
• Relative to measuring whole population
Surveys
Advantages of survey approach
• Surveys are:
• Cheap• Practical
• Can be done in a short time
• Can be done over phone or through mail or internet
Surveys
Advantages of survey approach
• Surveys are:
• Cheap• Practical• Accurate
• Results true of population within a small margin of error
Surveys
Margin of Error
• A range of values• True population value is
likely to be in this range.• If range is large, survey
results are not useful
• Size of range depends upon sample size and confidence level chosen for estimating population value.
Value obtained from sample
Margin of error = range of valueswhich containspopulation value
%
Surveys
How to create a survey
• Research questions• Design• Survey questions• Sampling procedure• Administration procedure• Analyzing the results
Surveys
Research question
• Start with a behavioral theory.
• Use it to generate research questions.
• Specify type of information that will (or won’t) meet your scientific objectives
Surveys
Research question
• Be as specific as possible• Think about interpretation
• In reports, you won’t discuss the actual questions on your survey instrument – you’ll discuss some theoretical construct
• What is it?
Surveys
Design
Longitudinal • Same samples measured at different times
• Potential problem with loss of subjects
Surveys
Design
• Longitudinal• Cross-sectional
• One or more samples measured at one time
• If groups differ, it’s difficult to say why
Surveys
Design
• Longitudinal• Cross-sectional• Successive independent samples
• Different samples at different times
• Samples supposed to be drawn from same population – but that may be arguable if interval is long
Surveys
Survey questions
• If a survey exists that does the job, use it
• Otherwise…
• Write a first draft
Surveys
Survey questions
• If a survey exists that does the job, use it
• Otherwise…
• Write a first draft• Get feedback and
revise draft
Surveys
Survey questions
• If a survey exists that does the job, use it
• Otherwise…
• Write a first draft• Get feedback and
revise draft• Pretest your
questionnaire
Surveys
Survey questions
• If a survey exists that does the job, use it
• Otherwise…
• Write a first draft• Get feedback and
revise draft• Pretest your
questionnaire• Edit questions
Surveys
Survey questions
• If a survey exists that does the job, use it
• Otherwise…
• Write a first draft• Get feedback and
revise draft• Pretest your
questionnaire• Edit questions• Formalize procedures
Surveys
Survey questions
• Avoid bias • “Do you believe in killing unborn babies?”
• “Should women be forced to bear unwanted children?”
Surveys
Question from an ABC poll
• “Schiavo suffered brain damage and has been on life support for 15 years. Doctors say she has no consciousness and her condition is irreversible. Her husband and her parents disagree about whether she would have wanted to be kept alive. Florida courts have sided with the husband and her feeding tube was removed on Friday. What's your opinion on this case - do you support or oppose the decision to remove Schiavo's feeding tube?”
Surveys
A Gallup Poll question the same week
• “As you may know, on Friday the feeding tube keeping Terri Schiavo alive was removed. Based on what you have heard or read about the case, do you think that the feeding tube should or should not have been removed?”
Surveys
Fox News poll 18 days earlier
• “Terri Schiavo has been in a so-called 'persistent vegetative state' since 1990. Terri's husband says his wife would rather die than be kept alive artificially and wants her feeding tube removed. Terri's parents believe she could still recover and want the feeding tube to remain. If you were Terri's guardian, what would you do? Would you remove the feeding tube or would you keep the feeding tube inserted?”
Surveys
Issues
• Is “requiring tube feeding” the same as “being on life-support”?
• What does the public understand from the term “life-support”?
• Does the term bias the result?
Surveys
Issues
• Were results influenced by fact poll conducted in one evening?
• Were results influenced by whether the person involved was referred to as “Schiavo” or “Terri”?
Surveys
Survey questions
• Be careful about social desirability
• Think about question sequence
• Use filter questions
Surveys
Survey questions
• Don’t write two questions as one
• What would “No” mean in answer to this question:
• Do you support the Administration’s decisions to increase class sizes and start classes earlier in the day?
Surveys
Survey questions
• Do the people you’re surveying have the information required to answer your questions?
• Check to see before you ask your questions
Surveys
Survey questions
• “Are you familiar with the Liberal Party’s platform for this election?”
before
• “In your view, will the Liberal Party’s platform produce positive or negative effects on the economy?”
Surveys
Survey questions
• Is the meaning of your question clear?
• If you refer to “young people” do you mean 9 year olds? 25 year olds?
Surveys
Survey questions
• Is the meaning of your question clear?
• “Which newspaper do you read?”
• Does this mean everyday? Sometimes?
• What if the respondent reads more than one?
Surveys
Survey questions
• Is the meaning of your question clear?
• Be clear about the time frame of the behavior you are asking about
• E.g., CFQ asks about frequency of mental slips over the last six months
Surveys
Sampling procedure
• Major task: to select a representative sample
• Representative sample – shares distribution of relevant characteristics with population
Surveys
Sampling procedure
• Some technical terms you need to know
• Population• Sampling frame• Sample• Element
Population
Sampling frameSample
Element
Surveys
Types of sampling procedure
• Non-probability sampling
• Elements vary in probability of being chosen
• Those probabilities are not known
• Accidental samples• Purposive samples
Surveys
Types of sampling procedure
Non-probability sampling
Probability sampling
• Researcher knows the probability of inclusion for each element in population.
• Simple random sampling
• Stratified random sampling
Surveys
Administration procedures
• Phone • Cheap & fast• But who is home?• Verbal instructions
and response alternatives may be hard to remember
Surveys
Administration procedures
• Phone• Face-to-face
• Expensive• But maximal control• Interviewer can see
whether respondent understands question
• Respondent may work harder
Surveys
Administration procedures
• Phone• Face-to-face• Mail
• Inexpensive• Significant problem
with response bias
Surveys
Analyze the Results
• Select a data analysis procedure before you collect data.
• Correlations• Path analysis• Factor analysis