Date post: | 14-Dec-2015 |
Category: |
Documents |
Upload: | norman-hamer |
View: | 213 times |
Download: | 0 times |
© 2009 by The McGraw-Hill Companies, Inc.
Research Methods in Psychology
Survey Research
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research
Survey results• describe
opinions, attitudes, preferences
• allow predictions about behavior
Survey research uses• questionnaires
predetermined set of questions
• a sample represents a population
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research, continued
Surveys can be• limited, specific in scope• more global in their goals
Are surveys always biased?• Don’t assume bias just because a specific
organization or company has sponsored the survey
• Examine the survey procedures and analyses
© 2009 by The McGraw-Hill Companies, Inc.
Correlational Research
Assess relationships among naturally occurring variables• for example: attitudes, preferences, personality traits,
feelings, age, sex Correlation coefficients
• strength and direction of predictive relationship between two variables
–1.00 0 +1.00
negative no positive relationship
© 2009 by The McGraw-Hill Companies, Inc.
Correlational Research, continued
Do these relationships indicate positive or negative correlations?• As the number of years in which individuals smoke
cigarettes increases, the likelihood of lung cancer increases
• As the frequency of participating in volunteer activities increases, occasions of depressed mood decrease
• As arousal level increases, the likelihood of retaliation following an offense increases
© 2009 by The McGraw-Hill Companies, Inc.
Obtaining a Sample
Researchers are not interested simply in the responses of those who complete a survey• They seek to describe the larger population
from which the survey was drawn
Careful selection of a sample allows researchers to generalize findings from the sample to the population
© 2009 by The McGraw-Hill Companies, Inc.
Basic Terms of Sampling
Population• set of all cases of interest• examples:
current students at your school current residents of your city citizens of the United States
Sampling Frame• list of the members of a population
example: registrar’s list of registered students
© 2009 by The McGraw-Hill Companies, Inc.
Basic Terms of Sampling, continued
Sample• subset of the population used to represent the
entire population example: students in this class as a sample of all
students at this school (or this city)
Element• each member of the population
© 2009 by The McGraw-Hill Companies, Inc.
Goal of Sampling
sample should represent the population• characteristics of participants in sample
should be similar to those of the entire population example: Which sample represents a population that is 30%
freshmen, 30% sophomore, 20% junior, 20% senior?
Sample A Sample B30 freshmen, 30 sophomores, 60 freshmen, 60 sophomores,
20 juniors, 20 seniors 40 juniors, 40 seniors
© 2009 by The McGraw-Hill Companies, Inc.
Biased Samples
A biased sample • occurs when characteristics of the sample
differ systematically from those of the population
• samples can overrepresent or underrepresent a segment of a population samples made up of psychology students overrepresent
college students and underrepresent people not in college most research underrepresents individuals from diverse
cultures
© 2009 by The McGraw-Hill Companies, Inc.
Biased Samples
Two sources• Selection bias
occurs when a researcher’s procedures for selecting a sample result in one or more segments of the population being under- or overrepresented
• Response-rate bias occurs when individuals selected for the initial
sample do not complete and return the survey• lack of interest, worried about privacy, don’t have time
© 2009 by The McGraw-Hill Companies, Inc.
Approaches to Sampling
“Sampling” refers to procedures used to obtain a sample
Two basic approaches:• nonprobability sampling• probability sampling
© 2009 by The McGraw-Hill Companies, Inc.
Approaches to Sampling, continued
Nonprobability sampling• no guarantee each member of population has
an equal chance to be in sample• “convenience sampling”
researcher selects individuals who are available and willing to respond to the survey
example: magazine surveys, call-in radio surveys
© 2009 by The McGraw-Hill Companies, Inc.
Approaches to Sampling, continued
Probability sampling• all members of population have an equal
chance of being selected for the survey• “simple random sample”• Two common methods:
choose randomly from the sampling frame of the population
random-digit dialing
© 2009 by The McGraw-Hill Companies, Inc.
Approaches to Sampling, continued
Probability sampling• Stratified random sample: divide population
into subpopulations, called strata• Draw random samples from the strata
best to select samples proportional to the strata size
• Stratified random sampling increases the likelihood the sample will represent the population
© 2009 by The McGraw-Hill Companies, Inc.
Survey Methods
Four methods for obtaining survey data:• mail surveys• personal interviews• telephone interviews• Internet surveys
Each method has advantages and disadvantages
Choose method based on research question
© 2009 by The McGraw-Hill Companies, Inc.
Survey Methods, continued
Mail surveys• quick, convenient, self-administered, best for
highly personal or embarrassing topics• problem of response rate:
people selected for sample don’t complete and return the survey
final sample may be biased—not representative of the population
• little control over how people respond to the questions
© 2009 by The McGraw-Hill Companies, Inc.
Survey Methods, continued
Personal interviews• researchers gain more control over how
survey is administered• interviewers can seek clarification of answers,
ask questions potential problem: interviewer bias
• occurs when interviewers record only selected portions of answers or changes wording of questions and answers.
• interviews are costly; interviewers must be highly motivated, carefully trained, supervised
© 2009 by The McGraw-Hill Companies, Inc.
Survey Methods, continued
Telephone interviews• complete brief surveys efficiently and with greater
access to population• random-digit dialing to select random samples• supervise interviewers easily• Problems
selection bias: no phone or multiple phone numbers response-rate bias: willingness to answer questions on
phone interviewer bias: changes in survey questions and responses
© 2009 by The McGraw-Hill Companies, Inc.
Survey Methods, continued
Internet surveys• efficient, low-cost, potential for very large
samples• samples can be very diverse and access
typically underrepresented samples• problems:
selection bias: access to Internet response-rate bias: willingness to respond lack of control over research environment
© 2009 by The McGraw-Hill Companies, Inc.
Survey Methods, continued
Ways to increase response rate• questionnaire has a “personal touch”
use name, not “Dear student”
• responding requires minimal effort• topic of survey is interesting to respondents• respondents identify with organization or
sponsor of survey
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs
“Research design”• a plan for conducting a research project• choose research method best suited for
answering a particular question
Three types of survey designs• cross-sectional design• successive independent samples design• longitudinal design
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs, continued
Cross-sectional survey design• select sample from one or more populations
at one time choose population of interest use probability sampling or convenience sampling
• probability sampling leads to a more representative sample
respondents complete a survey
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs, continued
Cross-sectional survey design• Survey responses are used to
describe population (descriptive statistics) make predictions for the population (correlations)
at that one point in time• If samples are drawn from different
populations, compare the populations• cannot assess change over time with cross-
sectional designs
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs, continued
Successive independent samples design• a series of cross-sectional designs over time• a different sample from the population
completes the survey each time• each sample is selected from the same
population• responses from each sample are used to
describe the population at each point in time
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs, continued
Successive independent samples design• compare survey responses from each sample
to see how the population changes over time• cannot determine whether individuals change
over time• Problem of noncomparable samples
If different populations are sampled each time, responses may differ because of true changes over time or because different populations were sampled
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs, continued
Longitudinal survey design• same sample of individuals completes the
survey at different points in time• can assess how individuals change over time• responses from the sample are generalized to
describe changes over time in the population
© 2009 by The McGraw-Hill Companies, Inc.
Survey Research Designs, continued
Longitudinal survey design: Problems• Attrition: people drop out of the study
sample no longer represents population from which it was selected
• Reactivity: respondents may strive to be consistent over time or become sensitized to the topic
• longitudinal surveys can’t tell why people change over time (only correlations)
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research
Questionnaires • most frequently used to collect survey data• measure different types of variables
demographic variables using checklists preferences and attitudes
• self-report scales• respond using rating scales (assume interval level of
measurement)
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
All measures must be reliable and valid Reliability refers to consistency of
measurement• Test-retest reliability
administer measure two times to same sample individuals’ scores should be consistent over time a high correlation between the two sets of scores
indicates good test-retest reliability (r > .80) individuals’ scores need not be identical each time,
only same place in the distribution of scores
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
How to improve reliability?• more items• greater variability among individuals on the
factor being measured• testing situation free of distractions• clear instructions
A measure can be reliable but not valid
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
Validity refers to the truthfulness of a measure
A valid measure assesses what it is intended to measure• Construct validity refers to whether an
instrument measures the theoretical construct it was designed to measure
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
Example of construct validity:• Intelligence: Do these questions from a
common measure of intelligence assess a person’s intelligence in a valid manner? comprehension: “Why would people use a secret ballot?” vocabulary: “What does dilatory mean?” similarities: “How are a telephone and a radio alike?”
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
Establishing construct validity:• convergent validity
extent to which two measures of the same construct are correlated (go together)
• discriminant validity extent to which two measures of different
constructs are not correlated (do not go together)
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
Construct validity example• new measure of self-esteem • Which constructs should show convergent
validity with self-esteem measure and which would show discriminant validity?
Constructs:
depression, well-being, intelligence, extraversion, age, sensation-seeking, social anxiety, life satisfaction, grade point average,
reading comprehension, artistic ability
© 2009 by The McGraw-Hill Companies, Inc.
Measures in Survey Research, continued
Correlations demonstrating construct validity and reliability are shown in a correlation matrix:
(1) (2) (3) (4)(1) new self-esteem, Time 1 1.0 --- --- ---(2) new self-esteem, Time 2 .85 1.0 --- ---(3) measure of positive affect .90 .90 1.0 ---(4) measure of artistic ability .10 .10 .15 1.0
test-retest reliabilityconvergent validitydiscriminant validity
© 2009 by The McGraw-Hill Companies, Inc.
Constructing a Questionnaire
Best choice for selecting a measure• use measure already shown to be reliable and
valid in previous research
If no suitable measure is found• create a questionnaire or measure
Creating a reliable and valid questionnaire is not easy
© 2009 by The McGraw-Hill Companies, Inc.
Constructing a Questionnaire, continued
Important first steps• Decide what information should be sought• Decide what type of questionnaire should be used
(e.g., self-administered?)• Write a first draft of the questionnaire• Have experts review questionnaire and then revise it
based on their suggestions• Pretest the questionnaire using sample and
conditions similar to the planned administration of the survey
• Review results and edit the questionnaire
© 2009 by The McGraw-Hill Companies, Inc.
Constructing a Questionnaire, continued
Next steps: Establish reliability and validity• Reliability
Test and retest questionnaire using sample and conditions similar to planned survey.
• Validity Convergent: Administer questionnaire with
measures of theoretically related constructs Discriminant: Administer questionnaire with
measures of theoretically unrelated constructs
© 2009 by The McGraw-Hill Companies, Inc.
Constructing a Questionnaire, continued
Guidelines for Writing Survey Questions• Choose how participants will respond
free-response (open-ended)• greater flexibility in responses• difficult to code
closed-response (multiple-choice, true-false)• responses are quick and easy• easy to score• may not accurately describe individuals’ responses
• Use simple, familiar vocabulary; keep questions short
© 2009 by The McGraw-Hill Companies, Inc.
Constructing a Questionnaire, continued
Guidelines for Writing Survey Questions• Write clear and specific questions
avoid double-barreled questions place conditional phrases at the beginning of
sentences avoid leading questions avoid loaded (emotion-laded) questions
© 2009 by The McGraw-Hill Companies, Inc.
Constructing a Questionnaire, continued
Ordering of questions• self-administered questionnaires
most interesting questions first
• personal and telephone interviews demographic questions first
• use funnel questions start with general questions and move to more
specific questions on a given topic
• use filter questions direct respondents to appropriate questions
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research
Reported vs. Actual Behavior• Survey responses may not be truthful
Reactivity• not truthful because the information is being recorded
Social desirability• not truthful because responding as they “should”
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Measuring social desirability• sample items from Marlowe-Crowne (1964)
Social Desirability Scale
What is the socially desirable response?
1. No matter who I’m talking to, I’m always a good listener T F
2. I like to gossip at times T F
3. I’m always willing to admit it when I make a mistake T F
4. I have almost never felt the urge to tell someone off T F
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Guidelines for social desirability• Accept people’s responses as truthful unless
there’s reason to suspect otherwise• Because actual behavior doesn’t always
match questionnaire responses, use a multimethod approach to answering research questions
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Correlation and causality• “correlation does not imply causality”• example:
correlation between being socially active (A) and life satisfaction (B)
three possible causal relationships• A causes B • B causes A• variable C causes both A and B (e.g., number of friends)
= spurious relationship
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis• statistical procedure to tease apart complex
correlational relationships among variables• Mediators
variables used to explain a correlation between two variables
• Moderators variables that affect direction or strength of
correlation between two variables
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Example of path analysis• Evans et al. (2005) observed a positive
correlation between measures of poverty and measures of psychological distress among children
Poverty → Psychological distress
• This is called a direct relationship or path a
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis example• Evans et al. sought to explain why this
relationship exists. level of chaos in the home as possible mediator chaotic living environment
• measured using concepts such as unpredictability, confusion, lack of structure, noise, overcrowding, and poor-quality housing
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis example• Evans et al. observed two important
correlations Measures of poverty were positively correlated
with greater chaos in the home (Path b) Greater chaos in the home was positively
correlated with psychological distress (Path c)
• These two correlations represent the indirect relationship between poverty and psychological distress
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis example• Diagram of direct and indirect relationships
Chaos
path b path c
Poverty Psychological
path a distress
“Chaos” mediates the relationship between poverty and psychological distress
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis example• What if the relationships between poverty,
chaos, and psychological distress are not observed for all children?
• A moderator variable may affect these relationships moderators affect the direction and strength of
relationships
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis example• Possible moderators:
sex of the child• the relationships between poverty, chaos, and
psychological distress may exist for boys, but not girls population density
• the relationships between poverty, chaos, and psychological distress may exist in urban areas, but not rural areas
personality features of children• the relationships may exist for low-resilient children, but
not high-resilient children
© 2009 by The McGraw-Hill Companies, Inc.
Thinking Critically About Survey Research, continued
Path analysis• helps us to understand relationships among
variables• but these relationships are still correlational• cannot make definitive causal statements• other untested variables are related to
children’s psychological distress