Sampling & External Validity 1 KNR 497 Research Methods Sampling Slide 1 Chapter 2 part 2
Transcript
Slide 1
Slide 2
Sampling & External Validity 1 KNR 497 Research Methods
Sampling Slide 1 Chapter 2 part 2
Slide 3
2 1 KNR 497 Research Methods: Sampling Slide 2 The 65, 95, 99
Percent Rule
Slide 4
KNR 497 Research Methods: Sampling Slide 3 Estimating the
Population Using a Sampling Distribution 2 1
Slide 5
1 The rest of the slides Types of sampling Probability based
Non-probability based KNR 497 Research Methods: Sampling Slide 4
2
Slide 6
3 1 KNR 497 Research Methods: Sampling Slide 5 Probability
Sampling Utilizes some form of random selection All units in the
population have equal probability of being chosen Nomenclature: N =
number of cases in the sampling frame n = number of cases in the
sample NCn = number of combinations of n from N f = n/N and is the
sampling fraction 2 4
Slide 7
KNR 497 Research Methods: Sampling Slide 6 Probability Sampling
Simple random sampling Stratified random sampling Systematic random
sampling Cluster (area) random sampling Multistage sampling 1
Slide 8
KNR 497 Research Methods: Sampling Slide 7 Probability
Sampling: Simple Random Sampling Objective: To select n units out
of N such that each NCn has an equal chance of being selected
Procedure: Use a table of random numbers or computer random- number
generator Example: N = 1000 n (desired) = 100 f = n/N = 100/1000
=.10 or 10% Randomly select 100 units (10%) Generalizable; may not
be representative of subgroups 3 1 2
Slide 9
KNR 497 Research Methods: Sampling Slide 8 Probability
Sampling: Stratified Random Sampling Objective: To select n units
out of N such that key subgroups of n are representative of
subgroups of N Procedure: Divide the population into nonoverlapping
groups (strata) N 1, N 2, N 3 N i, such that N 1 + N 2 + N 3 + N i
= N. Then do simple random sample of f = n/N in each strata
Disproportionate stratified random sampling can be used to
oversample small groups. 3 1 2 4
Slide 10
KNR 497 Research Methods: Sampling Slide 9 Probability
Sampling: Systematic Random Sampling Objective: To systematically
select n units out of N such that n is a random sample of N
Procedure: Number units in the population from 1 to N (NOTE: Units
must be randomly ordered) Decide on the n that you need Calculate k
= N/n = the interval size Randomly select an integer between 1 and
k Take every kth unit (diagram on next slide illustrates this) 1
2
Slide 11
KNR 497 Research Methods: Sampling Slide 10 Probability
Sampling: Systematic Random Sampling 1
Slide 12
KNR 497 Research Methods: Sampling Slide 11 Probability
Sampling: Cluster (Area) Random Sampling Objective: To obtain a
representative sample from N when N is spread out over a large
geographic area Procedure: Divide the population into clusters
Randomly sample clusters Measure all units within sampled clusters
Clusters are usually divided along geographical boundaries. 3 1
2
Slide 13
KNR 497 Research Methods: Sampling Slide 12 Probability
Sampling: Multistage Sampling Objective: To obtain a representative
sample from N by combining several sampling techniques to create a
more efficient or effective sample than the use of any one sampling
type can achieve on its own Example: 1. National sample of school
districts stratified by economics 2. Simple random selection of
schools within districts 3. Simple random selection of classes
within schools 3 1 2
Slide 14
KNR 497 Research Methods: Sampling Slide 13 Nonprobability
Sampling Does not involve random selection May be representative
but cannot depend upon the rationale of probability theory Used
when it is not feasible, practical, or theoretically sensible to
use random sampling Accidental versus purposive 3 1 2 4
Slide 15
Nonprobability Sampling: Accidental, Haphazard, or Convenience
Sampling One of the most common methods of sampling Man on the
street Volunteers or subjects who are conveniently available No
evidence that sample is representative KNR 497 Research Methods:
Sampling Slide 14 1 2
Slide 16
KNR 497 Research Methods: Sampling Slide 15 Nonprobability
Sampling: Purposive Sampling Sampling with a purpose in mind Useful
in reaching a targeted sample quickly Target population is reached
but with over- representation of subgroups that are more readily
accessible Types: Modal instance Expert Quota Heterogeneity
Snowball 3 1 2
Slide 17
KNR 497 Research Methods: Sampling Slide 16 Nonprobability
Sampling: Purposive Sampling Modal Instance Sampling the most
frequent case or typical case Difficult to define what a typical
case is Probably only useful for informal sampling contexts (or
perhaps even more dangerous for those) 1 2 3
Slide 18
KNR 497 Research Methods: Sampling Slide 17 Nonprobability
Sampling: Purposive Sampling Expert Sampling Assembling of a sample
of persons with known or demonstrable expertise in some area Panel
of experts May be useful for providing evidence as to the validity
of another sampling approach you have chosen 1 2
Slide 19
KNR 497 Research Methods: Sampling Slide 18 Nonprobability
Sampling: Purposive Sampling Quota Sampling Sample selected
nonrandomly according to some fixed quota Proportional quota
sampling used to represent the major characteristics of the
population of interest by sampling a proportional amount of each
Nonproportional quota sampling used to supply a minimum number of
units in each category but not concerned with proportions 1 2
Slide 20
KNR 497 Research Methods: Sampling Slide 19 Nonprobability
Sampling: Purposive Sampling Heterogeneity Sampling Used to provide
a sample that will include all the view or opinions without regard
to proportional representation Sampling for diversity Can be
thought of as the opposite of modal instance sampling 1 2
Slide 21
KNR 497 Research Methods: Sampling Slide 20 Nonprobability
Sampling: Purposive Sampling Snowball Sampling People meeting the
criteria for inclusion in the sample are identified and then they
recommend others they know who meet the criteria Useful when trying
to reach inaccessible or hard to find populations Examples may
include the homeless, drug users, etc. 1 2
Slide 22
Threats to External Validity Interaction of selection and
treatment Interaction of setting and treatment Interaction of
history and treatment Maybe it is just these people. Maybe it is
just these places. Maybe it is just these times.
Slide 23
Guiding Questions for Critiquing the External Validity of
Research 1. What are the main results of the study (e.g., positive
or negative relationship, group differences, effectiveness of the
intervention or treatment)? 2. Do the researchers explicitly state
or imply that similar results would hold for other: (a) people, (b)
places or situations, and/or (c) times? If so, what is the
population/place/time they are attempting to generalize to? 3. If
the researchers are generalizing their results, how reasonable are
these conclusions given the sample, sampling procedures, and
settings used? [This is the key External Validity question] 4. What
specifically might lead you to question these conclusions? In other
words, if they did suggest the results were generalizable, why
might you think otherwise? [The more convincing of a rationale you
can generate, the more you should question the external
validity]
Slide 24
Use the guiding questions to evaluate the external validity of
the following study Prior research has found that (a)
intercollegiate athletes are especially at-risk for excessive
alcohol consumption (e.g., Nelson & Wechsler, 2001), and (b)
sport-type differences exist among college athletes in terms of
yearly drinking prevalence rates (National Collegiate Athletic
Association, 2001). No studies, however, have examined sport-type
differences on more specific measures of alcohol consumption (i.e.,
drinks per week). In the present study, data were analyzed on 298
intercollegiate athletes from two different NCAA Division III
universities. Results indicated significant sport type differences
on alcohol consumption variables, with athletes from the sports of
swimming and diving and wrestling reporting the highest levels of
alcohol consumption (M = 5.20, SD = 4.00) and soccer and football
reporting the lowest (M = 4.02, SD = 3.25). Results suggest college
athletes participating in individual sports are at-risk for future
alcohol abuse.