27
Chapter 4 Chapter 4 Slide Slide 1 Psy B07 SAMPLING DISTRIBUTIONS SAMPLING DISTRIBUTIONS AND HYPOTHESIS TESTING AND HYPOTHESIS TESTING
Chapter 4Chapter 4 Slide Slide 11
Psy B07
SAMPLING DISTRIBUTIONS SAMPLING DISTRIBUTIONS AND HYPOTHESIS TESTINGAND HYPOTHESIS TESTING
Chapter 4Chapter 4 Slide Slide 22
Psy B07
Sampling Distributions revisitedSampling Distributions revisited Hypothesis TestingHypothesis Testing Using the Normal Distribution to Using the Normal Distribution to
test Hypothesestest Hypotheses Type I and Type II ErrorsType I and Type II Errors One vs. Two Tailed TestsOne vs. Two Tailed Tests
OutlineOutline
Chapter 4Chapter 4 Slide Slide 33
Psy B07
Statistics is ArguingStatistics is Arguing
Typically, we are arguing either 1) that Typically, we are arguing either 1) that some value (or mean) is different from some value (or mean) is different from some other mean, or 2) that there is a some other mean, or 2) that there is a relation between the values of one relation between the values of one variable, and the values of another.variable, and the values of another.
Thus, we typically first produce some null Thus, we typically first produce some null hypothesis (i.e., no difference or relation) hypothesis (i.e., no difference or relation) and then attempt to show how and then attempt to show how improbably something is given the null improbably something is given the null hypothesis.hypothesis.
Chapter 4Chapter 4 Slide Slide 44
Psy B07
Sampling DistributionsSampling Distributions
Just as we can plot distributions of Just as we can plot distributions of observations, we can also plot observations, we can also plot distributions of statistics (e.g., means).distributions of statistics (e.g., means).
These distributions of sample statistics These distributions of sample statistics are called are called sampling distributionssampling distributions..
For example, if we consider the 24 For example, if we consider the 24 students in a tutorial who estimated my students in a tutorial who estimated my weight as a population, their guesses weight as a population, their guesses have an x of 168.75 and an have an x of 168.75 and an of 12.43 ( of 12.43 (22 = 154.51)= 154.51)
Chapter 4Chapter 4 Slide Slide 55
Psy B07
Sampling DistributionsSampling Distributions
If we repeatedly sampled groups of 6 If we repeatedly sampled groups of 6 people, found the x of their estimates, people, found the x of their estimates, and then plotted the x’s, the distribution and then plotted the x’s, the distribution might look like:might look like:
0
5
10
15
20
25
30
35
155 157.5 160 162.5 165 167.5 170 172.5 175 177.5 180 182.5 185
Chapter 4Chapter 4 Slide Slide 66
Psy B07
Hypothesis TestingHypothesis Testing
What I have previously called “arguing” What I have previously called “arguing” is more appropriately called is more appropriately called hypothesis hypothesis testingtesting..
Hypothesis testing normally consists of Hypothesis testing normally consists of the following steps:the following steps:
1) some 1) some research hypothesisresearch hypothesis is proposed is proposed (or (or alternate hypothesisalternate hypothesis) - ) - HH1.1.
2) the 2) the null hypothesisnull hypothesis is also proposed - is also proposed - HH0.0.
Chapter 4Chapter 4 Slide Slide 77
Psy B07
Hypothesis TestingHypothesis Testing
3) the relevant sampling distribution is 3) the relevant sampling distribution is obtained under the assumption that obtained under the assumption that HH00 is correct.is correct.
4) I obtain a sample representative of 4) I obtain a sample representative of HH11 and calculate the relevant statistic (or and calculate the relevant statistic (or observation).observation).
5) Given the sampling distribution, I 5) Given the sampling distribution, I calculate the probability of observing calculate the probability of observing the statistic (or observation) noted in the statistic (or observation) noted in step 4, step 4, by chanceby chance..
6) On the basis of this probability, I make 6) On the basis of this probability, I make a decisiona decision
Chapter 4Chapter 4 Slide Slide 88
Psy B07
The Beginnings of an The Beginnings of an ExampleExample
One of the students in the tutorial One of the students in the tutorial guessed my weight to be 200 lbs. I guessed my weight to be 200 lbs. I think that said student was fooling think that said student was fooling around. That is, I think that guess around. That is, I think that guess represents something different that represents something different that do the rest of the guesses.do the rest of the guesses.
HH00 - the guess is not really - the guess is not really different.different.
HH11 - the guess is different. - the guess is different.
Chapter 4Chapter 4 Slide Slide 99
Psy B07
The Beginnings of an The Beginnings of an ExampleExample
1) obtain a sampling distribution of 1) obtain a sampling distribution of HH 0. 0.
2) calculate the probability of 2) calculate the probability of guessing 200, given this distributionguessing 200, given this distribution
3) Use that probability to decide 3) Use that probability to decide whether this difference is just whether this difference is just chance, or something more.chance, or something more.
Chapter 4Chapter 4 Slide Slide 1010
Psy B07
A Touch of PhilosophyA Touch of Philosophy
Some students new to this idea of Some students new to this idea of hypothesis testing find this whole hypothesis testing find this whole business of creating a null hypothesis business of creating a null hypothesis and then shooting it down as a tad on the and then shooting it down as a tad on the weird side, why do it that way?weird side, why do it that way?
This dates back to a philosopher named This dates back to a philosopher named Karl Popper who claimed that it is very Karl Popper who claimed that it is very difficult to prove something to be true, difficult to prove something to be true, but no so difficult to prove it to be but no so difficult to prove it to be untrue.untrue.
Chapter 4Chapter 4 Slide Slide 1111
Psy B07
A Touch of PhilosophyA Touch of Philosophy
So, it is easier to prove HSo, it is easier to prove H00 to be to be wrong, than to prove Hwrong, than to prove HAA to be right. to be right.
In fact, we never really prove HIn fact, we never really prove H11 to to be right. That is just something we be right. That is just something we imply (similarly Himply (similarly H00).).
Chapter 4Chapter 4 Slide Slide 1212
Psy B07
Using the Normal Using the Normal Distribution to test Distribution to test
HypothesesHypotheses
The “Marty’s Weight” example The “Marty’s Weight” example begun earlier is an example of a begun earlier is an example of a situation where we want to situation where we want to compare one observation to a compare one observation to a distribution of observations.distribution of observations.
This represents the simplest This represents the simplest hypothesis-testing situation hypothesis-testing situation because the sampling distribution because the sampling distribution is simply the distribution of the is simply the distribution of the individual observations.individual observations.
Chapter 4Chapter 4 Slide Slide 1313
Psy B07
Using the Normal Using the Normal Distribution to test Distribution to test
HypothesesHypotheses
Thus, in this case we can use the stuff we Thus, in this case we can use the stuff we learned about z-scores to test hypotheses learned about z-scores to test hypotheses that some individual observation is either that some individual observation is either abnormally high (or abnormally low).abnormally high (or abnormally low).
That is, we use our mean and standard That is, we use our mean and standard deviation to calculate the a z-score for the deviation to calculate the a z-score for the critical value, then go to the tables to find critical value, then go to the tables to find the probability of observing a value as the probability of observing a value as high or higher than (or as low or lower high or higher than (or as low or lower than) the one we wish to test.than) the one we wish to test.
Chapter 4Chapter 4 Slide Slide 1414
Psy B07
Finishing the ExampleFinishing the Example
= 168.75= 168.75 Critical = 200Critical = 200
= 12.43 = 12.43
51.243.12
75.168200
xz
Chapter 4Chapter 4 Slide Slide 1515
Psy B07
Finishing the ExampleFinishing the Example
From the z-table, the area of the From the z-table, the area of the portion of the curve above a z of portion of the curve above a z of 2.51 (i.e., the smaller portion) is 2.51 (i.e., the smaller portion) is approximately .0060.approximately .0060.
Thus, the probability of observing a Thus, the probability of observing a score as high or higher than 200 score as high or higher than 200 is .0060is .0060
Chapter 4Chapter 4 Slide Slide 1616
Psy B07
Making Decisions given Making Decisions given ProbabilitiesProbabilities
It is important to realize that all our test It is important to realize that all our test really tells us is the probability of some really tells us is the probability of some event given some null hypothesis.event given some null hypothesis.
It does not tell us whether that probability It does not tell us whether that probability is sufficiently small to reject His sufficiently small to reject H00, that , that decision is left to the experimenter.decision is left to the experimenter.
In our example, the probability is so low, In our example, the probability is so low, that the decision is relatively easy. There that the decision is relatively easy. There is only a .60% chance that the observation is only a .60% chance that the observation of 200 fits with the other observations in of 200 fits with the other observations in the sample. Thus, we can reject Hthe sample. Thus, we can reject H00 without much worry.without much worry.
Chapter 4Chapter 4 Slide Slide 1717
Psy B07
Making Decisions given Making Decisions given ProbabilitiesProbabilities
But what if the probability was 10% But what if the probability was 10% or 5%? What probability is small or 5%? What probability is small enough to reject Henough to reject H00??
It turns out there are two answers It turns out there are two answers to that:to that:
the real answer.the real answer. the “conventional” answer.the “conventional” answer.
Chapter 4Chapter 4 Slide Slide 1818
Psy B07
The “Real” AnswerThe “Real” Answer
First some terminology. . . .First some terminology. . . .
The probability level we pick as The probability level we pick as our cut-off for rejecting Hour cut-off for rejecting H00 is is referred to as our referred to as our rejection levelrejection level or or our our significance levelsignificance level..
Any level below our rejection or Any level below our rejection or significance level is called our significance level is called our rejection regionrejection region
Chapter 4Chapter 4 Slide Slide 1919
Psy B07
The “Real” AnswerThe “Real” Answer
OK, so the problem is choosing an OK, so the problem is choosing an appropriate rejection level.appropriate rejection level.
In doing so, we should consider the four In doing so, we should consider the four possible situations that could occur when possible situations that could occur when we’re hypothesis testing.we’re hypothesis testing.
Real state of the World Decision H0 true H0 false Reject H0 Type I error Correct Fail to Correct Type II error reject H0
Chapter 4Chapter 4 Slide Slide 2020
Psy B07
Type I and Type II ErrorsType I and Type II Errors
Type I error is the probability of rejecting Type I error is the probability of rejecting the null hypothesis when it is really true.the null hypothesis when it is really true.
Example: saying that the person who Example: saying that the person who guessed I weigh 200 lbs was just guessed I weigh 200 lbs was just screwing around when, in fact, it was screwing around when, in fact, it was an honest guess just like the others.an honest guess just like the others.
We can specify exactly what the We can specify exactly what the probability of making that error was, in probability of making that error was, in our example it was .60%.our example it was .60%.
Chapter 4Chapter 4 Slide Slide 2121
Psy B07
Type I and Type II ErrorsType I and Type II Errors
Usually we specify some “acceptable” Usually we specify some “acceptable” level of error before running the study.level of error before running the study.
then call something significant if it is then call something significant if it is below this level.below this level.
This acceptable level of error is typically This acceptable level of error is typically denoted as denoted as
Before setting some level of it is Before setting some level of it is important to realize that levels of important to realize that levels of are are also linked to Type II errorsalso linked to Type II errors
Chapter 4Chapter 4 Slide Slide 2222
Psy B07
Type I and Type II ErrorsType I and Type II Errors
Type II error is the probability of Type II error is the probability of failing to reject a null hypothesis failing to reject a null hypothesis that is really false.that is really false.
Example: judging OJ as not guilty Example: judging OJ as not guilty when he is actually guilty.when he is actually guilty.
The probability of making a Type II The probability of making a Type II error is denoted as error is denoted as
Chapter 4Chapter 4 Slide Slide 2323
Psy B07
Type I and Type II ErrorsType I and Type II Errors
Unfortunately, it is impossible to precisely Unfortunately, it is impossible to precisely calculate calculate because we do not know the because we do not know the shape of the sampling distribution under Hshape of the sampling distribution under H1.1.
It is possible to “approximately” measure It is possible to “approximately” measure , , and we will talk a bit about that in Chapter 8.and we will talk a bit about that in Chapter 8.
For now, it is critical to know that there is a For now, it is critical to know that there is a trade-off between trade-off between and and , as one goes down, , as one goes down, the other goes up.the other goes up.
Thus, it is important to consider the situation Thus, it is important to consider the situation prior to setting a significance level.prior to setting a significance level.
Chapter 4Chapter 4 Slide Slide 2424
Psy B07
The Conventional AnswerThe Conventional Answer
While issues of Type I versus Type II error While issues of Type I versus Type II error are critical in certain situations, are critical in certain situations, psychology experiments are not typically psychology experiments are not typically among them (although they sometimes among them (although they sometimes are).are).
As a result, psychology has adopted the As a result, psychology has adopted the standard of accepting =.05 as a standard of accepting =.05 as a conventional level of significance.conventional level of significance.
It is important to note, however, that It is important to note, however, that there is nothing magical about this value there is nothing magical about this value (although you wouldn’t know it by (although you wouldn’t know it by looking at published articles).looking at published articles).
Chapter 4Chapter 4 Slide Slide 2525
Psy B07
One vs. Two Tailed TestsOne vs. Two Tailed Tests
Often, we want to determine if some critical Often, we want to determine if some critical difference (or relation) exists and we are not difference (or relation) exists and we are not so concerned about the direction of the so concerned about the direction of the effect.effect.
That situation is termed That situation is termed two-tailedtwo-tailed, meaning , meaning we are interested in extreme scores at either we are interested in extreme scores at either tail of the distribution.tail of the distribution.
Note, that when performing a two-tailed test Note, that when performing a two-tailed test we must only consider something significant we must only consider something significant if it falls in the bottom 2.5% or the top 2.5% if it falls in the bottom 2.5% or the top 2.5% of the distribution (to keep of the distribution (to keep at 5%). at 5%).
Chapter 4Chapter 4 Slide Slide 2626
Psy B07
One vs. Two Tailed TestsOne vs. Two Tailed Tests
If we were interested in only a high or If we were interested in only a high or low extreme, then we are doing a low extreme, then we are doing a one-one-tailedtailed or or directionaldirectional test and look only to test and look only to see if the difference is in the specific see if the difference is in the specific critical region encompassing all 5% in the critical region encompassing all 5% in the appropriate tail.appropriate tail.
Two-tailed tests are more common Two-tailed tests are more common usually because either outcome would be usually because either outcome would be interesting, even if only one was interesting, even if only one was expected.expected.
Chapter 4Chapter 4 Slide Slide 2727
Psy B07
Other Sampling Other Sampling DistributionsDistributions
The basics of hypothesis testing The basics of hypothesis testing described in this chapter do not described in this chapter do not change.change.
All that changes across chapters is All that changes across chapters is the specific sampling distribution the specific sampling distribution (and its associated table of values).(and its associated table of values).
The critical issue will be to realize The critical issue will be to realize which sampling distribution is the which sampling distribution is the one to use in which situation.one to use in which situation.
