Post on 09-Feb-2022
transcript
Hypothesis Tests
• Objective decision-making tools
• Are used to answer very specific questions.
For example, “Are the two gum groups equivalent in terms of mean
change in DMFS?”
• Decision rules for possible answers to your question are based on how consistent the possible answers are with the observed data.
The Null Hypothesis, H0
• A hypothesis test focuses on a single question
• That question is formulated in terms of a “TRUE or FALSE” statement called the Null Hypothesis, also denoted by H0.
• The evidence from a hypothesis test is in the form of how sure we are that H0 is not true.
• Thus, the null hypothesis is formulated in such a way that it is the statement that we are expecting to disprove.
Example: Chewing Gum Study
• We are hoping to find evidence that the gums are different in terms of DMFS progression.
• Thus, we would phrase the null hypothesis:
• We hope to prove the gums are different by disproving H0
• Disproving H0 = “rejecting” the null hypothesis
H0: The mean change in DMFS in both groups is the same
Creating the Decision Rule
We will base our decision about H0 on a statistic that:
1. is a function of the observed data.
2. gives information about H0, the question of interest.
3. will be predictable (we know it’s probability distribution) when H0 is true
Example: change in facial height
• Study to determine whether facial height of
people aged 21-26 years changes over a ten
year period.
• Facial heights were measured from a sample
of graduate students.
• After ten years, the measurements were taken
again on 84 of the same students.
Example: change in facial height
The average change in facial height was �� = −1.7 mm
with s= 6.8 mm.
We define our null hypothesis to be:
H0: µ = 0,
where μ = mean change in facial height.
Evidence against H0 will mean evidence that facial
heights change.
The t statisticTo test H0: µ = 0 we use the t statistic
• If H0 is true, then we would expect the sample mean,
�, to be close to µ = 0.
• If H0 is not true, then we would expect the � to be
far from µ = 0.
• Thus, values of the t statistic far from zero will
indicate evidence against H0.
� = � − 0
��
The t statisticTo test H0: µ = 0 we use the t statistic
� =� − 0
��
Furthermore, If the null hypothesis is true, then we know that T has t distribution with n-1 degrees of freedom.
0
Making the Decision
• Compute the statistic from the observed data
• Calculate the probability of seeing the observed statistic if the null hypothesis were true
• If the observed statistic would have low probability of occurring if the null hypothesis were true this will be considered evidence against the null hypothesis
Example: change in facial height
The average change in facial height for the 84 former
grad students was X0 = -1.7 mm with s = 6.8 mm.
The t statistic is:
� =−1.7 − 0
6.8 84⁄= −2.29
Example: change in facial height
The probability of observing T = -2.29 or something
even more unlikely when H0 is true would be
P(t83 < -2.29) + P(t83 > 2.29) = 2 × 0.012 = 0.024*
t83distribution
-2.29 2.29
prob = 0.012prob = 0.012
* Computed using Excel function “T.DIST.2T”
Example: change in facial height
Because the probability p = 0.024 is fairly low (less than
1 in 40), we consider this good evidence against H0.
This probability, p = 0.024, is an example of a p-value
t83distribution
-2.29 2.29
prob = 0.012prob = 0.012
P-value• The final decision on whether or not we should believe the null
hypothesis is based upon the p-value.
• The p-value is defined as the probability that if one performed an experiment in which it was known that H0 was true, that we would still see data producing a statistic as “extreme” (or more extreme) as the one observed.
• By an “extreme” statistic we mean one that would be unlikely if H0were true. Or a statistic that appears to show evidence that H0 is not true.
• The smaller the p-value, the less likely our observed statistic would be if H0 were true.
• Since we know that the observed statistic is real, this means that small p-values are actually evidence against H0 .
Decision Rule•How small of a p-value do we require to decide that the null hypothesis is just too unlikely?
•We decide upon a threshold value a priori, before looking at the data.
•This threshold value is called the significance level and is usually denoted by α.
•If the p-value is less than α and we reject H0 this is often called “statistically significant”
•Standard choices for α are .05 or .01
Decision Rule
Reject H0 if the p-value < α
Example: change in facial height
Test the null hypothesis H0: µ = 0 at significance level α = 0.05.
• The p-value was p = 0.024. Since 0.024 is less than .05, we
REJECT H0.
“There is good evidence at the α = 0.05 significance level that
H0 is not true.”
• If the significance level was α = 0.01 instead, then we would
NOT REJECT H0.
“There is not good evidence at the α = 0.01 significance level
that H0 is not true.”
Computing the p-value
• If you have access to Excel, you can calculate the
probabilities from any t distribution
– http://courses.washington.edu/dphs568/course/excel-t.htm
• If not, you can use Table 4 in the course notes to tell
whether the p-value is below certain values
• Specifically, it will be good to know whether the p-
value is above or below α.
Using Table 4 to find out whether the p-
value is above or below α
• Look up tn-1, 1-α/2 , the 1-α/2th percentile of the tn-1
distribution, in Table 4.
• Compare T to tn-1, 1-α/2
• If |T| > tn-1, 1-α/2 , then p-value < α. REJECT H0
• If |T| < tn-1, 1-α/2 , then p-value > α. DO NOT REJECT H0
How does that work?
For the facial height data test
H0: µ = 0 at significance level α = 0.05.
t83,0.975 = 1.99, so
P(t83 < -1.99) + P(t83 > 1.99) = .05
Since |T|= 2.29 > 1.99, this means
P(t83 < -2.29) + P(t83 > 2.29)
< P(t83 < -1.99) + P(t83 > 1.99)
= .05
t83distribution
-1.99 1.99
prob = 0.05
t83distribution
-2.29 2.29
p-value
Example: chewing gum data
Do the data indicate at the α=0.01 significance level
that the mean DMFS has changed in Group A?
• Let µ = mean change in DMFS
• Test hypothesis H0: µ = 0
• Note: “µ = 0” would indicate “no change in DMFS”
• For Group A: n = 25, X0 = -0.72, and s = 5.37.
• The t statistic is:
� =−0.72 − 0
5.37 25⁄= −0.67
Example: chewing gum data
� =−0.72 − 0
5.37 25⁄= −0.67
• Since α=0.01, we look up the (1- α/2) = 99.5th
percentile of a t24 distribution.
• From Table 4, t24,0.995 = 2.80
• Since |T| = 0.67 < 2.80, DO NOT REJECT H0
• p-value = P(|t24| > .67) = 0.51t24distribution
-0.67 0.67
p-value
Properties of a hypothesis test
• The decision of a hypothesis test is based on the random sample.
• The decision of the test can be incorrect.
• There are two possible scenarios, and two possible test outcomes in each scenario:
H0 is true H0 is not true
test rejects H0
type Ierror
test rejects H0
OK
test does not reject
H0
OKtest does not reject
H0
type IIerror
•Two scenarios:•H0 is true, or H0 is not true
•Two test outcomes: •Reject H0, or Do not reject H0
Properties of a hypothesis testScenario #1 Scenario #2
Outcomes of hypothesis
test
H0 is true
test rejects H0
type Ierror
test does not reject
H0
OK
• If the test rejects H0, that is considered a type I error.
•The probability of making a type I error is equal to the significance level, α
•The significance level is chosen by the investigator
Scenario #1: H0 is true
H0 is not true
test rejects H0
OK
test does not reject
H0
type IIerror
Scenario #2: H0 is not true
• If the test does not reject H0, that is considered a type II error.
•The probability of being correct in scenario #2, of not making a type II error, is called the Power of the test
H0 is true H0 is not true
test rejects H0
type Ierror
test rejects H0
OK
test does not reject
H0
OKtest does not reject
H0
type IIerror
Test Rejects H0Scenario #1 Scenario #2
Test rejects
H0 is true H0 is not true
test rejects H0
type Ierror
test rejects H0
OK
• If we reject H0 with, say, α=.05, the implication is clear.
• When rejecting H0, our possible error would be a type I error.
• Because we set up our hypothesis test with a specified type I error probability of α=.05, we know that there is less than 5% chance that our decision is incorrect
Test Rejects H0Scenario #1 Scenario #2
Test rejects
H0 is true H0 is not true
test rejects H0
type Ierror
test rejects H0
OK
test does not reject
H0
OKtest does not reject
H0
type IIerror
Test does not reject H0Scenario #1 Scenario #2
Test doesnot reject
H0 is true H0 is not true
test does not reject
H0
OKtest does not reject
H0
type IIerror
Test does not reject H0
Test doesnot reject
• If we do not reject H0 then our possible error would be a type II error.
• The probability of type II error is not as easily established as the type I error.
• When failing to reject H0 , it is less clear how likely it is that our decision is correct or incorrect.
Recap
• A hypothesis test allows you to make a decision
• The null hypothesis is always formulated in such a way that it is the statement that we want to disprove.
• Reject H0 if the p-value < α (“statistically significant”)
• A hypothesis test only controls the type I error– If we reject H0 our evidence is clear– If we fail to reject H0 then it is not as clear