COMPARISONS INVOLVING TWO SAMPLE MEANS
Testing Hypotheses
Two types of hypotheses:
1. Ho: Null Hypothesis - hypothesis of no difference.
21 or 021
2. HA: Alternate Hypothesis – hypothesis of difference.
21 or 021
Two-tail vs. One-tail Tests
Two-tail tests have these types of hypotheses:
Ho: 21
HA: 21
Note the presence of the equal signs.
If you reject Ho: 21 , you don’t care which mean is greater.
1 can be either greater than or less than
2 .
One-tail tests my have one of the following types of hypotheses:
Ho: 21 Ho: 21
HA: 21 HA:
21
Note the presence of greater than or less than signs.
If you reject Ho:, you are being specific that 1 can only be on one side
of 2 .
t-Distribution
2/ 2/
Two-tail test One-tail test
Size of the rejection region = 2/ Size of the rejection region =
Types of Errors 1. Type I Error: To reject the null hypothesis when it is actually true.
The probability of committing a Type I Error is .
The probability of committing a Type I Error can be reduced by the investigator
choosing a smaller .
typical sizes of are 0.05 and 0.01.
The probability of committing a Type I Error also can be expressed as a
percentage (i.e. *100 %)
If =0.05, The probability of committing a Type I Error is 5%.
If =0.05, we are testing the hypothesis at the 95% level of confidence.
2. Type II Error: The failure to reject the null hypothesis when it is false.
The probability of committing a Type II Error is .
can be decreased by:
a. Increasing n (i.e. the number of observations per treatment).
b. Decreasing s2.
Increase n
Choose a more appropriate experimental design
Improve experimental technique.
Power of the Test
Power of the test is equal to: 1- .
Defined as the probability of accepting the alternate hypothesis when it is true.
We want the Power of the Test to be as large as possible.
Summary of Type I and Type II Errors
True Situation
Decision Null hypothesis is true Null hypothesis if false
Reject the null hypothesis Type I Error No error
Fail to reject the null hypothesis No error Type II Error
Hypothesis Testing
Summary of Testing a Hypothesis.
1. Formulate a meaningful null and alternate hypothesis.
2. Choose a level of .
3. Compute the value for the test statistic (i.e. t-statistic of F-statistic).
4. Look up the appropriate table value for the test statistic.
5. Formulate conclusions.
a. If the tabular statistic > calculated statistic, then you fail to reject Ho.
b. If the tabular statistic < calculated statistic, then you reject Ho.
Comparison of Two Sample Means (t-test)
Suppose we have two population means of and we want to test the hypothesis:
Ho: 21
HA: 21
This can be done using a t-test where:
21
21
YYs
YYt
Where: 1Y = mean of treatment 1
2Y = mean of treatment 2
21 YY
s
= standard deviation of the difference between two means.
Calculation of 21 YY
s
depends on three things.
1. Do the populations have a common variance (i.e. 2
2
2
1 )?
2. Are the two samples of equal size (i.e. n1 = n2)?
3. Are the observations meaningfully paired?
*Comparisons of Two Sample Means (n1 = n2) and 2
2
2
1
Given the following data:
iY 2
iY
Treatment 1 7 9 10 6 9 7 48 396
Treatment 2 2 5 3 1 5 2 18 68
Determine if the treatments means are significantly different at the 95% level of
confidence.
Step 1. Write the hypothesis to be tested:
Ho: 21
HA: 21
Step 2. Calculate 2
2
2
1 & ss
4.216
6
48396
2
2
1
s
8.216
6
1868
2
2
2
s
Step 2.1 Test to see if the two variances are homogeneous (i.e. Ho: 2
2
2
1 ).
The method of calculating 21 YY
s
will depend on whether you reject or fail to reject
Ho: 2
2
2
1 .
If you fail to reject Ho: 2
2
2
1 , the formula for 21 YY
s
is:
n
ss
p
YY
22
21
where: 2
ps = the pooled variance and
n = the number of observations in a treatment total.
If you reject Ho: 2
2
2
1 , the formula for 21 YY
s
is:
2
2
2
1
2
1
21 n
s
n
ss
YY
where: 2
1s = variance of Treatment 1
2
2s = variance of Treatment 2
1n = number of observations for treatment 1
2n = number of observations for treatment 2
Test Ho: 2
2
2
1 using an F-test where:
2
2arg
Smaller
erLF
If there are large differences between 2
2
2
1 & ss , F will become large and result in
the rejection of Ho: 2
2
2
1 .
We will be testing the hypothesis Ho: 2
2
2
1 at the 95% level of confidence.
This F-test is a two-tail test because we are not specifying which variance is
expected to be larger.
Thus, if you are testing 2/ = 0.05, then you need to use the F-table for α = 0.025
(Appendix Table IV, page 611).
This situation (i.e. testing Ho: 2
2
2
1 ) will be the only one used this semester in
which the F-test is a two-tail test.
When we use an F-test to test Ho: 21 , this will be a one-tail F-test because
the numerator of the test, the variance based on means ( 22
Tr ), is expected to
be larger than the denominator, the variance based on individual observations ( 2
). This one-tail F-test requires use of Appendix Table IV on page 610.
For this problem 167.14.2
8.2F
Step 2.2 Look up table F-value in Appendix Table IV (pages 611).
F
)1(),1(,2/ 21 nn = F alpha value/2; numerator df, denominator df
The table in our text is set up for one-tail tests. Thus, to use the table for a two-
tail test and you want to test at the 0.05 level, you need to look up the value of
0.005 in the table. The area under each of the two-tails is 0.025.
For this problem the table F for F0.05/2;5,5=7.15.
Step 2.3 Make conclusions:
Since the calculated value of F (1.167) is less than the Table-F value (7.15), we fail to
reject Ho: 2
2
2
1 at the 95% level of confidence.
Therefore, we can calculate 21 YY
s
using the formula:
n
ss
p
YY
22
21
Step 3. Calculate 2
ps
The following formula will work in 21 nn or
21 nn
)1()1(
)1()1(
21
2
22
2
112
nn
snsns p
where: 2
1s = variance of Treatment 1
2
2s = variance of Treatment 2
1n = number of observations for treatment 1
2n = number of observations for treatment 2
If 21 nn , you can calculate 2
ps using the formula
2
2
2
2
12 sss p
For this problem 6.22
)8.24.2(2
ps
Step 4. Calculate 21 YY
s
The following formula will work in 21 nn or
21 nn
)11
(21
2
21 nnss pYY
If 21 nn , you can calculate
21 YYs
using the formula:
n
ss
p
YY
22
21
Where n = the number of observations in a treatment total.
For this problem 9309.06
)6.2(221
YY
s
Step 5. Calculate t-statistic
21
21
YYs
YYt
371.5
9309.0/5
9309.0
6
18
6
48
t
Step 6. Look up table t-value.
df= )1()1( 21 nn =(6-1)+(6-1)=10
t.05/2;10df = 2.228
Step 6. Make conclusions.
-2228 2.228 5.371
Since t-calc (5.371) is > t-table (2.228) we
reject Ho: at the 95% level of
confidence.
Thus we can conclude that the mean of
treatment 1 is significantly different than that
of the mean of treatment 2at the 95% level of
confidence.
Comparison of Two Sample Means (Confidence Interval)
The formula for a confidence interval to test the hypothesis: Ho: 21 is:
212
21 )(YY
stYY
Using the data from the previous example:
212
21 )(YY
stYY
07.7
93.2
07.25
)931.0(228.2)38(
2
1
l
l
Comparison of Two Sample Means (F-test)
In conducting tests of significance for: 21
21
:
:
A
o
H
H we actually are conducting an F-
test of variances (i.e. the ratio of two variances.
F= estimate of σ2 based on means
estimate of σ2
based on individuals
=2
22
Tr
When the null hypothesis is rejected, the numerator of the F-test becomes large as
compared to the denominator. This causes the calculated vale for F to become large.
So far in class we have seen two different ways to estimate 2 :
1. nY
/22
We can estimate 2 by n* 2
Y (i.e. variance based on means).
2. We can estimate 2 by calculating 2 directly from individuals (variance based on
individuals).
Since the interval does not include
the value 0, we must reject Ho:
at the 95% level of
confidence.
The estimate of 2
Y approaches 2 only when you fail to reject Ho: because the treatment
effect 2
Tr in the expected mean square for the treatment source of variation ( 22
Tr )
approaches zero.
When Ho: 21 is rejected and the treatment variances are homogenous (i.e. 2
2
2
1 ),
the estimate of 2 based on means will over estimate 2 .
This occurs because the estimate of 2 based on means is affected by differences between
treatment means as well as differences due to random chance.
The estimate of 2 based on individuals is not affected by differences between treatment
means.
Note in the models below that the model for observations based on individuals does not
have a component for treatment.
This can be seen by looking at the linear models.
1. Linear model for observations based on individuals.
iiY
Where: Yi = the ith
observation of variable Y.
population mean.
i = random error
2. Linear model for samples from two or more treatments.
ijiijY
Where: Yij = the jth
observation of the ith
treatment.
population mean.
i = the ith
treatment
ij = random error
We can estimate 2 based on means by calculating a value called the Treatment
Mean Square (TRT MS).
We can estimate 2 based on individuals by calculating a value called the Error
Mean Square.
Given the linear model ijiijY , we can rewrite the components as:
can be rewritten as ..Y
i can be rewritten as ... YY i
ij can be rewritten as .iij YY
SOV Df SS MS F
Among trt (Trt) t-1 2... )( YYr i Trt SS/Trt df Trt MS/Error MS
Within trt (Error) t(r-1) 2.)( iij YY Error SS/Error df
Total tr-1 2..)( YYij
r = number of replicates
t = number of treatments
Example
Using the data from the previous t-test and CI problem
.iY 2
iY
Treatment 1 7 9 10 6 9 7 48 396
Treatment 2 2 5 3 1 5 2 18 68
Y..=66
Step 1. Calculate the Total SS
Total SS=
rt
YYYY
ij
ij ij
2
22..)(
Correction factor
Definition formula Working formula
=(72 + 9
2 + 10
2 + . . . + 2
2) - 66
2/(6*2)
= 464 – 363
= 101
Step 2. Calculate Trt SS
Trt SS=
rt
Y
r
YYYr
ijii
22
.2... )(
Correction factor
Definition formula Working formula
75
363438
12
66)
6
18
6
48(
222
Step 3. Calculate Error SS
Direct Method: Error SS= 26)6
1868()
6
48396()(
222
.2 i
i
j
ijr
YY
Indirect Method: Error SS = Total SS – Trt SS = 101-75 = 26
Step 4. Complete the ANOVA Table
SOV Df SS MS F
Trt (t-1) = 1 75.0 75.0 28.85**
Error t(r-1) = 10 26.0 2.6
Total rt-1 = 11
Step 5. Look up the Table F-value.
Fα; numerator df, denominator df = Fα; treatment df, error df
F0.05; 1,10 = 4.96
F0.01; 1,10 = 10.04
This Error MS is the same value as
sp2 in the t-test.
This is the estimate of (i.e. s2).
*,** Significant at
the 95% and 99%
levels of confidence,
respectively.
Step 6. Make conclusions.
0 4.96 10.04 28.85
Relationship Between the t-statistic and the F-statistic
When 2
2
2
1 , t2 = F.
Comparisons of Two Sample Means (21 nn ) and 2
2
2
1
Given the following data:
iY 2
iY
Treatment 1 3 9 6 10 28 226
Treatment 2 11 15 9 12 12 13 72 884
Determine if the treatments means are significantly different at the 95% level of confidence.
Step 1. Write the hypothesis to be tested:
Ho: 21
HA: 21
Step 2. Calculate 2
2
2
1 & ss
1014
4
28226
2
2
1
s
416
6
72884
2
2
2
s
Since F-calc (28.85) > 4.96 we reject
Ho: at the 95% level of
confidence.
Since F-calc (28.85) > 10.04 we
reject Ho: at the 99% level of
confidence.
Step 2.1 Test to see if the two variances are homogeneous (i.e. Ho:2
2
2
1 ).
2
2arg
Smaller
erLF
For this problem 5.24
10F
Step 2.2 Look up table F-value in Appendix Table IV (page 611).
F
)1(),1(,2/ 21 nn = F alpha value/2; numerator df, denominator df
For this problem the table F for F0.05/2;3,5= 7.76.
Step 2.3 Make conclusions:
Since the calculated value of F (2.5) is less than the Table-F value (7.76), we fail to reject
Ho: 2
2
2
1 at the 95% level of confidence.
Therefore, we can calculate 21 YY
s
using the formula:
)11
(21
2
21 nnss pYY
Step 3. Calculate 2
ps
)1()1(
)1()1(
21
2
22
2
112
nn
snsns p
where: 2
1s = variance of Treatment 1
2
2s = variance of Treatment 2
1n = number of observations for treatment 1
2n = number of observations for treatment 2
25.6)16()14(
4)16(10)14(2
ps
Step 4. Calculate 21 YY
s
614.16
1
4
125.6)
11(
21
2
21
nnss pYY
Step 5. Calculate t-statistic
21
21
YYs
YYt
098.3
614.1
127
t
Many people don’t line working with negative t-values.
If you are conducting a two-tail test, you can work with the absolute value of t since the
rejection regions are symmetrical about the axis of 0.
Therefore, 098.3098.3
Step 6. Look up table t-value.
Df= )1()1( 21 nn =(4-1)+(6-1)=8
t.05/2;8df = 2.306
Step 6. Make conclusions.
-2306 2.306 3.098
Since t-calc (3.098) is > t-table (2.306)
we reject Ho: at the 95% level of
confidence.
Thus we can conclude that the mean of
treatment 1 is significantly different than
that of the mean of treatment 2 at the
95% level of confidence.
Comparisons of Two Sample Means (n1 = n2 or 21 nn ) and 2
2
2
1
Given the following data:
2
iY iY
Treatment 1 15.9 13.8 16.5 28.3 28.2 26.1 17.6 3302.60 146.4
Treatment 2 17.6 16.4 14.1 16.2 16.5 14.1 17.7 1824.32 112.6
Determine if the treatments means are significantly different at the 95% level of confidence.
Step 1. Write the hypothesis to be tested:
Ho: 21
HA: 21
Step 2. Calculate 2
2
2
1 & ss
12.4017
7
4.1466.3302
2
2
1
s
18.217
7
6.11232.1824
2
2
2
s
Step 2.1 Test to see if the two variances are homogeneous (i.e. Ho: 2
2
2
1 ).
42.1818.2
12.40F
Step 2.2 Look up table F-value.
F0.05/2;6,6=5.82.
Step 2.3 Make conclusions:
Since the calculated value of F (18.42) is greater than the Table-F value (5.82), we reject
Ho: 2
2
2
1 at the 99% level of confidence.
Therefore, we have to calculate 21 YY
s
using the formula:
2
2
2
1
2
1
21 n
s
n
ss
YY
Step 3. Calculate 21 YY
s
458.27
18.2
7
12.40
2
2
2
1
2
1
21
n
s
n
ss
YY
Step 4. Calculate t’-statistic
Use t’ because t in this case is not distributed strictly as Student’s t.
21
21'
YYs
YYt
964.1
458.2
7
6.112
7
4.146
'
t
Step 5. Calculate effective degrees of freedom.
Calculation of the effective df allows us to use the t-table to look up values to test t’.
Effective df =
65.6)016.0475.5(
516.36
17
7
18.2
17
7
12.40
7
18.2
7
12.40
11
22
2
2
2
2
2
2
1
2
1
2
1
2
2
2
2
1
2
1
n
n
s
n
n
s
n
s
n
s
We can round the effective degrees of freedom of 6.65 to 7.0.
We then look up the t-value with 7 df.
t.05/2; 7df = 2.365
Step 6. Make conclusions.
-2.365 1.964 2.365
Paired Comparisons
Earlier it was stated that calculation of 21 YY
s
depends on three things:
4. Do the populations have a common variance (i.e. 2
2
2
1 )?
5. Are the two samples of equal size (i.e. n1 = n2)?
6. Are the observations meaningfully paired?
We will now discuss the consequences of using paired comparisons.
Since t’-calc (1.964) is < t-table (2.365) we
fail to reject Ho: at the 95% level
of confidence.
Thus we can conclude that the mean of
treatment 1 is not significantly different
than that of the mean of treatment 2at the
95% level of confidence.
Pairing of observations is done before the experiment is conducted.
Pairing is done to make the tests of significance more powerful.
If members of a pair tend to be large or small together, it may be possible to detect smaller
differences between treatments than would be possible without pairing.
The purpose of pairing is to eliminate an outside source of variation, that existing from pair
to pair.
Calculating the variance of differences rather than the variance of individuals controls the
variation.
Examples where pairing may be useful:
Drug or feed studies on the same animal.
Measurements done on the same individual at different times (before and after type
treatments).
DD
s
D
s
YYt
.2.1
where n
DD
jj YYD 21
n = the number of pairs.
n
s
n
ss DD
D
2
11
2
2
2
212
21
n
n
DD
n
n
YYYY
s
jj
jj
D
Example
Replicates Treatment 1 Treatment 2 Difference (D) Difference2 (D
2)
1 8 12 -4 16
2 12 17 -5 25
3 13 16 -3 9
4 10 16 -6 36
5 5 9 -4 16
48.1 Y 70.2 Y 22D 1022D
Step 1. Calculate Ds
14.115
5
22102
1
22
2
n
n
DD
sD
Step 2. Calculate D
s
510.05
14.1
Ds
Step 3. Calculate t-value
627.8510.0
4.4
510.0
5/22
Ds
Dt
Step 4. Look up Table t-value
df = number of pairs – 1
776.24;2/05.1;2/ tt dfn
Step 5. Make conclusions
-8.627 -2.776 2.776
REVIEW
You might be thinking there are many formulas to remember; however, there are some
simple ways to remember them.
We have talked about four types of variances:
s2 = variance based on individuals
2
Ys = variance based on means
2
21 YYs
= variance of the difference between two means
2
Ds = variance of paired observations
However, there is a relationship between all four variances that can help you in
remembering the formulas:
1/n
s2 2
Ys
2 2/n 2
2
Ds 2
21 YYs
1/n
Since t-calc (-8.627) is < t-table (-2.776)
we reject Ho: at the 95% level of
confidence.
Thus we can conclude that the mean of
treatment 1 is significantly different than
that of the mean of treatment 2at the 95%
level of confidence.
There are only two formulas for t:
Ys
Yt 0 and
21
21
YYs
YYt
You will need to remember how to calculate Y
s and21 YY
s
.
)1()1(
)1()1(
21
2
22
2
112
nn
snsns p regardless if
21 nn or 21 nn
)11
(21
2
21 nnss pYY
regardless if 21 nn or
21 nn
SAS Commands for the t-Test
t-test with equal variance
options pageno=1;
data ttest;
input trt yield;
datalines;
1 7
1 9
1 10
1 6
1 9
1 7
2 2
2 5
2 3
2 1
2 5
2 2
;
proc sort;
by trt;
run;
proc ttest;
class trt;
title 't-test with equal variance';
run;
t-test with unequal variance
options pageno=1;
data unequ;
input trt yield;
datalines;
1 15.9
1 13.8
1 16.5
1 28.3
1 28.2
1 26.1
1 17.6
2 17.6
2 16.4
2 14.1
2 16.2
2 16.5
2 14.1
2 17.7
;;
proc sort;
by trt;
run;
proc ttest;
class trt;
title 't-test with unequal variance';
run;
Paired t-test
options pageno=1;
data paired;
input a b;
datalines;
8 12
12 17
13 16
10 16
5 9
;;
proc ttest;
paired a*b;
title 'paired t-test';
run;
T-test to compare Treatments 1 and 2
(Equal variance)
The TTEST Procedure
Statistics
Variable Trt N
Lower CL
Mean Mean
Upper CL
Mean
Lower CL
Std Dev Std Dev
Upper CL
Std Dev
Std
Err
Mini-
mum
Maxi-
mum
Yield 1 6 6.37 8 9.62 0.96 1.54 3.79 0.63 6 10
Yield 2 6 1.24 3 4.75 1.04 1.67 4.1 0.68 1 5
Yield Diff (1-2) 2.92 5 7.07 1.12 1.61 2.82 0.93
T-Tests
Variable Method Variances DF t Value Pr > |t|
Yield Pooled Equal 10 5.37 0.0003
Yield Satterthwaite Unequal 9.94 5.37 0.0003
Equality of Variances
Variable Method Num DF Den DF F Value Pr > F
Yield Folded F 5 5 1.17 0.8698
If the variances are homogeneous, then we use this t-
test for comparing the means of the two treatments. If
the Pr>|t| value is 0.05 or less, then we reject the null
hypothesis. Thus, we can conclude the two treatment
means are different.
If the variances are not homogeneous, then we use this
t-test for comparing the means of the two treatments.
e two treatment means are different.
This F-test is testing to see if the variances of the two
treatments are homogeneous. If the Pr>F values is
greater than 0.001, then we fail to reject the
hypothesis that the two error variances are
homogeneous.
T-test to compare Treatments 1 and 2
(Unequal variance)
The TTEST Procedure
trt N Mean Std Dev Std Err Minimum Maximum
1 7 20.9143 6.3344 2.3942 13.8000 28.3000
2 7 16.0857 1.4758 0.5578 14.1000 17.7000
Diff (1-2) 4.8286 4.5991 2.4583
trt Method Mean 95% CL Mean Std Dev 95% CL Std Dev
1 20.9143 15.0559 26.7726 6.3344 4.0819 13.9488
2 16.0857 14.7208 17.4506 1.4758 0.9510 3.2499
Diff (1-2) Pooled 4.8286 -0.5276 10.1848 4.5991 3.2979 7.5918
Diff (1-2) Satterthwaite 4.8286 -1.0471 10.7042
Method Variances DF t Value Pr > |t|
Pooled Equal 12 1.96 0.0731
Satterthwaite Unequal 6.6495 1.96 0.0924
Equality of Variances
Method Num DF Den DF F Value Pr > F
Folded F 6 6 18.42 0.0025
Paired t-test
The TTEST Procedure
N Mean Std Dev Std Err Minimum Maximum
5 -4.4000 1.1402 0.5099 -6.0000 -3.0000
Mean 95% CL Mean Std Dev 95% CL Std Dev
-4.4000 -5.8157 -2.9843 1.1402 0.6831 3.2764
DF t Value Pr > |t|
4 -8.63 0.0010