T-test and ANOVA for Balanced Data
Ho Kim
GSPH, SNU
(hosting03.snu.ac.kr/~hokim)
Single Sample Analysis
dataset: peppers Peppers Dataset
Obs angle1 32 113 -74 25 36 87 -38 -29 13
10 411 712 -113 414 715 -116 417 1218 -319 720 521 322 -123 924 -725 226 427 828 -2
proc means data=peppers mean std stderr t probt;
run;
options
1. stderr: the standard error of the mean2. t: 3. probt: the significance probability of the t test
The MEANS Procedure
분석 변수 : angle
평균값 표준편차 표준오차 t값 Pr > |t|
-----------------------------------------------------------------
3.1785714 5.2988718 1.0013926 3.17 0.0037
-------------------------------------------------------------------
0 : 0H
Two Related Samples : paired t-test
dataset: pulse Pulse Dataset
Obs pre post d
1 62 61 1
2 63 62 1
3 58 59 -1
4 64 61 3
5 64 63 1
6 61 58 3
7 68 61 7
8 66 64 2
9 65 62 3
10 67 68 -1
11 69 65 4
12 61 60 1
13 64 65 -1
14 61 63 -2
15 63 62 1
d = pre-post
(difference in rate)
proc means data=pulse mean std stderr t probt;
var d;
run;
The MEANS Procedure
분석 변수 : d
평균값 표준편차 표준오차 t값 Pr > |t|-------------------------------------------------------------------
1.4666667 2.3258383 0.6005289 2.44 0.0285-------------------------------------------------------------------
Two-sided p-value
One-sided p-value=0.0285/2=0.0143 for vs.0 : 0H d 1 : 0H d
Two Independent Samples
dataset: bullets Bullets Dataset
Obs powder velocity
1 1 27.3
2 1 28.1
3 1 27.4
4 1 27.7
5 1 28.0
6 1 28.1
7 1 27.4
8 1 27.1
9 2 28.3
10 2 27.9
11 2 28.1
12 2 28.3
13 2 27.9
14 2 27.6
15 2 28.5
16 2 27.9
17 2 28.4
18 2 27.7
proc ttest data=bullets;
var velocity;class powder;
run;
The TTEST Procedure
Lower CL Upper CL Lower CL
Variable powder N Mean Mean Mean Std Dev
velocity 1 8 27.309 27.638 27.966 0.2596
velocity 2 10 27.841 28.06 28.279 0.2106
velocity Diff (1-2) -0.771 -0.422 -0.074 0.2582
Upper CL
Variable powder Std Dev Std Dev Std Err Minimum Maximum
velocity 1 0.3926 0.799 0.1388 27.1 28.1
velocity 2 0.3062 0.5591 0.0968 27.6 28.5
velocity Diff (1-2) 0.3467 0.5276 0.1644
Variable Method Variances DF t Value Pr > |t|
velocity Pooled Equal 16 -2.57 0.0206
velocity Satterthwaite Unequal 13.1 -2.50 0.0267
Equality of Variances
Variable Method Num DF Den DF F Value Pr > F
velocity Folded F 7 9 1.64 0.4782
For H0: Variances are equal, F = 1.64 DF = (7,9)
Multiple Comparisons and Pre-planed Comparisons
F-test: ‘All effects are zero.’
We still don’t know which effects are not zero even after rejecting Ho.
ex)
if is rejected, which is large and which is small ?
Multiple Comparisons
Several methods
LSD (least significant difference): Perform t tests for all possible pairs.
Duncan’s multiple range test: Compare the difference of means to the
pre-fixed values -> determine equality of means.
Inflating type I error is a major problem for multiple comparisons.
0 :H
0 1 2 3 4: 0H
0H i i
ex) let’s assume that is the significance level of a test.
multiple comparison for
For k=4,
overall = 0.1855, inflated type I error.
01 1 01 01
02 2 02 02
0 0 0 01 02
01
: 0 ( ) 1
: 0 ( ) 1
( ) where and
(
Let H p do not reject H H is true
H p do not reject H H is true
then p do not reject H H H H H
p do not reject H and do not reje
02 0
2
)
(1- ) (1- ) (1- )
ct H H
1 2 3 k
4
(1 ) (1 )
1 0.1855 0.8145 ( .95) .95
k
Bonferroni Correction : For m multiple comparisons,
individual significance level =
then over-all significance level becomes
Ex1. )m= 4
Ex2. ) If we have 10 hypothesis, perform individual test with
p= . Then over-all p=0.05 for the whole tests.
This is called “Bonferroni corrected p-value”.
m
40.05(1 ) 0.95 1 0.05
4
0.050.005
10
Multiple comparisons is performed after collecting data if
there were no planned hypotheses.
In many cases, we have preplanned comparisons when we
design a study. This is called preplanned comparisons.
We can do this by using CONTRAST, ESTIMATE, LSMEAN, etc
in SAS.
We don’t need to worry about type I error problems for
preplanned comparisons.
It is very important to learn how to use contrast statement.
BRAND Data
Explanatory var: brand
response var: wear
Preplanned Comparisons
model
i ij
i ij
ijy
ACMEiAJAXCHAMPTUFFYXTRA
Mean of i-th level
Veneer data set
Obs brand wear
1 ACME 2.3
2 ACME 2.1
3 ACME 2.4
4 ACME 2.5
5 CHAMP 2.2
6 CHAMP 2.3
7 CHAMP 2.4
8 CHAMP 2.6
9 AJAX 2.2
10 AJAX 2.0
11 AJAX 1.9
12 AJAX 2.1
13 TUFFY 2.4
14 TUFFY 2.7
15 TUFFY 2.616 TUFFY 2.7
17 XTRA 2.3
18 XTRA 2.5
19 XTRA 2.3
20 XTRA 2.4
Overall mean
Effect of i-th level
US vs Foreign
• Simultaneous testing
• estimate
0
1 1
3 2
1 1
3 2
: ( ) ( )
( ) ( ) 0
.333 .333 .333 - .5 - .5
2 ( ) 3 ( ) 0
ACME AJAX CHAMP TUFFY XTRA
ACME AJAX CHAMP TUFFY XTRA
ACME AJAX CHAMP TUFFY XTRA
H
2 2 2 - 3 - 3
Whole population is divided into homogeneous blocks.
We do randomization in each block.
We are not interested in the Block effects. We are interested in
the effect of the other factor.
ex) pesticide data
BLOCK=location: 1, 2, 3
5 kinds of BLEND: A, B, C, D, E
Interested in BLEND effects after controlling for BLOCK
effect.
Randomized-Blocks Design
2 factors
Pesticide Dataset
Obs block blend pctloss
1 1 B 18.2
2 1 A 16.9
3 1 C 17.0
4 1 E 18.3
5 1 D 15.1
6 2 A 16.5
7 2 E 18.3
8 2 B 19.2
9 2 C 18.1
10 2 D 16.0
11 3 B 17.1
12 3 D 17.8
13 3 C 17.3
14 3 E 19.8
15 3 A 17.5
ANOVA for Randomized-Blocks Design
proc anova data=pestcide;
class block blend;
model pctloss=block blend;
run;
The ANOVA ProcedureDependent Variable: pctloss
Sum ofSource DF Squares Mean Square F Value Pr > F
Model 6 13.20400000 2.20066667 2.52 0.1133Error 8 6.99200000 0.87400000Corrected Total 14 20.19600000
R-Square Coeff Var Root MSE pctloss Mean0.653793 5.329987 0.934880 17.54000
Source DF Anova SS Mean Square F Value Pr > F
block 2 1.64800000 0.82400000 0.94 0.4289blend 4 11.55600000 2.88900000 3.31 0.0705
3 factors: run position mat
Latin Square Design
Position
run 1 2 3 4
1 B D A C
2 D B C A
3 A C B D
4 C A D B
MAT: materials (A,B,C,D)
POS: (1, 2, 3, 4)
RUN: 4 level
WTLOSS: weight loss
SHRINK:
fewer # of repeats -> less expensive.
limit: Interactions can not be tested.
Garments Dataset
Obs run pos mat wtloss shrink
1 2 4 A 251 50
2 2 2 B 241 48
3 2 1 D 227 45
4 2 3 C 229 45
5 3 4 D 234 46
6 3 2 C 273 54
7 3 1 A 274 55
8 3 3 B 226 43
9 1 4 C 235 45
10 1 2 D 236 46
11 1 1 B 218 43
12 1 3 A 268 51
13 4 4 B 195 39
14 4 2 A 270 52
15 4 1 C 230 48
16 4 3 D 225 44
Analysis of ANOVA for Latin Square Design
proc anova data=garments; class run pos mat;
model wtloss shrink = run pos mat;
run;
The ANOVA Procedure
Dependent Variable: wtloss
Sum of
Source DF Squares Mean Square F Value Pr > F
Model 9 7076.500000 786.277778 12.84 0.0028
Error 6 367.500000 61.250000
Corrected Total 15 7444.000000
R-Square Coeff Var Root MSE wtloss Mean
0.950631 3.267740 7.826238 239.5000
Source DF Anova SS Mean Square F Value Pr > F
run 3 986.500000 328.833333 5.37 0.0390
pos 3 1468.500000 489.500000 7.99 0.0162
mat 3 4621.500000 1540.500000 25.15 0.0008
The ANOVA Procedure
Dependent Variable: shrink
Sum of
Source DF Squares Mean Square F Value Pr > F
Model 9 265.7500000 29.5277778 9.84 0.0058
Error 6 18.0000000 3.0000000
Corrected Total 15 283.7500000
R-Square Coeff Var Root MSE shrink Mean
0.936564 3.675439 1.732051 47.12500
Source DF Anova SS Mean Square F Value Pr > F
run 3 33.2500000 11.0833333 3.69 0.0813
pos 3 60.2500000 20.0833333 6.69 0.0242
mat 3 172.2500000 57.4166667 19.14 0.0018
ANOVA for One-Way Classification
proc anova data=veneer; class brand;
model wear=brand; run;
The ANOVA Procedure
Dependent Variable: wear
Sum of
Source DF Squares Mean Square F Value Pr > F
Model 4 0.61700000 0.15425000 7.40 0.0017
Error 15 0.31250000 0.02083333
Corrected Total 19 0.92950000
R-Square Coeff Var Root MSE wear Mean
0.663798 6.155120 0.144338 2.345000
Source DF Anova SS Mean Square F Value Pr > F
brand 4 0.61700000 0.15425000 7.40 0.0017
Least Significant Difference Comparisons of BRAND Mean
proc anova data=veneer;
class brand;
model wear=brand;
means brand/lsd;
run;The ANOVA Procedure
t Tests (LSD) for wear
노트: This test controls the Type I comparisonwise error rate, not the
experimentwise error rate.
Alpha 0.05Error Degrees of Freedom 15Error Mean Square 0.020833Critical Value of t 2.13145Least Significant Difference 0.2175
Means with the same letter are not significantly different.
T Grouping Mean N brandA 2.6000 4 TUFFYB 2.3750 4 XTRABB 2.3750 4 CHAMPBB 2.3250 4 ACMEC 2.0500 4 AJAX
Confidence Interval for BRAND Means
proc anova data=veneer; class brand;
model wear=brand;
means brand/lsd clm; run;
The ANOVA Procedure
t Confidence Intervals for wear
Alpha 0.05
Error Degrees of Freedom 15
Error Mean Square 0.020833
Critical Value of t 2.13145
Half Width of Confidence Interval 0.153824
brand N Mean 95% Confidence Limits
TUFFY 4 2.60000 2.44618 2.75382
XTRA 4 2.37500 2.22118 2.52882
CHAMP 4 2.37500 2.22118 2.52882
ACME 4 2.32500 2.17118 2.47882
AJAX 4 2.05000 1.89618 2.20382
US vs Foreign
• We need to input the coefficients of the model parameters
at intercept , brand in SAS
0
1 1
3 2
1 1
3 2
: ( ) ( )
( ) ( ) 0
.333 .333 .333 - .5 - .5
2 ( ) 3 ( ) 0
ACME AJAX CHAMP TUFFY XTRA
ACME AJAX CHAMP TUFFY XTRA
ACME AJAX CHAMP TUFFY XTRA
H
2 2 2 - 3 - 3
1 2 3 4 5, , , , ,
• Intercept Brand1 Brand2 Brand3 Brand4 Brand5
• 0 0.333 0.333 0.333 -0.5 -0.5
• 0 2 2 2 -3 -3
( )i i iE Y
i ij
i ij
ijy
ACMEiAJAXCHAMPTUFFYXTRA
Mean of i-th level
1 2 3 1 2 3
4 5 4 5
1 1
3 3
1 1
2 2
( ) ( ) (1)
( ) ( ) (2)
Ho: (1)-(2)=0
Simultaneous Contrasts among US BRAND Means
proc glm data=veneer;
class brand;
model wear=brand;
contrast 'US BRANDS' brand 1 -1 0 0 0, brand 1 0 -1 0 0;
run;
The GLM ProcedureDependent Variable: wear
Sum ofContrast DF Contrast SS Mean Square F Value Pr > F
US BRANDS 2 0.24500000 0.12250000 5.88 0.0130
1 2 3
1 2 1 3
1 2 1 3
1 2 1 3
: ( )
: and
=0 and 0
( ) 0 and ( ) 0
Ho
Ho
Int Brand1 Brand2 Brand3 Brand4 Brand5
0 1 -1 0 0 0 and
0 1 0 -1 0 0
Are the means of ACME AJAX CHAMP all the same ?
ANOVA and Contrast with PROC GLM
proc glm data=veneer; class brand;
model wear=brand;
contrast 'ACME vs AJAX' brand 1 -1 0 0 0;
run;The GLM Procedure
Dependent Variable: wearSum of
Source DF Squares Mean Square F Value Pr > F
Model 4 0.61700000 0.15425000 7.40 0.0017Error 15 0.31250000 0.02083333Corrected Total 19 0.92950000
R-Square Coeff Var Root MSE wear Mean0.663798 6.155120 0.144338 2.345000
Source DF Type I SS Mean Square F Value Pr > Fbrand 4 0.61700000 0.15425000 7.40 0.0017
Source DF Type III SS Mean Square F Value Pr > Fbrand 4 0.61700000 0.15425000 7.40 0.0017
Contrast DF Contrast SS Mean Square F Value Pr > FACME vs AJAX 1 0.15125000 0.15125000 7.26 0.0166
Contrasts among BRAND Means
proc glm data=veneer; class brand;
model wear=brand;
contrast 'US vs FOREIGN' brand 2 2 2 -3 -3;
contrast 'A-L vs C-L' brand 1 1 -2 0 0;
contrast 'ACME vs AJAX' brand 1 -1 0 0 0;
contrast 'TUFFY vs XTRA' brand 0 0 0 1 -1;
run;The GLM Procedure
Dependent Variable: wear
Contrast DF Contrast SS Mean Square F Value Pr > F
US vs FOREIGN 1 0.27075000 0.27075000 13.00 0.0026
A-L vs C-L 1 0.09375000 0.09375000 4.50 0.0510
ACME vs AJAX 1 0.15125000 0.15125000 7.26 0.0166
TUFFY vs XTRA 1 0.10125000 0.10125000 4.86 0.0435
1 2 3 1 2 3
1 2 3
1: ( ) ---> 0.5 0.5 - 0
2
0.5 0.5 0.5 0.5 ( ) 0
Ho
Estimating Difference between BRANDE mean
proc glm data=veneer; class brand;
model wear=brand;
estimate 'ACME vs AJAX' brand 1 -1 0 0 0;
run;
The GLM Procedure
Dependent Variable: wear
Standard
Parameter Estimate Error t Value Pr > |t|
ACME vs AJAX 0.27500000 0.10206207 2.69 0.0166
Estimating Mean of US Brand
proc glm data=veneer; class brand;
model wear=brand;
estimate 'US MEAN' intercept 3 brand 1 1 1 0 0/divisor=3;
run;
The GLM Procedure
Dependent Variable: wear
Standard
Parameter Estimate Error t Value Pr > |t|
US MEAN 2.25000000 0.04166667 54.00 <.0001
1 2 3 1 2 3
1 2 3
1 1( ) --> ( )
3 3
1 1 1
3 3 3
ACME - AJAX
proc glm data=veneer; class brand;
model wear=brand;
estimate 'Acme-Ajax' brand 1 -1 0 0 0;
run;
or
proc ttest data=veneer;
class brand;
var wear;
where brand in ('ACME', 'AJAX') ;
run;
Standard
Parameter Estimate Error t Value Pr > |t|
Acme-Ajax 0.27500000 0.10206207 2.69 0.0166
The TTEST Procedure
Lower CL Upper CL Lower CL
Variable brand N Mean Mean Mean Std Dev
wear ACME 4 2.0532 2.325 2.5968 0.0967
wear AJAX 4 1.8446 2.05 2.2554 0.0731
wear Diff (1-2) 0.0131 0.275 0.5369 0.0975
Statistics
Upper CL
Variable brand Std Dev Std Dev Std Err Minimum Maximum
wear ACME 0.1708 0.6368 0.0854 2.1 2.5
wear AJAX 0.1291 0.4814 0.0645 1.9 2.2
wear Diff (1-2) 0.1514 0.3334 0.107
T-Tests
Variable Method Variances DF t Value Pr > |t|
wear Pooled Equal 6 2.57 0.0424
wear Satterthwaite Unequal 5.58 2.57 0.0452
Equality of Variances
Variable Method Num DF Den DF F Value Pr > F
wear Folded F 3 3 1.75 0.6571
Homework
Perform ANOVA using estimate and contrast statements in this file.
Explain the meanings 1) using parameters of the ANOVA model, 2) in words (practical and realistic meaning).