Orthogonal Linear Contrasts

This is a technique for partitioning ANOVA sum of squares into individual degrees of freedom

Definition

Let x1, x2, ... , xp denote p numerical quantities computed from the data.

These could be statistics or the raw observations.

A linear combination of x1, x2, ... , xp is defined to be a quantity ,L ,computed in the following manner:

L = c1x1+ c2x2+ ... + cpxp

where the coefficients c1, c2, ... , cp are predetermined numerical values:

Definition

If the coefficients c1, c2, ... , cp satisfy:

c1+ c2 + ... + cp = 0,

Then the linear combination

L = c1x1+ c2x2+ ... + cpxp

is called a linear contrast.

Examples

p

xxxxL p

21

2354321 xxxxx

L

54321 2

1

2

1

3

1

3

1

3

1xxxxx

1.

pxp

xp

xp

11121

2.

3. L = x1 - 4 x2+ 6x3 - 4 x4 + x5

= (1)x1+ (-4)x2+ (6)x3 + (-4)x4 + (1)x5

A linear combination

A linear contrast

A linear contrast

Definition

Let A = a1x1+ a2x2+ ... + apxp and B= b1x1+ b2x2+ ... + bpxp be two linear contrasts of the quantities x1, x2, ... , xp. Then A and B are c called Orthogonal Linear Contrasts if in addition to:

a1+ a2+ ... + ap = 0 and

b1+ b2+ ... + bp = 0,

it is also true that:

a1b1+ a2b2+ ... + apbp = 0.

.

Example

Let

Note:

2354321 xxxxx

A

54321 2

1

2

1

3

1

3

1

3

1xxxxx

54321

2xxx

xxB

54321 1112

1

2

1xxxxx

01- 2

11

2

11-

3

1

2

1

3

1

2

1

3

1

Definition

Let A = a1x1+ a2x2+ ... + apxp,

B= b1x1+ b2x2+ ... + bpxp ,

..., and

L= l1x1+ l2x2+ ... + lpxp

be a set linear contrasts of the quantities x1, x2, ... , xp.

Then the set is called a set of Mutually Orthogonal Linear Contrasts if each linear contrast in the set is orthogonal to any other linear contrast..

Theorem:

The maximum number of linear contrasts in a set of Mutually Orthogonal Linear Contrasts of the quantities x1, x2, ... , xp is p - 1.

p - 1 is called the degrees of freedom (d.f.) for comparing quantities x1, x2, ... , xp .

Comments

1. Linear contrasts are making comparisons amongst the p values x1, x2, ... , xp

2. Orthogonal Linear Contrasts are making independent comparisons amongst the p values x1, x2, ... , xp .

3. The number of independent comparisons amongst the p values x1, x2, ... , xp is p – 1.

Definition

denotes a linear contrast of the p means

If each mean, , is calculated from n observations then:

The Sum of Squares for testing the Linear Contrast L, is defined to be:

pp xaxaxaL 2211

ixpxxx ,,2,1

222

21

2

= p

L +...+a+aa

n LSS

the degrees of freedom (df) for testing the Linear Contrast L, is defined to be

the F-ratio for testing the Linear Contrast L, is defined to be:

1Ldf

1

Error

L

MS

SS

F

Theorem:

Let L1, L2, ... , Lp-1 denote p-1 mutually orthogonal Linear contrasts for comparing the p means . Then the Sum of Squares for comparing the p means based on p – 1 degrees of freedom , SSBetween, satisfies:

121 p-L LLBetween + SS + ... + SS = SSSS

Comment

Defining a set of Orthogonal Linear Contrasts for comparing the p means

allows the researcher to "break apart" the Sum of Squares for comparing the p means, SSBetween, and make individual tests of each the Linear Contrast.

pxxx ,,2,1

The Diet-Weight Gain example

The sum of Squares for comparing the 6 means is given in the Anova Table:

,5.999.850.100 3 2 1 xxx ,,

7.789.832.79 6 5 4 xxx ,,

Five mutually orthogonal contrasts are given below (together with a description of the purpose of these contrasts) :

6543211 3

1

3

1xxxxxxL

(A comparison of the High protein diets with Low protein diets)

63412 2

1

2

1xxxxL

(A comparison of the Beef source of protein with the Pork source of protein)

(A comparison of the Meat (Beef - Pork) source of protein with the Cereal source of protein)

(A comparison representing interaction between Level of protein and Source of protein for the Meat source of Protein)

(A comparison representing interaction between Level of protein with the Cereal source of Protein)

5264313 2

1

4

1xxxxxxL

43614 2

1

2

1xxxxL

2645315 24

12

4

1xxxxxxL

The Anova Table for Testing these contrasts is given below:

Source: DF: Sum Squares: Mean Square: F-test:

Contrast L1 1 3168.267 3168.267 14.767

Contrast L2 1 2.500 2.500 0.012

Contrast L3 1 264.033 264.033 1.231

Contrast L4 1 0.000 0.000 0.000

Contrast L5 1 1178.133 1178.133 5.491

Error 54 11586.000 214.556

The Mutually Orthogonal contrasts that are eventually selected should be determine prior to observing the data and should be determined by the objectives of the experiment

Another Five mutually orthogonal contrasts are given below (together with a description of the purpose of these contrasts) :

63412 2

1

2

1xxxxL

(A comparison of the Beef source of protein with the Pork source of protein)

(A comparison of the Meat (Beef - Pork) source of protein with the Cereal source of protein)

5264311 2

1

4

1xxxxxxL

(A comparison of the high and low protein diets for the Beef source of protein)

(A comparison of the high and low protein diets for the Cereal source of protein)

(A comparison of the high and low protein diets for the Pork source of protein)

413 xxL

524 xxL

635 xxL

The Anova Table for Testing these contrasts is given below:

Source: DF: Sum Squares: Mean Square: F-test:

Beef vs Pork ( L1) 1 2.500 2.500 0.012

Meat vs Cereal ( L2) 1 264.033 264.033 1.231

High vs Low for Beef ( L3) 1 2163.200 2163.200 10.082

High vs Low for Cereal ( L4) 1 20.000 20.000 0.093

High vs Low for Pork ( L5) 1 2163.200 2163.200 10.082

Error 54 11586.000 214.556

Orthogonal Linear Contrasts

Polynomial Regression

Orthogonal Linear Contrasts for Polynomial Regression

k P o l y n o m i a l 1 2 3 4 5 6 7 8 9 1 0 a

i2

3 L i n e a r - 1 0 1 2 Q u a d r a t i c 1 - 2 1 6 4 L i n e a r - 3 - 1 1 3 2 0 Q u a d r a t i c 1 - 1 - 1 1 4 C u b i c - 1 3 - 3 1 2 0 5 L i n e a r - 2 - 1 0 1 2 1 0 Q u a d r a t i c 2 - 1 - 2 - 1 2 1 4 C u b i c - 1 2 0 - 2 1 1 0 Q u a r t i c 1 - 4 6 - 4 1 7 0 6 L i n e a r - 5 - 3 - 1 1 3 5 7 0 Q u a d r a t i c 5 - 1 - 4 - 4 - 1 5 8 4 C u b i c - 5 7 4 - 4 - 7 5 1 8 0 Q u a r t i c 1 - 3 2 2 - 3 1 2 8 7 L i n e a r - 3 - 2 - 1 0 1 2 3 2 8 Q u a d r a t i c 5 0 - 3 - 4 - 3 0 5 8 4 C u b i c - 1 1 1 0 - 1 - 1 1 6 Q u a r t i c 3 - 7 1 6 1 - 7 3 1 5 4

Orthogonal Linear Contrasts for Polynomial Regression

k P o l y n o m i a l 1 2 3 4 5 6 7 8 9 1 0 a

i2

8 L i n e a r - 7 - 5 - 3 - 1 1 3 5 7 1 6 8 Q u a d r a t i c 7 1 - 3 - 5 - 5 - 3 1 7 1 6 8 C u b i c - 7 5 7 3 - 3 - 7 - 5 7 2 6 4 Q u a r t i c 7 - 1 3 - 3 9 9 - 3 - 1 3 7 6 1 6 Q u i n t i c - 7 2 3 - 1 7 - 1 5 1 5 1 7 - 2 3 7 2 1 8 4 9 L i n e a r - 4 - 3 - 2 - 1 0 1 2 3 4 2 0 Q u a d r a t i c 2 8 7 - 8 - 1 7 - 2 0 - 1 7 - 8 7 2 8 2 7 7 2 C u b i c - 1 4 7 1 3 9 0 - 9 - 1 3 - 7 1 4 9 9 0 Q u a r t i c 1 4 - 2 1 - 1 1 9 1 8 9 - 1 1 - 2 1 1 4 2 0 0 2 Q u i n t i c - 4 1 1 - 4 - 9 0 9 4 - 1 1 4 4 6 8 1 0 L i n e a r - 9 - 7 - 5 - 3 - 1 1 3 5 7 9 3 3 0 Q u a d r a t i c 6 2 - 1 - 3 - 4 - 4 - 3 - 1 2 6 1 3 2 C u b i c - 4 2 1 4 3 5 3 1 1 2 - 1 2 - 3 1 - 3 5 - 1 4 4 2 8 5 8 0 Q u a r t i c 1 8 - 2 2 - 1 7 3 1 8 1 8 3 - 1 7 - 2 2 1 8 2 8 6 0 Q u i n t i c - 6 1 4 - 1 - 1 1 - 6 6 1 1 1 - 1 4 6 7 8 0

Example

Table Activation

Temperature 0 25 50 75 100 53 60 67 65 58 50 62 70 68 62 47 58 73 62 60 T.. Ti. 150 180 210 195 180 915 Mean 50 60 70 65 60

yij2 = 56545 Ti.2/n = 56475 T..2/nt = 55815

In this example we are measuring the “Life” of an electronic component and how it depends on the temperature on activation

The Anova Table

Source SS df MS FTreat 660 4 165.0 23.57

Linear 187.50 1 187.50 26.79Quadratic 433.93 1 433.93 61.99Cubic 0.00 1 0.00 0.00Quartic 38.57 1 38.57 5.51

Error 70 10 7.00Total 730 14

L = 25.00 Q2 = -45.00 C = 0.00 Q4 = 30.00

The Anova Tables for Determining degree of polynomial

Testing for effect of the factor

Source SS df MS F Treat 660 4 165 23.57 Error 70 10 7 Total 730 14

Testing for departure from Linear

S o u r c e S S d f M S F

L i n e a r 1 8 7 . 5 0 1 . 0 0 1 8 7 . 5 0 2 6 . 7 9 D e p a r t u r e f r o m L i n e a r 4 7 2 . 5 0 3 . 0 0 1 5 7 . 5 0 2 2 . 5 0 E r r o r 7 0 . 0 0 1 0 . 0 0 7 . 0 0

Testing for departure from Quadratic

S o u r c e S S d f M S F

L i n e a r 1 8 7 . 5 0 1 . 0 0 1 8 7 . 5 0 2 6 . 7 9 Q u a d r a t i c 4 3 3 . 9 3 1 . 0 0 4 3 3 . 9 3 6 1 . 9 9 D e p a r t u r e f r o m Q u a d r a t i c 3 8 . 5 7 2 . 0 0 1 9 . 2 9 2 . 7 6 E r r o r 7 0 . 0 0 1 0 . 0 0 7 . 0 0

y = 49.751 + 0.61429 x -0.0051429 x^2

40

45

50

55

60

65

70

0 20 40 60 80 100 120

Act. Temp

Lif

e

Multiple Testing

•Tukey’s Multiple comparison procedure•Scheffe’s multiple comparison procedure

Multiple Testing – a Simple Example

Suppose we are interested in testing to see if two parameters (1 and 2) are equal to zero.

There are two approaches

1. We could test each parameter separately

a) H0: 1 = 0 against HA: 1 ≠ 0 , then

b) H0: 2 = 0 against HA: 2 ≠ 0

2. We could develop an overall test

H0: 1 = 0, 2= 0 against HA: 1 ≠ 0 or 2 ≠ 0

1. To test each parameter separately

a)

then

b)

We might use the following test:

ˆ if Reject 1)1(

0 KH

0:against 0: 1)1(

1)1(

0 AHH

0:against 0: 2)2(

2)2(

0 AHH

ˆ if Reject 2)2(

0 KH

then

K is chosen so that the probability of a Type I errorof each test is .

2. To perform an overall test

H0: 1 = 0, 2= 0 against HA: 1 ≠ 0 or 2 ≠ 0

we might use the test

)(22

210

ˆˆ if Reject overallKH

)(overallK is chosen so that the probability of a Type I error is .

1̂ K 1̂

2̂

ˆ2 K

1̂

2̂

1̂ K

1̂

2̂

ˆ2 K

ˆ )(1

multipleK

1̂

2̂

ˆ )(2

multipleK

1̂

2̂

)(22

21

ˆˆ overallK

1̂

2̂

)(22

21

ˆˆ overallK

ˆ )(1

multipleK

ˆ )(2

multipleK

1̂

2̂

)(22

21

ˆˆ overallK

ˆˆ )(2211

ScheffeKcc

ˆˆ )(2211

ScheffeKcc

Post-hoc Tests

Multiple Comparison Tests

Post-hoc Tests

Multiple Comparison Tests

Suppose we have p means

An F-test has revealed that there are significant differences amongst the p means

We want to perform an analysis to determine precisely where the differences exist.

pxxx ,,2,1

Tukey’s Multiple Comparison Test

Let

Tukey's Critical Differences

Two means are declared significant if they differ by more than this amount.

n

MS

n

s Error

n

MSq

n

sqD Error

ixdenote the standard error of each

q = the tabled value for Tukey’s studentized range p = no. of means, = df for Error

Scheffe’s Multiple Comparison Test

Scheffe's Critical Differences (for Linear contrasts)

A linear contrast is declared significant if it exceeds this amount.

222

21,11 paaa

n

spFpS

222

21,11 p

Error aaan

MSpFp

= the tabled value for F distribution (p -1 = df for comparing p means, = df for Error)

,1pF

Scheffe's Critical Differences (for comparing two means)

ji xxL

2,11n

MSpFpS Error

Two means are declared significant if they differ by more than this amount.

Table 5: Critical Values for the multiple range Test , and the F-distribution

q.05 q.01 F.05 F.01

Length 3.84 4.80 2.92 4.51 Temp,Thickness,Dry 4.60 5.54 2.33 3.30

Table 6: Tukey's and Scheffe's Critical Differences Tukeys Scheffés

= .05 = .01 = .05 = .01 Length 1.59 1.99 2.05 2.16 Temp, Thickness, Dry 3.81 4.59 4.74 5.64

S u m m a r y o f t h e C o m p a r i s o n s : C o m p a r i s o n o f t h e m e a n f i l m L u s t r e f o r d i f f e r e n t c o m b i n a t i o n s o f l e v e l s o f t h e t h r e e f a c t o r s - T h i c k n e s s , T e m p e r a t u r e a n d D r y i n g p r o c e d u r e ( U s i n g T u k e y ' s c r i t i c a l D i f f e r e n c e w i t h = 0 . 0 1 ) :

C o m p a r i s o n o f t h e m e a n f i l m L u s t r e f o r d i f f e r e n t l e v e l s o f L e n g t h o f d r y i n g T i m e ( U s i n g T u k e y ' s c r i t i c a l D i f f e r e n c e w i t h = 0 . 0 1 ) :

4.25 4.44 5.66 15.45 17.43 28.76 29.95

4.25 0.19 1.41 11.2 13.18 24.51 25.74.44 1.22 11.01 12.99 24.32 25.515.66 9.79 11.77 23.1 24.29

15.45 1.98 13.31 14.517.43 11.33 12.5228.76 1.1929.95

Table of differences in means

Underlined groups have no significant differences

There are many multiple (post hoc) comparison procedures

1. Tukey’s

2. Scheffe’,

3. Duncan’s Multiple Range

4. Neumann-Keuls

etc

Considerable controversy:“I have not included the multiple comparison methods of D.B. Duncan because I have been unable to understand their justification” H. Scheffe, Analysis of Variance