Statistical Analysis

1

Statistical Analysis

Professor Lynne StokesDepartment of Statistical Science

Lecture 6Solving Normal Equations andEstimating Estimable Model

Parameters

2

Regression Models

Residuals

Least Squares

Solution: Solve the Normal Equations

ˆXyr

)X-(y)X-(y)( Minimize toˆ Choose

yXˆXX

ModeleXy

Sumof SquaredResiduals

3

Regression Solution

Under usual assumptions, the least squares estimator is Unique Unbiased Minimum Variance Consistent Known sampling distribution Universally used

yXXXˆ 1

4

Analysis of Completely Randomized Designs

Fixed Factor EffectsFactor levels specifically chosenInferences desired only on the factor levels

included in the experimentSystematic, repeatable changes in the mean response

5

Flow Rate Experiment

Filter Flow RatesA 0.233 0.197 0.259 0.244B 0.259 0.258 0.343 0.305C 0.183 0.284 0.264 0.258D 0.233 0.328 0.267 0.269 MGH Fig 6.1

Fixedor

Random?

6


A B C D

0.30

0.25

0.20

AverageFlowRate

Conclusion ?

Filter Type

0.35 Filter EffectsA -0.028B 0.030C -0.014D 0.013

7

Statistical Model for Single-Factor, Fixed Effects Experiments

Response OverallMean

(Constant)

MainEffect

for Level

i

Error

Modelyij = + i + eij i = 1, ..., a; j = 1, ..., ri

i: Effect of Level i = change in the mean response

8

Statistical Model for Single-Factor, Fixed Effects Experiments

Cell Means Model

yij = i + eij i = 1, ..., a; j = 1, ..., ri

yˆ ii

Effects Modelyij = + i + eij i = 1, ..., a; j = 1, ..., ri

Fixed Effects ModelsConnection: i = + i

yy~ , yˆ ii

9

Solving the Normal EquationsSingle-Factor, Balanced Experimentyij = + i + eij i = 1, ..., a j = 1, ..., r n = ar

Matrix Formulation

y = X + e

y = (y11 y12 ... y1r ... ya1 ya2 ... yar)’

]1I : )11[(

1001............01010011

=X raar

rrrr

rrrr

rrrr

1

...

a

10

Solving the Normal Equations

Residuals

Least Squares

Solution: Solve the Normal Equations

~Xyr

)X-(y)X-(y)( Minimize to~ Choose

yX~XX

11


X X = X y~

Normal Equations

n r r rr rr r

r r

yyy

ya a

.........

... ... ... ......

...

~~~

...~

...

0 00 0

0 0

12

12

Check

12


Normal Equations

Check

Linearly Dependent

a + 1 Parameters, a Linearly Independent Equations

aa

22

11

a21

y~r + ~r...

y ~r+ ~ry ~r+~ry~r + ... + ~r ~r+~n

Infinite Number of Solutions

13


Normal Equations

One Solution

ii

a

ii

a

i iy

0 01 1

= yy

~

~~

aa

22

11

y~r + ~r...

y ~r+ ~ry ~r+~ry ~n

iii y~~ˆ

14


Normal Equations

Another Solution

ii y~0~0

iii y~~ˆ

aa

22

11

a21

y~r ...

y ~r y ~r y~r ... ~r ~r

15


~ ~~

a

i = 1, . . . , a - 1

0 yy ya

i i a

Another Solution

Normal Equations

iii y~~ˆ

)1a(1-a

22

11

1-a21

y ~r+ ~r...

y ~r+ ~ry ~r+~ry~r + ... + ~r ~r+~n

16


Solutions are not estimates Estimable Functions

All solutions provide one unique estimator Estimators are unbiased

All solutions to the normal equationsproduce the same estimates of “estimable functions”

of the model means

17


Two-Factor, Balanced Experiment

Matrix Formulationy = X + e

yijk = ij + eijk = + i + j + ()ij + eijk i = 1, ..., aj = 1, ..., bk = 1, ..., r

X = [ 1 : XA : XB : XAB ]

1a1b11ab

n = abr

18


Two-Factor, Balanced Experiment

Matrix Formulationy = X + e

yijk = ij + eijk = + i + j + ()ij + eijk i = 1, ..., aj = 1, ..., bk = 1, ..., r

X = [ 1 : XA : XB : XAB ]

Numberof

Parameters1 + a + b + ab

rank( X ) < 1+a+b+ab

n = abr

1a1b11ab

19


X X = X y~

Normal Equations

ab

1

1

ab

1

1

y...

y...

yy

...

~...~~

r...0...00..................0...0...brbrr...ar...brn

Check

20


Matrix Linear Dependencies One Solution 1n None XA 1 : Columns of XA Sum to 1n a= 0

Eliminates a columnFrom XA

a – 1 “degrees of freedom”

21


Matrix Linear Dependencies One Solution 1n None XA 1 : Columns Sum of XA to 1n a= 0 XB 1 : Columns Sum of XB to 1n b = 0

Eliminates a columnFrom XB

b – 1 “degrees of freedom”

22


Matrix Linear Dependencies One Solution 1n None XA 1 : Columns sum to 1n a= 0 XB 1 : Columns sum to 1n b = 0 XAB 1 + (a - 1) + (b - 1) :

Sum over all columns = 1n ()ab = 0Eliminates a column

from XAB

23


Matrix Linear Dependencies One Solution 1n None XA 1 : Columns Sum to 1n a= 0 XB 1 : Columns Sum to 1n b = 0 XAB 1 + (a - 1) + (b - 1) :

Sum over all columns = 1n ()ab = 0 Sums of columns over each i = 1,...,a-1 & each j = 1,...,b-1 ()ib = 0 equal one of the remaining i=1,...,a-1 columns of XA and XB ()aj = 0

j=1,...,b-1(a – 1)(b – 1) “degrees of freedom”

24


Matrix Linear Dependencies One Solution XA 1 : Columns sum to 1n a= 0 XB 1 : Columns sum to 1n b = 0 XAB 1 + (a - 1) + (b - 1) :

Sum over all columns = 1n ()ab = 0 Sums of columns over each i = 1,...,a-1 & each j = 1,...,b-1 ()ib = 0 equal one of the remaining i=1,...,a-1 columns of XA and XB ()aj = 0

j=1,...,b-1Constraints : 1 + 1 + {1 + (a - 1) + (b - 1)} = a + b + 1Degrees of Freedom : (1 + a + b + ab) - (a + b + 1)

= ab = 1 + (a - 1) + (b - 1) + (a - 1)(b - 1)

25


~~ ~~~ ~~

~ ~ ~

yy

y

y

ab

i ib

a

j aj

b

ij i j

i = 1, . . . , a - 10

j = 1, . . . , b - 1

0

( ) i a ; j b

( ) 0 i = a or j = bij

ij

Check

26


j i, yyyy)(

b , ... 1, = j yy~a , ... 1, = i yy~

y~

jiijij

jj

ii

Check

Another Solution

27


Filter Flow RatesA 0.233 0.197 0.259 0.244B 0.259 0.258 0.343 0.305C 0.183 0.284 0.264 0.258D 0.233 0.328 0.267 0.269 MGH Fig 6.1

Fixedor

Random?

28

Quantifying Factor Effects

EffectChange in average response

due to changes in factor levels

1y 2y 3y ky y

1 2 3 k. . .

. . .

Factor Level

Average

Overall Average

Effect of Level t : ty y-

29


EffectChange in average response

due to changes in factor levels

1 2 3 k. . .Factor Level

Average

Overall Average

Effect of changing from Level s to Level t :

st

st

y -y =)y-y( - )yy(

1y 2y 3y ky y. . .

30


Main Effects for Factor Ay i y Change in average response due to

changes in the levels of Factor A

Main Effects for Factor By j y Change in average response due to

changes in the levels of Factor B

Interaction Effects for Factors A & B(y - y ) - (yij j i y )

Effect of Level i ofFactor A at Level jof Factor B

Effect of Level iof Factor A

31


Main Effects for Factor A

Main Effects for Factor B

Interaction Effects for Factors A & B

y i y

y j y

(y - y ) - (y

y - y - yij

ij i

j i

j

y

y

)

Change in average response due tochanges in the levels of Factor A

Change in average response due tochanges in the levels of Factor B

Change in average response duejoint changes in Factors A & Bin excess of changes in the maineffects

32

Two-Level Factors

Common to Use y - y2 1

Note: If r1 = r2 , y y1 = - (y y2 )

Effect of Level 1:Effect of Level 2:

y y1

y y2

33

Factors at Two Levels

Most common choice for designs involving many factors

Many efficient fractional factorial and screening designs available

Can use p two-level factors in place of factors whose number of levels is 2p

34

Calculating Two-Level Factor Effects: Pilot Plant Study

Main EffectDifference between the average responses at thetwo levels

M(Temp) = Average @ 180o - Average @ 160o

= 75.8 - 52.8 = 23.0

M(Conc) = Average @ 40% - Average @ 20%= 61.8 - 66.8 = -5.0

M(Catalyst) = Average @ C2 - Average @ C1= 65.0 - 63.5 = 1.5 BHH Section 10.3

MGH Section 5.3

35

Calculating Two-Level Factor Effects

Two-Factor Interaction EffectHalf the difference between the main effects of onefactor at each level of the second factor

M(Conc @ C2) = Average @ 40%&C2 - Average @ 20%&C2= 62.5 - 67.5 = -5.0

M(Conc @ C1) = Average @ 40%&C1 - Average @ 20%&C1= 61.0 - 66.0 = -5.0

I(Conc,Cat) = {M(Conc @ C2) - M(Conc @ C1)} / 2= 0

BHH Section 10.4MGH Section 5.3

36

Calculating Two-Level Factor EffectsTwo-Factor Interaction Effect

Half the difference between the main effects of onefactor at each level of the second factor

M(Temp @ C2) = Average @ 180o&C2 - Average @ 160o&C2= 81.5 - 48.5 = 33.0

M(Temp @ C1) = Average @ 180o&C1 - Average @ 160o&C1= 70.0 - 57.0 = 13.0

I(Temp,Cat) = {M(Temp @ C2) - M(Temp @ C1)} / 2= (33.0 - 13.0) / 2 = 10.0

37

Cell Means and Effects Model Estimability

Three-Factor Balanced Experiment

yijkl = ijk + eijkl i = 1 , ... , a ; j = 1 , ... , b ;k = 1, ... , c ; l = 1 , ... , r

ijk = + i + j + k + ()ij + ()ik + ()jk + ()ijk

38

Cell Means Models: Estimable Functions

ijk ijky

All cell means are estimable

39


ijk ijky


All linear combinations of cell means are estimable

c

c yijk ijk

ijk ijk

(includes , , etc. ) i ij

Does not dependon parameter constraints

40


ijk ijky


Some linear combinations of cell means are uninterpretable

1 23

1 2

Some linear combinations of cell means are essential

41

Cell Means and Effects Models

i

iiii

)()()()(

Imposing parameter constraintssimplifies the relationships;

makes the parameters more interpretable

42

Parameter Equivalence:Effects Representation & Cell Means Model

Parameter constraints i

iij

ij . . . = = . . . = ( ) 0ijk

ijk( )

Means and mean effects

i i

ij i j ij

( )

i i

ij ij i j

ij j i

( )

( ) ( )

43

Contrasts

k

1jj

k

1jjj a with a

Zero toSum tsCoefficien whoseParameters ofn CombinatioLinear A Contrast

Contrasts often eliminatenuisance parameters; e.g.,

44

Contrasts

i i i i

ij i j ij

ij il kj kl ij il kj kl

( )

( ) ( ) ( ) ( )

Main Effects

Interactions

4 2 2cr ijij

ij il kj klijkl

( ) {( ) ( ) ( ) ( ) }

Show

Date post:	06-Feb-2016
Category:	Documents
Upload:	jorryn
View:	26 times
Download:	0 times

Statistical Analysis

Documents