Statistical Analysis

transcript

Professor Lynne StokesDepartment of Statistical Science

Lecture 6Solving Normal Equations andEstimating Estimable Model

Parameters

Regression Models

Residuals

Least Squares

Solution: Solve the Normal Equations

)X-(y)X-(y)( Minimize toˆ Choose

yXˆXX

ModeleXy

Sumof SquaredResiduals

Regression Solution

Under usual assumptions, the least squares estimator is Unique Unbiased Minimum Variance Consistent Known sampling distribution Universally used

yXXXˆ 1

Analysis of Completely Randomized Designs

Fixed Factor EffectsFactor levels specifically chosenInferences desired only on the factor levels

included in the experimentSystematic, repeatable changes in the mean response

Flow Rate Experiment

Filter Flow RatesA 0.233 0.197 0.259 0.244B 0.259 0.258 0.343 0.305C 0.183 0.284 0.264 0.258D 0.233 0.328 0.267 0.269 MGH Fig 6.1

Fixedor

Random?

A B C D

AverageFlowRate

Conclusion ?

Filter Type

0.35 Filter EffectsA -0.028B 0.030C -0.014D 0.013

Statistical Model for Single-Factor, Fixed Effects Experiments

Response OverallMean

(Constant)

MainEffect

for Level

Modelyij = + i + eij i = 1, ..., a; j = 1, ..., ri

i: Effect of Level i = change in the mean response

Statistical Model for Single-Factor, Fixed Effects Experiments

Cell Means Model

yij = i + eij i = 1, ..., a; j = 1, ..., ri

yˆ ii

Effects Modelyij = + i + eij i = 1, ..., a; j = 1, ..., ri

Fixed Effects ModelsConnection: i = + i

yy~ , yˆ ii

Solving the Normal EquationsSingle-Factor, Balanced Experimentyij = + i + eij i = 1, ..., a j = 1, ..., r n = ar

Matrix Formulation

y = X + e

y = (y11 y12 ... y1r ... ya1 ya2 ... yar)’

]1I : )11[(

1001............01010011

=X raar

Solving the Normal Equations

Residuals

Least Squares

Solution: Solve the Normal Equations

)X-(y)X-(y)( Minimize to~ Choose

X X = X y~

Normal Equations

n r r rr rr r

.........

... ... ... ......

0 00 0

Normal Equations

Linearly Dependent

a + 1 Parameters, a Linearly Independent Equations

y~r + ~r...

y ~r+ ~ry ~r+~ry~r + ... + ~r ~r+~n

Infinite Number of Solutions

Normal Equations

One Solution

0 01 1

y~r + ~r...

y ~r+ ~ry ~r+~ry ~n

iii y~~ˆ

Normal Equations

Another Solution

ii y~0~0

iii y~~ˆ

y~r ...

y ~r y ~r y~r ... ~r ~r

i = 1, . . . , a - 1

0 yy ya

Another Solution

Normal Equations

iii y~~ˆ

)1a(1-a

y ~r+ ~r...

y ~r+ ~ry ~r+~ry~r + ... + ~r ~r+~n

Solutions are not estimates Estimable Functions

All solutions provide one unique estimator Estimators are unbiased

All solutions to the normal equationsproduce the same estimates of “estimable functions”

of the model means

Two-Factor, Balanced Experiment

Matrix Formulationy = X + e

yijk = ij + eijk = + i + j + ()ij + eijk i = 1, ..., aj = 1, ..., bk = 1, ..., r

X = [ 1 : XA : XB : XAB ]

1a1b11ab

n = abr

Two-Factor, Balanced Experiment

Matrix Formulationy = X + e

yijk = ij + eijk = + i + j + ()ij + eijk i = 1, ..., aj = 1, ..., bk = 1, ..., r

X = [ 1 : XA : XB : XAB ]

Numberof

Parameters1 + a + b + ab

rank( X ) < 1+a+b+ab

n = abr

1a1b11ab

X X = X y~

Normal Equations

~...~~

r...0...00..................0...0...brbrr...ar...brn

Matrix Linear Dependencies One Solution 1n None XA 1 : Columns of XA Sum to 1n a= 0

Eliminates a columnFrom XA

a – 1 “degrees of freedom”

Matrix Linear Dependencies One Solution 1n None XA 1 : Columns Sum of XA to 1n a= 0 XB 1 : Columns Sum of XB to 1n b = 0

Eliminates a columnFrom XB

b – 1 “degrees of freedom”

Matrix Linear Dependencies One Solution 1n None XA 1 : Columns sum to 1n a= 0 XB 1 : Columns sum to 1n b = 0 XAB 1 + (a - 1) + (b - 1) :

Sum over all columns = 1n ()ab = 0Eliminates a column

from XAB

Matrix Linear Dependencies One Solution 1n None XA 1 : Columns Sum to 1n a= 0 XB 1 : Columns Sum to 1n b = 0 XAB 1 + (a - 1) + (b - 1) :

Sum over all columns = 1n ()ab = 0 Sums of columns over each i = 1,...,a-1 & each j = 1,...,b-1 ()ib = 0 equal one of the remaining i=1,...,a-1 columns of XA and XB ()aj = 0

j=1,...,b-1(a – 1)(b – 1) “degrees of freedom”

Matrix Linear Dependencies One Solution XA 1 : Columns sum to 1n a= 0 XB 1 : Columns sum to 1n b = 0 XAB 1 + (a - 1) + (b - 1) :

Sum over all columns = 1n ()ab = 0 Sums of columns over each i = 1,...,a-1 & each j = 1,...,b-1 ()ib = 0 equal one of the remaining i=1,...,a-1 columns of XA and XB ()aj = 0

j=1,...,b-1Constraints : 1 + 1 + {1 + (a - 1) + (b - 1)} = a + b + 1Degrees of Freedom : (1 + a + b + ab) - (a + b + 1)

= ab = 1 + (a - 1) + (b - 1) + (a - 1)(b - 1)

~~ ~~~ ~~

ij i j

i = 1, . . . , a - 10

j = 1, . . . , b - 1

( ) i a ; j b

( ) 0 i = a or j = bij

j i, yyyy)(

b , ... 1, = j yy~a , ... 1, = i yy~

jiijij

Another Solution

Filter Flow RatesA 0.233 0.197 0.259 0.244B 0.259 0.258 0.343 0.305C 0.183 0.284 0.264 0.258D 0.233 0.328 0.267 0.269 MGH Fig 6.1

Fixedor

Random?

Quantifying Factor Effects

EffectChange in average response

due to changes in factor levels

1y 2y 3y ky y

1 2 3 k. . .

Factor Level

Average

Overall Average

Effect of Level t : ty y-

EffectChange in average response

due to changes in factor levels

1 2 3 k. . .Factor Level

Average

Overall Average

Effect of changing from Level s to Level t :

y -y =)y-y( - )yy(

1y 2y 3y ky y. . .

Main Effects for Factor Ay i y Change in average response due to

changes in the levels of Factor A

Main Effects for Factor By j y Change in average response due to

changes in the levels of Factor B

Interaction Effects for Factors A & B(y - y ) - (yij j i y )

Effect of Level i ofFactor A at Level jof Factor B

Effect of Level iof Factor A

Main Effects for Factor A

Main Effects for Factor B

Interaction Effects for Factors A & B

(y - y ) - (y

y - y - yij

Change in average response due tochanges in the levels of Factor A

Change in average response due tochanges in the levels of Factor B

Change in average response duejoint changes in Factors A & Bin excess of changes in the maineffects

Two-Level Factors

Common to Use y - y2 1

Note: If r1 = r2 , y y1 = - (y y2 )

Effect of Level 1:Effect of Level 2:

Factors at Two Levels

Most common choice for designs involving many factors

Many efficient fractional factorial and screening designs available

Can use p two-level factors in place of factors whose number of levels is 2p

Calculating Two-Level Factor Effects: Pilot Plant Study

Main EffectDifference between the average responses at thetwo levels

M(Temp) = Average @ 180o - Average @ 160o

= 75.8 - 52.8 = 23.0

M(Conc) = Average @ 40% - Average @ 20%= 61.8 - 66.8 = -5.0

M(Catalyst) = Average @ C2 - Average @ C1= 65.0 - 63.5 = 1.5 BHH Section 10.3

MGH Section 5.3

Calculating Two-Level Factor Effects

Two-Factor Interaction EffectHalf the difference between the main effects of onefactor at each level of the second factor

M(Conc @ C2) = Average @ 40%&C2 - Average @ 20%&C2= 62.5 - 67.5 = -5.0

M(Conc @ C1) = Average @ 40%&C1 - Average @ 20%&C1= 61.0 - 66.0 = -5.0

I(Conc,Cat) = {M(Conc @ C2) - M(Conc @ C1)} / 2= 0

BHH Section 10.4MGH Section 5.3

Calculating Two-Level Factor EffectsTwo-Factor Interaction Effect

Half the difference between the main effects of onefactor at each level of the second factor

M(Temp @ C2) = Average @ 180o&C2 - Average @ 160o&C2= 81.5 - 48.5 = 33.0

M(Temp @ C1) = Average @ 180o&C1 - Average @ 160o&C1= 70.0 - 57.0 = 13.0

I(Temp,Cat) = {M(Temp @ C2) - M(Temp @ C1)} / 2= (33.0 - 13.0) / 2 = 10.0

Cell Means and Effects Model Estimability

Three-Factor Balanced Experiment

yijkl = ijk + eijkl i = 1 , ... , a ; j = 1 , ... , b ;k = 1, ... , c ; l = 1 , ... , r

ijk = + i + j + k + ()ij + ()ik + ()jk + ()ijk

Cell Means Models: Estimable Functions

ijk ijky

All cell means are estimable

ijk ijky

All linear combinations of cell means are estimable

c yijk ijk

ijk ijk

(includes , , etc. ) i ij

Does not dependon parameter constraints

ijk ijky

Some linear combinations of cell means are uninterpretable

Some linear combinations of cell means are essential

Cell Means and Effects Models

)()()()(

Imposing parameter constraintssimplifies the relationships;

makes the parameters more interpretable

Parameter Equivalence:Effects Representation & Cell Means Model

Parameter constraints i

ij . . . = = . . . = ( ) 0ijk

ijk( )

Means and mean effects

ij i j ij

ij ij i j

ij j i

( ) ( )

Contrasts

1jjj a with a

Zero toSum tsCoefficien whoseParameters ofn CombinatioLinear A Contrast

Contrasts often eliminatenuisance parameters; e.g.,

Contrasts

i i i i

ij i j ij

ij il kj kl ij il kj kl

( ) ( ) ( ) ( )

Main Effects

Interactions

4 2 2cr ijij

ij il kj klijkl

( ) {( ) ( ) ( ) ( ) }

Statistical Analysis

Documents