method of least aquares.ppt

8/10/2019 method of least aquares.ppt

1/56

1

Introduction to two-levelexperimentation

1. Two-factor two-level experiments

Let us study the impact of two factors on a single variable.Example: (direct mail offering study)

Variable: response rateTwo factors: envelope size, postageSimple situation: each factor at two fixed levels

Low level High level

Factor A: envelope #10 9x12Factor B: postage 3 rd class 1 st class

It does not matter which level of a factor is labeled low andwhich labeled high; low and high are arbitrary labels. But once

this decision is made, the labels must be retained.


2/56

2

There are four possible experimentalcombinations. These are called treatments,or treatment combinations:

a 0b 0 A at low level, B at low levela 0b 1 A at low level, B at high levela 1b o A at high level, B at low levela 1b 1 A at high level, B at high level


3/56

3

For now we will imagine that each of the fourtreatments is run once; the experiment will becalled a two-factor two-level factorial design(without replication).

For each treatment the response rate is computed.

The four observed response rates are called yields or responses . The symbols used for treatments are

also used for yields (or yield m eans fo r wi threpl icat ions case ). Thus, a 0b 0 also represents theyield when the treatment a 0b 0 is run.


4/56

4

2. Estimating effects in two-factor

two-level experimentsEstimate of the effect of Aa 1b 1 - a 0b 1 estimate of effect of A at high Ba 1b 0 - a 0b 0 estimate of effect of A at low B

sum/2 estimate of effect of A over all B

Estimate of the effect of Ba 1b 1 - a 1b 0 estimate of effect of B at high Aa 0b 1 - a 0b 0 estimate of effect of B at low A

sum/2 estimate of effect of B over all A


5/56

5

Estimate the interaction of A and B

a 1b 1 - a 0b 1 estimate of effect of A at high Ba 1b 0 - a 0b 0 estimate of effect of A at low Bdifference/2 estimate of the effect of B on the effect of A

Called th e in teract ion of A and B

a 1b 1 - a 1b 0 estimate of effect of B at high Aa 0b 1 - a 0b 0 estimate of effect of B at low A

difference/2 estimate of the effect of A on the effect of BCalled th e in teract ion of B and A


6/56

6

Note that the two differences in theinteraction estimate are identical; bydefinition, the interaction of A and Bis the same as the interaction of B

and A. In a given experiment one ofthe two literary statements ofinteraction may be preferred by the

experimenter to the other; but bothhave the same numerical value.


7/56


8/56

8

Remarks on interactionMany people feel the need for experiments which

will reveal the effect, on the variable under study, of factorsacting jointly. This is what we have called in teract ion . Thesimple experimental design discussed here provides a wayof estimating such interaction, with the latter defined in away which corresponds to what many scientists and

managers have in mind when they think of interaction.

It is us eful to note that in teract ion w as not inv entedby s ta t is t ic ians . It is a jo int effect exis t ing, of tenpro m inent ly, in the real wo rld .

Statisticians have (wonderfully enough!!) providedways and means to measure it.


9/56

9

4. Symbolism and languageA is called a main effect. Our estimate of A is

often simply written A.B is called a main effect. Our estimate of B is

often simply written B.AB is called an interaction effect. Our estimate of

AB is often simply written AB.So the same letter is used, generally without

confusion, to describe the factor, to describe itseffect, and to describe our estimate of its effect.

Keep in mind that it is only for economy in speakingand writing that we sometimes speak/write about aneffect rather than an estimate of the effect." Weshould always remember that all quantities formedfrom the yields are, OF COURSE, estimates.


10/56

10

5. Table of signsThe following table is useful:

A B ABa 0b 0 - - +a 0b 1 - + -a 1b 0 + - -a 1b 1 + + +

Notice than in estimating A, the two treatments with A athigh level are compared to the two treatments with A at low level.Similarly B. This is, of course, logical.

Note also that the signs of treatments in the estimate ofAB are the products of the signs of the corresponding treatmentsof A and B.

Note, finally, that in each estimate, plus and minus signsare equal in number.

Effect = Ave of + Ave of - .


11/56

11

6B

low high

10 1213 15

lowhigh

A

Example 1 Blow high

10 1515 15

lowhigh

A

Example 2

Blow high

10 1313 10

lowhigh

A

Example 3 B

low high

12 1212 12

lowhigh

A

Example 4

A B AB1 3 2 02 2.5 2.5 -2.53 0 0 -3

4 0 0 0

Discussion of examples:Notice that in Examples 2 & 3interaction is as large as orlarger than main effects.

?


12/56

12

Change of scale, by multiplying each yield

by a constant (3 inches 3x2.54 cm),

multiplies each estimate by the constant

but does not affect the relationship of

estimates to each other. Addition of a

constant to each yield does not affect the

estimates. The numerical magnitude of

estimates is not important here; it is their

relationship to each other.


13/56

14

Earlier we formed:

estimate of 2A

estimate of 2B

estimate of 2AB

1 a b ab

-1

-1

1

1

-1

-1

-1

1

-1

1

1

1

A = (-1+a-b+ab)/2

B = (-1-a+b+ab)/2

AB = (1-a-b+ab)/2


14/56

15

Which for present purposes we replace by:

1 a b ab Z12

1

2

12

.

.

.

.

.

.

.

.

12

1

2

12

12

1

2

12

12

1

2

12

-

-

-

-

--

Now we can see that these coefficients of the threecontrasts are orthogonal and thus A, B and AB constituteorthogonal estimates and their SSs can be foundaccordingly. (For example, SSA = RxZ A^2.)

A

B

AB


15/56

16

8. Three factors each at two levelsThe dependent variable is response rate of a direct mailoffering.

low highA postage 3rd class 1st classB price $9.95 $12.95

C envelope size #10 9 x 12

Treatments (also yields) (a) old notation (b) new notation.(a) a 0b 0c 0 a 0b 0c 1 a 0b 1c 0 a 0b 1c 1 a 1b 0c 0 a 1b 0c 1 a 1b 1c 0 a 1b 1c 1(b) 1 c b bc a ac ab abc

Yates (standard) order : (add factors one after one)1 a b ab c ac bc abc


16/56

17

9. Estimating effects in three-factor

two-level designsEstimate of A

(1) a - 1 estimate of A, with B low and C low(2) ab - b estimate of A, with B high and C low(3) ac - c estimate of A, with B low and C high(4) abc - bc estimate of A, with B high and C high

= (a+ab+ac+abc-1-b-c-bc)/4,

= (-1+a-b+ab-c+ac-bc+abc)/4,(in Yates order)


17/56

18

Estimate of AB (the effect of B on the effect of A)

effect of A with B high - effect of A with B low, all at C high

plus

effect of A with B high - effect of A with B low, all at C low

Note that interaction are averages. Just as ourestimate of A is an average of response to A over allB and all C, so our estimate of AB is an average

response to AB over all C.AB = {[(4)-(3)] + [(2) - (1)]}/4

= {1-a-b+ab+c-acbc+abc)/4, in Yates order.


18/56

19

Estimate of ABC (the effect of C on AB)

interaction of A and B, at C high

minus

interaction of A and B at C low

ABC = {[(4) - (3)] - [(2) - (1)]}/4

= (-1+a+b-ab+c-acbc+abc)/4, in Yates order.


19/56

20

This is our first encounter with a three-factorinteraction. It measures the impact on the responserate of interaction AB as C (envelope size) goes from#10 to 9 x 12. Or, it measures the impact on responserate of interaction AC as B (price) goes from $9.95 to

$12.95. Or, finally, it measures the impact on theresponse rate of interaction BC as A (postage) goesfrom 3rd class to 1st class.

As with two-factor two-level factorial designs,the formation of estimates in three-factor two-levelfactorial designs can be summarized in a table:


20/56

21

A B AB C AC BC ABC

1 - - + - + + -a + - - - - + +

b - + - - + - +

ab + + + - - - -

c - - + + - - +

ac + - - + + - -

bc - + - + - + -

abc + + + + + + +

Plus-Minus Table


21/56

22

10. DATA ANALYSIS

1 a b ab c ac bc abc.062 .074 .010 .020 .057 .082 .024 .027

A = main effect of postage = .0125B = main effect of price = -.0485AB = interaction of A and B = -.0060C = main effect of envelope size = .0060AC = interaction of A and C = .0015BC = interaction of B and C = .0045ABC = interaction of A, B, and C = -.0050

NOTE: ac = largest yield; AC = smallest effect.


22/56

23

We describe several of these estimates, though onlater analysis of this example, taking into accountthe unreliability of estimates based on a smallnumber (eight) of data values, some estimates mayturn out to be so small in magnitude as not to

reject the null hypothesis that the correspondingtrue effect is zero. The largest estimate is -.0485,the estimate of B; an increase in price, from $9.95to $12.95, is associated with a decline in response

rate. The interaction AB = -.0060; an increase inprice from $9.95 to $12.95 reduces the effect of A,whatever it is (A = .0125), on response rate. Orequivalently,


23/56

24

an increase in postage from 3rd class to 1st classreduces (makes more negative) the alreadynegative effect (B = -.0485) of price. Finally, ABC =-.0050. Going from #10 to 9 x 12 envelope, thenegative interaction effect AB on response rate

becomes even more negative. Or, going from lowto high price, the positive interaction effect AC isreduced. Or, going from low to high postage, thepositive interaction effect (BC) is reduced. All three

descriptions of ABC have the same numericalvalue, but the direct marketer would select one ofthem, and then say it better!


24/56

25

11. Number and kinds of effects

We introduce the notation 2 k. This means a k-factor design with each factor at two levels. The

number of treatments in an unreplicated 2k

design is 2 k.

The following table shows the number of eachkind of effect for each of the six two-leveldesigns shown across the top.


25/56

26

22 2 3 2 4 2 5 26 27

2 3 4 5 6 71 3 6 10 15 21

1 4 10 20 351 5 15 35

1 6 211 7

1

main effect2 factor interaction3 factor interaction4 factor interaction5 factor interaction6 factor interaction7 factor interaction

In a 2 k design the number of r-factor effects is C k = k!/[r!(k - r)!]r

3 7 15 31 63 127


26/56

27

Notice that the total number of effects estimated in anydesign is always one fewer than the number oftreatments:

One need not repeat the earlier logic to determine theforms of estimates in 2 k designs for higher values of k.

A table going up to 2 5 is on P.265, Table 9.4.

in a 2 2 design there are 2 2=4 treatments; we estimate 2 2-1=3 effects,

in a 2 3 design there are 2 3=8 treatments; we estimate 2 3-1=7 effects.


27/56

28

Exercise:

Write down the plus-minus table for the24 design.


28/56

29

Note: for 2k designs with replications

All terms (1, a, b, ) are treatment combinations or

yield means of the treatment combinations.

In the previous example, yields are response rates andthus (1, a, b, ) are average response rates for the

corresponding treatment combinations.


29/56

30

12. Yates forward algorithm 1. Applied to Complete Factorials (Yates, 1937)

A systematic method of calculating estimates

of effects. For complete factorials firstarrange the yields in Yates (standard) order.Addition, then subtraction of adjacent yields.The addition and subtraction operations arerepeated until 2 k terms appear in each line:for a 2 k there will be k columns ofcalculations.


30/56

31

Example: 2 3 Yield 1st. Column 2nd. Column 3rd. Column

1

a

b

ab

c

acbc

abc

a+1

ab+b

ac+c

abc+bc

a-1

ab - bac - c

abc -bc

ab+b +a+1

abc+bc+ac+ c

ab-b+a-1

abc-bc+ ac- c

ab+b-a-1

abc+bc- ac-cab-b-a + 1

abc-bc-ac+ c

abc+ bc+ac+ c+ab +b+a+1

abc - bc+ac - c+ab - b+a -1

abc+ bc- ac - c+ab+ b -a -1

abc - bc- ac+ c+ab - b -a+1

abc+ bc+ac+ c -ab - b -a-1

abc - bc+ac - c-ab+ b -a+1abc+ bc- ac - c-ab- b+a+1

abc - bc- ac+ c-ab+ b+a-1

Checking in our 2 3 table of signs, entries in the third columnestimate, respectively,

=1) A B AB C AC BC ABCNote the line-by-line correspondence between yields (lower case lettersin the left column of the table) and factors estimated (upper case lettersdirectly above). Treatments and estimates of effects are in Yates order.


31/56

32

Yates Forward AlgorithmEXAMPLE:

23 already used.

1 .062 .136 .166 .356 -

a .074 .030 .190 .050 estimate of 4Ab .010 .139 .022 -.194 estimate of 4Bab .020 .051 .028 -.024 estimate of 4ABc .057 .012 -.106 .024 estimate of 4Cac .082 .010 -.088 .006 estimate of 4AC

bc .024 .025 -.002 .018 estimate of 4BCabc .027 .003 -.022 -.020 estimate of 4ABC

Again, note the line-by-line correspondence between treatmentsand estimates; both are in Yates order.

Yield 1 st Col 2 nd Col 3 rd ColTr.


32/56

33

2k

1 a b ab c ac bc abc

.062 .074 .010 .020 .057 .082 .024 .027

.062 1.00 1.00 1.00

.074 2.00 1.00 1.00

.010 1.00 2.00 1.00

.020 2.00 2.00 1.00

.057 1.00 1.00 2.00

.082 2.00 1.00 2.00

.024 1.00 2.00 2.00

.027 2.00 2.00 2.00

SPSS is not oriented toward providing output in a form traditionally associated

with tw o-level exper imentation . In fact, the ou tpu t does not, literally, provid e the

effects. For examp le, for factor A (imm ediately below , VAR0002), the ou tpu t tells

us th at th e mean is .0507 for high A , .0383 for low A. The d ifference between th e

two values, .0124, is the effect of A. The value resulting from Yates algorithm in

the p rev ious section was .0125 (i.e., 4 A = .05, A = .0125); the d ifferen ce is

rounding error, as the .0383 below is actually .03825, while the .0507 is below

actually .05075.


33/56

34

- - Description of Subpop ulations - -

Summa ries of VAR00001

By levels of A

Variable Value Label Mean Std Dev Cases

For Entire Pop u lation .0445 .0274 8

A 1.00 .0383 .0253 4A 2.00 .0507 .0318 4 Total Cases = 8

- - Description of Subp opu lations - -

Summa ries of VAR00001By levels of B



B 1.00 .0688 .0114 4 B 2.00 .0203 .0074 4

Total Cases = 8

D i ti f S b l ti


34/56

35

- - Description of Subpop ulations - -

Summa ries of VAR00001By levels of C



C 1.00 .0415 .0313 4 C 2.00 .0475 .0274 4 Total Cases = 8

DESIGN EASE A B C1 1 -1.000000 -1.000000 -1.000000 0.0620002 1 -1.000000 -1.000000 1.000000 0.057000

3 1 1.000000 -1.000000 -1.000000 0.0740004 1 -1.000000 1.000000 -1.000000 0.0100005 1 -1.000000 1.000000 1.000000 0.0240006 1 1.000000 1.000000 1.000000 0.0270007 1 1.000000 -1.000000 1.000000 0.0820008 1 1.000000 1.000000 -1.000000 0.020000

(first column is coun ter, second colum n = # rep licates)

INTERCEPT 0.0445000A 0.0062500B -0.0242500C 0.0030000AB -0.0030000AC 0.0007500BC 0.0022500ABC -0.0025000

NOTE: the va lues a re half of what we call th e effects


35/56

36

13. Main effects in the face

of large interactionsSeveral writers have cautioned againstmaking statements about main effectswhen the corresponding interactionsare large; interactions describe thedependence of the impact of one factoron the level of another; in the presenceof large interaction, main effects maynot be meaningful.


36/56

37

EXAMPLE Yields are purchase intent for cigarettes.

low level high levelSex Male FemaleBrand Frontiersman April

The yields are1 = 4.44 s = 2.04 b = 3.50 sb = 4.52The estimates areS = -.69 B = +.77 NP = +1.71.

In the face of such high interaction we now specialize the main

effect of each factor to particular levels of the other factor.Effect of B at high level S = sb - s = 4.52 - 2.04 = 2.48Effect of B at low level S = b - 1 = 3.50 - 4.44 = -.94,which appear to be more valuable for branding strategy thanthe mean (.77) of such disparate numbers.


37/56

38

Note that answers to these specializedquestions are based on fewer than 2 k yields. Inour numerical example, with interaction SBprominent, we have only two of the four yields inour estimate of B at each level of S.

In general we accept high interactionswherever found and seek to explain them; in theprocess of explanation, main effects (and lower-order interactions) may have to be replaced in ourinterest by more meaningful specialized effects.


38/56

39

14. Levels of factorsThe responses or yields are conjec tured to follow the curves

1815

20222930

47

58

t1 t2 t3 t4

Yield

Temperature

p 1

p 2

Compare P effect at (t 1, t 2) vs (t 3, t 4) or others.


39/56

40

At P effect (P 2-P 1)t1 22 - 30 = -8t2 29 - 58 = -29t3 47 - 20 = +27t4 18 - 15 = +3

at (t 1, t 3) ?

P= 27+3 = 15

T= (18-47)+(15-20) = -17PT= 3-27 = -29-(-5) = -12

2

2

2 2

at (t 3, t 4):

P= -8-29 = -18.5

T=(29-22)+(58-30)

= +17.5PT= -29-(-8) = 7-28 = -10.5

2

2

2 2

at (t 1, t 2):


40/56

41

It is only when the conjecturedresponses in the diagram are in fact linear

and parallel that choice of levels isunimportant.

One must acknowledge the essentiallycircular nature of the discussion. Oneneeds to have a good idea of the responsecurves in order to fix the levels of anexperiment which seeks essentially todiscover the response curves. But this kindof circularity characterizes all experimentalscience.


41/56

42

15. Factorial designs vs. designs

varying one factor at a timeExample: Variable: ProfitabilityTwo factors each at two levels:

Time Frame: Past Year FutureMode: Numerical Non-numerical

Vary one factor at a time. Hold Time Frame at

past year and take two observations onprofitability at each mode; we take twoobservations to facilitate comparison with afactorial design. Then we take two more

observations at (Numerical, Future):


42/56

43

Mode

Time Frame

Num.

Non-N.

Past Future


43/56

44

Now consider an unreplicated 2 2 factorial design.

Mode

Time Frame

Num.

Non-N.

Past Future


44/56

45

Comparison of the 2 2 factorial design and the one-factor-at-a-time design:

In the factorial design each estimate of a main effectis based on all four yields. Each estimate has asmuch supporting data (is as reliable) as the

corresponding estimate from the more costly six-yield one-factor-at-a-time design; the latter was ableto use only four of its six yields in each estimate.

In the factorial design, interaction = ( w hatever i t i s ),an effect not estimable from the one-factor-at-a-time design.

a.

b.


45/56

46

In the factorial design, each main effect is estimatedover both levels of the other factor, not at one levelas in the case of the one-factor-at-a-time design; thisincreased generality is usually, though, not always,attractive. If interaction is high, we may, as we have

seen, want the effect of each factor at each level ofthe other factor; this the one-factor-at-a-time designcan provide at two points (the Time Frame effect atNum. and the Mode effect at Past ) better than thefactorial design. But the one-factor-at-a-time designwill not reveal the magnitude of interaction in thefirst place!!

c.


46/56

47

An estimate of the effect of factors other than thetwo factors studied is possible in the 6-yieldexperiment. Thus, the differences in yields at agiven treatment combination cannot be due to TimeFrame, Mode, or their interaction since Time Frame

and Mode were fixed throughout each difference.These differences must be due to other factors.However, a replicated factorial experiment

can, of course, provide such an estimate.

d.


47/56

48

One-factor-at-a-time designs are less vulnerableto missing yields.

The general judgment, particularly in recentyears, is that factorial designs are definitelysuperior to one-factor-at-a-time experimentation.

f.


48/56

49

In a complete 2 5 design, we have 32 treatmentcombinations and, without replication, 32data values. Each data value contributes to

the estimate of each Effect. Thus, each Effecthas the reliability of 32 data values.

To achieve the same reliability doing one -at-a- time experimentation, we would need 96(NINETY-SIX!) data values:

Another Example


49/56

50

AL, B L, C L, DL, E L AH, B L, C L, DL, E L AL, B H, C L, DL, E L AL, B L, C H, DL, E L

AL, B L, C L, DH, E L AL, B L, C L, DL, E H

AB

C

D

E

Having 16 of each of these 6 treatment combinations

( = 96 data values in total) would give us estimates ofeach main Effect with the same reliability of 32 datavalues.

BUT, WHAT ABOUT INFORMING US ABOUT THEPRESENCE OF INTERACTIONS??


50/56

51

16. Factors not studied

In any experiment factors other than thosestudied may be influential. Their presence issometimes acknowledged under the title error.

They may be neglected, but the cost of neglectcould be high.It is important to deal explicitly with them;

even more, it is important to measure their impact.How?


51/56

52

1. Hold them constant.2. Randomize their effects.3. Estimate their magnitude by replicating

the experiment.4. Estimate their magnitude via side or

earlier experiments.5. Confound certain non-studied factors.


52/56


53/56

54

2. Effect of the number of factors on

the error of an estimateWhat is the variance of an estimate of aneffect? In a 2 k design, 2 k treatments go into eachestimate; the signs of the treatments are + or -,

depending on the effectbeing estimated. So any estimate

= 1 [generalized (+ or -) sum of 2 k treatments]

(any estimate) = 1 [2k 2] = 2 /2 k-2 .22k-2

NOTE: 2(kx)=k2 2(x)

2k-1

2


54/56

55

3. Effect of replication on the error ofan estimate

What is the effect of replication onthe error of an estimate? Consider a 2 k design with each treatment replicated rtimes.

1 a b abc d-

---

-

---

-

---

-

---

-

---

... ...


55/56

56

any estimate = 1 [sums of 2k

terms, all of themmeans ]

2

(any estimate) =1

[2k

2

= 2

/(r x 2k-2

) ;the larger the replication per treatment,the smaller the error of each estimate.

2k-1 based on samples of size r

22k-2 r


56/56

57

So, the error of an estimate depends onk (the number of factors studied) and r(the replication per treatmentcombination). It also (obviously)depends on 2.

The variance 2 can be reduced byholding some of the non-studied factors

constant. But, as has been noted, thisgain is offset by reduced generality ofany conclusions.

Date post:	02-Jun-2018
Category:	Documents
Upload:	richard-obinna
View:	217 times
Download:	0 times

method of least aquares.ppt

Documents