Psych 5500/6500

ANOVA:

Single-Factor Independent Means

Fall, 2008

ANOVA

ANOVA is short for ‘Analysis of Variance’, it is also known as the F test. It is applicable in a variety of experimental designs that involve the comparison of group means to determine whether or not the independent variable had an effect.

We will begin with the ‘single-factor independent means’ ANOVA.

‘Single-Factor’

This is similar to the ‘t test for independent means’. As in the t test there is one dependent variable and one independent variable (a ‘factor’ is an independent variable, thus ‘single-factor’ means one IV). But unlike in the t test, with the F test there can be 2 or more levels of the IV (i.e. two or more groups in the experiment).

Independent Means

The ANOVA (F test) we will begin with assumes that the scores are independent across groups. In other words, this could be used in a true experimental design, a quasi-experimental design, or a static group design (just like the t test for independent means).

Example 1:True Experimental Design

Does Vitamin C affect the length of people’s colds? Randomly divide subjects who are in their first day of a cold into 4 groups, then each group gets a different level of Vitamin C. Measure how long it takes each person to get over their cold (DV).

IV= Group 1: 0 mg, Group 2: 100 mg, Group 3: 500 mg, Group 4: 5000 mg

Example 2:Quasi-Experimental Design

Do three specific therapies differ in their ability to treat depression? Let subjects select the type of therapy they want (three different kinds are available), then measure their level of depression (DV) after 2 months of therapy (note the IV is manipulated by the experimenter).

IV= Group 1: Behavior Modification, Group 2: Gestalt, Group 3: Client-Centered

Example 3: Static Group Design

Does the size of a city affect the cancer rate in that city? Randomly select several small cities, several medium sized cities, and several large cities, measure their cancer rate per 10,000 citizens (DV). Note the IV is not manipulated, instead it is the criteria for assigning to groups;

IV= Group 1: Small Cities, Group 2: Medium Cities, Group 3: Large Cities

Relationship of t and FIf you have two groups (i.e. two levels of

your IV) you can use either a ‘t’ or ‘F’ test to analyze the results, and if you are testing a two-tailed hypothesis there is no difference between doing a t test or an F test.

c2cobt

2obt F tand Ft

The F test cannot test a directional (one-tailed) hypothesis. Thus, if you want to do a one-tailed test use t. The t test, however, cannot be used if you have more than two groups, then you must use F.

Hypotheses

If you have three levels of your IV then:H0: μ1= μ2= μ3 (one μ for each group)

You are saying that all the populations in the experiment have the same mean, and that any differences in the group sample means are just due to chance.

So what is HA?HA: μ1 μ2 μ3?

H0 and HA

No, HA: μ1 μ2 μ3 doesn’t work, as H0 and HA together must cover every possibility (e.g. what about μ1= μ2 μ3?). So, the correct answer is:

H0: μ1= μ2= μ3

HA: at least one μ is different than the rest

Test Statistic

We need a statistic whose value we know if H0 is true. With the t test for independent groups the way we tested whether μ1= μ2

was by using:If H0 is true then we expect But what if we have three or more groups? If

H0 is μ1= μ2= μ3 what would we expect if H0 is true?

21 YY

0YY 21

?YYY 321

We need a statistic that will measure whether several group means are about the same (H0 true and the means differ only due to chance) or if they differ more than you would expect if only chance were involved (i.e. if the independent variable made the populations—and thus the groups means—more different than you would expect if only random error were involved).

What statistic do we know measures how much a bunch of numbers (in this case group means but that doesn’t matter) differ from each other?

Analysis of VarianceThe essence of the F test for the one factor

independent group ANOVA is that it examines the variance of the group means to determine whether the group means differ more than you would expect if H0 were true. The logic of how we will do that is based upon ‘partitioning the Sums of Squares’

Setup

We will begin with a simple experiment with three groups, and three scores in each group.

Group 1 Group 2 Group 3

Y1 Y4 Y7

Y2 Y5 Y8

Y3 Y5 Y9

Symbols

groups ofnumber the a

scores) ofnumber total(the N total the N

j'' groupin scores ofnumber the N

scores) theof all ofmean (themean total the Y

j'' group ofmean the Y

score) which indicates i'(' score individualan Y

T

j

T

j

i

Group 1 Group 2 Group 3

Y1 Y4 Y7

Y2 Y5 Y8

Y3 Y5 Y9

N1=3 N2=3 N3=3

?Y1 ?Y2 ?Y3

9N

?Y

3a

T

T

Partitioning the Deviation

We begin by looking at how far each score is from themean of all of the scores:

Ti YY

Then we break (partition) that distance into two pieces,how far the score is from the mean of its group, and how far the mean of the group is from the mean total.

TjjiTi YYYYYY

Partitioning the SSNow we use those deviations to create three sums of squares.

2Tj

2

ji2

Ti YYYYYY

SSTotal = SSWithinGroups + SSBetweenGroups

SSTotal measures the squared deviations of the scores from the mean of all of the scores.SSWithin measures the squared deviations of the scores from the mean of the group they are in.SSBetween measures the squared deviations of the group means from the mean of all of the scores.

SS’sSums of squares are a way of measuring

variability. Consequently:SSTotal reflects how much all of the scores differ from each other

(if all the scores were the same they would all equal the total mean and the squared distances would all be zero).

SSWithin reflects how much the scores differ from other scores in the same group (if all the scores in a group are the same they would all equal the mean of their group and the squared distances would all be zero).

SSBetween reflects how much the group means differ from each other (if all of the group means were the same they would all equal the total mean and the squared distances would all be zero.

Example…

Refer to the handout on partitioning SS.

Partitioning the df

1Ndf TTotal

The total df for the experiment would be the total numberof scores – 1.

We are also going to partition that.

1aaN1N TT

dfTotal = dfWithin + dfBetween

What are d.f.s?

(Discuss in class)…

Mean Squares

A mean square is a Sum of Squares divided by its degrees of freedom.

Between

BetweenBetween

Within

WithinWithin

df

SSMS

df

SSMS

What is a Mean Square?

It is not normally computed as we won’t be needing it,but to make a conceptual point, let’s look at MSTotal.

1-N

)Y(Y

df

SSMS

2

Total

TotalTotal

Does that look familiar?

Error VarianceThe term error variance refers to the variance of the

population from which the scores were originally sampled. The use of the term ‘error’ will be clearer next semester, it refers to the error of using the mean to predict each score. For now just think of error variance as the variance of the population from which we sampled. The ANOVA assumes that each population in the study has the same variance.

Mean Square Within Groups

MSWithin is an estimate of error variance based upon how much the scores differ inside of each group. Essentially, it uses each group to estimate error variance, then pools those different estimates into one good estimate. If the N’s of each group are the same then MSWithin is literally the mean of the variance estimates from each group.

Mean Square Between Groups

MSBetween is an estimate of error variance based upon how much the group means differ from each other. Remember that the variance of the population affects the variance of the sample means (the standard error); well it also works the other way, the variance of the sample means tells us something about the variance of the population from which those means were drawn.

F

Within

Betweenobt MS

MSF

MSBetween and MSWithin are two, independent estimates of the same thing...error variance.

)dflargish (for 12df

dfμ true,is H0 If within

within

WithinF

Logic of the ANOVAWhen H0 is true: MSBetween and MSWithin are two, independent estimates of error variance.

00.1 est.

est.

MS

MSF

2Y

2Y

Within

Betweenobt

When H0 is false: the independent variable makes the group means differ more than they would if only chance were involved, which affects MSBetween making it larger. The independent variable—however—does not affect the variance inside of each group, thus MSWithin is not affected.

1.00σ est.

IV ofeffect σ est.

MS

MSF

2Y

2Y

Within

Betweenobt

Example

IV=type of therapy (control group, vs behavior modification vs psychoanalysis vs client-centered vs gestalt)

DV=level of depression after 2 months

H0: μC= μBM= μPA= μCC= μG

HA: at least one μ is different than the rest.

Data

Control Beh. Mod.

Psycho-analysis

Client-Centered

Gestalt

9 6 6 6 3

8 6 7 5 1

7 4 6 7 5

8Y1 33.5Y2 33.6Y3 6Y4 3Y5

Bar Graph

Control Behavior Mod Psycho Analysis

Client Centered Gestalt

X

0.00

2.00

4.00

6.00

8.00

Mea

n Y

ComputationsSSTotal=54.93

SSWithin=15.33

SSBetween=39.60

dfTotal=14

dfWithin=10

dfBetween=4

MSWithin=9.90

MSBetween=1.53

Fobt=6.46

Fc;.05,df1=4,df2=10=3.48

We reject H0

p Values & Expressing Results

The exact p value can be obtained by either performing the analysis using SPSS or by using my F tool and inputting the df’s and the value of Fobt. In this example p=.0078

The way the results are commonly expressed are as F(df1,df2)=Fobt, p=... In our example it would be: F(4,10)=6.46, p=.0078

Summary Table

Source SS df MS F p

Between 39.60 4 9.90 6.46 .0078

Within 15.33 10 1.53

Total 54.93 14

Another common way of expressing the results of the analysisis in a ‘Summary Table’.

DecisionH0: μC= μBM= μPA= μCC= μG

HA: at least one μ is different than the rest.

We have rejected H0, which means that we can conclude that at least one of the population means is different than the rest. It is tempting to say, for example, that the control group (which had a mean level of depression of 8) was more depressed than the Gestalt group (which had a mean level of depression of 3) but we cannot be that specific, we can only say that at least one group was different than the rest. We will learn in a future lecture how to make more specific tests among the group means.

Effect Size

Cohen’s d is not capable of determining an overall effect due to the independent variable when there are more than two groups as we can’t expect d to equal zero when H0 is true (i.e. when the independent variable has no effect):

?σ

μμμμd sCohen'

Y

4321

Effect Size (cont.)

For the overall effect of the independent variable we will have to turn to measures of association, which examine how much knowing what group the score is in helps us in predicting their score on the dependent variable. We will be covering that next semester. The measure we will be looking at then is called R², and to get a general idea of how it works...

Group 1 Group 2 Group 3

3 3 3

5 5 5

9 9 9

Group 1 Group 2 Group 3

3 5 9

3 5 9

3 5 9

R²=0 (knowing which group the score is in doesn’t help at all).

R²=1.00 (knowing which group the score is in allows us to knowexactly what the score will be).

You can see that R² will always be between 0 and 1

Computing R²

Total

Between2

SS

SSR

In our example:

72.093.54

6.39

SS

SSR

Total

Between2

Cohen’s fGPower uses Cohen’s f to express effect size.

While R² will be between 0 and 1, f expands that out to be between 0 and infinity.

2

2

R-1

Rf

The conventions for relating f to effect size are: .10=small effect .25=medium effect .40=large effect

Our Example

60.157.2.28

.72f

A whopping big effect size (because we are in Oakley land rather than using real data).

GPower and Cohen’s f

GPower will compute f for you if you give it the N’s of each group, the means of each group, and the standard deviation (the one you assume each group has in common), but the equation on the previous slide is much simpler. With this information you can then compute the power a priori and post hoc as you did with the t test.

In our example power=0.98 (ridiculously large for 3

scores per group, due to the big effect of the IV and the small amount of within-group variance, a byproduct of my making up the data).

Assumptions of This Use of the F test

1. Independence of scores (important).

2. All the populations are normally distributed (the F test is ‘robust’ to this assumption, particularly if N’s are large and roughly equal across groups).

3. Homogeneity of Variance (this can be violated if you have roughly equal N’s across the groups). Levenes’ test will evaluate this.

Homogeneity of Variance

If N’s are not equal, then the effects of violating this assumption are:

1. If larger sample size is associated with larger variances then alpha decreases (biased towards not making a type 1 error but at the expense of power).

2. If larger sample size is associated with smaller variances then alpha increases (biased towards making a type 1 error). If this is the case then either select a smaller significance level (e.g. .01 rather than .05) or ‘transform’ your data.

Levene’s TestLevene’s test for the inequality of variances can be

used to test whether a difference exists somewhere among the population variances. In our example with five types of therapy the hypotheses for Levene’s test would be:

H0: σ²1= σ²2= σ²3= σ²4= σ²5

Ha: at least one σ² is different than the rest.

Now that we know how ANOVA works it is easy to describe how Levene’s test works. Let’s begin with a simply study with just two groups.

Group 1 Group 2

9 26

10 28

10 32

11 34

Mean=10 Mean=30

S²=0.5 S²=10

The two groups have very different variances (10 vs. 30), we want to test to see whether it is reasonable to conclude that the populations these groups came from have different variances (our assumption is about populations).

H0: σ²1= σ²2 Ha: σ²1 σ²2

Group 1 Group 2

9 26

10 28

10 32

11 34

Mean=10 Mean=30

S²=0.5 S²=10

The first thing we do is to transform the original scores to deviation scores that reflect how far each score was from the mean of its group. Then we take the absolute values of those deviations

Group 1 Group 2

9 26

10 28

10 32

11 34

Mean=10 Mean=30

Group 1 Group 2

1 4

0 2

0 2

1 4

Original Scores Absolute deviations from the group mean.

We have changed the data to being a measure of how much each score differed from its group mean. In other words, each score is now a measure of variability. We can see in the absolute deviations that the original scores in group 1 did not vary much from their group mean (and thus didn’t vary much from each other).

Group 1 Group 2

9 26

10 28

10 32

11 34

Mean=10 Mean=30

Group 1 Group 2

1 4

0 2

0 2

1 4

Original Scores Absolute deviations from the group mean.

Levene’s test simply performs an ANOVA (with only 2 groups you could use a t test) to see if the mean of the deviations differ significant in the two groups, which tells you whether the variance of the original scores differ significantly.

H0: μ1= μ2

Ha: μ1 μ2

Group 1 Group 2

1 4

0 2

0 2

1 4

Mean=.5 Mean=3

Absolute deviations from the group mean.

Result of the ANOVA on the absolute deviation scores. F(1,6)=15.00, p=.008, so we can conclude that the difference in variances among the groups (original scores) was statistically significant.

Group 1 Group 2

9 26

10 28

10 32

11 34

Group 1 Group 2

1 4

0 2

0 2

1 4

Mean=.5 Mean=3

Original Scores Absolute deviations from the group mean.

Our ExampleControl B. M. P.A. C.C. Gestalt

9 6 6 6 3

8 6 7 5 1

7 4 6 7 5

Control B. M. P.A. C.C. Gestalt

1 .67 .33 0 0

0 .67 .67 1 2

1 1.33 .33 1 2

M=.67 M=.89 M=.44 M=.67 M=1.33

Originaldata

Absolutedeviationscores.

For the |deviations|, F(4,10)=0.782, p=.562

Levene’s: ConsiderationsThe previous example would actually have a problem

in real life, Levene’s test is not accurate when there are very small N’s in each group. This problem becomes negligible when you have 10 or more scores per group.

Also, you should know that since Levene’s procedure involves simply applying ANOVA to the absolute mean deviation scores, that Levene’s too has the assumption that the absolute deviation scores are normally distributed and that the groups have equal variances.

Levene’s: AssumptionsLevene’s test is fairly robust to violations of the

ANOVA assumptions. A study by Brown and Forsythe (1974), however, suggests that if the populations (of the original scores) are fat tailed that you use the ’10 percent trimmed mean’ instead of the mean when finding the absolute mean deviations, and if the populations are skewed that you find the absolute deviation from the median rather than the absolute deviation from the mean.

To find the 10 percent trimmed mean first chop off the 10% highest scores, and then the 10% lowest scores before finding the mean.

Levene’s: AssumptionsThe problem with using either the 10% trimmed mean

or the median is that SPSS will do Levene’s for you using the mean, if you want to use the 10 percent trimmed mean or the median you will have to do it yourself in a similar fashion to what I did in demonstrating how Levene’s works, you find the correct deviations and then do an ANOVA on them.

Brown, M. G. & Forsythe, A. B. (1974) Robust tests for the equality of variances. Journal of the American Statistical Association, 69, 364-367