Inferential StatisticsAnalysis of Variance – ANOVA

Faculty of Information TechnologyKing Mongkut’s University of Technology North Bangkok

ContentEstimationHypothesis testing

Forming hypothesisTesting population meansTesting population variancesTesting categorical data / proportionHypothesis about many population means

One-way ANOVA Two-way ANOVA

Analysis of Variance (ANOVA)Test if any of multiple means are different from each other

One-way ANOVA: 1 variables – 3 or more groupsDependent variable is assumed is of interval or ratio scale

Also used with ordinal scale dataCan describe the effect of independent variable on

dependent variableTwo-way ANOVA: two independent, one dependent variablesMANOVA: Two or more dependent variables

Can describe interaction between two independent variables

One-way ANOVATest the means (of dependent variable) between groups as

specified by an independent variable that are organized in 3 or more groups (dichotomous)Occupation: Student, Lecturer, Doctor (1 var - 3 groups)Salary: dependent variable

AssumptionsDependent variable is either an interval or ratio (continuous)Dependent variable is approximately normally distributed for each

category of the independent variableThere is equality of variances between the independent groups

(homogeneity of variances).Independence of cases.

One-way ANOVA ConceptTotal Variance = Between-Group Variance + Within-Group VarianceBetween-Group Variance

Describe the difference of means between groups, which is the effect on variable of interest

Within-Group VarianceDescribe the difference of means within each group, which is

the effect caused by other factors, called ErrorH0 : μ1 = μ2 = μ3 = … = μn

H1 : μ1 != μ2 != μ3 != … != μn (at least one different pair)

One-way ANOVA Table

SST = SSB + SSW

Source of Variance

Degree of Freedom (df) Sum Square (SS) Mean Square

(MS) F-ratio

Between Groups

(Treatment)

k-1

Within Groups (Error)

n-k

Total n-1

k: number of groups n: number of samplesdf: degree of freedom

MSW

MSBF

1k

SSBMSB

kn

SSWMSW

n

TXSSTij

n

i

K

j

22

11

K

j j

jn

iij

K

j n

TXSSW

j

1

2

1

2

1

One-way ANOVA: SPSSAnalyze -> Compare Means -> One-way ANOVAOption -> Tick…

Homogeneity of variance testDescriptive (optional)WelchPost Hoc - used when the result is significant (at least one

of the means is different) to find the group with the different mean

https://statistics.laerd.com/spss-tutorials/one-way-anova-using-spss-statistics.phphttp://academic.udayton.edu/gregelvers/psy216/spss/1wayanova.htm

ExampleDetermine if the means of total score are

different in the 5 SectionsH0 : μ1 = μ2 = μ3 = μ4 = μ5

H1 : μ1 != μ2 != μ3 != μ4 != μ5

At least one pair is different

Result: Descriptives and Variances

Check Levene test“Sig.” > = 0.05, thus variances are equal in all groups If not, need to refer to the Robust Tests of Equality of

Means Table (Welch) instead of the ANOVA Table

Result: ANOVA Table

Sig. = 0.013 < α, thus at least one of the group has different means

Use Post-Hoc tests To find the pair with different mean

Result: Post Hoc TestsThe pair that Sig. < α has

different meanSection 1 and 4Section 2 and 4Section 2 and 5Section 3 and 4Section 4 and 5

Two-way ANOVAUse to determine the effect of 2 or more factors (independent

variables) on one dependent variableOccupation: Student, Lecturer, DoctorAge: less than 20, 20-30, 31-40, 41 or olderSalary: dependent variable

AssumptionsDependent variable is either interval or ratio (continuous)The dependent variable is approximately normally distributed for

each combination of levels of the two independent variablesHomogeneity of variances of the groups formed by the different

combinations of levels of the two independent variables.Independence of cases

Two-way ANOVA ConceptTwo-way ANOVA compares

Means between columnsMeans between rowsMeans from the interaction of factors

Sum Square Row (SSR): variation effect of the 1st factorSum Square Column (SSC): variation effect of the 2nd factorSum Square Row Column (SSRC): variation effect of the

interaction of the two factorsSum Square Error (SSE): Error caused by external factorsSum Square Total (SST) = SSR + SSC + SSRC + SSE

Two-way ANOVA Table r: number of rows c: number of columnsn: number of samplesdf: degree of freedom

Two-way ANOVA: SPSSAnalyze -> General Linear Model -> Univariate

Multivariate is MANOVAAdd dependent variable and two or more factors

(independent variables)Option -> tick “Homogeneity tests” (optional “Descriptive”)Plot -> add one factor (containing more groups) to

“Horizontal Axis” and other to “Separate Lines” then click “Add”To obtain profile plot

Post Hoc to find pair that has different means (similar to One-way ANOVA, optional)https://statistics.laerd.com/spss-tutorials/two-way-anova-using-spss-statistics.php

ExampleDetermine the effect of major and gender on the total

score H0 : μ1 = μ2 = μ3 = μ4

H1 : μ1 != μ2 != μ3 != μ4

Result

Compare Error to Corrected TotalError should be less than 20% of corrected totalError is very large compared to corrected totalTotal score is effected by other external factors

Gender row Sig. = 0.024 < α, gender has effect on total scoreMajor row Sig. = 0.575 > α, major has no effect on total scoreMajor*Gender row Sig. = 0.298 > α, the interaction between

two factors has no effect on total score

Result: Profile Plot

ExampleDetermine the effect of section and gender on the total

score