1
11
psyc3010 lecture psyc3010 lecture 1111
One One and twoand two--way within way within participants participants anovaanova
Last week: Regression topicsnext week: mixed anova
22
announcementsannouncementsAssignment 2 due on Assignment 2 due on May 23 (next May 23 (next Monday)Monday)– submission via Turnitin (on Blackboard)– submission deadline = 12 noon– If in doubt, or submitting late, e-mail your tutor. You can cc me.
Quiz 2 marks available after Quiz 2 marks available after 5 5 pm pm Monday the 23Monday the 23rdrd (or (or earlier)earlier)
EExam xam review in Week 13 lecture (last class)review in Week 13 lecture (last class)–– exam content + structureexam content + structure–– how to study for the examhow to study for the exam–– review of course materialreview of course material
Evaluations Evaluations in Week 13 lecturein Week 13 lecture– course and lecturer
2
33
last lecture last lecture this lecturethis lecture
last lecture:– review of regression topics
this lecture:– back to ANOVA– one-way and two-way within-participants
designs
44
topics for this week topics for this week introduction to within-participants designs
research questionspowersources of variance
one-way within-participants ANOVAomnibus testsfollow-up tests (main effect comparisons)
two-way within-participants ANOVAomnibus testsfollow-up tests (simple effects and simple comparisons)
mixed-model: fixed and random effectssphericity: problem and solutions
3
55
66
anova anova –– a second looka second look
betweenbetween--participants participants designsdesigns–– each person serves in only each person serves in only oneone treatment/celltreatment/cell–– we then assume that any difference between them is due to our we then assume that any difference between them is due to our
experimental manipulation (or intrinsic features of the grouping experimental manipulation (or intrinsic features of the grouping variable, e.g., gender)variable, e.g., gender)
–– WithinWithin--cell variability is residual errorcell variability is residual errorwithinwithin--participants participants (repeated(repeated--measures) designsmeasures) designs– what if each participant served in each treatment?– violates the assumption of independence in factorial ANOVA
because scores for the participant are correlated across conditions
– but we can calculate and remove any variance due to dependence
– thus, we can reduce our error term and increase power
4
77
an illustrationan illustration
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
treatment means don’t differ by much – far more variability within
each group than between
88
an illustrationan illustration
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
most of this within-group variance is caused by the fact that some participants learn quickly, and some learn slowly – i.e.,
people are different
In between-participants design, all within-group variance is error, whereas repeated measures design remove individual
difference variation from the error term.
5
99
an illustrationan illustration
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
solution: firstly remove the between-participants variance (i.e., account for individual differences)
and then compare our treatment means
1010
Understanding RM versus BS Understanding RM versus BS designsdesigns
In between In between participants, participants, assign people assign people randomly to j conditionsrandomly to j conditions–– Total Variance = Between group + within Total Variance = Between group + within
groupgroup•• Treatment effect = between group varianceTreatment effect = between group variance•• Error = within group varianceError = within group variance
No No participant participant variability because each variability because each participant has only 1 data point (no participant has only 1 data point (no variance within individual)variance within individual)
6
1111
betweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance
total variationtotal variation
1-way between-participants anova:
residual/errorresidual/error
any individual differences within groups are considered
‘error’
1212
Understanding RM designsUnderstanding RM designsIn fully within In fully within participants participants design, people are design, people are tested in each of j conditions tested in each of j conditions “participant” “participant” factor is crossed with IV (e.g., factor factor is crossed with IV (e.g., factor A)A)End up with A x End up with A x P (Factor A x Participant) design P (Factor A x Participant) design with only 1 observation per cellwith only 1 observation per cellNo withinNo within--cell variance cell variance –– now a cell is one now a cell is one observation (for person observation (for person ii in condition j)in condition j)Weird Weird -- So So what is error?what is error?–– Interaction of A x Interaction of A x P P –– i.e., the changes (inconsistency) i.e., the changes (inconsistency)
in the effects of A across in the effects of A across participantsparticipants
7
1313
A A x P x P designdesign
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
Overall, treatment effect for 1 = 16 – 18.67 (-2.67)treatment effect for 2 = 19-18.67 (+0.33)treatment effect for 3 = 21 – 18.67 (+2.33)
1414
A A x P x P designdesign
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
For P1, treatment effect for 1 = 2 – 4.33 (-2.33)treatment effect for 2 = 4 – 4.33 (-0.33)treatment effect for 3 = 7 – 4.33 (+2.67)
8
1515
A A x P x P designdesign
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
For P2, treatment effect for 1 = 10 – 11.67 (-1.67)treatment effect for 2 = 12-11.67 (+0.33)treatment effect for 3 = 13-11.67 (+1.33)
1616
A x A x P P designdesign
treatmenttreatmentparticipant participant 11 2 2 3 3 meanmean
1 2 4 7 4.332 10 12 13 11.673 22 29 30 27.004 30 31 34 31.67
mean 16 19 21 18.67
For P3, treatment effect for 1 = 22 – 27 (-5)treatment effect for 2 = 29 – 27 (+2)treatment effect for 3 = 30 – 27 (+3)
9
1717
betweenbetween--participants participants variancevariance withinwithin--participants participants
variancevariance
1-way within-participants anova:
error/residual error/residual
[interaction [interaction P P x tr]x tr]betweenbetween--treatmentstreatments
any individual differences are removed first
total variationtotal variation
1818
10
1919
WithinWithin--participants participants designdesignTotal Variance = Between Total Variance = Between participants participants + + within within participantsparticipants
Between Between participants participants variance due to individual variance due to individual differences is partitioned out of error (and differences is partitioned out of error (and treatment)!treatment)!–– Within Within participants participants = between treatment = between treatment [treatment [treatment
effect] + effect] + treatment x treatment x participant participant interactioninteraction[residual error [residual error –– i.e., inconsistencies in the treatment i.e., inconsistencies in the treatment effect]effect]
–– F test = TR / TR x F test = TR / TR x PPAcknowledges reality that variability within Acknowledges reality that variability within conditions/groups and between conditions/groups are conditions/groups and between conditions/groups are both influenced by both influenced by participant participant factor [people doing study]factor [people doing study]
2020
the conceptual modelthe conceptual model
XXijij = = μμ + + ππii + + ττjj + e+ eijijfor i cases and j treatments:
Xij, any DV score is a combination of:μμ the grand mean,ππii variation due to the i-th person (μi - μ)τj variation due to the j-th treatment (μj - μ)eij error - variation associated with the i-th cases in
the j-th treatment – error = ππττijij (plus chance)(plus chance)
11
2121
partitioning the variancepartitioning the variance
error (TRxP)
treatment
Participants
2222
worked exampleworked example
basic learning studybasic learning study11--way way withinwithin--participants participants design (design (nn=5)=5)IV: blockIV: block–– 40 trials through whole experiment40 trials through whole experiment–– want to compare over 4 blocks of 10 to see if learning has want to compare over 4 blocks of 10 to see if learning has
occurredoccurred
DV = number of correct responses per blockDV = number of correct responses per block
12
2323
correct trials over 4 blocks of 10correct trials over 4 blocks of 10
block 1 block2 block3 block 4 P totalParticipant 1 4 3 6 5 18Participant 2 4 4 7 8 23Participant 3 1 2 1 3 7Participant 4 1 4 5 5 15Participant 5 5 7 6 9 27
block total 15 20 25 30 90block mean 3 4 5 6
2424
Definitional formulaeDefinitional formulaeTotal variability Total variability –– deviation of each observation from the deviation of each observation from the grand mean:grand mean:
Variability due to factor Variability due to factor –– deviation of factor group deviation of factor group means from grand mean:means from grand mean:
Variability due to Variability due to participants participants –– deviation of each deviation of each participant’s participant’s mean from the grand mean:mean from the grand mean:
Error Error –– changes (inconsistencies) in the effect of factor changes (inconsistencies) in the effect of factor across across participants participants (TR (TR x P x P interaction):interaction):
( )2. ..A jSS n Y Y= −∑
( )2. ..A jSS n Y Y= −∑
( )2. ..S iSS a Y Y= −∑∑
( )2..T ijSS Y Y= −∑
AxS T A SSS SS SS SS= − −( )2. . ..AxS i jSS Y Y Y Y= − − +∑∑ or
P
PP P
13
25
degrees of freedom dftotal = nj-1 = N-1 = 19 dfP = n-1 = 4 dftr = j–1 = 3 dferror = (n-1)(j-1) = 12
error df is different from between-participants anova – because error is now interaction of participant factor x treatment factor
Big N = Number of observations
number of participants * number of conditions
2626
the summary tablethe summary table
MSP = estimate of variance in DV attributable to INDIVIDUAL DIFFERENCES(averaged over treatment levels) – but ignore this & don’t report in write-up
MSTR = estimate of variance in DV attributable to TREATMENT(averaged over participants)
MSError = RESIDUAL: estimate of variance in DV not attributable to S or TR(interaction - the change in the treatment effect across participants = error)
Source SS df MS F
Between subjects (P) 59 4 14.75Treatment (TR) 25 3 8.33 6.66*Error 15 12 1.25
Total 99 19
* p <.05 F crit (3,12) = 3.49
14
2727
assuming the data was obtained from a assuming the data was obtained from a betweenbetween--participantsparticipants design . . . design . . .
Source SS df MS F
Treatment (TR) 25 3 8.33 1.80Error 74 16 4.63
Total 99 19
F crit (3,16) =3.24
in between-participants designs, individual differences are inseparable from error, hence contribute to the error term
in within-participants designs it is possible to partial out (i.e., remove) individual differences from the error term
smaller error term smaller error term greater POWER greater POWER ☺☺
2828
a note on error termsa note on error terms
hand calculations in hand calculations in withinwithin--participants participants anovaanova are are no differentno different to those in to those in betweenbetween--participants participants anovaanova–– only the only the error termerror term (and (and dfdf) changes) changes
in 1in 1--way way withinwithin--participants participants the error term the error term (and (and dfdf) is the treatment x ) is the treatment x participants participants interactioninteraction–– MSMSerrorerror = = MSMSTRxPTRxP
15
2929
Nobody made a greater mistake than he Nobody made a greater mistake than he who did nothing because he could do only who did nothing because he could do only a little.a little.
----Edmund Burke, statesman and writer Edmund Burke, statesman and writer (1729(1729--1797)1797)
3030
following up the main effect of treatment . . following up the main effect of treatment . . ..
in between-participants anova, MSerror is the term we would use to test any effect, including simple
comparisons [error = differences between participants –expect within-cell variance is the same across conditions]
but within-participants ANOVA we partition out and ignore the main effect of participants and compute an error term estimating inconsistency as participants change over WS
levels
Source SS df MS F
Treatment (TR) 25 3 8.33 6.66*Error 15 12 1.25
Total 40 19
* p <.05 F crit (3,12) = 3.49
16
3131
separate error terms: separate error terms: followingfollowing--up main effectsup main effects
–– We expect inconsistency in TR effect x We expect inconsistency in TR effect x participants participants so so in simple comparisons use only data for conditions in simple comparisons use only data for conditions involved in comparison & calculate involved in comparison & calculate separate error separate error termsterms each timeeach time
B2 vs B3
block 1 block2 block3 block 4 P totalParticipant 1 3 6 9Participant 2 4 7 11Participant 3 2 1 3Participant 4 4 5 9Participant 5 7 6 13
block total 20 25 45block mean 4 5
3232
separate error terms: separate error terms: followingfollowing--up main effectsup main effects
B1 vs B4
block 1 block2 block3 block 4 P totalParticipant 1 4 5 9Participant 2 4 8 12Participant 3 1 3 4Participant 4 1 5 6Participant 5 5 9 14
block total 15 30 45block mean 3 6
•• We expect inconsistency in TR effect x We expect inconsistency in TR effect x participants participants so in simple comparisons use only so in simple comparisons use only data for conditions involved in comparison & data for conditions involved in comparison & calculate calculate separate error termsseparate error terms each timeeach time
17
3333
betweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance
total variationtotal variation
Simple comparisons in between-participants anova:
residual/errorresidual/error
Partition treatment variance to follow-up, but use same error term (within-cell variance) for main effect (treatment) test
and for all follow-ups
Contrast 1
Contrast 2
Contrast 3
3434
withinwithin--participantsparticipantsbetweenbetween--participantsparticipants
Simple comparisons in RM designs:
total variationtotal variation
betweenbetween--treatmentstreatments residualsresiduals
C1C1
C2C2
C3C3
C1xPC1xP
C2xPC2xP
C3xPC3xP
Partition treatment variance and residual variance for follow-ups. Each contrast effect is
tested against error term = C x P interaction
18
3535
summary tablesummary table
Source SS df MS F
B2 vs B3 2.5 1 2.5 1.25Error 8 4 2
B1 vs B4 22.5 1 22.5 22.5*Error 4 4 1
these are the SScontrasts we can calculate in the same way as in between-participants anova
But SSPxcontrast terms we calculate separately for each within-participants effect
df for comparison is same as usual (i.e., 1)
dferror = (n-1)(j-1)= (5-1)(2-1)= 4
3636
19
3737
22--way way withinwithin--participants participants designsdesigns
calculations are similar to a 2calculations are similar to a 2--way way betweenbetween--participants participants ANOVAANOVA–– main effects for A and B are tested, as well as a main effects for A and B are tested, as well as a AxBAxB
interactioninteraction–– with a with a withinwithin--participants participants design, each effect tested design, each effect tested
has a has a separate error termseparate error term–– this error term simply corresponds to an this error term simply corresponds to an interaction interaction
between the effect due to between the effect due to participants, participants, and the and the treatment effecttreatment effect
• main effect of A error term is MSAxP• main effect of B error term is MSBxP• AxB interaction error term is MSABxP
3838
betweenbetween--groups variancegroups variance withinwithin--groups variancegroups variance
total variationtotal variation
2-way between-participants anova:
residual/errorresidual/error
Partition between-groups variance into A, B and AxB, but
use same error term (within-cell variance) for each test
(and all follow-ups)
A
B
AB
20
3939
withinwithin--participantsparticipantsbetweenbetween--participantsparticipants
2-way within-participants anova:
total variationtotal variation
betweenbetween--treatmentstreatments residualsresiduals
AA
BB
A x BA x B
AxPAxP
BxPBxP
AxBxPAxBxP
Partition treatment variance and residual variance for each effect. Each effect is tested
against error term = effect x P interaction
4040
22--way way withinwithin--participants participants exampleexample
another learning study: another learning study: 2 x 4 repeated2 x 4 repeated--measures factorial design (measures factorial design (nn=4)=4)first factor: phasefirst factor: phase–– phase 1: no reinforcement (100 trials)phase 1: no reinforcement (100 trials)–– phase 2: reward for correct response (100 trials)phase 2: reward for correct response (100 trials)
second factor: blocksecond factor: block–– each phase split into four blocks of 25each phase split into four blocks of 25–– enables us to compare performance for trials later in enables us to compare performance for trials later in
each phase with trials early in each phase each phase with trials early in each phase –– thereby thereby assessing assessing learninglearning
DV = number of correct responses per blockDV = number of correct responses per block
21
4141
Phase x Block repeated measures designPhase x Block repeated measures design[phase x block x [phase x block x participants]participants]
b1 b2 b3 b4 b1 b2 b3 b4Particip 1 3 4 3 7 5 6 7 11P2 6 8 9 12 10 12 15 18P3 7 13 11 11 10 15 14 15P4 0 3 6 6 5 7 9 11
Dataphase 1 phase 2
4242
summary table . . .summary table . . .
Source SS df MS F
Between participants 272.6 3 90.867
Phase 116.28 1 116.28 59.63*Phase x P 5.84 3 1.95
Block 129.6 3 43.20 12.24*Block x P 31.77 9 3.53
Phase x Block 3.34 3 1.11 3.26Ph x B x P 3.04 9 0.34
Critical F (1,3) = 10.13Critical F (3,9) = 3.86
22
4343
following up main effectsfollowing up main effectsas with oneas with one--way repeated measures designs, use of way repeated measures designs, use of error term for effect (e.g., error term for effect (e.g., MSMSBlockxPBlockxP) ) is not appropriate is not appropriate for followfor follow--up comparisonsup comparisonsa a separate error termseparate error term must be calculated for each must be calculated for each comparison comparison undertaken(undertaken(MSMSBlockBlockCOMPCOMPxPxP) )
Source SS df MS FB COMP 18.06 1 18.06 6.62
B COMPxP 8.19 3 2.73Critical F (1,3) = 10.13
4444
following up interactions . .following up interactions . . ..
again, separate error terms must be used again, separate error terms must be used for each effect testedfor each effect tested–– simple effectssimple effects
•• error term is error term is MSMSA at A at B1xPB1xP
•• the interaction between the the interaction between the AA treatment and treatment and participantsparticipants, , at at B1B1
–– simple comparisonssimple comparisons•• error term is error term is MSMS
AACOMPCOMP at at B1xPB1xP•• interaction between the interaction between the AA treatment (only the data treatment (only the data
contributing to the comparison, contributing to the comparison, AACOMPCOMP), and ), and participantsparticipants, , at at B1B1
23
4545
4646
2 approaches to 2 approaches to withinwithin--participants participants designsdesignsmixedmixed--modelmodel approachapproach–– what we have been doing with what we have been doing with examples so far calculationsexamples so far calculations–– treatment is a treatment is a fixedfixed factor, factor, participants participants is a is a randomrandom factorfactor
•• Fixed factor: Fixed factor: You chose the levels of the IV.You chose the levels of the IV.–– You have sampled all the levels of the IV You have sampled all the levels of the IV oror–– You have selected particular levels based on a theoretical reasonYou have selected particular levels based on a theoretical reason
•• Random factor: Random factor: The levels of the IV are chosen at randomThe levels of the IV are chosen at random•• Random factors have different error terms: all ANOVA we have done Random factors have different error terms: all ANOVA we have done
to date has assumed the IVs are fixed. For most of you, the to date has assumed the IVs are fixed. For most of you, the participant participant factor is the only random factor you will ever meet (be factor is the only random factor you will ever meet (be grateful). grateful). ☺☺ You can read up on random factor ANOVA models in You can read up on random factor ANOVA models in advanced textbooks if you need to (e.g., as a advanced textbooks if you need to (e.g., as a postgradpostgrad).).
–– powerful when assumptions are met powerful when assumptions are met –– mathematically usermathematically user--friendly friendly
•• just like a factorial just like a factorial anovaanova–– restrictive assumptions, but adjustments available if they are restrictive assumptions, but adjustments available if they are
violatedviolated
multivariate multivariate approachapproach which we will discuss briefly laterwhich we will discuss briefly later
24
4747
assumptions of mixedassumptions of mixed--model approachmodel approach
not dissimilar to not dissimilar to betweenbetween--participants participants assumptions:assumptions:1.1. sample is sample is randomly drawnrandomly drawn from populationfrom population2.2. DV scores are DV scores are normally distributednormally distributed in the in the
population population 3.3. compound symmetrycompound symmetry
•• homogeneity of variances in levels of homogeneity of variances in levels of repeatedrepeated--measures factormeasures factor
•• homogeneity of homogeneity of covariancescovariances(equal correlations/(equal correlations/covariancescovariances between between pairs of levels) pairs of levels)
4848
compound symmetrycompound symmetrythe variancethe variance--covariance matrix:covariance matrix:
T1T1 T2T2 T3T3T1T1 158.92158.92 163.33163.33 163.00163.00
ΣΣ == T2 T2 163.33163.33 172.67172.67 170.67170.67
T3T3 163.00163.00 170.67170.67 170.00170.00
25
4949
compound symmetrycompound symmetrythe variancethe variance--covariance matrix:covariance matrix:
T1T1 T2T2 T3T3T1T1 158.92158.92 163.33163.33 163.00163.00
ΣΣ == T2 T2 163.33163.33 172.67172.67 170.67170.67
T3T3 163.00163.00 170.67170.67 170.00170.00
compound symmetry requires that variances are roughly equal (homogeneity of variance)
5050
compound symmetrycompound symmetrythe variancethe variance--covariance matrix:covariance matrix:
T1T1 T2T2 T3T3T1T1 158.92158.92 163.33163.33 163.00163.00
ΣΣ == T2 T2 163.33163.33 172.67172.67 170.67170.67
T3T3 163.00163.00 170.67170.67 170.00170.00
compound symmetry requires that covariances are roughly equal (homogeneity of covariance)
26
5151
Mauchly’s test of sphericityMauchly’s test of sphericitycompound symmetry is a very restrictive compound symmetry is a very restrictive assumption assumption –– often violatedoften violatedsphericity sphericity is a more broad and less restrictive is a more broad and less restrictive assumptionassumptionSPSS SPSS –– Mauchley’s test of sphericityMauchley’s test of sphericity–– examines overall structure of covariance matrix examines overall structure of covariance matrix –– determines whether values in the main diagonal (variances) are determines whether values in the main diagonal (variances) are
roughly equal, and if values in the offroughly equal, and if values in the off--diagonal are roughly equal diagonal are roughly equal (covariances)(covariances)
–– evaluated as evaluated as χχ2 2 –– if significant, sphericity assumption is violatedif significant, sphericity assumption is violated–– not a robust test not a robust test AT ALLAT ALL –– very commonly fail to find very commonly fail to find
Mauchley’s sphericity is sig even when violations of Mauchley’s sphericity is sig even when violations of sphericity are present in the datasphericity are present in the data
5252
violations of sphericityviolations of sphericitywhen when sphericitysphericity doesn’tdoesn’t mattermatter
in between-participants designs, because treatments are unrelated (different participants in different treatments)– the assumption of homogeneity of variance still matters though
when within-participant factors have two levels, because only one estimate of covariance can be computed
when it when it doesdoes mattermatterin all other within-participants designs (3 + levels)when the sphericity assumption is violated, F-ratios are positively biased– critical values of F [based on df a – 1, (a – 1)(n – 1)] are too small– therefore, probability of type-1 error increases
27
5353
adjustments to degrees of freedomadjustments to degrees of freedom
Best to assume that have a problem and make Best to assume that have a problem and make adjustment proactively adjustment proactively –– change critical change critical FF by by adjusting degrees of freedomadjusting degrees of freedom
epsilon (epsilon (εε) adjustments) adjustments–– epsilon is simply a value by which the degrees of epsilon is simply a value by which the degrees of
freedom for the test of Ffreedom for the test of F--ratio is multipliedratio is multiplied–– equal to 1 when sphericity assumption is met (hence equal to 1 when sphericity assumption is met (hence
no adjustment), and < 1 when assumption is violatedno adjustment), and < 1 when assumption is violated–– the lower the epsilon value (further from 1), the more the lower the epsilon value (further from 1), the more
conservative the test becomesconservative the test becomes
5454
different types of epsilondifferent types of epsilonLowerLower--bound epsilonbound epsilon–– Act as if have only 2 treatment levels with maximal heterogeneityAct as if have only 2 treatment levels with maximal heterogeneity–– used for conditions of maximal heterogeneity, or worstused for conditions of maximal heterogeneity, or worst--case violation case violation
of sphericity of sphericity often too conservativeoften too conservative
GreenhouseGreenhouse--GeisserGeisser epsilonepsilon–– size of size of εε depends on degree to which sphericity is violateddepends on degree to which sphericity is violated–– 1 1 ≥≥ εε ≥≥ 1/(1/(kk--1)1) : varies between 1 (sphericity intact) and lower: varies between 1 (sphericity intact) and lower--bound bound
epsilon (worstepsilon (worst--case violation)case violation)–– generally recommended generally recommended –– not too stringent, not too laxnot too stringent, not too lax
28
5555
different types of epsilondifferent types of epsilonHuynhHuynh--Feldt epsilonFeldt epsilon–– an adjustment applied to the GGan adjustment applied to the GG--epsilonepsilon–– often results in epsilon exceeding 1, in which case it is set to 1 often results in epsilon exceeding 1, in which case it is set to 1 –– used when “true value” of epsilon is believed to be used when “true value” of epsilon is believed to be ≥≥ .75.75
5656
spss output from our previous examplespss output from our previous exampleMauchly's Test of Sphericityb
Measure: MEASURE_1
1.000 .000 0 . 1.000 1.000 1.000.111 3.785 5 .634 .587 1.000 .333.000 . 5 . .348 .370 .333
Within Subjects EffectPHASEBLOCKPHASE * BLOCK
Mauchly's WApprox.
Chi-Square df Sig.Greenhouse-Geisser Huynh-Feldt Lower-bound
Epsilona
Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables isproportional to an identity matrix.
May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in theTests of Within-Subjects Effects table.
a.
Design: Intercept Within Subjects Design: PHASE+BLOCK+PHASE*BLOCK
b.
No sphericity test for effects involving phase – only 2 levels
test for block is not significant (sphericity not violated) but we aren’t going to trust it!
29
5757
spss output from our previous examplespss output from our previous exampleMauchly's Test of Sphericityb
Measure: MEASURE_1
1.000 .000 0 . 1.000 1.000 1.000.111 3.785 5 .634 .587 1.000 .333.000 . 5 . .348 .370 .333
Within Subjects EffectPHASEBLOCKPHASE * BLOCK
Mauchly's WApprox.
Chi-Square df Sig.Greenhouse-Geisser Huynh-Feldt Lower-bound
Epsilona
Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables isproportional to an identity matrix.
May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in theTests of Within-Subjects Effects table.
a.
Design: Intercept Within Subjects Design: PHASE+BLOCK+PHASE*BLOCK
b.
compare the epsilon values
The lower the episilon, the greater the adjustment, the more conservative the test
5858
Measure: MEASURE_1
116.281 1 116.281 59.695 .005116.281 1.000 116.281 59.695 .005116.281 1.000 116.281 59.695 .005116.281 1.000 116.281 59.695 .005
5.844 3 1.9485.844 3.000 1.9485.844 3.000 1.9485.844 3.000 1.948
129.594 3 43.198 12.233 .002129.594 1.760 73.621 12.233 .011129.594 3.000 43.198 12.233 .002129.594 1.000 129.594 12.233 .04031.781 9 3.53131.781 5.281 6.01831.781 9.000 3.53131.781 3.000 10.5943.344 3 1.115 3.309 .0713.344 1.043 3.207 3.309 .1633.344 1.109 3.016 3.309 .1593.344 1.000 3.344 3.309 .1663.031 9 .3373.031 3.128 .9693.031 3.326 .9113.031 3.000 1.010
Sphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-bound
SourcePHASE
Error(PHASE)
BLOCK
Error(BLOCK)
PHASE * BLOCK
Error(PHASE*BLOCK)
Type III Sumof Squares df Mean Square F Sig.
• p values are different for Block despite same F of 12.233 – why?
• because each F value is associated with a different df
• thus, each calculated F value is compared to a different critical F value to determine whether it meets the criteria for statistical significance
30
5959
spss output from our previous examplespss output from our previous example
Measure: MEASURE_1
129.594 3 43.198 12.233 .002129.594 1.760 73.621 12.233 .011129.594 3.000 43.198 12.233 .002129.594 1.000 129.594 12.233 .04031.781 9 3.53131.781 5.281 6.01831.781 9.000 3.531
31.781 3.000 10.594
Sphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-bound
SourceBLOCK
Error(BLOCK)
Type IIISum ofSquares df Mean Square F Sig.
sphericity assumed – i.e., no adjustment
this is what we based our degrees of freedom on before, i.e., b-1 = 4-1 = 3, (n-1)(b-1) = 3 x 3 = 9 3,9
6060
spss output from our previous examplespss output from our previous example
Measure: MEASURE_1
129.594 3 43.198 12.233 .002129.594 1.760 73.621 12.233 .011129.594 3.000 43.198 12.233 .002129.594 1.000 129.594 12.233 .04031.781 9 3.53131.781 5.281 6.01831.781 9.000 3.531
31.781 3.000 10.594
Sphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-bound
SourceBLOCK
Error(BLOCK)
Type IIISum ofSquares df Mean Square F Sig.
Lower-bound – for worst case heterogeneity
i.e., df = 1, b-1 – here we come close to concluding non-significance (which could be a type-2 error)
31
6161
spss output from our previous examplespss output from our previous example
Measure: MEASURE_1
129.594 3 43.198 12.233 .002129.594 1.760 73.621 12.233 .011129.594 3.000 43.198 12.233 .002129.594 1.000 129.594 12.233 .04031.781 9 3.53131.781 5.281 6.01831.781 9.000 3.531
31.781 3.000 10.594
Sphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-bound
SourceBLOCK
Error(BLOCK)
Type IIISum ofSquares df Mean Square F Sig.
Greenhouse-Geisseradjustment does not change significance of result
6262
spss output from our previous examplespss output from our previous example
Measure: MEASURE_1
129.594 3 43.198 12.233 .002129.594 1.760 73.621 12.233 .011129.594 3.000 43.198 12.233 .002129.594 1.000 129.594 12.233 .04031.781 9 3.53131.781 5.281 6.01831.781 9.000 3.531
31.781 3.000 10.594
Sphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-boundSphericity AssumedGreenhouse-GeisserHuynh-FeldtLower-bound
SourceBLOCK
Error(BLOCK)
Type IIISum ofSquares df Mean Square F Sig.
Huynh-Feldt – adjusts GG
no different to ‘sphericity assumed’ – indicates that ε > 1
32
6363
Changes in Changes in participants’ participants’ learning with practice and with learning with practice and with or without reinforcement were explored in a 2 [phase] x 4 or without reinforcement were explored in a 2 [phase] x 4 [Block] repeated measures ANOVA. In these analyses, [Block] repeated measures ANOVA. In these analyses, the Huynhthe Huynh--FeldtFeldt correction was applied to the degrees of correction was applied to the degrees of freedom, however the full degrees of freedom are freedom, however the full degrees of freedom are reported here. reported here. Contrary to predictions, the interaction Contrary to predictions, the interaction was not significant, F(3,9) = 3.309, p = .159, was not significant, F(3,9) = 3.309, p = .159, eta2 eta2 = ??. = ??. However, as hypothesised, However, as hypothesised, participants participants learned more in learned more in the phase with reinforcement (the phase with reinforcement (M M = 42.5; = 42.5; SD SD = ??) than in = ??) than in the phase without (the phase without (MM = 27.25; = 27.25; SD SD = ??), = ??), FF(1, 3) = 59.70, (1, 3) = 59.70, pp = .005, eta2 = ??. A main effect of Block, = .005, eta2 = ??. A main effect of Block, FF(3,9) = (3,9) = 12.23, 12.23, pp = .002, eta2 = ??, was followed up with a series = .002, eta2 = ??, was followed up with a series of contrasts. These revealed that of contrasts. These revealed that
Writing upWriting up
6464
33
6565
multivariate approachmultivariate approachmultivariate analysis of variance multivariate analysis of variance ((manovamanova))–– creates a creates a linear composite linear composite of multiple DVsof multiple DVs–– In MANOVA approach to repeated measures In MANOVA approach to repeated measures
designs, our repeated measures variable is treated designs, our repeated measures variable is treated as multiple DVs and combined / weighted to as multiple DVs and combined / weighted to maximise the difference between levels of other maximise the difference between levels of other variables (similar to the approach regression uses variables (similar to the approach regression uses to combined multiple predictors)to combined multiple predictors)
•• multivariate tests multivariate tests –– Pillai’sPillai’s Trace, Trace, Hotelling’sHotelling’sTrace, Trace, Wilk’sWilk’s Lambda, Roy’s Largest RootLambda, Roy’s Largest Root
•• does not require restrictive does not require restrictive assumptions that mixed assumptions that mixed model within participants design doesmodel within participants design does
–– more more complex underlying mathcomplex underlying math
6666
multivariate approachmultivariate approachMultivariate Testsb
.952 59.695a 1.000 3.000 .005
.048 59.695a 1.000 3.000 .00519.898 59.695a 1.000 3.000 .00519.898 59.695a 1.000 3.000 .005
.992 43.017a 3.000 1.000 .112
.008 43.017a 3.000 1.000 .112129.050 43.017a 3.000 1.000 .112129.050 43.017a 3.000 1.000 .112
.990 102.333a 2.000 2.000 .010
.010 102.333a 2.000 2.000 .010102.333 102.333a 2.000 2.000 .010102.333 102.333a 2.000 2.000 .010
Pillai's TraceWilks' LambdaHotelling's TraceRoy's Largest RootPillai's TraceWilks' LambdaHotelling's TraceRoy's Largest RootPillai's TraceWilks' LambdaHotelling's TraceRoy's Largest Root
EffectPHASE
BLOCK
PHASE * BLOCK
Value F Hypothesis df Error df Sig.
Exact statistica.
Design: Intercept Within Subjects Design: PHASE+BLOCK+PHASE*BLOCK
b.
34
6767
Take home messageTake home messageWhat is MANOVA doing?What is MANOVA doing?–– Weighting the DV for each level of the repeated measures IV with Weighting the DV for each level of the repeated measures IV with
coefficients (like what happens to scores for each IV in multiple coefficients (like what happens to scores for each IV in multiple regression) to create a predicted DV score that maximises differences regression) to create a predicted DV score that maximises differences across the levels of the IVacross the levels of the IV
–– Problem:Problem: Instead of adapting model to observed DVs, selectively Instead of adapting model to observed DVs, selectively weight or discount DVs based on how they fit the model.weight or discount DVs based on how they fit the model.
•• AtheoreticalAtheoretical, over, over--capitalises on chancecapitalises on chanceTherefore, Therefore, don’t don’t use MANOVA approach to repeated measuresuse MANOVA approach to repeated measuresWith repeated measures designs, report the mixed model Fs not the With repeated measures designs, report the mixed model Fs not the MANOVA statisticsMANOVA statisticsUsually report GG Fs to ensure adjustment for sphericity violations Usually report GG Fs to ensure adjustment for sphericity violations which are common (regardless of which are common (regardless of Mauchley’sMauchley’s test, which is too test, which is too conservative and may not be sig. even when there are large conservative and may not be sig. even when there are large violations)violations)Personally I always use the GG or HF adjustment (HF can be more Personally I always use the GG or HF adjustment (HF can be more liberal) but report full liberal) but report full dfdf –– this is commonthis is common
6868
pros and conspros and cons
advantages of advantages of withinwithin--participants participants designs:designs:more efficientmore efficient–– nn Ps Ps in in jj treatments generate treatments generate njnj data pointsdata points–– simplifies proceduresimplifies procedure
more sensitivemore sensitive–– estimate individual differences estimate individual differences
((SSparticipantsSSparticipants) ) and remove from error termand remove from error term
35
6969
pros and conspros and consdisadvantages of disadvantages of withinwithin--participants participants designs:designs:
restrictive statistical assumptionsrestrictive statistical assumptionssequencing effects:sequencing effects:–– learning, practice learning, practice –– improved later regardless of manipulationimproved later regardless of manipulation–– Fatigue Fatigue –– deteriorating later regardless of manipulationdeteriorating later regardless of manipulation–– Habituation Habituation –– insensitivity to later manipulationsinsensitivity to later manipulations–– Sensitisation Sensitisation –– become more responsive to later manipulationsbecome more responsive to later manipulations–– Contrast Contrast –– previous treatment sets standard to which reactprevious treatment sets standard to which react–– Adaptation Adaptation –– adjustment to previous manipulations changes adjustment to previous manipulations changes
reaction to later reaction to later –– Direct carryDirect carry--over over –– learn something in previous that alters laterlearn something in previous that alters later–– Etc!Etc!
An essential methodological practice in RM designs is to An essential methodological practice in RM designs is to counterbalance to reduce sequencing effectscounterbalance to reduce sequencing effects–– i.e., half i.e., half participants participants receive order A1 then A2; half receive order A1 then A2; half
receive A2 then A1receive A2 then A1–– But can still get treatment x order interactionsBut can still get treatment x order interactions
7070
most important pointsmost important pointsin within in within participants participants anovaanova, the error term used for , the error term used for ANY effect is equal to ANY effect is equal to the interaction between that the interaction between that effect and the effect of effect and the effect of participantsparticipants (a random factor)(a random factor)–– this applies to:this applies to:
•• main effectsmain effects–– followfollow--up (main) comparisonsup (main) comparisons
•• interactionsinteractions–– simple effectssimple effects
followfollow--up (simple) comparisonsup (simple) comparisonsdue to problems causes by lack of compound due to problems causes by lack of compound symmetry/sphericity, adjustments (such as Greenhousesymmetry/sphericity, adjustments (such as Greenhouse--GeisserGeisser adjustment) to our degrees of freedom are adjustment) to our degrees of freedom are needed needed unless unless we used the we used the manovamanova approach, which we approach, which we shouldn’t, because it is inferiorshouldn’t, because it is inferior
36
7171
In class next week:In class next week:Mixed ANOVAMixed ANOVA
In the tutes:In the tutes:This weekThis week: Consult for A2: Consult for A2Next week: Next week: WithinWithin--participants and mixed participants and mixed designsdesigns
Readings :Readings :HowellHowell–– chapter 14chapter 14
FieldField–– Chapter 11Chapter 11