Aims of a Variance Components Analysis • Estimate the amount of variation between groups (level 2 variance) relative to within groups (level 1 variance) – How much variation is there in life expectancy between and within countries? – How much of the variation in student exam scores is between schools? i.e. is there within-school clustering in achievement? • Compare groups – Which countries have particularly low and high life expectancies? – Which schools have the highest proportion of students achieving grade A-C, and which the lowest? • As a baseline for further analysis
Transcript
Aims of a Variance Components Analysis
• Estimate the amount of variation between groups (level 2 variance) relative to within groups (level 1 variance)– How much variation is there in life expectancy between and within
countries?– How much of the variation in student exam scores is between
schools? i.e. is there within-school clustering in achievement?
• Compare groups– Which countries have particularly low and high life expectancies?– Which schools have the highest proportion of students achieving
grade A-C, and which the lowest?
• As a baseline for further analysis
Revision of Fixed Effects Approach
One way of allowing for group effects is to include as explanatory variables a set of dummy variables for groups. E.g. for 20 countries define 2021 ,,, DDD such that 1jD for
individuals in country j and 0jD otherwise. Denote the coefficient of
jD by j and the overall intercept by 0 as before. Model for outcome
of individual i in country j is:
ijk
kkij eDy
20
10
Cannot estimate 0 and all 20 j s. We usually estimate 0 and take
one country as the reference, e.g. fixing 20 =0 makes country 20 the reference.
Limitations of the Fixed Effects Approach
• When number of groups is large, there will be many extra parameters to estimate. (Only one in ML model.)
• For groups with small sample sizes, the estimated group effects may be unreliable. (In a ML model residual estimates for such
groups ‘shrunken’ towards zero.)
• Fixed effects approach originated in experimental design where number of groups is small (e.g. treatment vs. control) and all groups sampled. More generally, our groups may be a sample from a population. (The multilevel approach allows inferences to this population.)
Multilevel (Random Effects) Model
ijy is the value of y for the i th individual in the j th group. A model that
allows for (random) group effects is:
ijjij euy 0
0 is the overall mean of y (across all groups).
ju0 is the mean of y for group j
ju is the difference between group j ’s mean and the overall mean.
ije is the difference between the y-value for the i th individual and that
individual’s group mean, i.e. )( 0 jijij uye .
Individual (e) and Group (u) Residuals in a Variance Components Model
0
y42
e42
u2
u1
Partitioning Variance
Assume ),0(~ 2uj Nu and ),0(~ 2
eij Ne
2u is the between-group (level 2) variance and 2
e is the within-group (level 1) variance
Total variance = 2u + 2
e Variance partition coefficient is proportion of total variance due to differences between groups
22
2
eu
uVPC
= 0 if no group effects and 1 if no within-group differences.
Model Assumptions
Individual- and group-level residuals are normally distributed ),0(~ 2
uj Nu and ),0(~ 2eij Ne
Residuals at the same level are uncorrelated with one another
0),cov(
21jj uu for two different groups
0),cov(
2211jiji ee for two different individuals in the same or different groups
Residuals at different levels are uncorrelated with one another
0),cov(21
jij ue for the same or different groups
Intra-class Correlationρ = correlation between responses for 2 individuals in the same group:
)var()var(
),cov(),corr(
21
21
21jiji
jijijiji yy
yyyy
Now 2)var(),cov(),cov(),(co2121 ujjjjijjijjiji uuueueuyyv
because 0),cov(),cov(),(co2112
iijijjij eeeueuv
and 22)var()var()var( euijjij euy
so VPCeu
u
22
2
Examples of ρ
0
10
●●●●●
●●●●●
●●●●●
●●●●●
School1 2 3 4
0
10
●●●●●
●●●●●
●●●●●
●●●●●
School1 2 3 4
0
10●●●●●
●●●●●
●●●●●
●●●●●
School1 2 3 4
ρ ≈ 0 ρ ≈ 0.4 ρ ≈ 0.8
Note: Schools ranked according to mean of y
Example: Between-Country Differences in Hedonism
Parameter Estimate Standard error
0 -0.203 0.069 2u 0.094 0.030
2e 0.885 0.007
Total variance = 0.094+0.885=0.979 ρ = 0.094/(0.094 + 0.885) = 0.096. Thus 9.6% of the total variance in hedonism scores is due to between-country differences. Can also interpret 0.096 as the correlation between the hedonism scores for two randomly selected individuals from the same country.
Testing for Group Effects
We wish to test the null hypothesis (H0) that 2u =0, i.e. no between-
group variance. We compare a single-level (SL) model ii ey 0 and a multilevel (ML) model ijjij euy 0 in a likelihood ratio test.
Test statistic is LR = -2 log LSL – (-2 log LML) where LSL and LML are likelihood values for the two models.
The test statistic LR is compared with a chi-squared distribution. Here
we have 1 extra parameter ( 2u ) so d.f. = 1. [Sometimes take (p-
value)/2 as correction for fact that 2u must always be greater than 0]
Example: Testing for Country Differences in Hedonism
Null hypothesis is that there are no differences between countries in their mean hedonism scores, i.e. zero between-country variance. -2 log LSL = 102590 -2 log LML = 99303 LR = -2 log LSL – (-2 log LML) = 3287 which we compare to a chi-squared distribution on 1 d.f. The critical value for a test at the 5% significance level is 3.84, so strong evidence of betwen-country differences.
Residuals in a Variance Components Model
Group effects ju (level 2 residuals) are random variables assumed to
follow a normal distribution, so their distribution is summarised by two
parameters: the mean (fixed at zero) and variance 2u .
To make comparisons among groups we need to estimate ju . These
estimates are obtained after fitting the model. The total residual is ijj eu , estimated as 0ˆˆ ijijijij yyyr .
Need to split this into separate estimates of ju and ije .
Level 2 Residuals
A starting point for an estimate of ju would be to take the mean of
0ijy for group j . This is sometimes called the mean raw residual:
0 jj yr
where jy is the mean of ijy in group j .
We multiply this raw residual by a factor k called the shrinkage factor:
jj rku ˆ
where )/ˆ(ˆ
ˆ22
2
jeu
u
nk
and jn is sample size in group j
Shrinkage
k is always lies between 0 and 1 so jj ru ˆ
For large jn , k will be close to 1 and so ju will be close to jr
k also close to 1 when 2ˆe small relative to 2ˆu
Greater shrinkage (k closer to zero) when jn small or 2ˆe is large
relative to 2ˆu (high within-group variability), i.e. when we have little
information about the group. Then the group mean juˆ0 is pulled
towards the overall mean 0
Example: Mean Raw Residuals vs. Shrunken Residuals for Selected Countries
In this case, there is little shrinkage because nj is very large.
Example: Caterpillar Plot showing Country Residuals and 95% CIs
Residual Diagnostics
• Use normal Q-Q plots to check assumptions that level 1 and 2 residuals are normally distributed– Nonlinearities suggest departures from normality
• Residual plots can also be used to check for outliers at either level– Under normal distribution assumption, expect 95% of standardised residuals to lie between -2 and +2
– Can assess influence of a suspected outlier by comparing results after its removal
Example: Normal Plot of Individual (Level 1) Residuals
Linearity suggestsnormality assumption is reasonable
Example: Normal Plot of Country (Level 2) Residuals
