Developing a Mixed Effects Model Using SAS PROC

MIXED

Lauren Ackerman Katherine Morgan

Rai Oshima

Purpose of the Pilot Study

1. How accurately can participants identify gender from a handwriting sample?

2. Does accuracy improve with feedback?

Demographic Information

Group Feedbackn = 13

No Feedbackn=13

Waves Wave 1n = 21

Wave 2n = 25

Wave 3n = 24

Demographics AgeMean = 26.42Std Dev = 5.07Min = 17Max = 40

GenderM = 8F = 18

Dominant HandR = 23L = 1Missing = 2

SatisfactionY = 20N = 4Missing = 2

PredictY = 15N = 9Missing = 2

Missing Data!!!

Writing Samples

1. 2.

3. 4.

5. 6.

Why SAS:PROC SGPANEL

• Visualize change over time for each subject

proc sgpanel data = data_long;title 'Empirical Growth Plots of Score for Participants';label score = 'Score (# Correct out of 44)’ time = 'Time’;panelby id / columns = 3 rows = 5;reg y = score x = time;run;

PROC SGPLOT

proc sgplot data = data_long noautolegend;title 'OLS Trajectories Across Participants';yaxis min=0 max=50;reg x = time y = score / group = id

nomarkers lineattrs = (color = gray pattern = 1 thickness=1);

reg x = time y = score / nomarkers lineattrs = (color = red

pattern = 1 thickness=3);run;quit;

OLS Assumptions

1. Normality2. Homoscedasticity3. Zero Correlation

Why PROC MIXED?

Modeling Covariance Structure

Unstructured Covariance Model

Independence Covariance Model

cs20

220

20

20

20

220

20

20

20

2

Compound Symmetry Covariance Model

in

2

2

2

00

00

00

un

232313

232212

131221

Missing Data

PROC REG vs. PROC MIXED

MAR

Missing At Random

proc mixed data = hand_long method=ml;model score = time / solution;

run;

proc reg data = hand_long;model score = time;

run; quit;

TimeS 78.122.29ˆ

General Multilevel Model

)(

,0

0~),0(~

101000

2110

0120

1

02

1101

0000

10

ijiiijijij

i

iij

ii

ii

ijijiiij

TimeTimeS

NandNwhere

TimeS

Independence vs. Multilevel Model

0: 0121

200 H

Unconditional Growth Model

PROC MIXED Output

PROC REG Output

Covariances and Correlations

000.1

579.0000.1

553.0512.0000.1

273.16

199.14

124.12

)(

2

2Re

2Re

'21'01

20

ReRe

201

221

20

2Re

'

'

jj

jj

j

sidualsidual

jjjjsidualsidual

jjsidual

TimeTimeTimeTime

TimeTime

Covariances Correlation Matrix

Multilevel Model with Group

)(

,0

0~),0(~

1011100100

2110

0120

1

02

111101

001000

10

ijiiijijiijiij

i

iij

iii

iii

ijijiiij

TimeTimeGroupTimeGroupS

nandNwhere

Group

Group

TimeS

Fixed Effects Model AnalysisFixed Effects Parameter Model A Model B Model C Model D Model E Model F

Initial Status Π0i Intercept γ0031.0695

(0.6917)***

29.2597(0.8362)***

30.2060(0.7767)***

30.1538(0.7893)***

32.3241(1.3379)***

32.1517(1.1584)***

Happy(N = 1)

γ01 -5.3523(1.9405)*

-4.4149(1.7057)*

-4.5501(1.4950)**

-4.5436(1.4948)**

Centered Age(Age – 17)

γ02 -0.2769(0.1360)

-0.2568(0.1112)*

Rate of Change Π1i Interceptγ10 1.7341

(0.5303)**

1.5964(0.5696)*

1.6681(0.5295)**

1.4907(0.9542)

1.6977(0.5090)**

Happyγ11 1.2414

(1.4141)

Centered Age(Age – 17)

γ12 0.02611(0.1018)

Variance Components

Level 1 Within Person σε15.0951

(3.2023)***

12.1240(2.5832)***

11.6501(2.4676)***

12.1747(2.6187)***

11.0328(2.2713)***

11.0500(2.2724)***

Level 2In Initial Status

σ06.6775

(3.6172)*

5.6867(5.9618)

0.5907(4.1543)

0.9881(4.8273) 0 0

Rate of Change

σ1 0 0 0 0 0

Covariance σ011.0373

(3.0032)2.0230

(2.2735)2.3530

(3.1340)1.8423

(1.3173)1.8375

(1.3107)Ry,y 0.0930 0.2643 0.2037 0.2862 0.2857

Rε 0.1968 0.2282 0.1935 0.2691 0.2678

R0 0.1484 0.9115 0.8520 1.0000 1.0000

Deviance 408.9 399.0 387.7 393.0 365.9 366.0

AIC 414.9 409.0 405.7 407.0 379.9 378.0

BIC 418.6 415.3 417.0 415.8 388.2 385.1

2

2

2

2

2

2

Final Model

• Model F provided the best deviance statistic– Satisfaction with handwriting and age were the

only significant predictors for intercept– No significant predictor for slope besides time

ijiiij TimeAgeUnhappyS 70.1)17(26.054.415.32ˆ

Fit Statistics for Covariance Models

Independence

Standard

Unstructured

Compound Symmetry

Heterogeneous

CompoundSymmetry

First-OrderAutoregressi

ve

Heterogeneous

Autoregressive

Toeplitz

-2RLL 369.1 366.5 364.6 366.5 365.3 369.1 368.2 357.3

AIC 371.1 370.5 368.6 370.5 373.3 373.1 376.2 363.3

AICC 372.2 370.7 368.8 370.7 374.0 373.3 376.9 363.7

BIC 373.2 372.9 371.0 372.9 378.0 375.4 380.9 366.8Devia

nce

Sta

tisti

cs

20

220

20

20

20

220

20

20

20

2

Pilot Study Results

1. How accurately can participants identify gender from a handwriting sample?

Baseline 69.91% accuracy; 95% CI (65.60%,74.23%)

Time important predictor (Estimate 1.73, p<0.01)

2. Does accuracy improve with feedback?

Group not significant (Estimate 0.37, p = 0.79)

ConclusionWhy SAS?• Graphical and mixed effects modeling

capability

Why PROC MIXED?• Allows autocorrelation and

homoscedasticity• Flexibility in modeling the within subject

variability • Handles missing data• Inclusion of time-varying predictors