1

Regression as Moment Structure

2

Regression Equation Y = X + v

Observable Variables Y z = X

Moment matrix YY YX

= YX XX

Moment structure = 2XX +vv XX

= XX XX

Parameter vector = (, XX, vv )’

3

Sample: z1, z2, ..., zn n iid

Sample Moments

S = n-1 zi zi’

syy syx

S = syx sxx

Fitting S to =

Estimator S close to

3 moment equations•syy= 2XX +vv

•syx= XX •sxx= XX with 3 (unknown) parameters Parameter estimates

= (syx/sxx, sXX, syy - ()2sXX )’

is the same as theusual OLS estimate of

^ ^ ^

^ ^

^

4

Regression Equation Y = x + v X = x + u

Observable Variables Y z = X

Moment structure = 2XX +vv XX

= XX XX + uu

Parameter vector = (, XX, vv , uu )’

new parameter

5

Sample: z1, z2, ..., zn n iid

Sample Moments

S := n-1 zi zi’

syy syx

S = syx sxx

Fitting S to =

Estimator = S close to

3 moment equations•syy= 2xx +vv

•syx= xx •sxx= xx + uu

with 4 (unknown) parametersParameter estimates

= ??

is the same as theusual OLS estimate of

^ ^ ^

^

^

6

The effect of measurement error in regression

x Y

Xu

v

Y = (X -u)+ v = X + (v - u) = X + w, where w = v - u

Note that w is correlated with X, unless u or equals zero

So, the classical LS estimate b of is neither ubiased, neither consistent. In fact, b ---> YX/XX = xx/XX )= k

k is the so called Fiability coefficient (reliability of X). Since 0 k 1 b suffers from downward bias

7

Regression Equation Y = x1 + x2...+ pxp+ v Xk = xk + uk

Observable Variables b = SXX

-1SXY does not converge to

b* := (SXX - uu)-1 SXY

In multiple regression

Examples with EQS of regression with error in variables

Using suplementary information to assessing the magnitude of variances of errors in variables.

8

Path analysis & covariance structure

Example with ROS data

9

Sample covariance matrix

ROS92 ROS93 ROS94 ROS95ROS92 72.07ROS93 29.56 36.21ROS94 30.21 31.09 46.51ROS95 27.63 24.04 35.19 46.62

Mean: 6.27 7.35 10.02 8.80

n = 70

ROS92 ROS93 ROS94

F

b1 b2 b3

SEM:

bj = ?

It is a valid model ?

10

Calculations

b1b2= 29.56b1b3= 30.21b2b3= 31.09

b1b2/b1b3 = b2/b3 = 29.56/30.21--> b2 = .978b3

31.09 = b2b3= b3 (.978b3) --> b32= 31.09/.978

b3 = 5.64

In the same way, we obtain

b1=5.34b2=5.52

Model test in this case is CHI2 = 0, df = 0

11

Fitted Model

R92 R94R93

F

5.34 5.52 5.64

CHI2 = 0, df = 0

43.34 5.80 14.74

1

12

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS)/SPECIFICATIONS CAS=70; VAR=4; /LABEL V1=ROS92; V2=ROS93; V3=ROS94; V4=ROS95;/EQUATIONS V1 = *F1 + E1; V2 = *F1 + E2; V3 = *F1 + E3;/VARIANCES F1 = 1.0; E1 TO E3 = *;/COVARIANCES/MATRIX72.0729.56 36.2130.21 31.09 46.5127.63 24.04 35.19 46.62/END

13

ROS92 =V1 = 5.359*F1 + 1.000 E1 .974 5.504 ROS93 =V2 = 5.516*F1 + 1.000 E2 .650 8.482 ROS94 =V3 = 5.637*F1 + 1.000 E3 .753 7.482 VARIANCES OF INDEPENDENT VARIABLES ---------------------------------- E D --- --- E1 -ROS92 43.347*I I 8.205 I I 5.283 I I I I E2 -ROS93 5.789*I I 3.924 I I 1.475 I I I I E3 -ROS94 14.736*I I 4.693 I I 3.140 I I I I

14

… with the help of EQS

RESIDUAL COVARIANCE MATRIX (S-SIGMA) :   ROS92 ROS93 ROS94 V 1 V 2 V 3 ROS92 V 1 0.000 ROS93 V 2 0.000 0.000 ROS94 V 3 0.000 0.000 0.000

CHI-SQUARE = 0.000 BASED ON 0 DEGREES OF FREEDOM

STANDARDIZED SOLUTION:   ROS92 =V1 = .631*F1 + .776 E1 ROS93 =V2 = .917*F1 + .400 E2 ROS94 =V3 = .827*F1 + .563 E3

15

one - factor four- indicators model

R93 R95R94

F

* * *

CHI2 = ?, df = ?

p-value = ?

R92

*

****

16

… with the help of EQS /TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS)! This line is not read /SPECIFICATIONS CAS=70; VAR=4; /LABEL V1=ROS92; V2=ROS93; V3=ROS94; V4=ROS95;/EQUATIONS V1 = *F1 + E1; V2 = *F1 + E2; V3 = *F1 + E3; V4 = *F1 + E4;/VARIANCES F1 = 1.0; E1 TO E4 = *;/COVARIANCES/MATRIX72.0729.56 36.2130.21 31.09 46.5127.63 24.04 35.19 46.62/END

17

ROS92 =V1 = 4.998*F1 + 1.000 E1 .966 5.175 ROS93 =V2 = 4.837*F1 + 1.000 E2 .622 7.779 ROS94 =V3 = 6.417*F1 + 1.000 E3 .653 9.833 ROS95 =V4 = 5.393*F1 + 1.000 E4 .710 7.590 VARIANCES OF INDEPENDENT VARIABLES ---------------------------------- E D --- --- E1 -ROS92 47.090*I I 8.437 I I 5.581 I I I I E2 -ROS93 12.810*I I 2.775 I I 4.616 I I I I E3 -ROS94 5.332*I I 3.017 I I 1.767 I I I I E4 -ROS95 17.536*I I 3.682 I I 4.763 I I

… with the help of EQS

19

Fitted Model

R93 R95R94

F

4.84 6.42 5.40

CHI2 = 6.27, df = 2

p-value = .043

R92

4.99

17.545.3312.8147.10

20

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS)/SPECIFICATIONS CAS=70; VAR=4; /LABEL V1=ROS92; V2=ROS93; V3=ROS94; V4=ROS95;/EQUATIONS V1 = *F1 + E1; V2 = *F1 + E2; V3 = *F1 + E3; V4 = *F1 + E4;/VARIANCES F1 = 1.0; E1 TO E4 = *;/COVARIANCES/CONSTRAINTS (V1,F1)=(V2,F1)=(V3,F1)=(V4,F1); /MATRIX72.0729.56 36.2130.21 31.09 46.5127.63 24.04 35.19 46.62/END

21

… estimation results

ROS92 =V1 = 5.521*F1 + 1.000 E1 .528 10.450 ROS93 =V2 = 5.521*F1 + 1.000 E2 .528 10.450 ROS94 =V3 = 5.521*F1 + 1.000 E3 .528 10.450 ROS95 =V4 = 5.521*F1 + 1.000 E4 .528 10.450

CHI-SQUARE = 12.425 BASED ON 5 DEGREES OF FREEDOMPROBABILITY VALUE FOR THE CHI-SQUARE STATISTIC IS 0.02941

22

... EQS use an iterative optimization method

  ITERATIVE SUMMARY  PARAMETER ITERATION ABS CHANGE ALPHA FUNCTION 1 21.878996 1.00000 1.39447 2 5.741889 1.00000 0.43985 3 2.309283 1.00000 0.19638 4 0.477505 1.00000 0.18079 5 0.147232 1.00000 0.18014 6 0.056361 1.00000 0.18008 7 0.014530 1.00000 0.18007 8 0.005784 1.00000 0.18007 9 0.001423 1.00000 0.18007 10 0.000598 1.00000 0.18007

23

Exercise:

a) Write the covariance structure for the one - factor four- indicators modelb) From the ML estimates of this model, shown in previous slides, compute the fitted covariance matrix.c) In relation with b), compute the residual covariance matrix

Note: For c), use the following sample moments:

ROS92 ROS93 ROS94 ROS95ROS92 72.07ROS93 29.56 36.21ROS94 30.21 31.09 46.51ROS95 27.63 24.04 35.19 46.62

Mean: 6.27 7.35 10.02 8.80

n = 70

24

one - factor four- indicators model with means

R93 R95R94

F

* * *

CHI2 = ?, df = ?

p-value = ?

R92

*

****

1

**

* *

25

/TITLE FACTOR ANALYSIS MODEL (EXAMPLE ROS data)/SPECIFICATIONS CAS=70; VAR=4; ANALYSIS = MOMENT; /LABEL V1=ROS92; V2=ROS93; V3=ROS94; V4=ROS95;/EQUATIONS V1 = *V999+ *F1 + E1; V2 = *V999+ *F1 + E2; V3 = *V999+ *F1 + E3; V4 = *V999+ *F1 + E4;/VARIANCES F1 = 1.0; E1 TO E4 = *;/COVARIANCES/CONSTRAINTS ! (V1,F1)=(V2,F1)=(V3,F1)=(V4,F1);/MATRIX72.0729.56 36.2130.21 31.09 46.5127.63 24.04 35.19 46.62/MEANS6.27 7.35 10.02 8.80/END

26

ROS92 =V1 = 6.270*V999 + 4.998*F1 + 1.000 E1 1.022 .966 6.135 5.175 ROS93 =V2 = 7.350*V999 + 4.837*F1 + 1.000 E2 .724 .622 10.146 7.779 ROS94 =V3 = 10.020*V999 + 6.417*F1 + 1.000 E3 .821 .653 12.204 9.833 ROS95 =V4 = 8.800*V999 + 5.393*F1 + 1.000 E4 .822 .710 10.706 7.591 VARIANCES OF INDEPENDENT VARIABLES ---------------------------------- E D --- --- E1 -ROS92 47.092*I I 8.437 I I 5.582 I I I I E2 -ROS93 12.810*I I 2.775 I I 4.616 I I I I E3 -ROS94 5.332*I I 3.017 I I 1.767 I I I I E4 -ROS95 17.535*I I 3.682 I I 4.763 I I