Multiple Regression. The equation that describes how the dependent variable y is related to the independent variables :. x 1 , x 2 , . . . x p. and error term e is called the multiple regression model. y = b 0 + b 1 x 1 + b 2 x 2 + . . . + b p x p + e. - PowerPoint PPT Presentation
77
The equation that describes how the dependent variable y is related to the independent variables: Multiple Regression y = b 0 + b 1 x 1 + b 2 x 2 + . . . + b p x p + e e is a random variable called the error term x 1 , x 2 , . . . x p and error term e is called the multiple regression model . The equation that describes how the mean value of y is related to the p independent variables is called the multiple regression equation : E(y) = b 0 + b 1 x 1 + b 2 x 2 + . . . + b p x p where: b 0 , b 1 , b 2 , . . . , b p are parameters
Transcript
The equation that describes how the dependent variable y is related to the independent variables:
Multiple Regression
y = b0 + b1x1 + b2x2 + . . . + bpxp + e
e is a random variable called the error term
x1, x2, . . . xp
and error term e is called the multiple regression model.
The equation that describes how the mean value of y is related to the p independent variables is called the multiple regression equation:
E(y) = 0 + 1x1 + 2x2 + . . . + pxp
where: b0, b1, b2, . . . , bp are parameters
A simple random sample is used to compute sample statistics
y = b0 + b1x1 + b2x2 + . . . + bpxp
b0, b1, b2, . . . , bp
b0, b1, b2, . . . , bp
that are used as the point estimators of the parameters
The equation that describes how the predicted value of y is related to the p independent variables is called the estimated multiple regression equation:
Multiple Regression
Specification
1. Formulate a research question:
How has welfare reform affected employment of low-income mothers?
Issue 1: How should welfare reform be defined?
Since we are talking about aspects of welfare reform that influence the decision to work, we include the following variables:
• Welfare payments allow the head of household to work less.
tanfben3 = real value (in 1983 $) of the welfarepayment to a family of 3 (x1)
• The Republican lead Congress passed welfare reform twice, both of which were vetoed by President Clinton. Clinton signed it into law after the Congress passed it a third time in 1996. All states put their TANF programs in place by 2000.
2000 = 1 if the year is 2000, 0 if it is 1994 (x2)
Specification
1. Formulate a research question:
How has welfare reform affected employment of low-income mothers?
Issue 1: How should welfare reform be defined? (continued)
• Families receive full sanctions if the head of household fails to adhere to a state’s work requirement.
fullsanction = 1 if state adopted policy, 0 otherwise (x3)
Issue 2: How should employment be defined?
• One might use the employment-population ratio of Low-Income Single Mothers (LISM):
number of LISM that are employed
number of LISM livingepr
Specification
2. Use economic theory or intuition to determine what the true regression model might look like.
40
400
U0
55
550 U1
300
Leisure
Consumption
Receiving the welfare check
increases LISM’s leisure which decreases hours worked
Use economic graphs to derive testable hypotheses:
Economic theory suggests the following is not true:
Ho: b1 = 0
Specification
2. Use economic theory or intuition to determine what the true regression model might look like.
Use a mathematical model to derive testable hypotheses:
Economic theory suggests the following is not true:
Ho: b1 = 0
max ( , )
. .
80
U C L C L
s t
H L
C P wH
The solution of this problem is:
* 402
PL
w
* 10
2
L
P w
1 0 *
0H
P
Specification
3. Compute means, standard deviations, minimums and maximums for the variables.
state year epr tanfben3 fullsanction black dropo unemp
Alabama 1994 52.35 110.66 0 25.69 26.99 5.38
Alaska 1994 38.47 622.81 0 4.17 8.44 7.50
Arizona 1994 49.69 234.14 0 3.38 13.61 5.33
Arkansas 1994 48.17 137.65 0 16.02 25.36 7.50
West Virginia 2000 51.10 190.48 1 3.10 23.33 5.48
Wisconsin 2000 57.99 390.82 1 5.60 11.84 3.38
Wyoming 2000 58.34 197.44 1 0.63 11.14 3.81
Specification
3. Compute means, standard deviations, minimums and maximums for the variables.
4. Construct scatterplots of the variables. (1994, 2000)
Specification
5. Compute correlations for all pairs of variables. If | r | > .7 for a pair of independent variables, • multicollinearity may be a problem• Some say avoid including independent variables that are highly
correlated, but it is better to have multicollinearity than omitted variable bias.
epr fullsanction black dropo unemp
tanfben3 -0.03 -0.24 -0.53 -0.50 0.10
unemp -0.64 -0.51 0.16 0.47
dropo -0.44 -0.25 0.51
black -0.32 0.07
fullsanction 0.43
Estimation
Least Squares Criterion: 2 2min ( ) min ( )i i iy y ˆe
Computation of Coefficient Values:
In simple regression:
You can use matrix algebra or computer software packages to compute the coefficients
In multiple regression:
11
1
cov( , )var( )
x yb
x
0 1 1b y bx
1( )b XX Xy
0
1
p
b
b
b
Simple Regression
state yearepry
tanfben3
Alabama 1994 52.35 110.66
Alaska 1994 38.47 622.81
Arizona 1994 49.69 234.14
Arkansas 1994 48.17 137.65
West Virginia 2000 51.10 190.48
Wisconsin 2000 57.99 390.82
Wyoming 2000 58.34 197.44
Mean 50.23 250.04
Std dev 8.86 99.03
Covariance -24.70
11
1
1 2
1
cov( , )var( )
24.7099.03
0.0025
x yb
x
b
b
0 1 1
0
0
1
50.23 ( .0025)250.04
50.869
ˆ 50.869 .0025
b y bx
b
b
y x
1x
2( )x x SSE SSR
100 Squared
Residuals
epry
tanfben3
52.35 110.66 50.59 19425.76 4.47 3.10 0.13
38.47 622.81 49.28 138957.04 138.45 117.04 0.90
49.69 234.14 50.27 252.64 0.29 0.34 0.00
48.17 137.65 50.52 12630.56 4.25 5.51 0.08
51.10 190.48 50.38 3547.56 0.75 0.51 0.02
57.99 390.82 49.87 19820.99 60.19 65.87 0.13
58.34 197.44 50.37 2766.00 65.79 63.64 0.02
Sum 970931.62 7764.76 7758.48 6.28
1ˆ 50.869 .0025y x
y1x
21 1( )x x 2( )y y 2ˆ( )y y 2ˆ( )y y
SST
50.23y1 250.04x
Simple Regression
Test for Significance at the 5% level (a = 0.05)
t.025 = 1.984
1
1-statb
bt
s
s 2 = SSE/(n – 2)
8.898s
1 2( )
b
i
ss
x x
df = 100 – 2 = 98
We cannot reject
a = .05 a/2 = .025
= 7758.48/(100 – 2) = 79.17
8.898
970931.620.0090
.277
.0025.0090
0 1: 0H
-t.025 = -1.984
Simple Regression
• If estimated coefficient b1 was statistically significant, we would interpret its value as follows:
11
yb
x
.0025 -.0025
+1
Increasing monthly benefit levels for a family of three by $100 lowers the epr of LISM by 0.25 percentage points.
Simple Regression
100
100
-.25
+100
• However, since estimated coefficient b1 is statistically insignificant, we interpret its value as follows:
Increasing monthly benefit levels for a family of three
has no effect on the epr of LISM.
Our theory suggests that this estimate is biased towards zero
Simple Regression
Regression Statistics
Multiple R 0.0284
R Square 0.0008
Adjusted R Square -0.0094
Standard Error 8.8977
Observations 100
ANOVA
df SS MS F
Regression 1 6.281 6.281 0.079
Residual 98 7758.483 79.168
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 50.8687 2.427 20.961 0.000
tanfben3 -0.0025 0.009 -0.282 0.779
r 2·100% of the variability in y
can be explained by the model.
.08%epr of LISM
Error
Simple Regression
state yearepry
tanfben3_ln
Alabama 1994 52.35 4.71
Alaska 1994 38.47 6.43
Arizona 1994 49.69 5.46
Arkansas 1994 48.17 4.92
West Virginia 2000 51.10 5.25
Wisconsin 2000 57.99 5.97
Wyoming 2000 58.34 5.29
Mean 50.23 5.44
Std dev 8.86 0.41
Covariance 0.10
11
1
1 2
1
cov(ln , )var(ln )
0.100.41
0.6087
x yb
x
b
b
0 1 1
0
0
1
ln
50.23 (.6087)5.44
46.9192
ˆ 46.9192 .6026ln
b y b x
b
b
y x
1lnx
21 1(ln ln )x x SSE SSR
100 Squared
Residuals
epry
tanfben3_ln
52.35 4.71 49.78 0.543 4.47 6.57 0.20
38.47 6.43 50.84 0.982 138.45 153.01 0.36
49.69 5.46 50.24 0.000 0.29 0.30 0.00
48.17 4.92 49.92 0.269 4.25 3.05 0.10
51.10 5.25 50.11 0.038 0.75 0.97 0.01
57.99 5.97 50.55 0.275 60.19 55.33 0.10
58.34 5.29 50.14 0.025 65.79 67.36 0.01
Sum 16.28 7764.76 7758.73 6.03
1ˆ 46.9192 .6087 lny x
y1lnx
21 1(ln ln )x x 2( )y y 2ˆ( )y y 2ˆ( )y y
SST
50.23y1ln 5.44x
Simple Regression
Test for Significance at the 5% level (a = 0.05)
t.025 = 1.984
1
1-statb
bt
s
s 2 = SSE/(n – 2)
8.898s
1 2
1 1(ln ln )b
ss
x x
df = 100 – 2 = 98
We cannot reject
a = .05 a/2 = .025
= 7758.73/(100 – 2) = 79.17
8.89816.28
2.2055
.2760.60872.2055
0 1: 0H
-t.025 = -1.984
Simple Regression
• If estimated coefficient b1 was statistically significant, we would interpret its value as follows:
ˆ(292) 46.9192 .6087 ln(29ˆ(266) 46.91922 .6087 ln(26) 6)y y
.6087 ln(292 .6087 ln(266)ˆ )y
ln(292ˆ .6 ) ln(266)087y
ˆ .6087 l 292 6n( 2 6/ )y
ˆ .6087 ln(1.10)y
Suppose we want to know what happens to the epr of LISM if a state decides to increase its welfare payment by 10%.
When we use a logged dollar-valued independent variable we have to do the following first to interpret the coefficient:
ˆ .6087 ln(1.10) .058y . )10 .058
• If estimated coefficient b1 was statistically significant, we would interpret its value as follows:
Increasing monthly benefit levels for a family of three by 10% would result in a .058 percentage point increase in the average epr
of LISM
Simple Regression
• However, since estimated coefficient b1 is statistically insignificant, we interpret its value as follows:
Increasing monthly benefit levels for a family of three
has no effect on the epr of LISM.
Our theory suggests that this estimate has the wrong sign and is biased towards zero. This bias is called omitted variable bias.ˆ .6087 ln(1.10)y
Simple Regression
Regression Statistics
Multiple R 0.0279
R Square 0.0008
Adjusted R Square -0.0094
Standard Error 8.8978
Observations 100
ANOVA
df SS MS F
Regression 1 6.031 6.031 0.076
Residual 98 7758.733 79.171
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 46.9192 12.038 3.897 0.000
tanfben3_ln 0.6087 2.206 0.276 0.783
r 2·100% of the variability in y
can be explained by the model.
.08%epr of LISM
Error
Least Squares Criterion:
2 2min ( ) min ( )i i iy y ˆe
You can use matrix algebra or computer software packages to compute the coefficients
In multiple regression the solution is:
1( )b XX Xy
0
1
p
b
b
b
Multiple Regression
Multiple RegressionR Square 0.166
Adjusted R Square 0.149
Standard Error 8.171
Observations 100
ANOVA
df SS MS F
Regression 2 1288.797 644.398 9.652
Residual 97 6475.967 66.763
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 35.901 11.337 3.167 0.002
tanfben3_ln 1.967 2.049 0.960 0.339
2000 7.247 1.653 4.383 0.000
r 2·100% of the variability in y
can be explained by the model.
15%epr of LISM
Error
Multiple RegressionR Square 0.214
Adjusted R Square 0.190
Standard Error 7.971
Observations 100
ANOVA
df SS MS F
Regression 3 1664.635 554.878 8.732
Residual 96 6100.129 63.543
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 31.544 11.204 2.815 0.006
tanfben3_ln 2.738 2.024 1.353 0.179
2000 3.401 2.259 1.506 0.135
fullsanction 5.793 2.382 2.432 0.017
r 2·100% of the variability in y
can be explained by the model.
19%epr of LISM
Error
Multiple RegressionR Square 0.517
Adjusted R Square 0.486
Standard Error 6.347
Observations 100
ANOVA
df SS MS F
Regression 6 4018.075 669.679 16.623
Residual 93 3746.689 40.287
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 104.529 15.743 6.640 0.000
tanfben3_ln -5.709 2.461 -2.320 0.023
2000 -2.821 2.029 -1.390 0.168
fullsanction 3.768 1.927 1.955 0.054
black -0.291 0.089 -3.256 0.002
dropo -0.374 0.202 -1.848 0.068
unemp -3.023 0.618 -4.888 0.000
Error
Multiple RegressionR Square 0.517
Adjusted R Square 0.486
Standard Error 6.347
Observations 100
ANOVA
df SS MS F
Regression 6 4018.075 669.679 16.623
Residual 93 3746.689 40.287
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 104.529 15.743 6.640 0.000
tanfben3_ln -5.709 2.461 -2.320 0.023
2000 -2.821 2.029 -1.390 0.168
fullsanction 3.768 1.927 1.955 0.054
black -0.291 0.089 -3.256 0.002
dropo -0.374 0.202 -1.848 0.068
unemp -3.023 0.618 -4.888 0.000
Error
Multiple RegressionR Square 0.517
Adjusted R Square 0.486
Standard Error 6.347
Observations 100
ANOVA
df SS MS F
Regression 6 4018.075 669.679 16.623
Residual 93 3746.689 40.287
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 104.529 15.743 6.640 0.000
tanfben3_ln -5.709 2.461 -2.320 0.023
2000 -2.821 2.029 -1.390 0.168
fullsanction 3.768 1.927 1.955 0.054
black -0.291 0.089 -3.256 0.002
dropo -0.374 0.202 -1.848 0.068
unemp -3.023 0.618 -4.888 0.000
r 2·100% of the variability in y
can be explained by the model.
49%epr of LISM
Error
Multiple Regression
1 2 3 4 5 6ˆ 104.529 5.709ln 2.821 3.768 0.291 0.374 3.023y x x x x x x
Coefficients Standard Error t Stat P-value
Intercept 104.529 15.743 6.640 0.000
tanfben3_ln -5.709 2.461 -2.320 0.023
2000 -2.821 2.029 -1.390 0.168
fullsanction 3.768 1.927 1.955 0.054
black -0.291 0.089 -3.256 0.002
dropo -0.374 0.202 -1.848 0.068
unemp -3.023 0.618 -4.888 0.000
lnx1
x2
x3
x4
x5
x6
+
1. E(e) is equal to zero
2. Var() = s 2 is constant for all values of x1…xp
3. Error is normally distributed
4. The values of are independent
5. The true model is linear:
These assumptions can be addressed looking at the residuals:
Recall from chapter 14 that t and F tests are valid if the error term’s assumptions are valid:
ei = yi – yi^
y = b0 + b1∙ x1 + b2∙ x2 + … + bp∙ xp + e
Validity
The residuals provide the best information about the errors.
1. E(e) is probably equal to zero since E(e) = 0
2. Var() = s 2 is probably constant for all values of x1…xp if “spreads” in scatterplots of e versus y, time, x1…xp appear to be constant or White’s squared residual regression model is statistically insignificant
3. Error is probably normally distributed if the chapter 12 normality test indicates e is normally distributed
4. The values of are probably independent if the autocorrelation residual plot or Durbin-Watson statistics with various orderings of the data (time, geography, etc.) indicate the values of e are independent
5. The true model is probably linear if the scatterplot of e versus y is a horizontal, random band of points
^
Note: If the absolute value of the i th standardized residual > 2, the i th observation is an outlier.
Validity
^
E(e) is probably equal to zero since E(e) = 0
Zero Mean
1 2 3 4 5 6ˆ 104.529 5.709ln 2.821 3.768 0.291 0.374 3.023y x x x x x x
epry
tanfben3_ln 2000 fullsanction black dropo unemp epr hat residuale
52.35 4.71 0 0 25.69 26.99 5.38 43.83 8.52
38.47 6.43 0 0 4.17 8.44 7.50 40.76 -2.29
49.69 5.46 0 0 3.38 13.61 5.33 51.19 -1.50
48.17 4.92 0 0 16.02 25.36 7.50 39.60 8.57
51.10 5.25 1 1 3.10 23.33 5.48 49.31 1.79
57.99 5.97 1 1 5.60 11.84 3.38 55.14 2.85
58.34 5.29 1 1 0.63 11.14 3.81 59.44 -1.10
Sum 0
1lnx 2x 3x 4x 5x 6x y
Var() = s 2 is probably constant for all values of x1…xp if “spreads” inscatterplots of e versus y, t, x1…xp appear to be constant
• The only assumed source of variation on the RHS of the regression model is in the errors (ej ), and that residuals (ei ) provide the best information about them.
• The means of e and e are equal to zero.
• The variance of e estimates e:
≈
Constant Variance(homoscedasticity)
^
22
( 0)j
N
22 ( 0)
1ie
sn p
2j
N
2
1ie
n p
1
SSE
n p
• Non-constant variance of the errors is referred to as heteroscedasticity.
• If heteroscedasticity is a problem, the standard errors of the coefficients are wrong.
Heteroscedasticity is likely present if scatterplots of residuals versus t, y, x1, x2 … xp are not a random horizontal band of points.
Constant Variance(homoscedasticity)
^
-15
-10
-5
0
5
10
15
20
30 40 50 60 70
predicted epr
resid
ual
-15
-10
-5
0
5
10
15
20
0 10 20 30 40
black
resid
ual
-15
-10
-5
0
5
10
15
20
4 5 6 7
tanfben3_ln
resid
ual
-15
-10
-5
0
5
10
15
20
0 10 20 30
dropo
resid
ual
-15
-10
-5
0
5
10
15
20
0 2 4 6 8 10
unemp
resid
ual
Non-constant variance in black?
To test for heteroscedasticity, perform White’s squared residual regression by regress e2 on
x1, x2 … xp
x1x2, x1x3 … x1xp , x2x3, x2x4 … x2xp … xp – 1, xp
x1, x2 … xp
Constant Variance(homoscedasticity)
2 2 2
R Square 0.296
Adjusted R Square 0.058
Standard Error 51.205
Observations 100
ANOVA
Df SS MS F
Regression 25 81517 3261 1.24
Residual 74 194024 2622
Total 99 275541
If F-stat > F05 , we reject
H0: no heteroscedasticity
25
1.24
s 2 is probably constant
<
1.24
74
1.66
Constant Variance(homoscedasticity)
If heteroscedasticity is a problem,
• Estimated coefficients aren’t biased
• Coefficient standard errors are wrong
• Hypothesis testing is unreliable
In our example, heteroscedasticity does not seem to be a problem.
If heteroscedasticity is a problem, do one of the following:
• Use Weighted Least Squares with 1/xj or 1/xj0.5 as weights where xj is the
variable causing the problem
• Compute “Huber-White standard errors”
1
1-statb
bt
s
Normality
Error is probably normally distributed if the chapter 12 normality test indicates e is normally distributed
Histogram of residuals
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10
residuals
freq
uen
cy
-20 -16 -12 -8 -4 0 4 8 12 16 20
Ha: errors are not normally distribution
H0: errors are normally distributed
The test statistic:
k = 100/5 = 20 20 equal intervals.
has a chi-square distribution, if ei > 5.
22
1
( )-stat i
k
i
i
i
ef
e
To ensure this, we divide the normal distribution into k intervals all having the same expected frequency.
The expected frequency: ei = 5
Normality
Error is probably normally distributed if the chapter 12 normality test indicates e is normally distributed
z.
1/20 = .0500
The probability of being in this interval is
Normality
-1.645
Standardized residuals:mean = 0 std dev = 1
1.645
z.
2/20 = .1000
The probability of being in this interval is
Normality
-1.282
Standardized residuals:mean = 0 std dev = 1
1.282
z.
3/20 = .1500
The probability of being in this interval is
Normality
-1.036
Standardized residuals:mean = 0 std dev = 1
1.036
z.
4/20 = .2000
The probability of being in this interval is
Normality
-0.842
Standardized residuals:mean = 0 std dev = 1
0.842
z.
5/20 = .2500
The probability of being in this interval is
Normality
-0.674
Standardized residuals:mean = 0 std dev = 1
0.674
z.
6/20 = .3000
The probability of being in this interval is
Normality
-0.524
Standardized residuals:mean = 0 std dev = 1
0.524
z.
7/20 = .3500
The probability of being in this interval is
Normality
-0.385
Standardized residuals:mean = 0 std dev = 1
0.385
z.
8/20 = .4000
The probability of being in this interval is
Normality
-0.253
Standardized residuals:mean = 0 std dev = 1
0.253
z.
9/20 = .4500
The probability of being in this interval is
Normality
-0.126
Standardized residuals:mean = 0 std dev = 1
0.126
z.
10/20 = .5000
The probability of being in this interval is
Normality
0
Standardized residuals:mean = 0 std dev = 1
Normality
Observation Pred epr Residuals Std Res
1 54.372 -12.572 -2.044
2 55.768 -12.430 -2.021
3 55.926 -11.412 -1.855
4 54.930 -10.938 -1.778
5 62.215 -10.036 -1.631
6 59.195 -9.302 -1.512
7 54.432 -9.239 -1.502
8 37.269 -8.291 -1.348
9 48.513 -8.259 -1.343
10 44.446 -7.963 -1.294
11 43.918 -7.799 -1.268
99 50.148 15.492 2.518
100 58.459 16.259 2.643
-infinity to -1.645
f1 = 4
Count the number of residuals that are in the FIRST interval:
Normality
Observation Pred epr Residuals Std Res
1 54.372 -12.572 -2.044
2 55.768 -12.430 -2.021
3 55.926 -11.412 -1.855
4 54.930 -10.938 -1.778
5 62.215 -10.036 -1.631
6 59.195 -9.302 -1.512
7 54.432 -9.239 -1.502
8 37.269 -8.291 -1.348
9 48.513 -8.259 -1.343
10 44.446 -7.963 -1.294
11 43.918 -7.799 -1.268
99 50.148 15.492 2.518
100 58.459 16.259 2.643
-1.645 to -1.282
Count the number of residuals that are in the SECOND interval:
f2 = 6
Normality
LL UL f e f – e (f – e)2/e
−∞ -1.645 4 5 -1 0.2
-1.645 -1.282 6 5 1 0.2
-1.282 -1.036 4 5 -1 0.2
-1.036 -0.842 4 5 -1 0.2
-0.842 -0.674 9 5 4 3.2
-0.674 -0.524 7 5 2 0.8
-0.524 -0.385 5 5 0 0
-0.385 -0.253 3 5 -2 0.8
-0.253 -0.126 4 5 -1 0.2
-0.126 0.000 7 5 2 0.8
0.000 0.126 2 5 -3 1.8
Normality
LL UL f e f – e (f – e)2/e
0.126 0.253 3 5 -2 0.8
0.253 0.385 7 5 2 0.8
0.385 0.524 5 5 0 0
0.524 0.674 7 5 2 0.8
0.674 0.842 5 5 0 0
0.842 1.036 5 5 0 0
1.036 1.282 3 5 -2 0.8
1.282 1.645 5 5 0 0
1.645 ∞ 5 5 0 0
c 2-stat = 11.6
= .05 (column)
27.587
Do Not Reject H0 Reject H0
.05
2
There is no reason to doubt the assumption that the errors are normally
distributed.
2 -stat
df = 20 – 3 = 17 (row) 2.05 27.587
11.6 17
Normality
Normality
The previous test of normally distributed residuals was used because it was the test we conducted in chapter 12. There are a number of normality tests one can chose.
• The Jarque-Bera test involves using the skew and kurtosis of the residuals.
• The test statistic follows a chi-square distribution with 2 degrees of freedom:
kurtosis measures "peakedness" of the probability distribution. • High kurtosis → sharp peak, low kurtosis → flat peak.• involves raising standardized residuals to the 4th power • Excel: =kurt(A1:A100) → 0.0214
skewness measures asymmetry of the distribution. • 0 skew → symmetric distribution, negative skew → skewed left,
positive skew → skewed right• involves raising standardized residuals to the 3rd power• Excel: =skew(A1:A100) → 0.3276
2 22 2 2100-stat 1.791
6 4 6
.0214.3276
4
kurtw
nske
= .05 (column)
5.99
Do Not Reject H0 Reject H0
.05
2
There is no reason to doubt the assumption that the errors are normally
distributed.
2 -stat
df = 2 (row) 2.05 5.99
1.791 2
Normality
Normality
If the errors are normally distributed,
• parameter estimates are normally distributed
• F and t significance tests are valid
If the errors are not normally distributed but the sample size is large,
• parameter estimates are approximately normally distributed (CLT) • F and t significance tests are valid
If the errors are not normally distributed and the sample size is small,
• parameter estimates are not normally distributed
• F and t significance tests are not reliable
The values of are probably independent if the autocorrelation residual plot or if the Durbin-Watson statistic (DW-stat) indicate the values of e are independent
The DW-stat varies when the data’s order is altered
• If you have cross-sectional data, you really don’t have to worry about computing DW-stat
• If you have time series data, compute DW-stat after sorting by time
• If you have panel data, compute the DW-stat after sorting by state and then time.
Independence
no autocorrelation if DW-stat = 2
perfect "-" autocorrelation if DW-stat = 4
perfect "+" autocorrelation if DW-stat = 0
IndependenceState Year Residuals (ei - ei-1)2 ei
2
Alabama 1994 8.522 - 72.63
Alabama 2000 -4.110 159.57 16.89
Alaska 1994 -2.290 - 5.24
Alaska 2000 14.835 293.25 220.08
Arizona 1994 -1.497 - 2.24
Arizona 2000 -4.081 6.68 16.65
Arkansas 1994 8.567 - 73.39
Arkansas 2000 6.910 2.75 47.74
California 1994 -0.558 - 0.31
California 2000 -4.801 18.01 23.05
Colorado 1994 3.393 - 11.51
Colorado 2000 -10.036 180.35 100.73
Wyoming 1994 6.573 - 43.20
Wyoming 2000 -1.096 58.81 1.20
sum 2889 3747
2889DW-stat
3747
DW-stat 0.77
The errors may not be
independent.
Independence
-15.00
-10.00
-5.00
0.00
5.00
10.00
15.00
20.00
0 10 20 30 40 50 60 70 80 90 100
geographic order
resi
dual
Autocorrelation Residual Plot
If autocorrelation (or serial correlation) is a problem,
• Estimated coefficients aren’t biased, but
• Their standard errors may be inflated
• Hypothesis testing is unreliable
In our example, autocorrelation seems to be problematic.
If autocorrelation is a problem, do one of the following:
• Change the functional form
• Include an omitted variable
• Use Generalized Least Squares
• Compute “Newey-West standard errors” for the estimated coefficients.
Independence
1
1-statb
bt
s
e = -0.001y2 + 0.111y - 2.706
-15
-10
-5
0
5
10
15
20
30 35 40 45 50 55 60 65
epr-hat
resi
du
als
The true model is probably linear if the scatterplot of e versus y is a horizontal, random band of points
Linearity
^
If you fit a linear model to data which are nonlinearly related,
• Estimated coefficients are biased
• Predictions are likely to be seriously in error
In our example, nonlinearity does not seem to be a problem.
If the data are nonlinearly related, do one of the following:
• Rethink the functional form
• Transform one or more of the variables
Linearity
Since the autocorrelation assumption appears to be invalid t and F tests are unreliable.
1
1-statb
bt
s
Overall Significance of the Model
• In simple linear regression, the F and t tests provide the same conclusion.
• In multiple regression, the F and t tests have different purposes.
The F test is used to determine whether a significant relationship exists between the dependent variable and the set of all the independent variables.
…it is referred to as the test for overall significance.
-stat -statt F p-valuet-stat = p-valueF-stat
• Hypotheses: H0: 1 = 2 = . . . = p = 0
Ha: At least one parameter is not equal to zero.
• Test Statistic: F-stat = MSR/MSE
• Reject H0 if F-stat > Fa (Fa is in column dfMSR and row dfMSE & a)
Testing for Overall Significance
SSE SSR
epry
52.35 43.83 4.47 72.63 41.05
38.47 40.76 138.45 5.24 89.82
49.69 51.19 0.29 2.24 0.91
48.17 39.60 4.25 73.39 112.97
51.10 49.31 0.75 3.19 0.85
57.99 55.14 60.19 8.11 24.12
58.34 59.44 65.79 1.20 84.77
Sum 7764.76 3746.69 4018.07
y 2( )y y 2ˆ( )y y 2ˆ( )y y
SST
50.23y
1 2 3 4 5 6ˆ 104.529 5.709ln 2.821 3.768 0.291 0.374 3.023y x x x x x x
Testing for Overall Significance
R Square 0.517
Adjusted R Square 0.486
Standard Error 6.347
Observations 100
ANOVA
df SS MS F
Regression 6 4018.075 669.679 16.623
Residual 93 3746.689 40.287
Total 99 7764.764
Coefficients Standard Error t Stat P-value
Intercept 104.529 15.743 6.640 0.000
tanfben3_ln -5.709 2.461 -2.320 0.023
2000 -2.821 2.029 -1.390 0.168
fullsanction 3.768 1.927 1.955 0.054
black -0.291 0.089 -3.256 0.002
dropo -0.374 0.202 -1.848 0.068
unemp -3.023 0.618 -4.888 0.000
Error
Testing for Overall Significance
r 2·100% of the variability in y
can be explained by the model.
49%epr of LISM
F.05
Do not Reject H0 Reject H0
≈ 1
Hence, we reject H0.
There is insufficient evidence to conclude that the
coefficients are not all equal to zero simultaneously.
dfD = 93 and = .05 (row)
Testing for Overall Significance
H0: 1 = 2 = . . . = p = 0
dfN = 6 (column)
16.6232.20
-1.986 1.986
.025
-2.3 0t
.025
Reject H0 at a 5% level of significance.
Do Not Reject RejectReject
1
1-statb
bt
s-2.32
-5.7092.461
df = 100 – 6 – 1 = 93 (row)a = .05
Testing for Coefficient Significance
a /2 = .025 (column)
H0: 1 = 0
I.e., TANF welfare payments influence the decision to work.
-1.986 1.986
.025
-1.39 0t
.025
We cannot reject H0 at a 5% level of significance.
Do Not Reject RejectReject
2
2-statb
bt
s-1.39
-2.8212.029
df = 100 – 6 – 1 = 93 (row)a = .05
Testing for Coefficient Significance
a /2 = .025 (column)
H0: 2 = 0
I.e., welfare reform in general does not influence the decision to work.
-1.986 1.986
.025
1.960t
.025
Although we cannot reject H0 at a 5% level of significance, we can at the 10% level (p-value = .054).
Do Not Reject RejectReject
3
3-statb
bt
s1.96
3.7681.927
Testing for Coefficient Significance
H0: 3 = 0
I.e., full sanctions for failure to comply with work rules influence the decision to work.
