1Spring 02

First Derivatives

x

y

x

y

x

y

dy/dx = 0 dy/dx > 0 dy/dx < 0

x

yx

dx

dy

0lim

2Spring 02

First Derivatives

x

y

x

y x

y

x

y

dy/dx > 0

dy/dx > 0

dy/dx < 0

dy/dx < 0

3Spring 02

First and Second Derivatives

x

y

x

y

Y is maximized and minimized when dy/dx = 0

4Spring 02

Rules of Differentiation

1

0

n

n

nxdx

dy

xy

dx

dy

ky

dx

dunau

dx

dy

xuu

auy

naxdx

dy

axy

n

n

n

n

1

1

)(

5Spring 02

Summation Operators

0)(

)(

...

1

1

0

1

111

11

211

n

ii

n

i

n

i

n

ii

n

iii

n

ii

n

ii

n

ii

n

n

ii

XX

nkkXk

YXYX

XkkX

XXXX

i

6Spring 02

Summation Operators

XYYXn

YYXXn i

iii

ii 1

))((1

n

i

n

jij

n

i

n

jijj

n

i

n

ji

n

ji

n

iij

n

i

n

ji

ii

ii

YXYX

YXYX

XXn

XXn

1 11 11 1

111 1

222

)(

1)(

1

7Spring 02

Expectations Operators

22

1

2

1

)]([)]([ XEXEXEXp

XpXE

i

N

iiX

i

N

iix

8Spring 02

Expectation Operators

)()()(

))]())(([(),(

)(

)(])[(

)()(

2

222

YEXEYXE

YEYXEXEYXCov

VarXabaXVar

XEaaXE

bXaEbaXE

9Spring 02

Expectations Operation

),(2)()()( YXCovYVarXVarYXVar

If X,Y are independent:

0),(

)()()(

YXCov

YEXEXYE

10Spring 02

OLS Estimation

In reality:

OLS line:

Error term:•omitted variables •intrinsic randomness•errors in measurement

iii XY

ii bXaY ˆ

11Spring 02

OLS Estimation

Objectives Find average Y given X Test hypothesis Predict or forecast

12Spring 02

OLS Estimation

2

2

)(

)ˆ(

ii

i

ii

i

bXaYMin

YYMin

Minimize the sum of the errors squared:

Solve simultaneously:

XbYa

XXN

YXYXNb

i iii

i i iiiii

22 )(

13Spring 02

OLS Problem

Given the price and length of the following textbooks, find the relationship between length and price:

Book Price Length

1 $11 100

2 $15 225

3 $24 400

4 $20 350

14Spring 02

OLS Problem

0

5

10

15

20

25

30

0 100 200 300 400 500

Length

Price

15Spring 02

OLS Problem

Book Price LengthY X XY X̂ 2

1 11 100 1100 100002 15 225 3375 506253 24 400 9600 1600004 20 350 7000 122500

Sum 70 1075 21075 343125Mean 17.5 268.75

b= 0.041729a= 6.285303

16Spring 02

OLS Problem

If X=100, Yhat =$10.46

XY 0417.029.6ˆ

17Spring 02

Special Case

00*0

)(

0

2

2

222

bXbYa

X

YX

XN

X

YXN

YX

XXN

YXYXNb

YX

ii

iii

ii

iii

i iii

i i iiiii

18Spring 02

Special Case

0

0

0

2

a

x

yxb

y

x

YYy

XXx

ii

iii

ii

ii

Deviations from the mean

19Spring 02

Special Case

Book Price LengthY X y x xy x^2

1 11 100 -6.5 -168.75 1096.875 28476.562 15 225 -2.5 -43.75 109.375 1914.0633 24 400 6.5 131.25 853.125 17226.564 20 350 2.5 81.25 203.125 6601.563

Sum 70 1075 2262.5 54218.75Mean 17.5 268.75

b= 0.041729a= 6.285303

20Spring 02

Formal Exposition of Model

Model

X is nonstochastic

E(i) = 0

Var(i) = 2

E(i,j)=0 for i=j

iii XY

21Spring 02

Formal Exposition of Model

22Spring 02

Formal Exposition of Model

23Spring 02

Formal Exposition of Model

24Spring 02

Formal Exposition of Model

25Spring 02

Gauss Markov Theorem

The OLS estimators are BLUE: best, linear, unbiased estimators

),(~ˆ

ˆ

2

2

2

ii

ii

iii

xN

x

yx

2ˆ

2

22

N

es i

),(~2

22

ii

ii

xN

XN

26Spring 02

Gauss Markov Theorem

The variance of hat varies: Directly with the variance of Inversely with xi

2

The variance of hat varies: Directly with 2

27Spring 02

Book Example

Book Price LengthY X X̂ 2 y x xy x^2 yhat i=yi-yhati i^2

1 11 100 10000 -6.50 -168.75 1096.88 28476.56 -7.04 0.54 0.292 15 225 50625 -2.50 -43.75 109.38 1914.06 -1.83 -0.67 0.453 24 400 160000 6.50 131.25 853.13 17226.56 5.48 1.02 1.054 20 350 122500 2.50 81.25 203.13 6601.56 3.39 -0.89 0.79

Sum 70 1075 343125 2262.50 54218.75 2.59Mean 17.5 268.75

hat= 0.04 shat= 0.00hat= 6.29 shat= 1.43s^2= 1.29s= 1.14

28Spring 02

Confidence Intervals

Ho: =o

Ha: o

Ho: =o

Ha: o

ˆ

0ˆ

st

ˆ

0ˆ

st

29Spring 02

Confidence Intervals

Ho: =Ha:

Ho: =Ha:

ˆ

ˆ

st

ˆ

ˆ

st

30Spring 02

Book Example

Test if length is a significant explanatory variable of price Test if differs from zero

Ho: = Ha:

53.800489.

0417.ˆ

ˆ

s

t

31Spring 02

Book Example

Test if the intercept differs significantly from zero Test if differs from zero

Ho: = Ha:

39.443.1

29.6ˆ

ˆ

s

t

32Spring 02

Goodness of Fit

The residuals can help to explain how well the regression line fits the points.

Variation (not variance!) of Y can be broken down into the portion explained by the regression equation and the unexplained portion (error term) of the model

)ˆ()ˆ(

)()( 2

YYYYYY

YYYVariation

iiii

ii

33Spring 02

Goodness of Fit

34Spring 02

Goodness of Fit

10

1

1

)ˆ()ˆ()(

2

2

222

R

TSS

ESS

TSS

RSSR

TSS

RSS

TSS

ESS

RSSESSTSS

YYYYYYi

ii

iii

i

35Spring 02

Goodness of Fit

ii

ii

ii

ii

ii

ii

ii

ii

ii

ii

ii

yy

yR

yy

y

yy

2

2

2

2

2

2

2

2

2

222

1

ˆ

ˆ

1

ˆ

36Spring 02

Back to Book Example

Book Price LengthY X X̂ 2 y x xy x^2 yhat i=yi-yhati i^2 y^2

1 11 100 10000 -6.50 -168.75 1096.88 28476.56 -7.04 0.54 0.29 42.252 15 225 50625 -2.50 -43.75 109.38 1914.06 -1.83 -0.67 0.45 6.253 24 400 160000 6.50 131.25 853.13 17226.56 5.48 1.02 1.05 42.254 20 350 122500 2.50 81.25 203.13 6601.56 3.39 -0.89 0.79 6.25

Sum 70 1075 343125 2262.50 54218.75 2.59 97.00Mean 17.5 268.75

hat= 0.04 shat= 0.00 that= 8.54hat= 6.29 shat= 1.43 that= 4.39s^2= 1.29s= 1.14

R^2= 0.9733

37Spring 02

ANOVA or F-Test

F-distribution is the ratio of two 2 distributions divided by their respective degrees of freedom. A 2 distribution is the sum of squares of N independently distributed normal random variables. The basis of the F-test is the idea that the ratio of the explained variation to the unexplained variation should be high if the tested model is a reasonable approximation of the true model.

38Spring 02

ANOVA or F-Test

By dividing RSS and ESS by their degrees of freedom, we will convert the variations to variances.

As long as the error terms are normally distributed with a zero mean, the variances will follow a 2 distribution.

39Spring 02

ANOVA or F-Test

By calculating an F-statistic:

We are testing the hypothesis that: Ho: = Ha:

2

22ˆ

)2(

1s

x

nESS

RSSF i

i

40Spring 02

ANOVA or F-Test

Since in the bivariate model case, the null hypothesis for the t-test and the F-test are the same, both should give the same accept/reject answer to the null hypothesisIn fact

222,1 NN tF

41Spring 02

Back to Book Example

Book Price LengthY X X̂ 2 y x xy x^2 yhat i=yi-yhati i^2 y^2

1 11 100 10000 -6.50 -168.75 1096.88 28476.56 -7.04 0.54 0.29 42.252 15 225 50625 -2.50 -43.75 109.38 1914.06 -1.83 -0.67 0.45 6.253 24 400 160000 6.50 131.25 853.13 17226.56 5.48 1.02 1.05 42.254 20 350 122500 2.50 81.25 203.13 6601.56 3.39 -0.89 0.79 6.25

Sum 70 1075 343125 2262.50 54218.75 2.59 97.00Mean 17.5 268.75

hat= 0.0417 shat= 0.00 that= 8.54hat= 6.29 shat= 1.43 that= 4.39s^2= 1.29s= 1.14

R^2= 0.97332F= 72.9644

42Spring 02

Scaling and the Units of Measurement

Changing the scale of measurement of the dependent variable changes the corresponding scaling of all the regression coefficients (including residuals and standard errors).

Changing the scale of measurement of a single independent variable changes its coefficient and corresponding standard error but all other statistics are the same.

43Spring 02

Forecasting

Point estimateInterval estimate Standard error

The prediction variance varies directly with 2

The denominator of the third term is t-1 times the sample variance of X, so as the variance of X increase, the variance of the prediction decreases

The prediction variance all increase the farther the prediction is from the mean of X.

tt

tp

XX

XXNss

2

2122

)(

)(/11

44Spring 02

Formal Exposition of Model

45Spring 02

Book Example

How much will a book that has 300 pages sell for? Point Estimate

Yhat = 6.29 + .0417*300 = $18.80 95% Confidence Interval Estimate

95.)80.2378.12(

95.)28.1*303.429.1828.1*303.429.18(

28.1

64.154219

)8.268300(4/11[*29.1

22

YP

YP

s

s

p

p

46Spring 02

Causality

Direction of causality

Casual or causal relationship Spurious correlation

Simultaneity Crime and police officers