Chapter 4: Regression Analysis

© Stevenson, McGraw Hill, 2007- Assoc. Prof. Sami Fethi, EMU, All Right Reserved.

Regression Analysis; Chapter4

MGMT 405, POM, 2011/12. Lec Notes

Chapter 4: Regression Analysis

Department of Business Administration

FALL 2011-2012

Too complicated

by hand!

MGMT 405, POM, 2011/12. Lec Notes © Stevenson, McGraw Hill, 2007- Assoc. Prof. Sami Fethi, EMU, All Right Reserved.

Regression Analysis; Chapter 4

2

Outline: What You Will Learn . . .

Purpose of regression analysisSimple linear regression modelOverall significance concept- F-test Individual significance concept- t-testCoefficient of determination and correlation

coefficientConfident intervalMultiple regression ModelCompare and contrast simple linear regression

analysis and multiple regression Analysis



3

Purpose of Regression Analysis

Regression Analysis is Used Primarily to Model Causality and Provide PredictionPredict the values of a dependent (response) variable

based on values of at least one independent (explanatory) variable

Explain the effect of the independent variables on the dependent variable

The relationship between X and Y can be shown on a scatter diagram



4

Scatter Diagram

It is two dimensional graph of plotted points in which the vertical axis represents values of the dependent variable and the horizontal axis represents values of the independent or explanatory variable.

The patterns of the intersecting points of variables can graphically show relationship patterns.

Mostly, scatter diagram is used to prove or disprove cause-and-effect relationship. In the following example, it shows the relationship between advertising expenditure and its sales revenues.



5

Scatter Diagram

Scatter Diagram-Example

Year X Y

1 10 44

2 9 40

3 11 42

4 12 46

5 11 48

6 12 52

7 13 54

8 13 58

9 14 56

10 15 60



6

Scatter Diagram

Scatter diagram shows a positive relationship between the relevant variables. The relationship is approximately linear.

This gives us a rough estimates of the linear relationship between the variables in the form of an equation such as

Y= a+ b X



7

Regression Analysis

In the equation, a is the vertical intercept of the estimated linear relationship and gives the value of Y when X=0, while b is the slope of the line and gives an estimate of the increase in Y resulting from each unit increase in X.

The difficulty with the scatter diagram is that different researchers would probably obtain different results, even if they use same data points. Solution for this is to use regression analysis.



8

Regression Analysis

Regression analysis: is a statistical technique for obtaining the line that best fits the data points so that all researchers can reach the same results.

Regression Line: Line of Best FitRegression Line: Minimizes the sum of the squared

vertical deviations (et) of each point from the regression line.

This is the method called Ordinary Least Squares (OLS).



9

Regression Analysis

In the table, Y1 refers actual or observed sales revenue of $44 mn associated with the advertising expenditure of $10 mn in the first year for which data collected.

In the following graph, Y^1

is the corresponding sales revenue of the firm estimated from the regression line for the advertising expenditure of $10 mn in the first year.

The symbol e1 is the corresponding vertical deviation or error of the actual sales revenue estimated from the regression line in the first year. This can be expressed as e1= Y1- Y^

1.

Year X Y

1 10 44

2 9 40

3 11 42

4 12 46

5 11 48

6 12 52

7 13 54

8 13 58

9 14 56

10 15 60



10

Regression Analysis In the graph, Y^

1 is the

corresponding sales revenue of the firm estimated from the regression line for the advertising expenditure of $10 mn in the first year.

The symbol e1 is the corresponding vertical deviation or error of the actual sales revenue estimated from the regression line in the first year. This can be expressed as e1= Y1- Y^

1.



11

Regression Analysis

Since there are 10 observation points, we have obviously 10 vertical deviations or error (i.e., e1 to e10). The regression line obtained is the line that best fits the data points in the sense that the sum of the squared (vertical) deviations from the line is minimum. This means that each of the 10 e values is first squared and then summed.



12

Simple Regression Analysis

Now we are in a position to calculate the value of a ( the vertical intercept) and the value of b (the slope coefficient) of the regression line.

Conduct tests of significance of parameter estimates.

Construct confidence interval for the true parameter.

Test for the overall explanatory power of the regression.



13

Simple Linear Regression Model

Regression line is a straight line that describes the dependence of the average average value value of one variable on the other

ii iY X

Y Intercept SlopeCoefficient

Random Error

Independent (Explanatory) Variable

Regression

Line

Dependent (Response) Variable



14

Ordinary Least Squares (OLS)

Model: t t tY a bX e

ˆˆ ˆt tY a bX

ˆt t te Y Y



15


Objective: Determine the slope and intercept that minimize the sum of the squared errors.

2 2 2

1 1 1

ˆˆ ˆ( ) ( )n n n

t t t t tt t t

e Y Y Y a bX



16


Estimation Procedure

1

2

1

( )( )ˆ

( )

n

t tt

n

tt

X X Y Yb

X X

ˆa Y bX



17


Estimation Example

1 10 44 -2 -6 122 9 40 -3 -10 303 11 42 -1 -8 84 12 46 0 -4 05 11 48 -1 -2 26 12 52 0 2 07 13 54 1 4 48 13 58 1 8 89 14 56 2 6 12

10 15 60 3 10 30120 500 106

4910101149

30

Time tX tY tX X tY Y ( )( )t tX X Y Y 2( )tX X

10n

1

12012

10

nt

t

XX

n

1

50050

10

nt

t

YY

n

1

120n

tt

X

1

500n

tt

Y

2

1

( ) 30n

tt

X X

1

( )( ) 106n

t tt

X X Y Y

106ˆ 3.53330

b

ˆ 50 (3.533)(12) 7.60a



18


Estimation Example

10n 1

12012

10

nt

t

XX

n

1

50050

10

nt

t

YY

n

1

120n

tt

X

1

500n

tt

Y

2

1

( ) 30n

tt

X X

1

( )( ) 106n

t tt

X X Y Y

106ˆ 3.53330

b

ˆ 50 (3.533)(12) 7.60a



19

The Equation of Regression Line

The equation of the regression line can be constructed as follows:

Yt^=7.60 +3.53 XtWhen X=0 (zero advertising expenditures), the

expected sales revenue of the firm is $7.60 mn. In the first year, when X=10mn, Y1^= $42.90 mn.

Strictly speaking, the regression line should be used only to estimate the sales revenues resulting from advertising expenditure that are within the range.



20

Tests of Significance: Standard Error

To test the hypothesis that b is statistically significant (i.e., advertising positively affects sales), we need first of all to calculate standard error (deviation) of b^.

The standard error can be calculated in the following expression:



21

Tests of Significance

Standard Error of the Slope Estimate

2 2

ˆ 2 2

ˆ( )

( ) ( ) ( ) ( )t t

bt t

Y Y es

n k X X n k X X



22


Example Calculation

2 2

1 1

ˆ( ) 65.4830n n

t t tt t

e Y Y

2

1

( ) 30n

tt

X X

2

ˆ 2

ˆ( ) 65.48300.52

( ) ( ) (10 2)(30)t

bt

Y Ys

n k X X

1 10 44 42.90

2 9 40 39.37

3 11 42 46.43

4 12 46 49.96

5 11 48 46.43

6 12 52 49.96

7 13 54 53.49

8 13 58 53.49

9 14 56 57.02

10 15 60 60.55

1.10 1.2100 4

0.63 0.3969 9

-4.43 19.6249 1

-3.96 15.6816 0

1.57 2.4649 1

2.04 4.1616 0

0.51 0.2601 1

4.51 20.3401 1

-1.02 1.0404 4

-0.55 0.3025 9

65.4830 30

Time tX tYtY ˆ

t t te Y Y 2 2ˆ( )t t te Y Y 2( )tX X

Yt^=7.60 +3.53 Xt =7.60+3.53(10)= 42.90



23


Example Calculation

2

ˆ 2

ˆ( ) 65.48300.52

( ) ( ) (10 2)(30)t

bt

Y Ys

n k X X

2

1

( ) 30n

tt

X X

2 2

1 1

ˆ( ) 65.4830n n

t t tt t

e Y Y



24

Tests of SignificanceTests of Significance

Calculation of the t Statistic

ˆ

ˆ 3.536.79

0.52b

bt

s

Degrees of Freedom = (n-k) = (10-2) = 8

Critical Value (tabulated) at 5% level =2.306



25

Confidence interval

We can also construct confidence interval for the true parameter from the estimated coefficient.

Accepting the alternative hypothesis that there is a relationship between X and Y.

Using tabular value of t=2.306 for 5% and 8 df in our example, the true value of b will lies between 2.33 and 4.73

t=b^+/- 2.306 (sb^)=3.53+/- 2.036 (0.52)



26


Decomposition of Sum of Squares

2 2 2ˆ ˆ( ) ( ) ( )t t tY Y Y Y Y Y

Total Variation = Explained Variation + Unexplained Variation



27


Decomposition of Sum of Squares



28

Coefficient of Determination-R2

Coefficient of Determination: is defined as the proportion of the total variation or dispersion in the dependent variable that explained by the variation in the explanatory variables in the regression.

In our example, COD measures how much of the variation in the firm’s sales is explained by the variation in its advertising expenditures.



29


Coefficient of Determination

22

2

ˆ( )

( )t

Y YExplained VariationR

TotalVariation Y Y

2 373.840.85

440.00R



30

Coefficient of CorrelationCoefficient of Correlation--rr

Coefficient of Correlation (r): The square root of the coefficient of determination.

This is simply a measure of the degree of association or co-variation that exists between variables X and Y.

In our example, this mean that variables X and Y vary together 92% of the time.

The sign of coefficient r is always the same as the sign of coefficient of b^.



31


Coefficient of Correlation

2 ˆr R with the signof b

0.85 0.92r

1 1r

2 2 2ˆ ˆ( ) ( ) ( )t t tY Y Y Y Y Y



32

Simple Linear Regression: Example 1

The general manager of building materials production plant feels the demand for plasterboard shipments may be related to the number of construction permits issued in a town during the previous quarter. The maneger has collected the data shown in the table.

no const. Plast.

Permit Ship.

1 15 6

2 9 4

3 40 16

4 20 6

5 25 13

6 25 9

7 15 10

8 35 16



33

QuestionsQuestions

a. Derive a regression forecasting equation

b. Determine a point estimate for plasterboard shipment when the number of construction permits 30.

c. Compute the standard error of regression.

d. Compute the upper and lower limits for consruction when it is 30.

e. Compute the correlation coefficient, and determination of coefficient. Briefly interpret both.

f. Use t-test whether t-value of estimated b is significant at 5% level of significance.



Answer A

34

no Cons.Per. (x)

Plast.Ship (Y)

(X-X‾) (Y-Y‾) (X-X‾)² (Y-Y‾)² (X-X‾)(Y-Y‾)

1 15 6 -8 -4 64 16 322 9 4 -14 -6 196 36 843 40 16 17 6 289 36 1024 20 6 -3 -4 9 16 125 25 13 2 3 4 9 66 25 9 2 -1 4 1 -27 15 10 -8 0 64 0 08 35 16 12 6 144 36 72

Σ 0 0 774 150 30623 10

X‾ Y‾



Answer A-B

35

1

2

1

( )( )ˆ

( )

n

t tt

n

tt

X X Y Yb

X X

=0.395349

ˆa Y bX =0.906973

Y^=0.906973+0.395349X



Answer C-D

36

Y^ Y-Y^ (Y-Y^)² Y^-Y‾ (Y^-Y‾)²

6.832 -0.832 0.692224 -3.168 10.03622

4.462 -0.462 0.213444 -5.538 30.66944

16.707 -0.707 0.499849 6.707 44.98385

8.807 -2.807 7.879249 -1.193 1.423249

10.782 2.218 4.919524 0.782 0.611524

10.782 -1.782 3.175524 0.782 0.611524

6.832 3.168 10.03622 -3.168 10.03622

14.732 1.268 1.607824 4.732 22.39182

Σ 29.02386 120.7639



Answer C-D

37

2 2

ˆ 2 2

ˆ( )

( ) ( ) ( ) ( )t t

bt t

Y Y es

n k X X n k X X

29.02/8*(774)=0.079

There is 95 % probability that shipment for 30 permits will fall between 12.84 and 13.13 shipments and the rest will fall outside of these limits.

D)

ß= Y±t sb 13.1375 13+(1.94*(0.079))

12.8467 13-(1.94*(0.079))



Answer E

38

Y Y^ Y-Y^ (Y-Y^)² Y^-Y‾ (Y^-Y‾)² (Y-Y‾)²

6 6.832 -0.832 0.692224 -3.168 10.03622 16

4 4.462 -0.462 0.213444 -5.538 30.66944 36

16 16.707 -0.707 0.499849 6.707 44.98385 36

6 8.807 -2.807 7.879249 -1.193 1.423249 16

13 10.782 2.218 4.919524 0.782 0.611524 9

9 10.782 -1.782 3.175524 0.782 0.611524 1

10 6.832 3.168 10.03622 -3.168 10.03622 0

16 14.732 1.268 1.607824 4.732 22.39182 36

Σ 29.02386 120.7639 150



Answer E

39

=120.7639/150 0.805092413=0.81

r=√R² 0.897269421=0.90

22

2

ˆ( )

( )t

Y YExplained VariationR

TotalVariation Y Y



Answer F

40



41

Simple Linear Regression: Example 2

You wish to examine the linear dependency of the annual sales of produce stores on their sizes in square footage. Sample data for 7 stores were obtained. Find the equation of the straight line that fits the data best.

Annual Store Square Sales

Feet ($1000)

1 1,726 3,681

2 1,542 3,395

3 2,816 6,653

4 5,555 9,543

5 1,292 3,318

6 2,208 5,563

7 1,313 3,760



42

Scatter Diagram: Example

0

2000

4000

6000

8000

10000

12000

0 1000 2000 3000 4000 5000 6000

Square Feet

An

nu

al

Sa

les

($00

0)

Excel Output



43

Simple Linear Regression Equation: Example

0 1ˆ

1636.415 1.487i i

i

Y b b X

X

CoefficientsIntercept 1636.414726X Variable 1 1.486633657

From Excel Printout:



44

Graph of the Simple Linear Regression Equation: Example

0

2000

4000

6000

8000

10000

12000

0 1000 2000 3000 4000 5000 6000

Square Feet

An

nu

al

Sa

les

($00

0)

Y i = 1636.415 +1.487X i



45

Interpretation of Results: Example

ˆ 1636.415 1.487i iY X

The slope of 1.487 means that for each increase of one unit in X, we predict the average of Y to increase by an estimated 1.487 units.

The equation estimates that for each increase of 1 square foot in the size of the store, the expected annual sales are predicted to increase by $1487.



46

Crucial Assumptions

Error term is normally distributed.Error term has zero expected value or mean.Error term has constant variance in each time

period and for all values of X.Error term’s value in one time period is

unrelated to its value in any other period.



47

Multiple Regression Analysis

Model:

1 1 2 2 ' 'k kY a b X b X b X



48

Relationship between 1 dependent & 2 or more independent variables is a linear function

1 2i i i k ki iY X X X

Y-intercept Slopes Random error

Dependent (Response) variable

Independent (Explanatory) variables




49

Multiple Regression Model: Example

Develop a model for estimating heating oil used for a single family home in the month of January, based on average temperature and amount of insulation in inches.

Oil (Gal) Temp Insulation275.30 40 3363.80 27 3164.30 40 1040.80 73 694.30 64 6

230.90 34 6366.70 9 6300.60 8 10237.80 23 10121.40 63 331.40 65 10

203.50 41 6441.10 21 3323.00 38 352.50 58 10



50

Multiple Regression Model: Example

0 1 1 2 2i i i k kiY b b X b X b X Coefficients

Intercept 562.1510092X Variable 1 -5.436580588X Variable 2 -20.01232067

Excel Output

1 2ˆ 562.151 5.437 20.012i i iY X X

For each degree increase in temperature, the estimated average amount of heating oil used is decreased by 5.437 gallons, holding insulation constant.

For each increase in one inch of insulation, the estimated average use of heating oil is decreased by 20.012 gallons, holding temperature constant.



51


Adjusted Coefficient of Determination

2 2 ( 1)1 (1 )

( )

nR R

n k

Regression StatisticsMultiple R 0.982654757R Square 0.965610371Adjusted R Square 0.959878766Standard Error 26.01378323Observations 15

SST

SSRr ,Y 2

12



52

Interpretation of Coefficient of Multiple Determination

212 .9656Y

SSRr

SST

96.56% of the total variation in heating oil can be explained by temperature and amount of insulation

95.99% of the total fluctuation in heating oil can be explained by temperature and amount of insulation after adjusting for the number of explanatory variables and sample size

2adj .9599r



53

Testing for Overall Significance

Shows if Y Depends Linearly on All of the X Variables Together as a Group

Use F Test StatisticHypotheses:

H0: β 1 = β2 = … = βk = 0 (No linear relationship)H1: At least one βi 0 ( At least one independent variable

affects Y )

The Null Hypothesis is a Very Strong StatementThe Null Hypothesis is Almost Always Rejected



54


Analysis of Variance and F Statistic

/( 1)

/( )

Explained Variation kF

Unexplained Variation n k

2

2

/( 1)

(1 ) /( )

R kF

R n k



55

Test for Overall SignificanceExcel Output: Example

ANOVAdf SS MS F Significance F

Regression 2 228014.6 114007.3 168.4712 1.65411E-09Residual 12 8120.603 676.7169Total 14 236135.2

k -1= 2, the number of explanatory variables and dependent variable

n - 1p-value

k = 3, no of parameters



56

Test for Overall Significance:Example Solution

03.89

= 0.05

H0: 1 = 2 = … = k = 0

H1: At least one j 0

= .05

df = 2 and 12

Critical Value:

Test Statistic:

Decision:

Conclusion:

F 168.47

Reject at = 0.05.

There is evidence that at least one independent variable affects Y.



57

t Test StatisticExcel Output: Example

Coefficients Standard Error t Stat P-valueIntercept 562.1510092 21.09310433 26.65094 4.77868E-12Temp -5.436580588 0.336216167 -16.1699 1.64178E-09Insulation -20.01232067 2.342505227 -8.543127 1.90731E-06

t Test Statistic for X2 (Insulation)

t Test Statistic for X1 (Temperature)

i

i

b

bt

S



58

t Test : Example SolutionDoes temperature have a significant effect on monthly consumption of heating oil? Test at = 0.05.

H0: 1 = 0

H1: 1 0

df = 12

Critical Values:

Test Statistic:

t Test Statistic = -16.1699

Decision:

Reject H0 at = 0.05.

Conclusion:

There is evidence of a significant effect of temperature on oil consumption holding constant the effect of insulation.

Reject HReject H 00

.025 .025

-2.1788 2.17880



59

Problems in Regression Analysis

Multicollinearity: Two or more explanatory variables are highly correlated.

Heteroskedasticity: Variance of error term is not independent of the Y variable.

Autocorrelation: Consecutive error terms are correlated.

Functional form: Misspecified by the omission of a variable

Normality: Residuals are normally distributed or not



60

Steps in Demand Estimation

Model Specification: Identify VariablesCollect DataSpecify Functional FormEstimate FunctionTest the Results



61

Functional Form Specifications

Linear Function:

Power Function:

0 1 2 3 4X X YQ a a P a I a N a P e

1 2( )( )b bX X YQ a P P

Estimation Format:

1 2ln ln ln lnX X YQ a b P b P



62

Demand EstimationTo use these important demand relationship in

decision analysis, we need empirically to estimate the structural form and parameters of the demand function-Demand Estimation.

Qdx= (P, I, Pc, Ps, T)

(-, + , - , +, +) The demand for a commodity arises from the consumers’

willingness and ability to purchase the commodity. Consumer demand theory postulates that the quantity demanded of a commodity is a function of or depends on the price of the commodity, the consumers’ income, the price of related commodities, and the tastes of the consumer.



63

Demand Estimation

In general, we will seek the answer for the following In general, we will seek the answer for the following qustions:qustions:

How much will the revenue of the firm change after increasing the price of the commodity?

How much will the quantity demanded of the commodity increase if consumers’ income increase

What if the firms double its ads expenditure?What if the competitors lower their prices? Firms should know the answers the abovementioned

questions if they want to achieve the objective of maximizing thier value.



64

Dummy-Variable Models

When the explanatory variables are qualitative in nature, these are known as dummy variables. These can also defined as indicators variables, binary variables, categorical variables, and dichotomous variables such as variable D in the following equation:

eDcIcPccQ xx ......3210



65

Dummy-Variable Models

Categorical Explanatory Variable with 2 or More Levels

Yes or No, On or Off, Male or Female,

Use Dummy-Variables (Coded as 0 or 1)

Only Intercepts are Different

Assumes Equal Slopes Across Categories

Regression Model Has Same Form

Can the dependent variable be dummy?



66

Thanks

Date post:	04-Jan-2016
Category:	Documents
Upload:	gabriel-puckett
View:	103 times
Download:	7 times

Chapter 4: Regression Analysis

Documents