Statistics and Quantitative Analysis U4320 Segment 8 Prof. Sharyn O’Halloran.

transcript

Statistics and Quantitative Analysis U4320

Segment 8Prof. Sharyn O’Halloran

I. Introduction A. Overview

1. Ways to describe, summarize and display data.

2.Summary statements: Mean Standard deviation Variance

3. Distributions Central Limit Theorem

I. Introduction (cont.)

A. Overview

4. Test hypotheses

5. Differences of Means

B. What's to come?

1. Analyze the relationship between two or more variables with a specific technique called regression analysis.

I. Introduction (cont.)

A. Overview

B. What's to come?

2. This tools allows us to predict the impact of one

variable on another.

For example, what is the expected impact of a SIPA degree on income?

II. Causal Models Causal models explain how changes in one variable

affect changes in another variable.

Incinerator -------------------------> Bad Public Health

Regression analysis gives us a way to analyze precisely

the cause-and-effect relationships between variables.

Directional Magnitude

II. Causal Models (cont.)

A. Variables Let us start off with a few basic definitions.

1. Dependent Variable The dependent variable is the factor that we want

to explain. 2. Independent Variables

Independent variable is the factor that we believe causes or influences the dependent variable.

Independent variable-------> Dependent VariableCause ------------------> Effect

A. Variables

B. Voting Example Let us say that we have a vote in the House of

Representatives on health. And we want to know if party affiliation influenced individual members' voting decisions?

1. The raw data looks like this:

Vote (Dep)(Indep) YES NOParty DEM 220 65 285

REP 30 120 150250 185 435

A. Variables

B. Voting Example 2. Percentages look like this:

3. Does party affect voting behavior?

Given that the legislator is a Democrat, what is the chance of

voting for the health care proposal?

YES NODEM 50.6% 14.9% 65.5REP 6.9% 27.6% 34.5

57.5 42.5 100

A. Variables

B. Voting Example 3. Does party affect voting behavior? (cont.)

What is the Probability of being a democrat?

What is the Probability of being a Democrat and voting yes?

Vote (DepVar)Indep YES NOParty DEM

A. Variables

B. Voting Example 4. Casual Model

This is the simplest way to state a causal model

A-------------> B

Party ---------> Vote

5. Interpretation The interpretation is that if party influences vote, then as we

move from Republicans to Democrats we should see a move from a No vote to a YES vote.

A. Variables B. Voting Example

C. Summary 1. Regression analysis helps us to explain the impact

of one variable on another.

We will be able to answer such questions as what is the

relative importance of race in explaining one's income?

Or perhaps the influence of economic conditions on the levels

of trade barriers?

A. Variables B. Voting Example

C. Summary 2. Univariate Model

For now, we will focus on the univariate case, or the causal

relation between two variables.

We will then relax this assumption and look at the relation of

multiple variables in a couple of weeks.

III. Fitted Line Although regression analysis can be very

complicated, the heart of it is actually very simple.

It centers on the notion of fitting a line through the data.

1. Example

Suppose we have a study of how wheat yield depends on fertilizer. And

we observe this relation:

XFertilizer(lb/Acre)

YYield (bu/acre)

100 40200 50300 50400 70500 65600 65700 80

III. Fitted Line (cont.)

1. Example (cont.) The observed relation between Fertilizer and Yield then

can be plotted as follows:

Fertilizer

100 200 300 400 500 600 700

1. Example

2. What line best approximates the relation between these observations? a) Highest and Lowest Value

Fertilizer

100 200 300 400 500 600 700

Lowest & highest value

1. Example

2. What line best approximates the relation between these observations? (cont.) b) Median Value

Fertilizer

100 200 300 400 500 600 700

[Median]

1. Example 2. What line best approximates the relation between these

observations?

3. Predicted Values a) Example 1:

The line that is fitted to the data gives the predicted

value of Y for any give level of X.

observations?

3. Predicted Values (cont.)

a) Example 1:

Fertilizer

100 200 300 400 500 600 700

If X is 400 and all we know was the fitted line then we would expect the yield to

be around 65.

observations?

b) Example 2:

Many times we have a lot of data and fitting the line becomes

rather difficult.

observations?

b) Example 2:

Fertilizer

100 200 300 400 500 600 700

For example, if our plotted data looked like this:

IV. OLS Ordinary Least Squares We want a methodology that allows us to be able to

draw a line that best fits the data.

A. The Least Square Criteria

What we want to do is to fit a line whose equation is of the form:

This is just the algebraic representation of a line.

$Y a bX= +

IV. OLS Ordinary Least Squares (cont.)

A. The Least Square Criteria (cont.)

1. Intercept:

a represents the intercept of the line. That is, the point at which the line crosses the Y axis.

2. Slope of the line:

b represents the slope of the line. Yield

Fertilizer

100 200 300 400 500 600 700

a change in x

change in Y

A. The Least Square Criteria (cont.) 1. Intercept: 2. Slope of the line:

Remember: the slope is just the change in Y divided by the change in X. Rise/Run

3. Minimizing the Sum or Squares a) Problem:

How do we select a and b so that we minimize the pattern of

vertical Y deviations (predicted errors)?

We what to minimize the deviation:

d Y Y= −$

A. The Least Square Criteria (cont.) 1. Intercept: 2. Slope of the line: 3. Minimizing the Sum or Squares

b) There are several ways in which we can do this.

1. First, we could minimize the sum of d.

We could find the line that will give us the

lowest sum of all the d's. The problem of course is that some d's would be

positive and others would be negative and when we add them all up they would end up canceling

each other. In effect, we would be picking a line so that the

d's add up to zero.

A. The Least Square Criteria (cont.)

1. Intercept: 2. Slope of the line: 3. Minimizing the Sum or Squares

b) There are several ways in which we can do this.

2. Absolute Values

3. Sum of Squared Deviations

Minimize d Y -YΣ Σ= $()MinimizeYY dΣΣ2 2=−$

B. OLS Formulas 1. Fitted Line

The line that we what to fit to the data is:

This is simply what we call the OLS line. Remember: we are concerned with how to

calculate the slope of the line b and the intercept of the line

$Y a bX= +

2. OLS Slope

The OLS slope can becalculated from the formula:()()()bXXYYXX=−−−2

2. OLS Slope

In the book they use the abbreviations:xXXyYY…−…−⇒ b=

B. OLS Formulas 1. Fitted Line 2. OLS Slope

3. Intercept

Now that we have the slope b it is easy to calculate a

Note: when b=0 then the intercept is just the mean of the dependent variable.

aYbX=-

A. The Least Square Criteria B. OLS Formulas

C. Example 1: Fertilizer and Yield

Data Deviation Form ProductsX Y xXX…−y = YY−xy x2100 40 -300 -20 6000 90,000200 50 -200 -10 2000 40,000300 50 -100 -10 1000 10,000400 70 0 10 0 0500 65 100 5 500 10,000600 65 200 5 1000 40,000700 80 300 20 6000 90,000X= 400 Y= 60 x=0 y=0 xy=16,500 x2=280000

So to calculate the slope we solve:

We can then use the slope b to calculate the intercept

b = xyx=216500280000,, = .059

Remember:

Plugging these estimated values into our fitted line equation, we get:

$YabX=+⇒aYbX=−

a = 60-.059(400) = 36.4

$ . .Y X= +36 4 059

What is the predicted bushels produced with 400 lbs of fertilizer?

What if we add 700 lbs of fertilizer what would be the

expected yield?

$..()Y=+364059400= 60

A. The Least Square Criteria B. OLS Formulas C. Example 1: Fertilizer and Yield

D. Interpretation of b and a

1. Slope b

Change in Y that accompanies a unit change X.

The slope tells us that when there is a one unit change in the independent variable what is the predicted effect on the

dependent variable?

1. Slope b

The slope then tells us two things: i) The directional effect of the independent variable on the

dependent variable. There was a positive relation between fertilizer and yield.

1. Slope b

The slope then tells us two things: ii) It also tells you the magnitude of the effect on the

dependent variable. For each additional pound of fertilizer we expect an

increased yield of .059 bushels.

D. Interpretation of b and a 2. The Intercept

The intercept tells us what we would expect if there is no fertilizer

added, we expect a yield of 36.4 bushels.

So independent of the fertilizer you can expect 36.4 bushels. Alternatively, if fertilizer has no effect on yield, we would simply

expect 36.4 bushels. The yield we expected with no fertilizer.

$..()Y=+3640590= 36.4

A. The Least Square Criteria B. OLS Formulas C. Example 1: Fertilizer and Yield D. Interpretation of b and a

E. Example II: Radio Active Exposure 1. Casual Model

We want to know if exposure to radio active waste is linked to cancer?

Radio Active Waste --------------> Cancer

IV. OLS Ordinary Least Squares (cont.) A. The Least Square Criteria B. OLS Formulas C. Example 1: Fertilizer and Yield D. Interpretation of b and a

E. Example II: Radio Active Exposure 2. Data

Index of Radio deaths perActive Exposure 10,000

X Y xXX…−y = YY−xy x2

8.3 210 3.7 50 185 13.696.4 180 1.8 20 36 3.243.4 130 -1.2 -30 36 1.443.8 170 -0.8 10 -8 0.642.6 130 -2.0 -30 60 411.6 210 7.0 50 350 491.2 120 -3.4 -40 136 11.562.5 150 -2.1 -10 21 4.411.6 140 -3.0 -20 60 9X= 4.6 Y= 160 x=0 y=0 xy=876 x2=97.0

E. Example II: Radio Active Exposure 3. Graph

1 2 3 4 5 6 7 8 9 10 11 12

E. Example II: Radio Active Exposure 4. Calculate the regression line for predicting Y

from X

i) Slope

How do we interpret the slope coefficient?

For each unit of radioactive exposure, the cancer mortality rate rises by 9.03 deaths per 10,000 individuals.

b = xyx=2876970. = 9.03

E. Example II: Radio Active Exposure ii) Calculate the intercept

Plugging these estimated values into our fitted line equation, we get:

aYbX=−a = 160- 9.03 (4.6) = 118.5

$ . .Y X= +118 5 9 03

E. Example II: Radio Active Exposure 5. Predictions:

Let's calculate the mortality rate if X were 5.0.

How about if X were 0?

$..(.)Y=+118590350= 163.6

$..()Y=+11859030= 118.5

E. Example II: Radio Active Exposure How can we

interpret this result?

Even with no radioactive exposure, the mortality rate would be 118.5.

1 2 3 4 5 6 7 8 9 10 11 12

Y=118.5+9.03X

III. Advantages of OLS A. Easy

1. The least square method gives relative easy or at least

computable formulas for calculating a and b. $YabX=+b= xyx2aYbX=−

III. Advantages of OLS (cont.)

A. Easy

B. OLS is similar to many concepts we have already used.

1. We are minimizing the sum of the squared deviations. In effect, this is very similar to how we find the variance.

2. Also, we saw above that when b=0,

The interpretation of this is that the best prediction we can make of Y is just the sample mean .

This is the case when the two variables are independent.

$Y = a or $Y = Y

III. Advantages of OLS (cont.)

A. Easy B. OLS is similar to many concepts we have already used.

C. Extension of the Sample Mean

Since OLS is just an extension of the sample mean, it has

many of the same properties like efficient and unbiased.

D. Weighted Least Squares

We might want to weigh some observations more heavily

than others.

V. Homework Example In the homework assignment, you are asked to select two

interval/ratio level variables and calculate the fitted line that

minimizes the sum of the squared deviations (the regression line).

A. Choose 2 Variables

What effect does the number of years of education have on the

frequency that one reads the newspaper?

The independent variable is Education And the dependent variable is Newspaper reading.

V. Homework Example(cont.)

A. Choose 2 Variables

B. Coding the Variables

First, I made a new variable called PAPER.

Recode all the missing data values to a single value.

Remove missing values from the data set.

Then do the same for education

A. Choose 2 Variables B. Coding the Variables

C. Getting the number of valid observations

Next, see how many valid observations are left by using the “Summarize” command under the “Data” menu.

A. Choose 2 Variables B. Coding the Variables C. Getting the number of valid observations

D. Sampling five observations

1. So we randomly sample 5 from 1019.

2. As before, use the “Select” command under the “Data” menu to get 5 random observations.

3. Then go to the “Statistics” menu and use the “Summarize” > “List” command to get the entries for the variables of interest.

A. Choose 2 Variables B. Coding the Variables C. Getting the number of valid observations D. Sampling five observations

E. Calculate the OLS Line Finally, you will have to compute the fitted line for these data.

X= SMARTS Y= PAPER xXX…−y = YY−xy x2

15 1 1.6 -0.4 -0.64 2.568 2 -5.4 0.6 -3.24 29.1615 1 1.6 -0.4 -0.64 2.5613 2 -0.4 0.6 -0.24 0.1616 1 2.6 -0.4 -1.04 6.76X= 13.4 Y= 1.4 x=0 y=0 xy=-5.8 x2=41.2

E. Calculate the OLS Line

1. Calculate b =

2 . Calculate the intercept:

3 . Calculate the OLS line:

xy/ x2= -5.8/41.2 = -0.14

a =Y - bXa = 1.4- (-0.14)13.4

= 1.4 + 1.876 = 3.276$Y = a + bX$Y = 3.3 - 0.14X

4. Plot

5 10 15 20

Y=3.3=0.14X

5. Interpretation

A person with no education would read 3.3 newspapers a day.

5. Interpretation (cont.)

Our results further tell us that each additional year of education reduces the number of newspapers a person reads by 0.14.

So for every year of education you read 14% less.

5. Interpretation (cont.)

This example suggests some of the problems with drawing inferences about the underlying population

from small samples.

Statistics and Quantitative Analysis U4320 Segment 8 Prof. Sharyn O’Halloran.

Documents