Fundamentals of Probability II.pioneer.netserv.chula.ac.th/~achairat/Appendix B... · 2015. 7....

Fundamentals of Probability

Read Wooldridge, Appendix B

Fundamentals of Probability: Part One . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat

Outline

Part OneI. Random Variables and Their Probability DistributionsII. Joint Distribution, Conditional Distributions, and

Independence.III. Features of Probability Distributions

Part TwoIV. Features of Joint and Conditional DistributionsV. The Normal and Related Distributions

2Fundamentals of Probability . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat

I. Random Variables II. Joint & Conditional D’s III. Probability D’s IV. Features of Joint & Conditional D’s V. Normal & Related D’s

• Probability is of interest for students in business, economics, and other fields.

Example: Airline is trying to maximize profits from the available 100 seats.

• Strategy 1: Should the airline accept reservations at most 100 seats?

• There is a chance that those who book might not show up. This results in lost revenue to the airline

3

I. Random Variables and Their Probability Distributions

Fundamentals of Probability . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat



Introduction

• Strategy 2:Should the airline accept more than 100 reservations?– This policy runs the risk of the airline having to compensate people

who are bumped from the overbooked flight.

• Can we decide on the optimal or the best number of reservations the airline should make?

• We can use basic probability to arrive at a solution.




Experiment

• Experiment: flip a coin ten times and count the number of heads.

• Definition:

An experiment is the procedure that can be repeated and has a well‐defined set of outcomes.– Carrying out the coin‐flipping procedure again and again– Number of heads appearing in an integer from 0 to 10.




Random Variables

• DefinitionA random variable is one that takes on numerical values and has an outcome that is determined by an experiment.

• Example: a random number is the number of heads appearing in an experiment (10 flips of a coin).

A. DiscreteB. Continuous




Random Variables and Outcome

Example: Note that the random variable X takes on a value in a set {0, 1, 2, …, 10}

• NotationsRandom Variables – are denoted by an uppercase letters; eg, X,YParticular Outcomes – are denoted by a smallcaseletters; eg, x,y.

X =“number of heads” appearing in 10 flips of a coin – random variable

x = 6 – particular outcome






• Subscripts: we indicate large collections of random variables by using subscripts.

Example: Income of 20 randomly chosen households in the United States.– Random variables: X1, X2, … X20– Particular outcomes: x1, x2, … x20






Example: Consider tossing a single coin1) What are the outcomes?

heads and tails

2) This is an example of qualitative events.

3) Define the random variable as follows.X=1 if the coin turns up headsX=0 if the coin turns up tails.





Bernoulli or Binary Random Variable

• Definition:A random variable that can only take on the values zero or one (0 or 1) is called a Bernoulli (or binary) random variables.

event X=1 successevent X=0 failure





Discrete Random Variable

• Definition:A discrete random variable is one that takes on only a finite or countable infinite number of values.

• “Countable infinite” means even though an infinite number of values can be taken as a random variable, those values can be put in one‐to‐one correspondence with the positive integers.

• For our purpose,– we will concentrate on discrete random variables that takes on only a finite

number of values.





• Any discrete random variable is completely described by listing its possible values and associated probabilities that it takes on each value.

• A Bernoulli random variable is the simplest example of a discrete random variable.

Example: coin flippingP(X=1) = ½ “the probability that X equal one is one‐half”P(X=0) = ½


12

Discrete Random Variable and Probability Distribution




• Suppose X takes on k possible values {x1, x2,,…xk} and their associated probabilities {p1, p2,….,pk}, there are two properties.

(1) P(X=xj) = pj

“The probability that X takes on the value xj is equal to pj.”

(2) p1+ p2 + … + pk =1

The sum of all probabilities is equal to one (1).

Discrete Random VariableA. DiscreteB. Continuous




Example: airline example.

X=1 if the person shows up for the reservation.X=0 otherwise

– To completely describe the behavior of X, define as the probability that the customer shows up after making a reservation.

P(X=1) = (Interpret! when equals 0.75)P(X=0) = 1‐

• Special Notation: X = Bernoulli ().• It is read as “X has a Bernoulli distribution with probability of success equal to

.”






• Definition: pdf of XThe probability density function (pdf) of X summarizes the information concerning the possible outcomes of X and the corresponding probabilities:

f(xj) = pj ; j = 1, …, k

• Notes

1) f(x)=0 for any x not equal to xj for j

2) pdf provides information of the likely outcomes of the random variable.


15





Example: X is the number of free throws made out of two attempts.

• X can take on three values, {0, 1, 2}

• pdf is given byf(0) = 0.2; f(1) =0.44; f(2)=0.36. (see graph)

• What is the probability that the player makes at least one free throw.P(X 1) = P(X=1) + P(X=2) 0.44+0.36 = 0.80


16





Discrete Random VariableA. DiscreteB. Continuous




Continuous Random Variable

• DefinitionA variable X is a continuous random variable if it takes on any real value with probability zero (0).

• “A function that can be graphed without lifting your pencil from the paper.”

• Idea: a continuous random variable takes on so many possible values.





• Probability density function for continuous rv.– For positive probabilities, a continuous rv X lies between certain values (a and

b); ie.,

P(a<X<b) (Draw graph!)

Continuous Random VariableA. DiscreteB. Continuous




The area under the pdfbetween points a and b.

This is the integral of the function f between points a and b.

• Cumulative Distribution Function (cdf)If X is a random variable, then its cdf is defined for any value of x by

F(x) = P(X x)

• For discrete rv, cdf is obtained by summing the pdf over all values xj such that x<xj.

• For continuous rv, cdf is equal to the area under the pdf [f(x)] to the left of xj.





• A cdf is an increasing (or at least nondecreasing) function of x.Given x1<x2, then

P(X x1) < P(X x2); i.e., F(x1)<F(x2).

Properties:1) For any number c,

P(X>c) = 1‐F(c)

2) For any numbers a<b, P(a X b) = F(b) ‐ F(a)





• For econometrics, we will use cdf to compute probabilities only for continuous random variables; eg, normal distribution.

P(X ≥ c) = P(X > c)P(a X b) = P(a < X b) = P(a X < b) = P(a < X < b)

Problem B.1

B.1 Suppose that a high school student is preparing to take the SAT exam. – Explain why his or her eventual SAT score is properly viewed as a random variable.[ans.]





Problem B.1B.1

• Before the student takes the SAT exam, we do not know – nor can we predict with certainty – what the score will be.

• The eventual SAT score clearly satisfies the requirements of a random variable.

• The actual score depends on numerous factors, many of which we cannot even list, let alone know ahead of time. (The student’s innate ability, how the student feels on exam day, and which particular questions were asked, are just a few.)

23

ACK




24

Problem B.4B.4 For a randomly selected county in the United States, let

X represent the proportion of adults over age 65 who are employed, or the elderly employment rate. Then, X is restricted to a value between zero and one. Suppose that the cumulative distribution function for X is given by

F(x) = 3x2 – 2x3 for 0x 1.

Find the probability that the elderly employment rate is at least 0.6 (60%). [ans.]





Problem B.4B.4X: proportion of adults over age 65 who are employedFind the probability that the elderly employment rate at least 0.6

• We want P(X .6).

• Because X is continuous, this is the same as P(X > .6) = 1 – P(X .6)

= 1 ‐ F(.6)F(.6) = 3(.6)2 – 2(.6)3 = .648.

= 1 ‐ .648= .352

• One way to interpret this is that 35% of all counties have an elderly employment rate of .6 or higher.




II. Joint Distribution, Conditional Distributions, and Independence

• In business and economics, we may be interested in the occurrence of events of more than one random variables.

Example:

– The airline may be interested in the probability that a person makes a reservation and is a business traveler. This is an example of joint probability.

– The airline may be interested in the probability that a person makes a reservation, conditional that he is a business traveler. This is an example of conditional probability.

A. Joint Distrib.B. Conditional




Joint Distribution and Independence

• Let X and Y be discrete random variables. Then, (X,Y) have a joint distribution.

• The joint distribution can be fully described by the joint density function of (X,Y)

fX,Y(x,y) = P(X=x, Y=y) or

f(x,y) = P(X=x, Y=y)

• Given pdfs of X and Y, it is easy to obtain a joint pdf.






• Independence

• Random variables X and Y are said to be independent if and only if

fX,Y(x,y) = fX(x)fY(y)

fX(x) is the pdf of X and fY(y) is the pdf of Y.

• Note that in the context of more than one random variable, – fX(x) and fY(y) are marginal pdfs.

• Intuitively, independence refers to knowing the outcome of X does not change the probabilities of the possible outcomes of Y and vice versa.






• Given

fX,Y(x,y) = fX(x)fY(y)

• It is the same as

P(X=x,Y=y) = P(X=x)P(Y=y)

That is, the probability that X=x and Y=y is the product of the two probabilities P(X=x) and P(Y=y).





Example B.1: Free Throw ShootingExample: Free Throw Shooting

– Let X and Y be a Bernoulli random variable. Note that a Bernoulli random variable is a zero‐one random variable.

X = 1 he makes the first free throwY = 1 he makes the second free throw

– Suppose that she or he is an 80% free‐throw shooter; i.e.,

P(X=1) = P(Y=1) = 0.8.

Here we are interested in the number of success in a sequence of Bernoulli trials.

• What is the probability of the player making both free throws?

P(X=1)P(Y=1) = (0.8)(0.8) =0.64





Example B.1: Free Throw Shooting

Example: continue

• If the chance of making the second free throw depends on whether the first was make – that is, X and Y are not independent.

• Intuitively, independence refers to knowing the outcome of X does not change the probabilities of the possible outcomes of Y and vice versa.






• Useful fact:

If X and Y are independent and we define new variables, g(X) and h(Y)for any function g and h, then

• these two random variables are also independent.





• Given a random variable X and a function g(.), – create g(X) as a new random variable.

• Example– X is a random variable, so are X2, exp(X) and log(X).


Example: Airline

– Suppose the airline accepts n reservations. For each i=1, …, n,

– let Yi denote the Bernoulli random variable indicating whether the customer shows up: Yi = 1, if he shows upYi = 0, otherwise

– Let be the probability of success. Each Yi has a Bernoulli () distribution.






Example: Airline (continue)

– Define X be the number of customers showing up out of n reservationsX = Y1 + … + Yn

– Assume Yi has the probability of success and the Yi are independence. Then, the probability density function of X is

– When a random X has a pdf as above, we writeX Binomial (n, ).






– n! = n(n‐1)(n‐2) 1– 0! = 1

!!( )!

n nx x n x






Example: Let n=100, = 0.95• What is the probability that none shows up?

P(X=0) = 1*(.95)0*(.05)100 = 1*1*(7.8886E‐131) = 7.8866E‐131

Let n=100, = 0.5• What is the probability that at least one person shows up?

P(X1) = 1 – P(X=0) = 1 ‐ (7.8886E‐131) = 1

! 100! 1!( )! 100!

n nx x n x





Conditional Distributions

• Generally, Y is related to one or two or more variables.

• How X affects Y is contained in the conditional distribution of Y given X.






• This information is summarized in the conditional probability density function, defined as

fYX(yx) = fX,Y(x,y)/fX(x) f(yx) = f(x,y)/f(x)

for all values of x such that fX(x)>0.

• The conditional pdf when X and Y are discrete is

fYX(yx) = P(Y=yX=x) = P(X=x,Y=y)/P(X=x)






• When X and Y are independent, knowledge of the value taken by X tells us nothing about the probability that Y takes on various values, and vice versa. That is,

fYX(yx) = fY(y) (show !) f(yx) = f(y)

fXY(xy) = fX(x) f(xy) = f(x)





Example B.2 Free Throw Shooting

Example: Free Throw Shooting

• Suppose X is the first free throw and Y is the second free throw.

fYX(11) = .85– If the player makes the first free throw, the probability of making the

second free throw is 0.85.





Example B.2 Free Throw Shooting

fYX(11) = .85

• Interpret the following:1) fYX(10) = .702) fYX(01) = .153) fYX(00) = .304) P(X=1) = .805) P(X=1,Y=1)

• Can we compute P(X=1,Y=1)?

P(X=1,Y=1) = P(Y=1X=1)P(X=1) Why??= (0.85)*(0.8) = .65





III. Features of Probability Distributions

• We are interested in few aspects (concepts) of the distributions of random variables.

1) Measure of central tendency2) Measure of variability or spread3) Measure of association between two random variables.

A. Central tendencyB. Variability



III. Features of Probability Distribution

A Measure of Central Tendency

• Expected value– one of the most important probabilistic concepts – names: expected value, the expectation of X– notations: E(X), X,

• If X represents some variable in a population,– The expected value is the population mean.


43I. Random Variables II. Joint & Conditional D’s III. Probability D’s IV. Features of Joint & Conditional D’s V. Normal & Related D’s


• Suppose X is a random variable.

– The expected value of X is the weighted average of all possible values of X.

– The weights are determined by the probability density function.



Fundamentals of Probability . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat III. Features of Probability Distribution


• Precise (mathematical) definition

Suppose X is a discrete random variable taking on a finite number of values {x1, x2,..., xk} and f (x) denote the probability function. The expected value of X is




Example B.3 Computing an Expected Value

Example:

– X takes on the values ‐1, 0, 2 with probabilities 1/8, ½, and 3/8E(X) = ‐1(1/8) + 0(1/2) + 2(3/8)

= 5/8 E(X2)

Notes:1) The expected value can be a number that is not even a possible value of X2) It can be deficient for summarizing the central tendency of certain discrete random

variables.





• Suppose X is a continuous random variable, the expected value defined as an integral:

• Notes1) E(X) is always a number that is a possible outcome of X.2) It is computed using integration.





• For discrete random variable, the expected value of g(X) is simply a weighted average:

• For continuous random variable, the expected value of g(X) is




A Measure of Central TendencyExample: • Let g(X) =X2.

• The random variable X takes on the values ‐1, 0, 2 with probabilities 1/8, ½, 3/8

E(X2) = ‐12(1/8) + 02 (1/2) + 22 (3/8) = 13/8

• Note that E(X2) [E(X)]2Example: 13/8 [5/8]2

• This suggests that for a nonlinear function, g(X),E[g(x)] g[E(x)].





• Suppose X and Y are two discrete random variables. • The expected value of g(X,Y), given X and Y taking on values {x1, …, xk} and

{y1, …, ym}, respectively, is

where fX,Y(xh, yj) is the joint probability density function of (X,Y).

,1 1

[ ( , )] ( , ) ( , )k m

X Y h j h jh j

E g X Y f x y g x y




Properties of Expected Value

• There are a few simple rules that we need to know to appreciate certain manipulations for important results.

Property E.1For any constant c, E(c) = c




Property E.2• For any constants a and b,

E(aX + b) = aE(X) + b

Example: Let E(X) = Y = X‐E(Y) = E(X) – = 0Where a = 1 and b = ‐






Example: Y is in Fahrenheit and X is in Celsius.

Y = 32 + (9/5)X

• What is the expected temperature in Fahrenheit if the expected temperature in Bangkok is 25 degree Celsius?

E(X) = 25E(Y) = 77





Property E.3• If {a1, …, an} are constants and

{X1, …, Xn} are random variables, thenE(a1X1 + … + anXn) = a1E(X1) + ... + anE(Xn)

• Using summation notation,





• If ai=1, then

Read: The expected value of the sum is the sum of expected value.• This property is often used!




Example B.5 Finding Expected RevenueExample:Let X1, X2, X3 are the numbers of small, medium, and large pizzas with

E(X1)=25, E(X2)=57, E(X3)=40

The prices of small, medium, and large pizzas are $5.5, $7.6, an d $9.15

1)The expected revenue from pizza sales isE(5.5X1 +7.6X2 + 9.15X3 ) = 5.5E(X1) + 7.6E(X2) + 9.15E(X3)

=5.5(25) + 7.6(57) + 9.15(40) =137.5+433.2+366 = 936.70

2) The actual revenue on any particular day will generally differ from this value




Example B.5 Finding Expected Revenue

• Given X Binomial(n,). Then, E(X) = n

Show:1) Note that X = Y1+ …+Yn, where Yi Bernoulli ().

E(X) = E(Y1+ …+Yn) = E(Y1) + …+ E(Yn) = n– The expected number of successes in n Bernoulli trials is the number of trials times the

probability of success on any particular trial.

2) Suppose that =.85. What is the expected value of people showing up when there are 120 bookings?

E(X) = 102




Expected Value Summation

E.1 E(c) = c s.1

E.2 E(aX+b) = aE(X) + b s.2

E.3 E ∑ ∑If ai=1, E ∑ ∑ s.3

Summary: Expectation and Summation



The Median

• A median is another measure of central tendency. It is sometimes denoted as med(X).

• If X is discrete and takes on a finite number of values, the median is obtained by ordering the possible values of X and select the value in the middle.




The Median

Example: X takes on the values{‐4, 0, 2, 8, 10, 13, 17}• The median is 8

Example: {‐5, 3, 9, 17}• What is the median? (6)




The Median• What is one special case that mean and median are equal?




Ans. This could occur if the probability distribution of X is symmetrically distributed about the value µ.

Mathematically, the condition isf( + x) = f( - x)

Symmetric distribution

Measures of Variability

• Expected value – a measure of central tendency

• Variance – a measure of dispersion– Variance shows how the individual x values are spread or distributed

about its mean value.




VarianceSuppose there are two pdfs with the same mean. If the distribution of X is

more tightly centered about the mean than is the distribution of Y, – we say that the variance of X is smaller than that of Y.




Variance

• Given a random variable X and the population mean =E(X).

1) The expectation of X measures the central tendency2) The variance, a measure of variability, measures how far X from , on average.

• Variance tells us the expected distance from X to its mean:

Var(X) = E(X‐)2

1) We works with squared difference. 2) The squaring serves to eliminate sign from the distance measure.




Variance

• Notations for Variance2

x, or simply 2, when the context is clear.

• Show that 2 = E(X2) – 2

2 = E(X2 ‐2X – 2) = E(X2) ‐2E(X) – 2

= E(X2) – 2




Variance and Bernoulli

Example: [Binomial]– Given Y Bernoulli () then,

E(Y) =

– Since Y2=Y (say Y=1 if the person shows up for reservation) and E(Y2) = ; thus,

Var(Y) = – 2 = (1‐ )

• Note that Var(Y) = E(Y2) – 2




Variance and Properties

Property V.1• Var(X)=0 if and only if there is a constant c such that P(X=c)=1,

in which case, E(X)=c





Property V.2• For any constants, a and b,

Var(aX+b) = a2Var(X)

Notes:1) Adding a constant to a random variable does not change the variance.2) Multiplying a random variable by a constant increases the variance by a

factor equal to the square of that constant.





Example:• What is the variance of Y when

Y = 32 + (9/5)X

• Var(Y) = (9/5)2Var(X)= (81/25)Var(X)




Standard Deviation

• The standard deviation of a random variable, sd(X), is simply the positive square root of the variance:

sd(X) = +[VAR(X)] ½

• Notations for he standard deviation x or simply when the random variable is understood.




Standard Deviation

Property SD.1• For any constant c,

sd(c)=0




Standard DeviationProperty SD.2• For any constants, a and b,

sd(aX+b) = asd(X)

• If a>0, thensd(aX+b) = asd(X)

• Property SD.2 makes standard deviation more natural to work with than variance.




Standard Deviation

Example:• Let X be a random variable in thousand of dollars

Let Y=1000X; i.e., Y is a random variable in dollarsSuppose E(X) = 20 and sd(X) = 6, What are E(Y), sd(Y), and Var(Y)?

• Ans.E(Y) = 20,000 and sd(Y) = 6,000Var(Y) = (1000)2Var(X) = 1,000,000Var(X);

• Note that the variance of Y is one million times larger than the variance of X.




Standardizing a Random Variable

• Let X be a random variable. A new random variable can be obtained by – subtracting off its mean and – dividing by its standard deviation :





• This procedure is known as standardizing the random variable X. – Z is called standardized random variable. – In introductory statistic courses, it is called Z‐transform of X.





• A standardized random variable Z has a mean of zero and a variance equal to one.

Z = aX+b

where a = 1/ and b = ‐ /

• Z has zero MeanE(Z) = E(X/ – /)

= (1/)E(X) –(/) Note that E(X) = = 0





Z = aX+bwhere a = 1/ and b = ‐/

• Z has unit varianceVar(Z) = Var(X/ – /)

= (1/2)Var(X) Note that Var(X) = 2

= 1





Example: Let E(X)=2 and Var(X)=9• What is the standardized random variable? What are its mean and

variance?

• Ans.1) Standardized random variable:

Z = (X‐)/ = (x‐2)/32) Z has expected value zero.3) Z has variance one.

.




Expected Value: E(X)Variance: Var(X) = E(X‐)2

Standard Deviation: s.d(X) =

E.1 E(c) = c v.1 Var(c) = 0s.d.1 s.d.(c) = 0

E.2 E(aX+b) = aE(X) + b v.2 Var(aX+b) = a2Var(X)s.d.2 s.d.(aX+b) =as.d.(X)

E.3 E ∑ ∑If ai=1, E ∑ ∑

Summary: Expectation and Summation



Problem B.7

B.7 If a basketball player is a 74% free‐throw shooter, then, on average, how many free throws will he or she make in a game with eight free‐throw attempts? [ans.]




Problem B.7

B.7• In eight attempts the expected number of free throws is 8(.74) = 5.92, or about six free throws.



Problem B.8

• B.8 Suppose that a college student is taking three courses: a two‐credit course, a three credit course, and a four‐credit course. The expected grade in the two‐credit course is 3.5, while the expected grade in the three‐ and four‐credit courses is 3.0. What is the expected overall grade point average for the semester? [ans.](Remember that each course grade is weighted by its share of the total number of units.)




Problem B.8B.8The weights for the two‐, three‐, and four‐credit courses are

2/9, 3/9, and 4/9, respectively.

• Let Yj be the grade in the jth course, j = 1, 2, and 3, let X be the overall grade point average. Then

X = (2/9)Y1 + (3/9)Y2 + (4/9)Y3• The expected value is

E(X) = (2/9)E(Y1) + (3/9)E(Y2) + (4/9)E(Y3)= (2/9)(3.5) + (3/9)(3.0) + (4/9)(3.0)= (7 + 9 + 12)/9 3.11.



Problem B.9

• B.9 Let X denote the annual salary of university professors in the United States, measured in thousands of dollars. Suppose that the average salary is 52.3, with a standard deviation of 14.6. Find the mean and standard deviation when salary is measured in dollars. [ans.]




Problem B.9

B.9• If Y is salary in dollars then Y = 1000X, and so the expected value of Y is 1,000 times the expected value of X, and the standard deviation of Y is 1,000 times the standard deviation of X.

• Therefore, the expected value and standard deviation of salary, measured in dollars, are $52,300 and $14,600, respectively.



IV. Features of Joint and Conditional Distributions• The joint probability density function (pdf) of two random

variables completely described relationship between them.

• There are two summary measures that describe how, on average, X and Y vary with one another: covariance and correlation.

A. CovarianceB. Correlation CoefficientC. Conditional ExpectationD. Conditional Variance


Fundamentals of Probability . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat IV. Features of Joint and Conditional Distributions

Covariance

• The covariance between two random variables, X and Y, measures linear dependence between two variables. It is defined as the expected value of the product (X‐X) and (Y‐Y):

Cov(X,Y) = E(X‐X)(Y‐ Y)

• which is sometimes denoted as XY.




CovarianceLet X=E(X) and Y=E(Y) and consider the variable (X‐ X)(Y‐ Y).

– X > X implies “X is above its mean”

– Y > Y implies “Y is above its mean”

– If X > X and Y > Y, then (X‐ X)(Y‐ Y) > 0– If X < X and Y < Y, then (X‐ X)(Y‐ Y) > 0– If X > X and Y < Y, then (X‐ X)(Y‐ Y) < 0




Covariance• It can be shown that

Cov(X,Y) = E(XY) ‐ XY

• Show

Cov(X,Y) = E[(X ‐ X)(Y ‐ Y)]= E[(X ‐ X)Y]= E[X(Y‐ Y)]= E(XY) ‐ XY

• If E(X)=E(Y)=0, then Cov(X,Y) = E(XY).




Covariance and Properties

Property COV.1 [Binomial]• If X and Y are independent, then Cov(X,Y) = 0.

• Show Cov(X,Y) = E(XY) ‐ XY

= E(X)E(Y) ‐ XY = 0

Note that E(XY)=E(X)E(Y)] = XY

• The converse is not true.




Covariance and PropertiesThe converse is not true! If the covariance of X and Y is zero, this does not

imply that X and Y are independent.

Example: (See CE.5)• There are random variables X such that if Y=X2, then Cov(X,Y) = 0

– [Any random variable with E(X)=0 and E(X3)=0 has this property]– Show Cov(X,Y) = 0.

Cov(X,Y) = E(XY) ‐ XY

= E(X3) – 0*Y = 0

• If Y=X2, then X and X2 are clearly not independent: once we know X, we know Y.




Covariance and Properties

Property COV.2• If a1, a2, b1, and b2 are constants, then

Cov(a1X+b1,a2Y+b2) = a1a2Cov(X,Y)

• Covariance can be altered by multiplying by a constant




Correlation Coefficient

• Suppose we want to find the relationship between amount of earnings X and annual earnings Y. – We can find the covariance between X and Y, – but it depends on the units of measurement.

• The deficiency of the covariance measure is overcome by the correlation coefficient of X and Y:





• Notes

1) Notation: XY XY = XYXX

2) It sometimes called the population correlation.3) The sign depends on XY. 4) The magnitude of Corr(X,Y) is easier to interpret and invariant to the units

of measurement (see to Property Corr.1).5) If X and Y are independent, then Corr(X,Y) = 0.





Property Corr.1

‐1 Corr(X,Y) 1

• Notes

1) If Corr(X,Y) = 0, equivalently Cov(X,Y), then there is no linear relationship between X and Y, and X and Y are uncorrelated random variables.

2) If Corr(X,Y) = 1, there is a perfect positive linear relationship.

3) If Corr(X,Y) = ‐1, there is a perfect negative linear relationship.





Property Corr.2• For any constants, a1, b1, a2, and b2,(1) if a1a2 > 0, then

Corr(a1X+b1,a2Y+b2) = Corr(X,Y), (2) if a1a2 < 0, then

Corr(a1X+b1,a2Y+b2) = ‐ Corr(X,Y).

• Note that correlation coefficient does not depend on any units of measurement such as dollars, cents, years quarters, months, and so on.




CovarianceCov(X,Y) = E(X‐uX)(Y‐uY)

CorrelationCorr(X,Y) = ,. · .

Cov.1 If X, Y independent,Cov(X,Y) = 0 Corr.1 ‐1 Corr(X,Y) 1

Cov.2 Cov(a1X+b1,a2Y+b2) = a1a2Cov(X,Y)

Corr.2 If a1a2>0, Corr(a1X+b1, a2Y+b2) = Corr(X,Y)

If a1a2<0, Corr(a1X+b1, a2Y+b2) = – Corr(X,Y)

Summary: Covariance and Correlation



Variance of Sums of Random Variables

Property VAR.3• For constants a and b,

Var(aX+bY) = a2Var(X) + b2Var(Y) + 2abCov(X,Y)

Var(aX‐bY) = a2Var(X) + b2Var(Y) ‐ 2abCov(X,Y)





If X and Y are uncorrelated, Cov(X,Y)=0, then

1) The variance of sum is the sum of the variance.Var(X + Y) = Var(X) + Var(Y)

2) The variance of difference is the sum of the variance, not the difference in variances.

Var(X ‐ Y) = Var(X) + Var(Y)




Variance of Sums of Random VariablesExample:X denote Friday profits earned by a restaurant Y denote Saturday profits earned by a restaurant

Let E(X) = E(Y) = 300 and sd(X) = sd(Y) = 15

What is the expected profits for the two nights?

– E(Z) = E(X) + E(Y) = 300+300 = 600

If X and Y are independent, what is the standard deviation of total profits?

– Var(Z) = Var(X) + Var(Y) = 2*225 = 450– Standard deviation = 21.21





Property VAR.4• If {X1, …, Xn} are pairwise uncorrelated random variables and {ai : i= 1, …, n} are

constants, then

Var(a1X1 + …. + anXn) = a12Var(X1) + … + an2Var(Xn)

• The random variables {X1, …, Xn} are pairwise uncorrelated if each variable in the set is uncorrelated with every other variable in the set.





In summation notation,

A special case, if ai=1 for all i,




Variance of Sums of Random Variables Example:

• Let X Binomial(n,).

X = Y1 + … + Yn

where Yi are independent Bernoulli() random variables.

• Var(X) = Var(Y1) + … + Var(Yn)= n(1‐)

• Notes1) Var(Yi) = (1‐ )] (see IIIB.1)2) Independent random variables are uncorrelated (see COV.1)




Variance of Sums of Random Variables Example: Airline reservations

n=120 and =.85Yi=1, if person i shows up at the airport

• What is the variance of the number of passengers arriving for their reservation?Var(X) = n(1‐) = (120)(.85)(.15) = 15.3standard deviation = 3.9




Variance and Expectation


constants, then



105

Expected Value: E(X) Variance: Var(X) = E(X‐x)2

Var(X) = E(X)2 ‐ x2

E.1 E(c) = c v.1 Var(c) = 0

E.2 E(aX+b) = aE(X) + b v.2 Var(aX+b) = a2Var(X)

E.3 E ∑ ∑If ai=1, E ∑ ∑

v.3 Var(aX+bY) = a2Var(X) + b2Var(Y) + 2abCov(X,Y)

Var(aX–bY) = a2Var(X) + b2Var(Y) – 2abCov(X,Y)

v.4 Let {X1, …, Xn} be pairwise uncorrelated. Var ∑ ∑ 2If ai=1, Var ∑ ∑



Conditional Expectation

• In economics and business, we would like to explain one variable, called Y, in terms of another variable, say X.

• We have learn the concept of conditional probability function of Y given X.

• Note that the distribution of Y depends on X=x.





• We can summarize a relationship between Y and X by the conditional expectation of Y given X, sometimes called the conditional mean.

• We denote the conditional mean by E(YX=x) or E(Yx) for short.

• E(Yx) is a function x, which tells us how the expected value of Y varies with x.





• When Y is a discrete random variable taking on values {y1, … , ym}, then

• When Y is continuous, E(Yx) is defined by integrating yfYX(yx) over all possible values of y.

• The conditional expectation is simply the weighted average of possible values of Y, but now the weights take into account a particular value of X.

|1

( | ) ( | )m

j Y X jj

E Y X x y f y x




1( | ) ( | )

m

j jj

E Y x y f y x


• Conditional expectations can be linear and nonlinear functions.

Example:• E(wageeduc) = 4 +.60educ

• What is the average wage rate given employees have 16 years of education?E(wageeduc=16) = 4 +.60(16) = 4 + 9.6 =13.6





• Conditional expectations can be linear and nonlinear functions. Example: X is a random variable greater than zero

– E(Yx) = 10/x– see graph!

Notes:1) This could represent a demand function (Y=quantity; X=price)2) An analysis of linear association, such as correlation analysis, would be incomplete.




Properties of Conditional Expectation

Property CE.1• E[c(X)X] = c(X) for any function c(X).

• This implies that functions of X behave as constant when we compute the expectation conditional on X.

• Example 1: E(X2X) = X2 When we know X, we know X2

• Example 2: x = educE(xx) = xE(educeduc) = educ





Property CE.2.For functions a(X) and b(X),

E[a(X)Y+b(X)X) = a(X)E(YX) + b(X)

Example 1: Find the conditional expectation of a function, a+bxa+bxE(a + bxx) = a + bx

Example 2: Find the conditional expectation of a function, a+bx = 4 +.60educ4 +.60educ E(4 + .60educeduc) = 4 + .06educ

Example 3: Find the conditional expectation of a function, wage – 4 – .60educwage – 4 – .60educE(wage – 4 – .60educ educ) = E(wageeduc) – 4 – .60educ

Example 4: Find the conditional expectation of a function, XY + 2X2XY + 2X2E(XY + 2X2 X) = XE(YX) + 2X2




Properties of Conditional ExpectationProperty CE.3:• If X and Y are independent, E(YX) = E(Y)

Notes:1) If X and Y are independent, then the expected value of Y given X does not depend

on X.

2) Example: If wage and education are independent, then the average wages of high school and college graduates are the same.

3) A special case is that if U and X are independent and E(U)=0, then E(U) =0, andE(UX) = 0





• The law of iterated expectations says that the expected value of (X) is simply equal to the expected value of Y.

Property CE.4

E[E(YX)] = E(Y) [B.2] [See Problem B.10]

1) Find E(YX) as a function of X. Property CE.22) Take the expected value of E(YX) with respect to the distribution of X. 3) End up with E(Y)





Example:

1. Given E(educ)=11.5 andwage = 4 + .60educE(wageeduc) = 4 + .60educ Property CE.2

2. The law of iterated expectation implies that E[E(wageeduc)] = E(wage) = E(4+.60educ) Property CE.4

= 4+.60E(educ) = 4 + (.6)(11.5) = 10.9 or $10.9 an hour





Property CE.5• If E(Y X) = E(Y), then Cov(X,Y)=0 and Corr(X,Y)=0, or X and Y are

uncorrelated.

Notes:1. If knowledge of X does not change the expected value of Y, E(Y), then X

and Y are uncorrelated.2. If X and Y are correlated, then E(YX) must depend on X.

• The converse is not true: If X and Y are uncorrelated, E(YX) could still depend on X.




Expected Value and Conditional Expectation (CE)


constants, then



117

Expected Value: E(X) Conditional Expectation

E.1 E(c) = c CE.1 E[c(X) X] = c(X)

E.2 E(aX+b) = aE(X) + b CE.2 E[a(X)Y+b(X)X]= a(X)E[YX]+b(X)

E.3 E ∑ ∑If ai=1, E ∑ ∑ CE.3 If X and Y are independent,

E(YX)= E(Y)

CE.4 E[E(YX)] = E(Y)

CE.5 If E(YX)= E(Y), Cov(X,Y)=0



Conditional Variance

• Given random variables X and Y,Var(YX=x) = E{[Y‐E(Yx)]2x}

• The variance of Y, conditional on X=x is the variance associated with the conditional distribution of Y, given X=x.

Var(YX=x) = E{[Y‐E(Yx)]2x}= E(Y2x) – [E(Yx)]2





Example:Var(SAVINGINCOME) = 400 + .25INCOME

Notes:1) As income increases, the variance of saving levels also increases.2) The relationship between variance of SAVING and INCOME is separate from

that between the expected value of SAVING and INCOME.





Property CV.1• If X and Y are independent, then Var(YX)=Var(Y)

• This is obvious. If the distribution of Y given X does not depend on X. Var(YX) is just one feature of this distribution.

120




Problem B.10

B.10 Suppose that at a large university, college grade point average, GPA, and SAT score, SAT, are related by the conditional expectation

E(GPASAT) = .70 + .002SAT.

(i) Find the expected GPA when SAT 800. Find E(GPASAT = 1,400). Comment on the difference. [ans.]




Problem B.10 continue…

(ii) If the average SAT in the university is 1,100, what is the average GPA? [ans.](Hint: Use Property CE.4. E[E(YX)] = E(Y)]

(iii) If a student’s SAT score is 1,100, does this mean he or she will have the GPA found in part (ii)? Explain. [ans.]




Problem B.10 (i)

(i) • E(GPA|SAT = 800) = .70 + .002(800) = 2.3.

Similarly, • E(GPA|SAT = 1,400) = .70 + .002(1400) = 3.5.

• The difference in expected GPAs is substantial, and the difference in SAT scores is also rather large.



Problem B.10 (ii)

(ii) • Following the hint, we use the law of iterated expectations.

• Since E(GPA|SAT) = .70 + .002 SAT,

• the (unconditional) expected value of GPA is E(GPA) = 70 + .002 E(SAT)

= .70 + .002(1100) = 2.9.I. Random Variables II. Joint & Conditional D’s III. Probability D’s IV. Features of Joint & Conditional D’s V. Normal & Related D’s


Problem B.10 (iii)

(iii)

• No, 2.9 is an average GPA. It does not mean a particular person will obtain GPA of 2.9.



V. The Normal and Related Distributions

• We will rely on normal and related distributions to make inference in statistics and econometrics.

• The normal distribution and its related distribution is most widely used distribution in statistics and econometrics.

A. NormalB. Standard NormalC. Chi SquareD. t distributionF. F distribution


Fundamentals of Probability . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat V. The Normal and Related Distributions

The Normal Distribution

• Suppose X is a normal random variable. The pdf of X can be written as

• where =E(X) and 2=Var(X). • We say that X has a normal distribution with expected value and

variance 2, written as X Normal(,2)




Notes on the Normal Distribution

1) A normal random variable is a continuous random variable.

2) Its pdf has a familiar bell shape.




3) A normal distribution is symmetric; thus is also the median of X.

4) The normal distribution is sometimes called the Gaussian distribution

The Normal Distribution

• Certain random variables appear to follow a normal distribution Examples: heights and weights.

• Some distribution is skewed towards the upper tail, i.e., income. However, a variable can be transformed to achieve normality. A popular transformation is the natural log. Let Y be log of income X.

Y=log(X) We say that X has a lognormal distribution.




The Standard Normal Distribution

• Suppose that a random variable Z is normally distributed with zero expected value (=0) and unit variance (2=1). Then, Z has a standard normal distribution. Z Normal(0,1)

• The pdf of a standard normal variable, denoted by (z),

and read “phi”





• The standard normal cumulative distribution function, denoted by (z), is obtained by the area under to the left of z. (see graph)

(z) = P(Z<z)

– Most software packages include simple commands for computing values of the standard normal cdf.

Example:– P(Z<‐3.1) = .001– P(Z<3.1) = .999

Note that P(Z>3.1) = .001




- 3.10 │ │ 3.10


P(Z<-3.1) = .001

P(Z<3.1) = .999








• Given (z) = P(Z<z), three important formulas for standard normal cdfs:1) P(Z>z) = 1‐(z)2) P(Z<‐z) = P(Z>z) 3) P(a<Z<b) = (b) –(a)




4) Another useful expression is that, for any c>0,

Notes:– 1) The probability P(Z>c) is simply twice the probability P(Z>c).– 2) This reflects the symmetry of the standard normal distribution.


• Examples:1) P(Z>.44) = 1 ‐ P(Z<.44) = 1 ‐ .67 = .332) P(Z<‐.92) = P(Z>.92) = 1‐ P(Z<.92) = 1 ‐ .821 = .1793) P(‐1<Z<.5) = P(Z<.5) ‐ P(Z<‐1) = .692 ‐ .159 = .53284) P(│Z│>.44) = P(Z>.44) + P(Z<.‐44) + = 2*P(Z>.‐44) = 2*(.33) = .66







Properties of the Normal Distribution

Property Normal.1• If X Normal(,2), then

(X‐)/ Normal(0,1).• Z Normal(0,1).

Example:• Suppose X Normal(3,4). Compute P(X1)

= P(Z‐1) = (‐1) = .159

- 3( 1) ( - 3 1 - 3) 12

xP X P X P







Probabilities for a Normal Random Variable

Example:Compute P(2X 6) when X Normal(4,9)

= (.67)‐ (‐.67) = .749 ‐ .251 = .498

2 - 4 - 4 6 - 4P(2<X 6) <3 3 32 2................... ( )3 3

xP

P Z



Fundamentals of Probability . Intensive Course in Mathematics and Statistics . Chairat Aemkulwat V. The Normal and Related Distributions 140



Probabilities for a Normal Random VariableExample:Compute P(X>2) when X Normal(4,9P(X>2) = P(X>2)+P(X<‐2)

= P[X−4 > 2−4] + P [X−4 < ‐ 2−4] = P(Z> ‐ 23)+P(Z<‐2)= 1 ‐(‐ 23) + (‐2)= 1 ‐.251+.023= .772







Property Normal.2• If X Normal(,2), then

aX+b Normal(a+b, a22)

Example: Let Y = 2X + 3If X Normal(1,9) then Y Normal(5,36)

Y = 2X + 3E(Y) = 2E(X) + 3 = 2*1 =5

Var(Y) = 22Var(X) = 4*9 = 36sd(Y) = 6





Property Normal.3• Let X and Y are jointly normally distributed. X and Y are independent if

and only if Cov(X,Y)=0.

• Zero correlation and independence are not the same. But in the case of normally distributed random variables, it turns out that zero correlation suffices for independence.





Property Normal.4• Any linear combination of independently identically distributed (i.i.d.)

normal variables has a normal distribution.

Example:• Let Xi be i.i.d. normal variables. Note that it is distributed as Normal(,2)

Y = X1 + 2X2 – 3X3Mean: E(Y) = + 2 ‐ 3 = 0Variance Var(Y) = 2 +42 + 92 = 142





Example:• Suppose Y1, Y2, … , Yn are i.i.d. normal random variables and each is distributed

as Normal(,2). Then

Show:= (1/n)[Y1+ ….+Yn]

E( ) = (1/n)[ + … +] = Var( ) = (1/n2)[Var(Y1) + … + Var(Yn)]

= (1/n2)[2 + … + 2] = 2/n




The Chi‐Square Distribution

• The chi‐square distribution is obtained from independent, standard normal random variables.

• Let Zi, i =1, …, n be independent random variables, each distributed standard normal. Define a new variable X as the sum of the squared of the Zi:




• X has what is known as a chi‐square distribution with n degrees of freedom. – df (the number of terms in the sum)

• We write this as X 2n

Notes: The Chi‐Square Distribution

3) If X 2n then, • the expected value of X is n• the variance of X is 2n

4) For df in excess of 100, the variable can be treated as a standard normal variable, where k is the df




1) A chi‐square random variable is always nonnegative

2) It is not symmetric about any point. It is skewed to the right for small df.




The t Distribution • The t distribution is the workhorse in classical statistics and multiple regression

analysis.

• Let Z have a standard normal distribution and X has a chi‐square distribution with n degrees of freedom. Assume that Z and X are independent. Then, the random variable T is




Notes: The t DistributionA. NormalB. Standard NormalC. Chi SquareD. t distributionF. F distribution



3) Expected value and Variance:Expected value = 0 (for n>1) Variance = n/(n‐2) (for n>2)

4) t distribution is symmetric about its mean, but has more area in tails.

1) The random variable T has t distribution with n degrees of freedom.– We denote this as T tn

2) t distribution gets its degrees of freedom from the chi‐square variable.




The F Distribution

• A F distribution is used for testing hypothesis in the context of multiple regression analysis.

• Let X1k1 and X2k2 and assume that X1 and X2 are independent. Then, the random variable F




has a F distribution with (k1,k2) DFs.

• We denote this as FFk1,k2k1 – the numerator degrees of freedom because it is associated with chi‐

square variable in the numerator.k2 – is the denominator degrees of freedom




Notes: The F Distribution 1) Relationship between t and F statistic

t2k2 = F1,K2

2) Relationship between 2and F statisticFk1,K2 = 1 as k2




3) Like the chi‐square distribution, the F distribution is skewed to the right.

4) As k1 and k2 become large, the F distribution approaches the normal distribution.




Notes: The F Distribution

In passing, note that since for large df (or n ), the t, chi-square, and F distributions approaches the normal distribution.




Problem B.2

B2. Let X be a random variable distributed as Normal(5,4). Find the probabilities of the following events:

(i) P(X 6) [ans.]

(ii) P(X > 4) [ans.]

(iii) P(X – 5> 1) [ans.]




Problem B.2 (i)

(i) Normal (5,4)

• P(X 6) = P[(X – 5)/2 (6 – 5)/2]= P(Z .5) (See Table G.1) .692

where Z denotes a Normal (0,1) random variable. [We obtain P(Z .5) from Table G.1.]



Problem B.2 (ii)

(ii) Normal (5,4)

• P(X > 4) = P[(X – 5)/2 > (4 – 5)/2]= P(Z > .5)= 1 ‐ P(Z .5) (See Table G.1)= 1 – 0.3085 .692.



Problem B.2 (iii)(iii) Normal (5,4) • P(|X – 5| > 1)

= P(X – 5 > 1) + P(X – 5 < –1)= P(X > 6) + P(X < 4)= P[(X‐5)/2 > (6‐4)/2] + P[(X‐5)/2 < (4‐6)/2)= P(Z > .5) + P(Z < ‐.5)

• P(|X – 5| > 1) = P(Z < ‐.5) + P(Z < ‐.5)= .3085 +.3085 = .617 (Table G.1)

• P(|X – 5| > 1) = P(Z > .5) + P(Z < ‐.5)= (1 – .692) + (1 – .692) = .616

where we have used answers from parts (i) and (ii).



Table G.1 B.1(i) B.2(ii) B.3(iii)



Date post:	10-Sep-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

Fundamentals of Probability II.pioneer.netserv.chula.ac.th/~achairat/Appendix B... · 2015. 7....

Documents