1

Probability and Statistical Inference (9th Edition)

Chapter 4

Bivariate Distributions

November 4, 2015

2

Joint Probability Mass Function

Let X and Y be two discrete random variables defined on the same outcome set. The probability that X=x and Y=y is denoted by PX,Y(x,y)= P(X=x,Y=y) and is called the joint probability mass function (joint pmf) of X and Y

PX,Y(x,y) satisfies the the following 3 properties:

S. ofsubset a isA where,,,Pr )3(

1, )2(

1,0 )1(

,,

,,

,

AyxYX

SyxYX

YX

yxPAYXob

yxP

yxP

3

Example: Roll a pair of unbiased dice. For each of the 36 possible outcomes, let X denote the smaller number and Y denote the larger number

The joint pmf of X and Y is:

61 36/2

61 36/1,, yx

yxyxP YX

Joint Probability Mass Function

4

Note that we can always create a common outcome set for any two or more random variables. For example, let X and Y correspond to the outcomes of the first and second tosses of a coin, respectively. Then, the outcome set of X is {head up, tail up} and the outcome set of Y is also {head up, tail up}. The common outcome set of X and Y is {(head up, head up),(head up, tail up),(tail up, head up),(tail up, tail up)}

Joint Probability Mass Function

5

Another Example: Assume that we toss a dice once. Let random variable X correspond to whether the outcome is less than or equal to 2, and random variable Y correspond to whether the outcome is an even number. Then, the joint pmf of X and Y is shown on the next page

Joint Probability Mass Function

6

1

0 1PXY(0,0)=1/3

PXY(0,1)=1/3

PXY(1,1)=1/6

PXY(1,0)=1/6

Outcome 1 2 3 4 5 6

X 1 1 0 0 0 0

Y 0 1 0 1 0 1

X

Y

Joint Probability Mass Function

7

Marginal Probability Mass Function

Let PXY(x,y) be the joint pmf of discrete random variables X and Y. Then

is called the marginal pmf of X Similarly,

is called the marginal pmf of Y

j

j

yiXY

yiX

yxP

yYxXobxXobxP

),(

,PrPr

ix

iYXY yxPyP ,,

8

Independent Random Variables

Two discrete random variables X and Y are said to be independent if and only if

Otherwise, X and Y are said to be dependent

.,, yPxPyxP YXYX

9

Uncorrelated Random Variables

Let X and Y be two random variables. Then, E[(X-µX)(Y-µY)] is called the covariance of X and Y (denoted by Cov(X,Y))

Covariance is a measure of how much two random variables change together

A positive value of Cov(X,Y) indicates that Y tends to increase as X increases

Two discrete random variables X and Y are said to be uncorrelated if Cov(X,Y)=0

Otherwise, X and Y are said to be correlated

10

Independent Implies Uncorrelated Cov(X,Y) = E[(X-µX)(Y-µY)]

= E[XY- µYX- µXY+ µXµY]

= E[XY]- µYE[X]- µXE[Y]+E[µXµY]

= E[XY]- µXµY

If X and Y are independent, then

Therefore, if X and Y are independent, then Cov(X,Y)=0 The converse statement is not true (example later)

][][

)()(

)()(

),(][

YEXE

yyPxxP

yPxxyP

yxxyPXYE

yY

xX

x yYX

x yXY

11

Correlation Coefficient

Correlation coefficient of X and Y:

Insights: If X and Y are above or below their respective means simultaneously, then ρXY > 0. If X is above µX whenever Y is below µY, and X is below µX whenever Y is above µY, then ρXY < 0

12

Addition of Two Random Variables

Let X and Y be two random variables. Then, E[X+Y]=E[X]+E[Y]

Note that the above equation holds even if X and Y are dependent

Proof of the discrete case:

][][)()(

),(),(

),(),(

))(,(][

YEXEyyPxxP

yxPyyxPx

yxPyyxPx

yxyxPYXE

yY

xX

y xXY

x yXY

x yXY

x yXY

x yXY

13

On the other hand,

#

2222

2222

222

22

2

])[][][(2][][

)][(2)][()][(

2][2][][

)(2)(])[(

)])((2)()[(

]))()[((

][

YEXEXYEYVarXVar

XYEYEXE

XYEYEXE

YXE

YXYXE

YXE

YXVar

yxyx

yxyx

yxyx

yxyx

yx

Addition of Two Random Variables

14

Note that if X and Y are independent, then E[XY]=E[X]E[Y]

Therefore, if X and Y are independent, then Var[X+Y]=Var[X]+Var[Y]

Addition of Two Random Variables

15

Examples of Correlated Random Variables

Assume that a supermarket collected the following statistics of customers’ purchasing behavior:

Purchasing

Wine

Not Purchasing

Wine

Male 45 255

Female 70 630

Purchasing

Juice

Not Purchasing

Juice

Male 60 240

Female 210 490

16

Examples of Correlated Random Variables

Let random variable M correspond to whether a customer is male, random variable W correspond to whether a customer purchases wine, random variable J correspond to whether a customer purchases juice

17

The joint pmf of M and W is

Cov(M,W)= E[MW] - E[M]E[W]= 0.045 – 0.3*0.115 = 0.0105 > 0M and W are positively correlated (outcome M=1 makes it more likely that W=1)

W

M

PMW (1,1) = 0.045

PMW (1,0) = 0.255

PMW (0,1) = 0.07

PMW (0,0) = 0.63

Examples of Correlated Random Variables

18

The joint pmf of M and J is

Cov(M,J)= E[MJ] - E[M]E[J]= 0.06 – 0.3*0.27 = -0.021 < 0M and J are negatively correlated

W

M

PMJ (1,1) = 0.06

PMJ (1,0) = 0.24

PMJ (0,1) = 0.21

PMJ (0,0) = 0.49

Examples of Correlated Random Variables

19

Example of Uncorrelated Random Variables

Assume X and Y have the following joint pmf:PXY(0,1)= PXY(1,0)= PXY(2,1)= 1/3

We can derive the following marginal pmfs:

xXYY

x xXYYXYX

yXYX

yXYX

xPP

xPPyPP

yPPyPP

3/2)1,(1

3/1)0,(0 ; 3/1),2(2

3/1),1(1 ; 3/1),0(0

20

Example of Uncorrelated Random Variables

Since PXY(0,1) = 1/3, andPX(0) x PY(1) = 1/3 x 2/3 = 2/9,X and Y are not independent

However,Cov(X,Y) = E[XY] – E[X]E[Y] = [2/9 x 1 + 2/9 x 2] – [1 x 2/3] = 0.Thus, X and Y are uncorrelated

Thus, uncorrelated does not imply independence

21

Conditional Distributions

Let X and Y be two discrete random variables. The conditional probability mass function (pmf) of X, given that Y=y, is defined by

Similarly, if X and Y are continuous random variables, then the conditional probability density function (pdf) of X, given that Y=y, is defined by

Y. of Space y that provided ,

,

yP

yxPyxP

Y

XYYX

.

,

yf

yxfyxf

Y

XYYX

22

Conditional Distributions

Assume that X and Y are two discrete random variables. Then,

Similarly, for two continuous random variables X and Y, we have

.1

,,

Y. of Space y that provided ,0,

x x Y

Y

Y

xXY

Y

XYYX

Y

XYYX

yP

yP

yP

yxP

yP

yxPyxPb

yP

yxPyxPa

.1

Y. of Space y that provided ,0,

XS

yxfb

yf

yxfyxfa

YX

Y

XYYX

23

Conditional Distributions

The conditional mean of X, given that Y=y, is defined by

The conditional variance of X, given that Y=y, is defined by

. EYX x

YX yxxPyx

. 2

YX x

YXYX yxPx

24

Example 1

Let X and Y have the joint pmf

It can be easily shown that

Then, the conditional pmf of X, given that Y=y, is

25

Example 1 (Cont.)

Similarly, the conditional pmf of Y, given that X=x, is

26

Example 2

3 blue balls (labeled A, B, C) and 2 red balls (labeled D, E) are in a bag

Randomly taking a ball out of the bag, what is the probability of getting a blue ball? (Ans: 3/5)

What is the probability of getting A? (Ans: 1/5) What is the probability of getting A, given that the bal

l we get is a blue ball? (Ans: 1/3)

X = label of the ball we getY = color of the ball we getP(X=A | Y=blue) = P(X=A, Y=blue) / P(Y=blue) = (1/5) / (3/5) = 1/3

27

Bivariate Normal Distribution

The joint pdf of bivariate normal

The joint pdf of multivariate normal

where in the case of bivariate

and | | denotes the determinant of a matrix

28

Bivariate Normal Distribution

-3-2

-10

12

3

-2

0

2

0

0.1

0.2

0.3

0.4

x1x2

Prob

abili

ty D

ensi

ty

-3-2

-10

12

3

-2

0

2

0

0.1

0.2

0.3

0.4

x1x2

Prob

abili

ty D

ensi

ty

Graphic representations of bivariate (2D) normal

29

Bivariate Normal Distribution

x1

x2

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

x1

x2

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

30

Bivariate Normal Distribution

-3-2

-10

12

3

-2

0

2

0

0.1

0.2

0.3

0.4

x1x2

Prob

abili

ty D

ensi

ty

-3-2

-10

12

3

-2

0

2

0

0.1

0.2

0.3

0.4

x1x2

Prob

abili

ty D

ensi

ty

31

Bivariate Normal Distribution

x1

x2

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

x1

x2

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

32

Bivariate Normal Distribution

-3-2

-10

12

3

-2

0

2

0

0.1

0.2

0.3

0.4

x1x2

Prob

abili

ty D

ensi

ty

x1

x2

-3 -2 -1 0 1 2 3-3

-2

-1

0

1

2

3

33

Example

Let us assume that in a certain population of college students, the respective grade point average (GPA)—say X and Y—in high school and the first year in college have an approximate bivariate normal distribution with parameters

Then, for example,

where

34

Example (Cont.)

The conditional pdf of Y, given that X=x, is normal, with mean

and variance

35

Example (Cont.)

Since the conditional pdf of Y, given that X=3.2, is normal with mean

and standard deviation

we have

36

Correlations and Independence for Normal Random Variables

In general, random variables may be uncorrelated but statistically dependent (i.e., uncorrelated does not imply independence)

But if a random vector has a multivariate normal distribution, then any two or more of its components that are uncorrelated are independent (i.e., uncorrelated does imply independence in this case)

37

The fact that two random variables X and Y both have a normal distribution does not imply that the pair (X, Y) has a joint normal distribution.

Example: Suppose X has a normal distribution with expected value 0 and variance 1. Let

where c is a positive number X and Y are not jointly normally distributed,

even though they are separately normally distributed

Correlations and Independence for Normal Random Variables

38

If X and Y are normally distributed and independent, this implies they are "jointly normally distributed", i.e., the pair (X, Y) must have multivariate normal distribution

However, a pair of jointly normally distributed variables need not be independent (would only be so if uncorrelated)

Correlations and Independence for Normal Random Variables