Post on 19-Jan-2016
transcript
1
Probability and Statistical Inference (9th Edition)
Chapter 4
Bivariate Distributions
November 4, 2015
2
Joint Probability Mass Function
Let X and Y be two discrete random variables defined on the same outcome set. The probability that X=x and Y=y is denoted by PX,Y(x,y)= P(X=x,Y=y) and is called the joint probability mass function (joint pmf) of X and Y
PX,Y(x,y) satisfies the the following 3 properties:
S. ofsubset a isA where,,,Pr )3(
1, )2(
1,0 )1(
,,
,,
,
AyxYX
SyxYX
YX
yxPAYXob
yxP
yxP
3
Example: Roll a pair of unbiased dice. For each of the 36 possible outcomes, let X denote the smaller number and Y denote the larger number
The joint pmf of X and Y is:
61 36/2
61 36/1,, yx
yxyxP YX
Joint Probability Mass Function
4
Note that we can always create a common outcome set for any two or more random variables. For example, let X and Y correspond to the outcomes of the first and second tosses of a coin, respectively. Then, the outcome set of X is {head up, tail up} and the outcome set of Y is also {head up, tail up}. The common outcome set of X and Y is {(head up, head up),(head up, tail up),(tail up, head up),(tail up, tail up)}
Joint Probability Mass Function
5
Another Example: Assume that we toss a dice once. Let random variable X correspond to whether the outcome is less than or equal to 2, and random variable Y correspond to whether the outcome is an even number. Then, the joint pmf of X and Y is shown on the next page
Joint Probability Mass Function
6
1
0 1PXY(0,0)=1/3
PXY(0,1)=1/3
PXY(1,1)=1/6
PXY(1,0)=1/6
Outcome 1 2 3 4 5 6
X 1 1 0 0 0 0
Y 0 1 0 1 0 1
X
Y
Joint Probability Mass Function
7
Marginal Probability Mass Function
Let PXY(x,y) be the joint pmf of discrete random variables X and Y. Then
is called the marginal pmf of X Similarly,
is called the marginal pmf of Y
j
j
yiXY
yiX
yxP
yYxXobxXobxP
),(
,PrPr
ix
iYXY yxPyP ,,
8
Independent Random Variables
Two discrete random variables X and Y are said to be independent if and only if
Otherwise, X and Y are said to be dependent
.,, yPxPyxP YXYX
9
Uncorrelated Random Variables
Let X and Y be two random variables. Then, E[(X-µX)(Y-µY)] is called the covariance of X and Y (denoted by Cov(X,Y))
Covariance is a measure of how much two random variables change together
A positive value of Cov(X,Y) indicates that Y tends to increase as X increases
Two discrete random variables X and Y are said to be uncorrelated if Cov(X,Y)=0
Otherwise, X and Y are said to be correlated
10
Independent Implies Uncorrelated Cov(X,Y) = E[(X-µX)(Y-µY)]
= E[XY- µYX- µXY+ µXµY]
= E[XY]- µYE[X]- µXE[Y]+E[µXµY]
= E[XY]- µXµY
If X and Y are independent, then
Therefore, if X and Y are independent, then Cov(X,Y)=0 The converse statement is not true (example later)
][][
)()(
)()(
),(][
YEXE
yyPxxP
yPxxyP
yxxyPXYE
yY
xX
x yYX
x yXY
11
Correlation Coefficient
Correlation coefficient of X and Y:
Insights: If X and Y are above or below their respective means simultaneously, then ρXY > 0. If X is above µX whenever Y is below µY, and X is below µX whenever Y is above µY, then ρXY < 0
12
Addition of Two Random Variables
Let X and Y be two random variables. Then, E[X+Y]=E[X]+E[Y]
Note that the above equation holds even if X and Y are dependent
Proof of the discrete case:
][][)()(
),(),(
),(),(
))(,(][
YEXEyyPxxP
yxPyyxPx
yxPyyxPx
yxyxPYXE
yY
xX
y xXY
x yXY
x yXY
x yXY
x yXY
13
On the other hand,
#
2222
2222
222
22
2
])[][][(2][][
)][(2)][()][(
2][2][][
)(2)(])[(
)])((2)()[(
]))()[((
][
YEXEXYEYVarXVar
XYEYEXE
XYEYEXE
YXE
YXYXE
YXE
YXVar
yxyx
yxyx
yxyx
yxyx
yx
Addition of Two Random Variables
14
Note that if X and Y are independent, then E[XY]=E[X]E[Y]
Therefore, if X and Y are independent, then Var[X+Y]=Var[X]+Var[Y]
Addition of Two Random Variables
15
Examples of Correlated Random Variables
Assume that a supermarket collected the following statistics of customers’ purchasing behavior:
Purchasing
Wine
Not Purchasing
Wine
Male 45 255
Female 70 630
Purchasing
Juice
Not Purchasing
Juice
Male 60 240
Female 210 490
16
Examples of Correlated Random Variables
Let random variable M correspond to whether a customer is male, random variable W correspond to whether a customer purchases wine, random variable J correspond to whether a customer purchases juice
17
The joint pmf of M and W is
Cov(M,W)= E[MW] - E[M]E[W]= 0.045 – 0.3*0.115 = 0.0105 > 0M and W are positively correlated (outcome M=1 makes it more likely that W=1)
W
M
PMW (1,1) = 0.045
PMW (1,0) = 0.255
PMW (0,1) = 0.07
PMW (0,0) = 0.63
Examples of Correlated Random Variables
18
The joint pmf of M and J is
Cov(M,J)= E[MJ] - E[M]E[J]= 0.06 – 0.3*0.27 = -0.021 < 0M and J are negatively correlated
W
M
PMJ (1,1) = 0.06
PMJ (1,0) = 0.24
PMJ (0,1) = 0.21
PMJ (0,0) = 0.49
Examples of Correlated Random Variables
19
Example of Uncorrelated Random Variables
Assume X and Y have the following joint pmf:PXY(0,1)= PXY(1,0)= PXY(2,1)= 1/3
We can derive the following marginal pmfs:
xXYY
x xXYYXYX
yXYX
yXYX
xPP
xPPyPP
yPPyPP
3/2)1,(1
3/1)0,(0 ; 3/1),2(2
3/1),1(1 ; 3/1),0(0
20
Example of Uncorrelated Random Variables
Since PXY(0,1) = 1/3, andPX(0) x PY(1) = 1/3 x 2/3 = 2/9,X and Y are not independent
However,Cov(X,Y) = E[XY] – E[X]E[Y] = [2/9 x 1 + 2/9 x 2] – [1 x 2/3] = 0.Thus, X and Y are uncorrelated
Thus, uncorrelated does not imply independence
21
Conditional Distributions
Let X and Y be two discrete random variables. The conditional probability mass function (pmf) of X, given that Y=y, is defined by
Similarly, if X and Y are continuous random variables, then the conditional probability density function (pdf) of X, given that Y=y, is defined by
Y. of Space y that provided ,
,
yP
yxPyxP
Y
XYYX
.
,
yf
yxfyxf
Y
XYYX
22
Conditional Distributions
Assume that X and Y are two discrete random variables. Then,
Similarly, for two continuous random variables X and Y, we have
.1
,,
Y. of Space y that provided ,0,
x x Y
Y
Y
xXY
Y
XYYX
Y
XYYX
yP
yP
yP
yxP
yP
yxPyxPb
yP
yxPyxPa
.1
Y. of Space y that provided ,0,
XS
yxfb
yf
yxfyxfa
YX
Y
XYYX
23
Conditional Distributions
The conditional mean of X, given that Y=y, is defined by
The conditional variance of X, given that Y=y, is defined by
. EYX x
YX yxxPyx
. 2
YX x
YXYX yxPx
24
Example 1
Let X and Y have the joint pmf
It can be easily shown that
Then, the conditional pmf of X, given that Y=y, is
25
Example 1 (Cont.)
Similarly, the conditional pmf of Y, given that X=x, is
26
Example 2
3 blue balls (labeled A, B, C) and 2 red balls (labeled D, E) are in a bag
Randomly taking a ball out of the bag, what is the probability of getting a blue ball? (Ans: 3/5)
What is the probability of getting A? (Ans: 1/5) What is the probability of getting A, given that the bal
l we get is a blue ball? (Ans: 1/3)
X = label of the ball we getY = color of the ball we getP(X=A | Y=blue) = P(X=A, Y=blue) / P(Y=blue) = (1/5) / (3/5) = 1/3
27
Bivariate Normal Distribution
The joint pdf of bivariate normal
The joint pdf of multivariate normal
where in the case of bivariate
and | | denotes the determinant of a matrix
28
Bivariate Normal Distribution
-3-2
-10
12
3
-2
0
2
0
0.1
0.2
0.3
0.4
x1x2
Prob
abili
ty D
ensi
ty
-3-2
-10
12
3
-2
0
2
0
0.1
0.2
0.3
0.4
x1x2
Prob
abili
ty D
ensi
ty
Graphic representations of bivariate (2D) normal
29
Bivariate Normal Distribution
x1
x2
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
x1
x2
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
30
Bivariate Normal Distribution
-3-2
-10
12
3
-2
0
2
0
0.1
0.2
0.3
0.4
x1x2
Prob
abili
ty D
ensi
ty
-3-2
-10
12
3
-2
0
2
0
0.1
0.2
0.3
0.4
x1x2
Prob
abili
ty D
ensi
ty
31
Bivariate Normal Distribution
x1
x2
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
x1
x2
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
32
Bivariate Normal Distribution
-3-2
-10
12
3
-2
0
2
0
0.1
0.2
0.3
0.4
x1x2
Prob
abili
ty D
ensi
ty
x1
x2
-3 -2 -1 0 1 2 3-3
-2
-1
0
1
2
3
33
Example
Let us assume that in a certain population of college students, the respective grade point average (GPA)—say X and Y—in high school and the first year in college have an approximate bivariate normal distribution with parameters
Then, for example,
where
34
Example (Cont.)
The conditional pdf of Y, given that X=x, is normal, with mean
and variance
35
Example (Cont.)
Since the conditional pdf of Y, given that X=3.2, is normal with mean
and standard deviation
we have
36
Correlations and Independence for Normal Random Variables
In general, random variables may be uncorrelated but statistically dependent (i.e., uncorrelated does not imply independence)
But if a random vector has a multivariate normal distribution, then any two or more of its components that are uncorrelated are independent (i.e., uncorrelated does imply independence in this case)
37
The fact that two random variables X and Y both have a normal distribution does not imply that the pair (X, Y) has a joint normal distribution.
Example: Suppose X has a normal distribution with expected value 0 and variance 1. Let
where c is a positive number X and Y are not jointly normally distributed,
even though they are separately normally distributed
Correlations and Independence for Normal Random Variables
38
If X and Y are normally distributed and independent, this implies they are "jointly normally distributed", i.e., the pair (X, Y) must have multivariate normal distribution
However, a pair of jointly normally distributed variables need not be independent (would only be so if uncorrelated)
Correlations and Independence for Normal Random Variables