29
Chapter 4 Multivariate Normal Distribution
Chapter 4Multivariate Normal
Distribution
4.1 Random Vector4.1 Random Vector4.1 Random Vector4.1 Random Vector Random Variable
Random Vector
X
,1
pX
X
X
X1, , Xp are random variables
A. Cumulative Distribution Function (c.d.f.)
Random Variable
F(x) = P(X x)
F(x) = F(x1,,xp) = P(X1 x1, , Xp xp)
Marginal distribution
• F(x1) = P(X1x1) = P(X1x1, X2, , Xp) = F(x1, , , )
• F(x1, x2) = P(X1x1 ,X2x2) = F(x1, x2, , , )
Random Vector
B. Density Random Variable
Random Vector
)()(
)()(F
xF'xf
xdttfxx
p
pp
ppp
x x
pp
xx
xxxxf
xxdtdtttfxxp
1
11
1111
),,(F),,(
),,(),,(),,(F1
R
C. Conditional Distribution
Random Variable
Random Vector
P(A)
P(B)
B)P(A)BP(A
Conditional Probability of A given B
when A and B are not independent
Conditional Density of x1,, xq given xq+1= xq+1, , xp= xp .
),,|,,( 11 pqq xxxxf ),,(
),,,,,(
1
11
pq
pqq
xxf
xxxxf
h
g where h: the joint density of x1,, xp ;
g: the marginal density of xq+1, , xp .
D. Independence Random Variable
Random Vector
t.independen are Y and X
,)P(Y)P(X
)Y,P(X
yxyx
yx
(X1,X2) ~ F(x1, x2)
If
F(x1, x2)= F1 (x1) F2 (x2) , x1, x2
x1 and x2 are said to be independent.
(X1,,Xp) ~ F(x1, ,xp)
If
X1, ,Xp are said to be mutually independent.
pixxxx i
p
iiip ,,1,)(F),,(F
11
(X1,X2) ~ F(x1, x2)
If
F(x1, x2)= F1 (x1) F2 (x2) , x1, x2
x1 and x2 are said to be independent.
(X1,,Xp) ~ F(x1, ,xp)
If
X1, ,Xp are said to be mutually independent.
pixxxx i
p
iiip ,,1,)(F),,(F
11
Random Vector
X ~ F(x1, ,xp), Y ~ G(y1, , yq) X and Y are independent if
.,,,,,any for
,,,,,,,,,
11
111111
pp
qpqqpp
RyyRxx
yyGxxFyYyYxXxXP
E. Expectation Random Variable
Random Vector
)()E( xdFxx
pnxX
pnxX
ij
ij
:)E()E(
:)(
Some Properties:
E(AX) = AE(X)
E(AXB + C) = AE(X)B + C
E(AX + BY) = AE(X) + BE(Y)
E(tr AX) = tr(AE(X))
F. Variance - Covariance Random Variable
Random Vector
2
2
)(EE
)()(E)Var(
xx
xxxx
dF
qjpiYX
YYXX
ji
qp
,,1,,,1),,Cov(),Cov(
)',,()',,( 11
yx
yx
.tindependenareandif0),(Cov yxyx
Other Properties: Cov(x) = Cov(x, x)
Cov(Ax , By) = A Cov(x, y) B
Cov(Ax) = A Cov(x) A
Cov(x - a, y - b) = Cov(x, y), where a and b are constant vectors
Cov(x - a) = Cov(x) , where a is constant vector
E(xx) = Cov(x) + E(x)E(x)
E(x - a)(x - a) = Cov(x) + (E(x)- a)(E(x)- a) a Rn
Assume that E(x)= and Cov(x) = exist, and A is an pp constant matrix, then
E(xAx) = tr(A) + A
G. Correlation Random Variable
Random Vector
x = (X1, ,Xp)
that is called correlation matrix of x .
21
Var(Y) Var(X)
Y)Cov(X,Y)Corr(X,
Corr(x) = (Corr(Xi,Xj)): pp
4.2 Multivariate Normal Distribution4.2 Multivariate Normal Distribution4.2 Multivariate Normal Distribution4.2 Multivariate Normal Distribution
Random Variable:
X ~ N(,2)
N(0,1)~YY,Xd
22
)(2
1
2
1...
xefdp
2
21
2
1...
yefdp
Definition of Multivariate Normal DistributionDefinition of Multivariate Normal DistributionDefinition of Multivariate Normal DistributionDefinition of Multivariate Normal Distribution
standard normal: y = (Y1,,Yq), Y1,,Yq i.i.d, N(0, 1) y ~ Nq(0, Iq)
xxexpx 'p
q
y 21
21
:Density
qIy
y
cov
E:Moments 0
Definition of Multivariate Normal DistributionDefinition of Multivariate Normal DistributionDefinition of Multivariate Normal DistributionDefinition of Multivariate Normal Distribution
definite negative-non ,
,pp:DD ,μ N~x
qp:D 1,p:μ,Dyμx
p
'ΣΣ
μ-xΣ μ-xΣ x
Σ
x-1
2
1exp
2
1
0when :Density
21
'pp
Σx
μx
cov
E:Moments
4.3 The bivariate normal distribution4.3 The bivariate normal distribution4.3 The bivariate normal distribution4.3 The bivariate normal distribution
2
1
2
1
,xX
X
2221
2121
2221
1211
Σ
2221
21 Var ,Var XX
2121 ,Cov XX
21, XXCorr
1 0 001 2122
221 ,,Σ
2221
212
12
2221
2122
222
21
1
1
1
1
1
1
1
Σ
The density function x is
2
1
1122
21
21 12
1
12
1
x
xxp exp,
2
2
22
2
22
1
112
xxx
The contour of p(x1, x2) is an ellipsoid
μxΣμx 1'c
4.4 Marginal and conditional distributions4.4 Marginal and conditional distributions
Theorem 4.4.1
.,
.,
:Proof
.,~Then
and ,~ Assume
'N
''N
'N
N
m
m
m
p
A Σ AAμd
ADD AAμd~ADyAμdyμAdz
A Σ AAμdz
p,m:A p;m:d Ax,dzΣμx
Corollary 1
.Σ,μ~x
.,:Σ,:μ,:x
ΣΣ
ΣΣΣ,
μ
μμ,
x
xx
Σ μ x,Σ,μ~x
1111
1111
2221
1211
2
1
2
1
:Then
0 1 1 where
into andPartition . Assume
m
p
N
pmmmmm
N
Corollary 2
All marginal distributions of are still normal distributions.
Σ μ,x pN~
Ax
3
2
1
32
21
X
X
X
XX
XX
110
011
Example 4.4.1 Σ μ,x 3~ N
Then,
32
21
3
2
1
μμ
μμ
μ
μ
μ
Aμ110
011
-
-
33232213222312
13222312221211
332323221312
231322121211
332313
232212
131211
2
210
11
0110
11
01
110
011
-
-'A Σ A
The distribution of Ax is multivariate normal with mean
And covariance matrix
Theorem 4.4.2
Let x be a p × 1 random vector. Then x has a multivariate normal distribution if and only if a’x follows a normal distribution for any
.
Note: Σaaxaxa
μaxa
'''
''E
covvar
pRa
Theorem 4.4.3
The assumption is the same as in corollary 1 of Theorem 4.4.1. Then the conditional distribution of x1 given x2 = x2 is
where
21121 .. Σ,μ mN
211211211
212121
Σ Σ ΣΣΣ
μx ΣΣμμ 1-
22.
21-
22.
.,μμ~|
.Σ,μ~x
22
212
112222
12221
2
1
xNxXX
N
Example 4.4.2
Example 1
Let x = (x1, …, xs) be some body characteristics of women, wher
e x1: Hight (身高 )
x2: Bust (胸圍 )
x3: Waist (腰圍 )
x4: Height below neck (頸下高度 )
x5: Buttocks (臀圍 )
363272135703205321933610
0337227254033589
85939536258471
530305146
66029
.....
....
...
..
Σ,
91.52
61.32
70.26
83.39
154.98
μ
.
with~known isIt Σ μ, x sN
The correlation of R can be computer from Σ
00013760627067603630
0001133024206480
000173200540
00012160
0001
.....
....
...
..
.
R
Take x(1) = (x1, x2, x3), x(1) = (x4) and x(3) = (x5).
52.9119.032.61
52.9176.026.70
52.9171.039.83
52.9138.098.154
52.91363.27
213.5
703.20
532.19
336.10
32.61
26.70
39.83
98.154
5
5
5
5
51
5
4
3
2
1
x
x
x
x
xx
x
x
x
x
E
0406717118103897
717119524758109735
181075810588168640
38979735864075625
2135 70320 53219 3361036327
2135
70320
53219
336100337227254033589
227285939536258471
540353625530305146
35898471514666029
1
5
4
3
2
1
....
....
....
....
.,.,.,..
.
.
.
.....
....
....
....
x
x
x
x
x
Cov
70723707108733
582166430
71716
1.717- 1810 38970406
7171
1810
3897
19524758109735
75810588168640
9735864075625
1
5
4
4
3
2
1
...
..
.
,.,..
.
.
.
...
...
...
x
x
x
x
x
x
Cov
Homework 3.5. Please directly compute and computer it by the recursion formula.
5
4
3
2
1
x
x
x
x
x
E
We see that
1540.0195.0
10386.0
1
859.39707.23,|
530.30582.16,|
660.29717.16,|
5
4
3
2
1
3543
2542
1541
x
x
x
x
x
xxxx
xxxx
xxxx
Corr
VarVar
VarVar
VarVar
4.5 Independent4.5 Independent4.5 Independent4.5 Independent
Theorem 4.5.1
.Σxx
ΣΣ
ΣΣΣ,
μ
μμ,
x
xx
Σμ x,Σ,μ~x
1221
2221
1211
2
1
2
1
0 ifonly and ift independen are and Then
into andPartition . Assume
pN
Corollary 1
t.independen are - and
tindependen are - and
21
221212
11
112121
xΣΣxx
xΣΣxx
