Multivariate Normal Distribution – I • We will almost always assume that the joint distribution of the p × 1 vectors of measurements on each sample unit is the p-dimensional multivariate normal distribution. • The MVN assumption is often appropriate: – Variables can sometimes be assumed to be multivariate normal (perhaps after transformation) – Central limit theorem tells us that distribution of many multivariate sample statistics is approximately normal, re- gardless of the form of the population distribution. • As a bonus, the MVN assumption leads to tractable results (but mathematical convenience should not be the reason for choosing the MVN as the probability model). 107
Transcript
Multivariate Normal Distribution – I
• We will almost always assume that the joint distribution ofthe p×1 vectors of measurements on each sample unit is thep-dimensional multivariate normal distribution.
• The MVN assumption is often appropriate:
– Variables can sometimes be assumed to be multivariatenormal (perhaps after transformation)
– Central limit theorem tells us that distribution of manymultivariate sample statistics is approximately normal, re-gardless of the form of the population distribution.
• As a bonus, the MVN assumption leads to tractable results(but mathematical convenience should not be the reason forchoosing the MVN as the probability model).
107
Multivariate Normal Distribution
• The MVN is a generalization of the univariate normal
distribution for the case p ≥ 2.
• Recall that if X is normal with mean µ and variance σ2,
and its density function is given by
f(x) =1
(2πσ2)1/2exp[−
1
2σ2(x− µ)2], −∞ < x <∞.
• We can write the kernel of the density as:
exp[−1
2σ2(x− µ)2] = exp[−
1
2(x− µ)′(σ2)−1(x− µ)].
108
Multivariate Normal Distribution
Now consider a p× 1 random vector X = [X1, X2, . . . , Xp]′.
The kernel shown above generalizes to
exp[−1
2(x− µ)′Σ−1(x− µ)],
where
x =
x1x2...xp
and µ =
µ1µ2...µp
and Σ =
σ11 σ12 · · · σ1pσ21 σ22 · · · σ2p
... ... . . . ...σp1 σp2 · · · σpp
is the p× p population covariance matrix. This is the kernel
of the MVN distribution.
109
Multivariate Normal Distribution
• The quadratic form (x − µ)′Σ−1(x − µ) in the kernel is astatistical distance measure, of the type we described ear-lier. For any value of x, the quadratic form gives the squaredstatistical distance of x from µ accounting for the fact thatthe variances of the p variables may be different and that thevariables may be correlated.
• This quadratic form is often referred to as Mahalanobisdistance.
• The density function of the MVN distribution is:
f(x) =1
(2π)p/2|Σ|1/2exp[−
1
2(x− µ)′Σ−1(x− µ)],
where the normalizing constant (2π)p/2|Σ|1/2 makes thevolume under the MVN density equal to 1.
110
Multivariate Normal Distribution
• We use the notation
X ∼ Np(µ,Σ) or X ∼MVN(µ,Σ)
to indicate that the p-dimensional random vector X has
a multivariate normal distribution with mean vector µ and
covariance matrix Σ.
111
Example: Bivariate Normal Densityσ11 = σ22, and ρ12 = 0
112
Example: Bivariate Normal Densityσ11 = σ22, and ρ12 > 0
113
Density Contours
• The x values that yield a constant height for the density formellipsoids centered at µ.
• The MVN density is constant on surfaces or contours where
(x− µ)′Σ−1(x− µ) = c2.
• Definition of a constant probability density contour is all x’sthat satisfy the expression above.
• The axes of the ellipses are in the directions of theeigenvectors of Σ and the length of the j − th longest axis isproportional to (
√λj), where λj is the eigenvalue associated
with the j − th eigenvector of Σ.
114
Density Contours• Recall that if (λj, ej) is an eigenvalue-eigenvector pair for Σ
and Σ is positive definite, then (λ−1j , ej) is an eigenvalue-
eigenvector pair of Σ−1.
• The jth axis is ±c√λjej, for j = 1, ..., p.
• We show later that if
c2 = χ2p(α),
where χ2p(α) is the upper (100α)th percentile of a χ2 distri-
bution with p degrees of freedom, then the probability is 1−αthat the value of a random vector will be inside the ellipsoiddefined by
(x− µ)′Σ−1(x− µ) ≤ χ2p(α)
115
Bivariate Normal Density Contours
• Eigenvalues and eigenvectors of Σ are obtained from
|Σ− λI| = 0. Using σ12 = σ21 = ρ√σ11√σ22, we have
0 =
∣∣∣∣∣[σ11 σ12σ12 σ22
]−[λ 00 λ
] ∣∣∣∣∣=
∣∣∣∣∣ σ11 − λ ρ√σ11√σ22
ρ√σ11√σ22 σ22 − λ
∣∣∣∣∣= (σ11 − λ)(σ22 − λ)− ρ2σ11σ22
= λ2 − (σ11 + σ22)λ+ σ11σ22(1− ρ2).
116
Bivariate Normal Density Contours
• Solutions to this quadratic equation are:
λ1 =1
2
[σ11 + σ22 +
√(σ11 + σ22)2 − 4σ11σ22(1− ρ2)
]
λ2 =1
2
[σ11 + σ22 −
√(σ11 + σ22)2 − 4σ11σ22(1− ρ2)
]
Solving quadratic equations:
ax2 + bx+ c = 0
The solutions are
x =−b+
√b2 − 4ac
2aand x =
−b−√b2 − 4ac
2a117
Bivariate Normal Density Contours
• An eigenvector associated with λ1 satisfies[σ11 σ12σ12 σ11
] [e11e12
]= λ1
[e11e12
],
where eij denotes the jth element in the ith eigenvector. We
• If σ12 > 0, the major axis of the ellipse will be in the direction
of the 45o line. The actual value of σ12 does not matter. If
σ12 < 0, then the major axis will be perpendicular to the 45o
line.
122
Bivariate Normal Contours (σ11 = σ22)
123
More Bivariate Normal Examples
• 50% and 90% contours of two bivariate normal densities.
Density is the highest when x = µ.
124
Central (1− α)× 100% Region of aBivariate Normal Distribution
• The ratio of the lengths of the major and minor axes is
Length of major axis
Length of minor axis=
√λ1√λ2
• If 1 − α is the probability that a randomly selected memberof the population is observed inside the ellipse, then the half-length of the axes are given by√
χ22(α)
√λi
• This is the smallest region that has probability 1− α ofcontaining a randomly selected member of the population
125
Central (1− α)× 100% Region of aBivariate Normal Distribution
• The area of the ellipse containing the central (1−α)×100%
of a bivariate nornal population is
area = πχ22(α)
√λ1
√λ2 = πχ2
2(α)|Σ|1/2
• Note that
det(Σ) = det
σ11 σ12
σ21 σ22
= det
σ11 ρσ1σ2
ρσ1σ2 σ22
= σ11σ22(1−ρ2)
126
Central (1− α)× 100% Region of aBivariate Normal Distribution
• For fixed variances, σ11 and σ22, area of the ellipse is largestwhen ρ = 0
• The area becomes smaller as ρ approaches 1 or as ρapproches −1.
• For σ11 = σ22 and ρ = 0 the contours of constant densityare concentric circles and λ1 = λ2
• For σ11 > σ22 the axes of the ellipse are parallel to thecoordinate axes, with the major axis parallel to the horizontalaxis.
• For σ22 > σ11 the axes of the ellipse are parallel to thecoordinate axes, with the major axis parallel to the verticalaxis.
127
Central (1− α)× 100% Region of aMultivariate Normal Distribution
• For a p-dimensional normal distribution, the smallest region
such that there is probability 1− α that a randomly selected
![STATISTICAL APPLICATIONS OF THE MULTIVARIATE SKEW NORMAL ... · arXiv:0911.2093v1 [stat.ME] 11 Nov 2009 STATISTICAL APPLICATIONS OF THE MULTIVARIATE SKEW-NORMAL DISTRIBUTION A.Azzalini](https://static.documents.pub/doc/80x56/5b40be297f8b9a91078d8f73/statistical-applications-of-the-multivariate-skew-normal-arxiv09112093v1.jpg)
