8/2/2019 Oller - Fisher Information Metric

1/8

Indian Statistical Institute

Information Metric for Extreme Value and Logistic Probability DistributionsAuthor(s): Jose M. OllerReviewed work(s):Source: Sankhy: The Indian Journal of Statistics, Series A, Vol. 49, No. 1 (Feb., 1987), pp. 17-23Published by: Springer on behalf of the Indian Statistical InstituteStable URL: http://www.jstor.org/stable/25050618 .

Accessed: 21/11/2011 17:42

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of

content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms

of scholarship. For more information about JSTOR, please contact support@jstor.org.

SpringerandIndian Statistical Institute are collaborating with JSTOR to digitize, preserve and extend access to

Sankhy: The Indian Journal of Statistics, Series A.

http://www.jstor.org

http://www.jstor.org/action/showPublisher?publisherCode=springerhttp://www.jstor.org/action/showPublisher?publisherCode=indstatinsthttp://www.jstor.org/stable/25050618?origin=JSTOR-pdfhttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/page/info/about/policies/terms.jsphttp://www.jstor.org/stable/25050618?origin=JSTOR-pdfhttp://www.jstor.org/action/showPublisher?publisherCode=indstatinsthttp://www.jstor.org/action/showPublisher?publisherCode=springer

8/2/2019 Oller - Fisher Information Metric

2/8

Sankhy? :The Indian Journal of Statistics1987, Volume 49, Series A, Pt. 1, pp. 17-23.

INFORMATION METRIC FOR EXTREME VALUE ANDLOGISTIC PROBABILITY DISTRIBUTIONS

By JOSE M. OLLERUniversity of Barcelona, Spain

SUMMARY. In this paper we discuss the differential geometric properties induced bythe information metric over the parameter space of some parametric families of extreme valueprobability distributions (Gumbel, Cauchy-Fr?cheb, Weibull) and the logistic probability distribution. The Gaussian curvature, the geodesies and the Rao distances between distributions areobtained explicitly and related properties are discussed.

1. IntroductionDistance measures between probability distributions have been used ina wide variety of studies in problems of statistical inference and in practical

applications to study affinities among given sets of populations : analysis ofbiological data, in economics, in sociology and other fields of study ; see, forinstance, Matusita (1955), Prevosti et al. (1975), Rao (1948, 1973, 1982). Inall cases, the distance could be interpreted as a measure of the informationdissimilarity between two probability distributions.

Frequently, a statistical model is specified by a family of parametricprobability distributions. One method of setting the difference betweentwo of them is by defining a covariant symmetric tensor field of the secondorder and positive definite over the parameter space of the given family ofdistributions, which may be regarded as a metric tensor field of this space,with a Riemannian manifold structure. The distance between two distributions is then computed following the usual methods of Riemannian geometry, by resolving the corresponding geodesic differential equations.

The metric tensor field can be defined via the Fisher information matrix,Rao (1945), Atkinson and Mitchell (1981), or more generally through the^-entropy functional, Burbea and Rao (1982, 1984) and Burbea (1984). Themethod has been applied for well known families of parametric probabilitydistributions (see the above cited papers, and also Oiler and Cuadras,1985).

AMS (1980) subject classification : 62H30, 94A17, ?3A35.Key words and phrases : Information metric, geodesic distance in probability, extreme value

distributions.A 1-3

8/2/2019 Oller - Fisher Information Metric

3/8

18 J. M. OLLERIn this paper, the method is applied to extreme value probability dis

tributions of the Gumbel (exponential), Cauchy-Fr?chet, and Weibull (bounded) types, and to the Weibull and the logistic distribution. A comprehensive study of these distributions may be found in Johnson and Kotz (1970).

Here we obtain the Gaussian curvature, the geodesies and the Rao distancefor each family of distributions. Some other related properties are alsodiscused.

2. GUMBEL EXTREME VALUE DISTRIBUTIONSThe Gumbel (exponential) extreme value family of distributions is defined

by the cumulative distribution functionF(x) = exp (-e-/0),

where a and d are parameters (6> 0). We let ?2= {(oc, 6) e722 :6 > 0},and thus

p(x\a, 6) = exp (?e-'?) -/*-L

is the probability density function defined on 72 Xii, where 72 is the samplespace and ?2 is the parameter space.

First of all we have to compute the metric tensor field over the parameter space through the ^-entropy functional, as in Burbea and Rao (1982),by taking (?>(x) x log x. Then the metric tensor components are the elements of the Fisher information matrix, which, under regularity conditions,as in the present case, are

F/d2logff\ _ __? _ w (d*\ogp\ _ yd2logf>v

By observing that the moment generating function ijr(t)= E(etx) of therandom variable X is given by :

x?r(t) e?*T(l~dt) (6t ni) = y*+ ?,where y is the Euler constant, we find that the metric tensor field components are :

S'il ?. Pu= 9n =-

(??p-. 022 ((1-')')2+^-) 02 - (2>1)

8/2/2019 Oller - Fisher Information Metric

4/8

INFORMATION METRIC FOR EXTREME VALUE DISTRIBUTIONS 19Thus, the information metric may be expressed as

ds2= ~ ((da)2-2a(dot) (dd)+(a2+b2) (dO)2)where

a = 1?7, b= nj^/Qand (a, 6) is in the parameter space ?1.

This metric can be factored, resulting in

ds2= ^ (da?(a?ib)dd) (da-(a+ib)dd).The linear change of parameters

u = a?ad, v = bd

constitutes a diffeomorphism (a, 0)-> (u, v) of Q, onto itself, with the Jacobianb= tt/a/g. Thus, in the parameters (u, v) (u ef?, v > 0), the information

metric becomesb2ds2 = -g (du+idv) (du?idv)v

or b2ds2 = -j- (?w)2+(^)2.This is, effectively, the Poincar? metric of the upper-half plane

Q = {w e $ : mw > 0}, w = u-\-iv, and thus all relevant quantities are easilydeduced, and in fact well-known (see pp. 24-25 of Burbea (1984)). TheGaussian curvature is then lc= ? Ijb2 ? ? 6/n2, the geodesies curves arethe semi circles w = A+re** (r> 0, 0 < ^ < n) where A is a real constant.

The Rao distance between two points (ocv 0X) and (a2, 62), s(l, 2) is

-?1'2) ^e|rl = v?itenh-1^2) - Where *M 2?= /[(?.-?i)-?^?-^)]^^,-^)' \*? ?v*' ' l[(a2-a1)-a(?2-?1)P+62(?2+^; - ^^which is also a distance on Q, called the "M?bius distance".

3 CaUCHY-FRECHET AND WEIBULL EXTREME VALUE DISTRIBUTIONSThe Cauchy-Fr?chet extreme-value family of distributions is defined by

the cumulative distribution function F, with F(y) = 0 for y < 0 and with

i%)= exp(-(|p)

8/2/2019 Oller - Fisher Information Metric

5/8

20 J. M. OLLERfor y > 0, where ? and ? are positive parameters. We let ?2= 72% where72+ = (0, oo). Then p(.\?, A), where p(y\?, A) = 0 for y < 0 and

~A \ / V \~(A+1) A*IA*)--p(-(JD(?)- )for y > 0, is the probability density function defined on 72X?,, where 72is the sample space and ?2 is the parameter space.

This distribution can be reduced to the former Gumbel extreme valuedistribution with the change of the random variables X = log Y forY > 0 and X ? 0 for Y < 0, and with the change of parameters a = log ?and 6 = 1/A. Thus it follows that the metric tensor field in the new systemof coordinates (parameters), (a, d), is given by (2.1). This is so because theinformation metric is invariant under the admissible transformations of therandom variables, as well as of the parameters. Therefore, the Gaussiancurvature over the parameter space ?2 is also a constant and equal tok = ?6/n2. The Rao distance between two points (?v Ax) and (?2, A2)in ?2 is

,(1, 2)= -JL log [?^i| =? tanh-i 8(1, 2) ... (3.1)Where ?n - _ / Pog (A/?)+?(A2-A1)/A1 AJ'+^-AJ'/AfAj **( ' ' \ Pog (?/A)+a(A,-A1)/A1AJ?+6*(A8+A1)*/Af| /and a = 1?y, 6 = nj\/e.

The Weibull (bounded) extreme value family of distributions is defined bythe cumulative distribution function F, with F(z) = 1 for z ^ 0 and

*(Z)= e*p(-(-?f)for 2< 0, and where ? and A are positive parameters in the parameter space?2= 7?2. Again, this distribution can be reduced to that of the Gumbel

extreme value distribution by the following change of random variables andparameters :

f -log(-Z), Z o

and a = ?log ?, 6 = 1/A.

8/2/2019 Oller - Fisher Information Metric

6/8

INFORMATION METRIC FOR EXTREME VALUE DISTRIBUTIONS 21It follows that the Rao distance between two points (?v ?-J and (/?2, A2) in Q,corresponding to these Weibull distributions, is given by

?< 2>w-log SSrl -w tonh"'*" 2) - (u, v) given by u = \/3/(7r2+3) a and v=?constitutes a diffeomorphism of Q onto itself, resulting in

dsz= it+V ?^(du)>+{dvnwhich is, essentially the Poincar? metric for the upper half-plane Q. It follows

8/2/2019 Oller - Fisher Information Metric

7/8

22 J. M. OLLERthat the Gaussian curvature is k = ? 9/(7r2+3), and that the Rao distancebetween (a, ?x) and (a2, ?2) is

?1, 2)L logI?f^> = 2-^|?i tanh-,(1,2)Where m 2)= /(3/(^+3))(a2-q1)2+(/?2-/g1)2^' > \(3l{^+Z))(oc2-a1f+(?,+?1r> '

5. Some extensions and concluding bemabksLet Xv ..., Xn be an independent set of random variables. Each dis

tributed as the Gumbel (exponential) extreme value probability distributionwith parameters a* and di > 0 (i= 1, ..., n). The parameter space is then(/?X^+)n, i.e. (av d1; ... ; an, dn) e(7ex7S+)n. The Rao distance is then

where 8jc(l, 2) is suitably defined through (2.3). Very similar results, withminor modifications, hold for the other distributions discussed before.

It should be noted that the informative geometry induced by the abovedistributions is essentially that of the univariate normal distributions (seeBurbea and Rao (1982) and Burbea (1984)).

In practice, each distribution considered is characterized by maximumlikelihood estimates of the parameters. Substituting the latter in (2.2) or(3.1), gives the maximum likelihood estimate of 5(1, 2). In some practicalapplications, as demography and survival time studies it is possible to identifyeach group or individual through one of the distributions considered, andhence one can compare them using the previously determined Rao distances.Then one can apply multidimensional scaling (MDS) techniques to obtain arepresentation of the distributions as points in a plane. It is also possibleto obtain a hierarchic classification of the distributions and their graphicoutputs, or dendrograms, by using numerical taxonomy methods. It is,of course, also possible to construct some statistical hypothesis test basedon the above Rao distances.

ReferencesAtkinson, C. and Mitchell, A. F. S. (1981) : Rao's distance measure. Sankhy?, 43, A, 345-365.Burbea, J. (1984) : Informative geometry of probability spaces. Technical Report No. 84-52,

University of Pittsburgh.Burbea, J. and Rao, C. R. (1982) : Fmtropy differential metric, distance and divergence mea

sures in probability spaces : a unified approach. J. Multivariate Anal., 12, 575-596.

8/2/2019 Oller - Fisher Information Metric

8/8

INFORMATION METRIC FOR EXTREME VALUE DISTRIBUTIONS 23-(1984) : Differential metrics in probability spaces. Probability Math. Statist., 3,

Fase. 2, 241-258.Cuadras, C. M. (1981) : M?todos de An?lisis Mi?tivariante., Eunibar, Barcelona.Hicks, N. J. (1965) : Noies on Differential Geometry, Van Nostran, Princeton, N. J..Johnson, N. L. and Kotz, S. (1970) : Distributions in Statistics, Wiley, New York.

Matusita. K. (1957) : Decision rule based on the distance for the classification problem. Ann.Inst. Statist. Math., 8, 67-77.

Oller, J. M. and Cuadras, C. M. (1985) : Rao's distance for negative multinomial distributions.Sankhy?, 47, A, 75-83.

Pbevosti, A., Ocana, J. and Alonso, G. (1975) : Distances between populations of Drosophilamelanogaster, based on chromosome arrangement frequencies. Theoretical Appl. Genetic,45, 231-241.

Rao, C. R. (1945) : Information and accuracy attainable in the estimation of parameters. Bull.Calcutta Math. Soc, 37, 81-91.

-? (1948): The utilization of multiple measurements in problems of biological classification. J. Roy Statist. Soc, B, 10, 159-193.

- (1973) : Linear Statistical Inference and its Applications, Wiley, New York.- (1982): Diversity and Dissimilarity Coefficients: A unified approach. J. Theoreti

cal Population Biology, 21, 24-43.Spivak, M. (1979): A Comprehensive Introduction to Differential Geometry, Publish or Perish,

Berkeley, CA.

Paper received : October, 1985.Revised : January, 1986.