Mu1 tivaria t e Statistical Analysis Second Edition, Revised and Expanded Narayan C. Giri University of Montreal Montreal, Quebec, Canada M A R C E L MARCEL DEKKER, INC. D E K K E R NEW YORK BASEL Although great care has been taken to provide accurate and current information, neither theauthor(s) nor the publisher, nor anyone else associated with this publication, shall be liablefor any loss, damage, or liability directly or indirectly caused or alleged to be caused by thisbook. The material contained herein is not intended to provide specic advice orrecommendations for any specic situation.Trademark notice: Product or corporate names may be trademarks or registered trademarksand are used only for identication and explanation without intent to infringe.Library of Congress Cataloging-in-Publication DataA catalog record for this book is available from the Library of Congress.ISBN: 0-8247-4713-5This book is printed on acid-free paper.HeadquartersMarcel Dekker, Inc., 270 Madison Avenue, New York, NY 10016, U.S.A.tel: 212-696-9000; fax: 212-685-4540Distribution and Customer ServiceMarcel Dekker, Inc., Cimarron Road, Monticello, New York 12701, U.S.A.tel: 800-228-1160; fax: 845-796-1772Eastern Hemisphere DistributionMarcel Dekker AG, Hutgasse 4, Postfach 812, CH-4001 Basel, Switzerlandtel: 41-61-260-6300; fax: 41-61-260-6333World Wide Webhttp://www.dekker.comThe publisher offers discounts on this book when ordered in bulk quantities. For moreinformation, write to Special Sales/Professional Marketing at the headquarters addressabove.Copyright # 2004 by Marcel Dekker, Inc. All Rights Reserved.Neither this book nor any part may be reproduced or transmitted in any form or by anymeans, electronic or mechanical, including photocopying, microlming, and recording, orby any information storage and retrieval system, without permission in writing from thepublisher.Current printing (last digit):10 9 8 7 6 5 4 3 2 1PRINTED IN THE UNITED STATES OF AMERICASTATISTICS: Textbooks and Monographs D. B. Owen Founding Editor, 1972-1991 Associate Editors Statistical Computing/ Multivariate Analysis Professor Anant M. Kshirsagar University of Michigan Nonparametric Statistics Professor William R. Schucany Southern Methodist Universig Probability Quality ControllReliability Professor Edward G. Schilling Rochester Institute of Technology Professor Marcel F. Neuts University of Arizona Editorial Board Applied Probability Dr. Paul R. Garvey The MITRE Corporation Statistical Distributions Professor N. Balakrishnan McMaster University Economic Statistics Statistical Process Improvement Professor David E. A. Giles University of Victoria Professor G. Geoffrey Vining Virginia Polytechnic Institute Experimental Designs Stochastic Processes Mr. Thomas B. Barker Rochester Institute of Technology Professor V. Lakshrmkantham Florida Institute of Technology Multivariate Analysis Survey Sampling Professor Subir Ghosh University of Calgornia-Riverside Professor Lynne Stokes Southern Methodist University Time Series Sastry G. Pantula North Carolina State University 1. The Generalized Jackknife Statistic, H. L. Gray and W. R. Schucany 2. Multivariate Analysis, Anant M. Kshirsagar 3. Statistics and Society, Walter T. Federer 4. Multivariate Analysis: A Selected and Abstracted Bibliography, 1957-1 972, Kocher- lakota Subrahmaniam and Kathleen Subrahmaniam 5. Design of Experiments: A Realistic Approach, Vi gi l L. Anderson and Robert A. McLean 6. Statistical and Mathematical Aspects of Pollution Problems, John W. Pratt 7. Introduction to Probability and Statistics (in two parts), Part I: Probability; Part I I : Statistics, Narayan C. Gin' 8. Statistical Theory of the Analysis of Experimental Designs, J. Ogawa 9. Statistical Techniques in Simulation (in two parts), Jack P. C. Kleijnen 10. Data Quality Control and Editing, Joseph 1. Naus 11. Cost of Living Index Numbers: Practice, Precision, and Theory, Kali S. Banejee 12. Weighing Designs: For Chemistry, Medicine, Economics, Operations Research, Statistics, Kali S. Banejee 13. The Search for Oil: Some Statistical Methods and Techniques, edited by D. B. Owen 14. Sample Size Choice: Charts for Experiments with Linear Models, Robert E. Odeh and Martin Fox 15. Statistical Methods for Engineers and Scientists, Robert M. Bethea, Benjamin S. Duran, and Thomas L. Boullion 16. Statistical Quality Control Methods, Irving W. Bun 17. On the History of Statistics and Probability, edited by D. B. Owen 18. Econometrics, Peter Schmidt 19. Sufficient Statistics: Selected Contributions, Vasant S. Huzurbazar (edited by Anant M. Kshirsagar) 20. Handbook of Statistical Distributions, Jagdish K. Patel, C. H. Kapadia, and D. 8. Owen 21. Case Studies in Sample Design, A. C. Rosander 22. Pocket Book of Statistical Tables, compiled by R. E. Odeh, D. B. Owen, Z. W. Birnbaum, and L. Fisher 23. The Information in Contingency Tables, D. V. Gokhale and Solomon Kullback 24. Statistical Analysis of Reliability and Life-Testing Models: Theory and Methods, Lee J. Bain 25. Elementary Statistical Quality Control, Irving W. Burr 26. An Introduction to Probability and Statistics Using BASIC, Richard A. Gmeneveld 27. Basic Applied Statistics, 8. L. Raktoe and J. J. Hubert 28. A Primer in Probability, Kathleen Subrahmaniarn 29. Random Processes: A First Look, R. Syski 30. Regression Methods: A Tool for Data Analysis, Rudolf J. Freund and Paul D. Minton 31. Randomization Tests, Eugene S. Edgington 32. Tables for Normal Tolerance Limits, Sampling Plans and Screening, Robert E. Odeh and D. B. Owen 33. Statistical Computing, William J. Kennedy, Jr., and James E. Gentle 34. Regression Analysis and Its Application: A Data-Oriented Approach, Richard F. Gunst and Robert L. Mason 35. Scientific Strategies to Save Your Life, 1. D. J. Bross 36. Statistics in the Pharmaceutical Industry, edited by C. Ralph Buncher and Jia-Yeong Tsay 37. Sampling from a Finite Population, J. Hajek 38. Statistical Modeling Techniques, S. S. Shapiro and A. J. Gross 39. Statistical Theory and Inference in Research, T. A. Bancroff and C.-P. Han 40. Handbook of the Normal Distribution, Jagdish K. Patel and Campbell B. Read 41. Recent Advances in Regression Methods, Hrishikesh D. Vinod and Aman Ullah 42. Acceptance Sampling in Quality Control, Edward G. Schilling 43. The Randomized Clinical Trial and Therapeutic Decisions, edited by Niels Tygstrup, John M Lachin, and Erik Juhl 44. Regression Analysis of Survival Data in Cancer Chemotherapy, Walter H. Carter, Jr., Galen L. Wampler, and Donald M. Stablein 45. A Course in Linear Models, Anant M. Kshirsagar 46. Clinical Trials: Issues and Approaches, edited by Stanley H. Shapiro and Thomas H. Louis 47. Statistical Analysis of DNA Sequence Data, edited by B. S. Weir 48. Nonlinear Regression Modeling: A Unified Practical Approach, David A. Ratkowsky 49. Attribute Sampling Plans, Tables of Tests and Confidence Limits for Proportions, Rob- ert . Odeh and D. B. Owen 50. Experimental Design, Statistical Models, and Genetic Statistics, edited by Klaus Hinkelmann 51. Statistical Methods for Cancer Studies, edited by Richard G. Comell 52. Practical Statistical Sampling for Auditors, Arthur J. Wilbum 53. Statistical Methods for Cancer Studies, edited by Edward J. Wegman and James G. Smith 54. Self-organizing Methods in Modeling: GMDH Type Algorithms, edited by Stanley J. Fadow 55. Applied Factorial and Fractional Designs, Ro6ert A. McLean and Virgil L. Anderson 56. Design of Experiments: Ranking and Selection, edited by Thomas J. Santner and Ajit C. Tamhane 57. Statistical Methods for Engineers and Scientists: Second Edition, Revised and Ex- panded, Robert M. Bethea, Benjamin S. Duran, and Thomas L. Boullion 58. Ensemble Modeling: Inference from Small-Scale Properties to Large-Scale Systems, Alan . Gelfand and Crayton C. Walker 59. Computer Modeling for Business and Industry, Bruce L. Boweman and Richard T. 0 'Connell 60. Bayesian Analysis of Linear Models, Lyle D. Broemeling 61. Methodological Issues for Health Care Surveys, Brenda Cox and Steven Cohen 62. Applied Regression Analysis and Experimental Design, Richard J. Brook and Gregory C. Arnold 63. Statpal: A Statistical Package for Microcomputers-PC-DOS Version for the IBM PC and Compatibles, Bruce J. Chalmer and David G. Whitmore 64. Statpal: A Statistical Package for Microcomputers-Apple Version for the I I , II+, and I le, David G. Whitmore and Bruce J. Chalmer 65. Nonparametric Statistical Inference: Second Edition, Revised and Expanded, Jean Dickinson Gibbons 66. Design and Analysis of Experiments, Roger G. Petersen 67. Statistical Methods for Pharmaceutical Research Planning, Sten W. Bergman and John C. Giffins 68. Goodness-of-Fit Techniques, edited by Ralph B. D'Agostino and Michael A. Stephens 69. Statistical Methods in Discrimination Litigation, edited by D. H. Kaye and Mike/ Aickin 70. Truncated and Censored Samples from Normal Populations, Helmut Schneider 71. Robust Inference, M. L. Tiku, W. Y. Tan, andN. Balakrishnan 72. Statistical Image Processing and Graphics, edited by Edward J. Wegman and Douglas J. DePriest 73. Assignment Methods in Combinatorial Data Analysis, Lawrence J. Hubert 74. Econometrics and Structural Change, Lyle D. Broemeling and Hiroki Tsurumi 75. Multivariate Interpretation of Clinical Laboratory Data, Adelin Albert and Eugene K. Hanis 76. Statistical Tools for Simulation Practitioners, Jack P. C. Kleijnen 77. Randomization Tests: Second Edition, Eugene S. Edgington 78. A Folio of Distributions: A Collection of Theoretical Quantile-Quantile Plots, Edward 5. Fowlkes 79. Applied Categorical Data Analysis, Daniel H. Freeman, Jr. 80. Seemingly Unrelated Regression Equations Models: Estimation and Inference, Viren- dra K. Snvastava and David E. A. Giles 81. Response Surfaces: Designs and Analyses, Andre 1. Khun' and John A. Cornell 82. Nonlinear Parameter Estimation: An Integrated System in BASIC, John C. Nash and Mary Walker-Smith 83. Cancer Modeling, edited by James R. Thompson and Bany W. Brown 84. Mixture Models: Inference and Applications to Clustering, Geoffrey J. McLachlan and Kaye E. Basford 85. Randomized Response: Theory and Techniques, Anjit Chaudhun' and Rahul Mukedee 86. Biopharmaceutical Statistics for Drug Development, edited by Karl f. Peace 87. Parts per Million Values for Estimating Quality Levels, Robert E. Odeh and D. B. Owen 88. Lognormal Distributions: Theory and Applications, edited by Edwin L. Crow and Kunio Shimizu 89. Properties of Estimators for the Gamma Distribution, K. 0. Bowman and L. R. Shenton 90. Spline Smoothing and Nonparametric Regression, Randall L. Eubank 91. Linear Least Squares Computations, R. W Farebrother 92. Exploring Statistics, Damaraju Raghavarao 93. Applied Time Series Analysis for Business and Economic Forecasting, Sufi M. Nazem 94. Bayesian Analysis of Time Series and Dynamic Models, edited by James C. Spall 95. The Inverse Gaussian Distribution: Theory, Methodology, and Applications, Raj S. Chhikara and J. Leroy Folks 96. Parameter Estimation in Reliability and Life Span Models, A. Clifford Cohen and Betty Jones Whitten 97. Pooled Cross-Sectional and Time Series Data Analysis, Teny E. Dielman 98. Random Processes: A First Look, Second Edition, Revised and Expanded, R. Syski 99. Generalized Poisson Distributions: Properties and Applications, P. C. Consul 100. Nonlinear L,-Norm Estimation, Rene Gonin and Arthur H. Money 101. Model Discrimination for Nonlinear Regression Models, Dale S. Bomwiak 102. Applied Regression Analysis in Econometrics, Howard E. Doran 103. Continued Fractions in Statistical Applications, K. 0. Bowman and L. R. Shenton 104. Statistical Methodology in the Pharmaceutical Sciences, Donald A. Beny 105. Experimental Design in Biotechnology, Peny D. Haaland 106. Statistical Issues in Drug Research and Development, edited by Karl 15. Peace 107. Handbook of Nonlinear Regression Models, David A. Ratkowsky 108. Robust Regression: Analysis and Applications, edited by Kenneth D. Lawrence and Jeffrey L. Arthur 109. Statistical Design and Analysis of Industrial Experiments, edited by Subir Ghosh 11 0. U-Statistics: Theory and Practice, A. J. Lee 111. A Primer in Probability: Second Edition, Revised and Expanded, Kathleen Subrah- maniam 112. Data Quality Control: Theory and Pragmatics, edited by Gunar E. Liepins and V. R. R. 11 3. Engineering Quality by Design: Interpreting the Taguchi Approach, Thomas B. Barker 114. Survivorship Analysis for Clinical Studies, Eugene K. Hanis and Adelin Albert 115. Statistical Analysis of Reliability and Life-Testing Models: Second Edition, Lee J. Bain and Max Engelhardt 116. Stochastic Models of Carcinogenesis, Wai-Yuan Tan 117. Statistics and Society: Data Collection and Interpretation, Second Edition, Revised and Expanded, Walter T. Federer 11 8. Handbook of Sequential Analysis, B. K. Ghosh and P. K. Sen 11 9. Truncated and Censored Samples: Theory and Applications, A. Clifford Cohen 120. Survey Sampling Principles, E. K. Foreman 121. Applied Engineering Statistics, Robert M. Bethea and R. Russell Rhinehart 122. Sample Size Choice: Charts for Experiments with Linear Models: Second Edition, Robert E. Odeh and Martin Fox 123. Handbook of the Logistic Distribution, edited by N. Balakrishnan 124. Fundamentals of Biostatistical Inference, Chap T. Le 125. Correspondence Analysis Handbook, J.-P. Benzecri Uppuluri 126. Quadratic Forms in Random Variables: Theory and Applications, A. M. Mathai and Serge B. Provost 127. Confidence Intervals on Variance Components, Richard K. Burdick and Franklin A. Graybill 128. Biopharmaceutical Sequential Statistical Applications, edited by Karl E. Peace 129. Item Response Theory: Parameter Estimation Techniques, Frank B. Baker 130. Survey Sampling: Theory and Methods, Anjit Chaudhuri and Horst Stenger 131. Nonparametric Statistical Inference: Third Edition, Revised and Expanded, Jean Dick- inson Gibbons and Subhabrata Chakraborti 132. Bivariate Discrete Distribution, Subrahmaniam Kocherlakota and Kathleen Kocher- lakota 133. Design and Analysis of Bioavailability and Bioequivalence Studies, Shein-Chung Chow and Jen-pei Liu ' 134. Multiple Comparisons, Selection, and Applications in Biometry, edited by Fred M. 135. Cross-Over Experiments: Design, Analysis, and Application, David A. Ratkowsky, Marc A. Evans, and J. Richard Alldredge 136. Introduction to Probability and Statistics: Second Edition, Revised and Expanded, Narayan C. Giri 137. Applied Analysis of Variance in Behavioral Science, edited by Lynne K. Edwards 138. Drug Safety Assessment in Clinical Trials, edited by Gene S. Gilbert 139. Design of Experiments: A No-Name Approach, Thomas J. Lorenzen and Virgil L. An- derson 140. Statistics in the Pharmaceutical Industry: Second Edition, Revised and Expanded, edited by C. Ralph Buncher and Jia-Yeong Tsay 141. Advanced Linear Models: Theory and Applications, Song-Gui Wang and Shein-Chung Chow 142. Multistage Selection and Ranking Procedures: Second-Order Asymptotics, Nitis Muk- hopadhyay and Tumulesh K. S. Solanky 143. Statistical Design and Analysis in Pharmaceutical Science: Validation, Process Con- trols, and Stability, Shein-Chung Chow and Jen-pei Liu 144. Statistical Methods for Engineers and Scientists: Third Edition, Revised and Expanded, Robert M. Bethea, Benjamin S. Duran, and Thomas L. Boullion 145. Growth Curves, Anant M. Kshirsagar and William Boyce Smith 146. Statistical Bases of Reference Values in Laboratory Medicine, Eugene K. Hanis and James C. Boyd 147. Randomization Tests: Third Edition, Revised and Expanded, Eugene S. Edgington 148. Practical Sampling Techniques: Second Edition, Revised and Expanded, Ranjan K. Som 149. Multivariate Statistical Analysis, Narayan C. Giri 150. Handbook of the Normal Distribution: Second Edition, Revised and Expanded, Jagdish K. Patel and Campbell B. Read 151. Bayesian Biostatistics, edited by Donald A. Berry and Dalene K. Stangl 152. Response Surfaces: Designs and Analyses, Second Edition, Revised and Expanded, Andre 1. Khuri and John A. Cornell 153. Statistics of Quality, edited by Subir Ghosh, William R. Schucany, and William B. Smith 154. Linear and Nonlinear Models for the Analysis of Repeated Measurements, Edward f . Vonesh and Vernon M. Chinchilli 155. Handbook of Applied Economic Statistics, Aman Ullah and David E. A. Giles 156. Improving Efficiency by Shrinkage: The James-Stein and Ridge Regression Estima- tors, Marvin H. J. Gruber 157. Nonparametric Regression and Spline Smoothing: Second Edition, Randall L. Eu- bank 158. Asymptotics, Nonparametrics, and Time Series, edited by Subir Ghosh 159. Multivariate Analysis, Design of Experiments, and Survey Sampling, edited by Subir Ghosh HoPPe 160. 161. 162. 163. 164. 165. 166. 167. 168. 169. 170. 171. 172. Statistical Process Monitoring and Control, edited by Sung H. Park and G. Geoffrey Vining Statistics for the 21st Century: Methodologies for Applications of the Future, edited by C. R. Rao and Gabor J. Szekely Probability and Statistical Inference, Nitis Mukhopadhyay Handbook of Stochastic Analysis and Applications, edited by D. Kannan and V. Lak- shmikantham Testing for Normality, Henry C. Thode, Jr. Handbook of Applied Econometrics and Statistical Inference, edited by Aman Ullah, Alan T. K. Wan, and Anoop Chaturvedi Visualizing Statistical Models and Concepts, R. W. Farebrother Financial and Actuarial Statistics: An Introduction, Dale S. Borowiak Nonparametric Statistical Inference: Fourth Edition, Revised and Expanded, Jean Dickinson Gibbons and Subhabrata Chakraborti Computer-Aided Econometrics, edited by David E. A. Giles The EM Algorithm and Related Statistical Models, edited by Michiko Watanabe and Kazunon Yamaguchi Multivariate Statistical Analysis: Second Edition, Revised and Expanded, Narayan C. Gin Computational Methods in Statistics and Econometrics, Hisashi Tanizaki Additional Volumes in Preparation To Nilima, Nabanita, and NandanPreface to the Second EditionAs in the rst edition the aim has been to provide an up-to-date presentation ofboth the theoretical and applied aspects of multivariate analysis using theinvariance approach for readers with a basic knowledge of mathematics andstatistics at the undergraduate level. This new edition updates the original bookby adding new results, examples, problems, and references. The following newsubsections are added. Section 4.3 deals with the symmetric distributions: itsproperties and characterization. Section 4.3.6 treats elliptically symmetricdistributions (multivariate) and Section 4.3.7 considers the singular symmetricaldistribution. Regression and correlations in symmetrical distributions arediscussed in Section 4.5.1. The redundancy index is included in Section 4.7. InSection 5.3.7 we treat the problem of estimation of covariance matrices and theequivariant estimation under curved model of mean, and covariance matrix istreated in Section 5.4. Basic distributions in symmetrical distributions are givenin Section 6.12. Tests of mean against one-sided alternatives are given in Section7.3.1. Section 8.5.2 treats multiple correlation with partial information andSection 8.1 deals with tests with missing data. In Section 9.5 we discuss therelationship between discriminant analysis and cluster analysis.A new Appendix A dealing with tables of chi-square adjustments to the Wilkscriterion U (Schatkoff, M. (1966), Biometrika, pp. 347358, and Pillai, K.C.S.and Gupta, A.K. (1969), Biometrika, pp. 109118) is added. Appendix B lists thepublications of the author.In preparing this volume I have tried to incorporate various comments ofreviewers of the rst edition and colleagues who have used it. The comments ofvmy own students and my long experience in teaching the subject have also beenutilized in preparing the Second Edition.Narayan C. Girivi Preface to the Second EditionPreface to the First EditionThis book is an up-to-date presentation of both theoretical and applied aspects ofmultivariate analysis using the invariance approach. It is written for readers withknowledge of mathematics and statistics at the undergraduate level. Variousconcepts are explained with live data from applied areas. In conformity with thegeneral nature of introductory textbooks, we have tried to include many examplesand motivations relevant to specic topics. The material presented here isdeveloped from the subjects included in my earlier books on multivariatestatistical inference. My long experience teaching multivariate statistical analysiscourses in several universities and the comments of my students have also beenutilized in writing this volume.Invariance is the mathematical term for symmetry with respect to a certaingroup of transformations. As in other branches of mathematics the notion ofinvariance in statistical inference is an old one. The unpublished work of Huntand Stein toward the end of World War II has given very strong support to theapplicability and meaningfulness of this notion in the framework of the generalclass of statistical tests. It is now established as a very powerful tool for provingthe optimality of many statistical test procedures. It is a generally acceptedprinciple that if a problem with a unique solution is invariant under a certaintransformation, then the solution should be invariant under that transformation.Another compelling reason for discussing multivariate analysis throughinvariance is that most of the commonly used test procedures are likelihoodratio tests. Under a mild restriction on the parametric space and the probabilityviidensity functions under consideration, the likelihood ratio tests are almostinvariant.Invariant tests depend on the observations only through maximal invariant. Tond optimal invariant tests we need to nd the explicit form of the maximalinvariant statistic and its distribution. In many testing problems it is not alwaysconvenient to nd the explicit form of the maximal invariant. Stein (1956) gave arepresentation of the ratio of probability densities of a maximal invariant byintegrating with respect to a invariant measure on the group of transformationsleaving the problem invariant. Stein did not give explicitly the conditions underwhich his representation is valid. Subsequently many workers gave sufcientconditions for the validity of his representation. Spherically and ellipticallysymmetric distributions form an important family of nonnormal symmetricdistributions of which the multivariate normal distribution is a member. Thisfamily is becoming increasingly important in robustness studies where the aim isto determine how sensitive the commonly used multivariate methods are to themultivariate normality assumption. Chapter 1 contains some special resultsregarding characteristic roots and vectors, and partitioned submatrices of real andcomplex matrices. It also contains some special results on determinants andmatrix derivatives and some special theorems on real and complex matrices.Chapter 2 deals with the theory of groups and related results that are useful forthe development of invariant statistical test procedures. It also contains results onJacobians of some important transformations that are used in multivariatesampling distributions.Chapter 3 is devoted to basic notions of multivariate distributions and theprinciple of invariance in statistical inference. The interrelationship betweeninvariance and sufciency, invariance and unbiasedness, invariance and optimaltests, and invariance and most stringent tests are examined. This chapter alsoincludes the Stein representation theorem, Hunt and Stein theorem, androbustness studies of statistical tests.Chapter 4 deals with multivariate normal distributions by means of theprobability density function and a simple characterization. The second approachsimplies multivariate theory and allows suitable generalization from univariatetheory without further analysis. This chapter also contains some characterizationsof the real multivariate normal distribution, concentration ellipsoid and axes,regression, multiple and partial correlation, and cumulants and kurtosis. It alsodeals with analogous results for the complex multivariate normal distribution,and elliptically and spherically symmetric distributions. Results on vec operatorand tensor product are also included here.Maximum likelihood estimators of the parameters of the multivariate normal,the multivariate complex normal, the elliptically and spherically symmetricdistributions and their optimal properties are the main subject matter of Chapter5. The JamesStein estimator, the positive part of the JamesStein estimator,viii Preface to the First Editionunbiased estimation of risk, smoother shrinkage estimation of mean with knownand unknown covariance matrix are considered here.Chapter 6 contains a systematic derivation of basic multivariate samplingdistributions for the multivariate normal case, the complex multivariate normalcase, and the case of symmetric distributions.Chapter 7 deals with tests and condence regions of mean vectors ofmultivariate normal populations with known and unknown covariance matricesand their optimal properties, tests of hypotheses concerning the subvectors of m inmultivariate normal, tests of mean in multivariate complex normal andsymmetric distributions, and the robustness of the T2-test in the family ofelliptically symmetric distributions.Chapter 8 is devoted to a systematic derivation of tests concerning covariancematrices and mean vectors, the sphericity test, tests of independence, the R2-test,a special problem in a test of independence, MANOVA, GMANOVA, extendedGMANOVA, equality of covariance matrice in multivariate normal populationsand their extensions to complex multivariate normal, and the study of robustnessin the family of elliptically symmetric distributions.Chapter 9 contains a modern treatment of discriminant analysis. A briefhistory of discriminant analysis is also included here.Chapter 10 deals with several aspects of principal component analysis inmultivariate normal populations.Factor analysis is treated in Chapter 11 and various aspects of canonicalcorrelation analysis are treated in Chapter 12.I believe that it would be appropriate to spread the materials over two three-hour one-semester basic courses on multivariate analysis for statistics graduatestudents or one three-hour one-semester course for graduate students innonstatistic majors by proper selection of materials according to need.Narayan C. GiriPreface to the First Edition ixContentsPreface to the Second Edition vPreface to the First Edition vii1 VECTOR AND MATRIX ALGEBRA 11.0 Introduction 11.1 Vectors 11.2 Matrices 41.3 Rank and Trace of a Matrix 71.4 Quadratic Forms and Positive Denite Matrix 71.5 Characteristic Roots and Vectors 81.6 Partitioned Matrix 161.7 Some Special Theorems on Matrix Derivatives 211.8 Complex Matrices 24Exercises 25References 272 GROUPS, JACOBIAN OF SOME TRANSFORMATIONS,FUNCTIONS AND SPACES 292.0 Introduction 292.1 Groups 292.2 Some Examples of Groups 302.3 Quotient Group, Homomorphism, Isomorphism 312.4 Jacobian of Some Transformations 332.5 Functions and Spaces 38References 39xi3 MULTIVARIATE DISTRIBUTIONS AND INVARIANCE 413.0 Introduction 413.1 Multivariate Distributions 413.2 Invariance in Statistical Testing of Hypotheses 443.3 Almost Invariance and Invariance 493.4 Sufciency and Invariance 553.5 Unbiasedness and Invariance 563.6 Invariance and Optimum Tests 573.7 Most Stringent Tests and Invariance 583.8 Locally Best and Uniformly Most Powerful Invariant Tests 583.9 Ratio of Distributions of Maximal Invariant, Steins Theorem 593.10 Derivation of Locally Best Invariant Tests (LBI) 61Exercises 63References 654 PROPERTIES OF MULTIVARIATE DISTRIBUTIONS 694.0 Introduction 694.1 Multivariate Normal Distribution (Classical Approach) 704.2 Complex Multivariate Normal Distribution 844.3 Symmetric Distribution: Its Properties and Characterizations 914.4 Concentration Ellipsoid and Axes (Multivariate Normal) 1104.5 Regression, Multiple and Partial Correlation 1124.6 Cumulants and Kurtosis 1184.7 The Redundancy Index 120Exercises 120References 1275 ESTIMATORS OF PARAMETERS AND THEIR FUNCTIONS 1315.0 Introduction 1315.1 Maximum Likelihood Estimators of m, S in Npm; S 1325.2 Classical Properties of Maximum Likelihood Estimators 1415.3 Bayes, Minimax, and Admissible Characters 1515.4 Equivariant Estimation Under Curved Models 184Exercises 202References 2066 BASIC MULTIVARIATE SAMPLING DISTRIBUTIONS 2116.0 Introduction 2116.1 Noncentral Chi-Square, Students t-, F-Distributions 2116.2 Distribution of Quadratic Forms 2136.3 The Wishart Distribution 2186.4 Properties of the Wishart Distribution 2246.5 The Noncentral Wishart Distribution 231xii Contents6.6 Generalized Variance 2326.7 Distribution of the Bartlett Decomposition (RectangularCoordinates) 2336.8 Distribution of Hotellings T22346.9 Multiple and Partial Correlation Coefcients 2416.10 Distribution of Multiple Partial Correlation Coefcients 2456.11 Basic Distributions in Multivariate Complex Normal 2486.12 Basic Distributions in Symmetrical Distributions 250Exercises 258References 2647 TESTS OF HYPOTHESES OF MEAN VECTORS 2697.0 Introduction 2697.1 Tests: Known Covariances 2707.2 Tests: Unknown Covariances 2727.3 Tests of Subvectors of m in Multivariate Normal 2997.4 Tests of Mean Vector in Complex Normal 3077.5 Tests of Means in Symmetric Distributions 309Exercises 317References 3208 TESTS CONCERNING COVARIANCE MATRICES ANDMEAN VECTORS 3258.0 Introduction 3258.1 Hypothesis: A Covariance Matrix Is Unknown 3268.2 The Sphericity Test 3378.3 Tests of Independence and the R2-Test 3428.4 Admissibility of the Test of Independence and the R2-Test 3498.5 Minimax Character of the R2-Test 3538.6 Multivariate General Linear Hypothesis 3698.7 Equality of Several Covariance Matrices 3898.8 Complex Analog of R2-Test 4068.9 Tests of Scale Matrices in Epm; S 4078.10 Tests with Missing Data 412Exercises 423References 4279 DISCRIMINANT ANALYSIS 4359.0 Introduction 4359.1 Examples 4379.2 Formulation of the Problem of Discriminant Analysis 4389.3 Classication into One of Two Multivariate Normals 4449.4 Classication into More than Two Multivariate Normals 468Contents xiii9.5 Concluding Remarks 4739.6 Discriminant Analysis and Cluster Analysis 473Exercises 474References 47710 PRINCIPAL COMPONENTS 48310.0 Introduction 48310.1 Principal Components 48310.2 Population Principal Components 48510.3 Sample Principal Components 49010.4 Example 49210.5 Distribution of Characteristic Roots 49510.6 Testing in Principal Components 498Exercises 501References 50211 CANONICAL CORRELATIONS 50511.0 Introduction 50511.1 Population Canonical Correlations 50611.2 Sample Canonical Correlations 51011.3 Tests of Hypotheses 511Exercises 514References 51512 FACTOR ANALYSIS 51712.0 Introduction 51712.1 Orthogonal Factor Model 51812.2 Oblique Factor Model 51912.3 Estimation of Factor Loadings 51912.4 Tests of Hypothesis in Factor Models 52412.5 Time Series 525Exercises 526References 52613 BIBLIOGRAPHY OF RELATED RECENT PUBLICATIONS 529Appendix A TABLES FOR THE CHI-SQUARE ADJUSTMENTFACTOR 531Appendix B PUBLICATIONS OF THE AUTHOR 543Author Index 551Subject Index 555xiv Contents1Vector and Matrix Algebra1.0. INTRODUCTIONThe study of multivariate analysis requires knowledge of vector and matrixalgebra, some basic results of which are considered in this chapter. Some of theseresults are stated herein without proof; proofs can be obtained from Besilevsky(1983), Giri (1993), Graybill (1969), Maclane and Birkoff (1967), Markus andMine (1967), Perlis (1952), Rao (1973), or any textbook on matrix algebra.1.1. VECTORSA vector is an ordered p-tuple x1; . . . ; xp and is written asx x1...xp

:Actually it is called a p-dimensional column vector. For brevity we shall simplycall it a p-vector or a vector. The transpose of x is given by x0 x1; . . . ; xp. If allcomponents of a vector are zero, it is called the null vector 0. Geometrically ap-vector represents a point A x1; . . . ; xp or the directed line segment 0A!with1the point A in the p-dimensional Euclidean space Ep. The set of all p-vectors isdenoted by Vp. Obviously Vp Epif all components of the vectors are realnumbers. For any two vectors x x1; . . . ; xp0 and y y1; . . . ; yp0 we denethe vector sum x y x1y1; . . . ; xpyp0 and scalar multiplication by aconstant a byax ax1; . . . ; axp0:Obviously vector addition is an associative and commutative operation, i.e.,x y y x; x y z x y z where z z1; . . . ; zp0, and scalarmultiplication is a distributive operation, i.e., for constants a; b; a bx ax bx. For x; y [ Vp; x y and ax also belong to Vp. Furthermore, for scalarconstants a; b; ax y ax ay and abx bax abx:The quantity x0y y0x p1 xiyi is called the dot product of two vectors x; yin Vp. The dot product of a vector x x1; . . . ; xp0 with itself is denoted bykxk2 x0x, where kxk is called the norm of x. Some geometrical signicances ofthe norm are1. kxk2is the square of the distance of the point x from the origin in Ep,2. the square of the distance between two points x1; . . . ; xp; y1; . . . ; yp isgiven by kx yk2,3. the angle u between two vectors x; y is given by cos u x=kxk0 y=kyk.Denition 1.1.1. Orthogonal vectors. Two vectors x; y in Vpare said to beorthogonal to each other if and only if x0y y0x 0. A set of vectors in Vpisorthogonal if the vectors are pairwise orthogonal.Geometrically two vectors x; y are orthogonal if and only if the angle betweenthem is 908. An orthogonal vector x is called an orthonormal vector if kxk2 1.Denition 1.1.2. Projection of a vector. The projection of a vector x on y=0,both belonging to Vp, is given by kyk2x0 yy. (See Fig. 1.1.)If 0A! x; 0B! y, and P is the foot of the perpendicular from the point A on0B, then 0P! kyk2x0 yy where 0 is the origin of Ep. For two orthogonalvectors x; y the projection of x on y is zero.Denition 1.1.3. A set of vectors a1; . . . ; ak in Vpis said to be linearlyindependent if none of the vectors can be expressed as a linear combination of theothers.Thus if a1; . . . ; ak are linearly independent, then there does not exist a set ofscalar constants c1; . . . ; ck not all zero such that c1a1 ckak 0. It may beveried that a set of orthogonal vectors in Vpis linearly independent.2 Chapter 1Denition 1.1.4. Vector space spanned by a set of vectors. Let a1; . . . ; ak be aset of k vectors in VP. Then the vector space V spanned by a1; . . . ; ak is the set ofall vectors which can be expressed as linear combinations of a1; . . . ; ak and thenull vector 0.Thus if a; b [ V, then for scalar constants a; b; aa bb and aa also belongto V. Furthermore, since a1; . . . ; ak belong to Vp, any linear combination ofa1; . . . ; ak also belongs to Vpand hence V , Vp. So V is a linear subspace of Vp.Denition 1.1.5. Basis of a vector space. A basis of a vector space V is a setof linearly independent vectors which span V.In Vpthe unit vectors e1 1; 0; . . . ; 00; e2 0; 1; 0; . . . ; 00; . . . ; ep 0; . . . ; 0; 10 form a basis of Vp. If A and B are two disjoint linear subspaces of Vpsuch that A V =fg has a compact closure.Denition 3.9.4. (Proper action). Let G be a group of transformations actingtopologically from the left on the space x and let h be a mapping onG X !X Xgiven byhg; x gx; x; x [ X; g [ G:The group G acts properly on X if for every compactC , X Xh1C is compact. If G acts properly on X then X is a called Cartan G-space. Theaction is proper if for every pair (A; B) of compact subsets of XA; B fg [ GjgA >B =fgis closed. If G acts properly on X then X is a Cartan G-space. It is not known ifthe converse is true. Wijsman (1967) has studied the properness of several groupsof transformations used in multivariate testing problems. We refer to this paperand the references contained therein for the verication of these two concepts.Theorem 3.9.1. (Stein (1956)). Let G be a group of transformations g operatingon a topological space (X; A) and l a measure on X which is left-invariant underG. Suppose that there are two given probability densities p1; p2 with respect to lsuch thatP1A Ap1xdlxP2A Ap2xdlx60 Chapter 3for A [ A and P1; P2 are absolutely continuous. Let TX : X !X be amaximal invariant under G. Denote by Pi , the distribution of TX when X hasdistribution Pi; i 1; 2. Then under certain conditionsdP2TdP1TG p2gxdmgG p1gxdmg 3:20where m is a left invariant Haar measure on G.An alternative form of (3.20) is given bydP2TdP1TG f2gxXgdngG f1gxXgdng 3:21where figx; i 1; 2 denote the probability density function with respect torelatively invariant measure n with left multiplier Xg. Stein gave the statementof Theorem 3.9.1 without giving explicitly the conditions under which it holds.However this theorem was successfully used by Giri (1961, 1964, 1965) andSchwartz (1967). Schwartz (1967) gave also a set of conditions (rathercomplicated) which must be satised for this theorem to hold. Wijsman (1967a)gave a sufcient condition for this theorem using the concept of Cartan G-space.Koehn (1970) gave a generalization of the results of Wijsman (1967). Bonder(1976) gave a condition for (3.21) through topological arguments. Anderson(1982) obtained certain conditions for the validity of (3.20) in terms of properaction on groups. Wijsman (1985) studied the properness of several groups oftransformations commonly used for invariance in multivariate testing problems.The presentation of materials in this section is very sketchy. We refer toreferences cited above for further reading and for the proof of Theorem 3.9.1.3.10. DERIVATION OF LOCALLY BEST INVARIANT TESTS(LBI)Let X be the sample space of X and let G be a group of transformations g actingon the left of X. Assume that the problem of testing H0 : u [ V0 against thealternative H1 : u [ V1 is invariant under G, transforming X !gX and let TXbe a maximal invariant on X under G. The ratio R of the distributions of TX, foru1[ V1; u0[ V0 (by (3.21)) is given byR dPu1TdPu0T D1Gfu1gxXgdng 3:22Multivariate Distributions and Invariance 61whereD Gfu0gxXgdng:Letfux buqcxju; u [ V 3:23where bu, c and q are known functions and q is 0; 1 to 0; 1. Formultivariate normal distributions qz expz and cxj0 is a quadraticfunction of x. Assuming q and b are continuously twice differentiable we expandfu1xfu1x bu1fqcxju0 q1cxju0cxju1 cxju012q2zcxju1 cxju02oku1u0kg3:24where bu1 bu0 oku1u0k; z acxju0 1 acxju1; 0 a 1; ku1u0k is the norm of u1u0 and qix diq=dxi. From (3.22)and (3.24)R 1 D1Gq1cgxju0cgxju1cgxju0xgndg Mx; u1; u03:25where M is the remainder term.Assumptions1. The second term in the right-hand side of (3.25) is a function lu1; u0Sx,where Sx is a function of Tx.2. Any invariant test fX of size a satises Eu0fXMX; u1; u0 oku1u0k uniformly in f.Under above assumptions the power function Eu1fX satisesEu1f a Eu0fXlu1; u0SX oku1u0k 3:26By the Neyman-Pearson lemma the test based Sx is LBI.The following simple characterization of LBI test has been given by Giri(1968). Let R1; . . . ; Rp be maximal invariant in the sample space and letu1; . . . ; up be the corresponding maximal invariant in the parametric space. Fornotational convenience we shall write (R1; . . . ; Rp) as a vector R and (u1; . . . ; up)as a vector u though R and u may very well be diagonal matrices with diagonalelements R1; . . . ; Rp and u1; . . . ; up respectively.For xed u suppose that pr;u is the pdf of R with respect to the Lebesguemeasure. For testing H0 : u u0 u01; . . . ; u0p against alternatives62 Chapter 3H1 : u u1; . . . ; up =u0, suppose thatpr;upr;u0 1 pi1uiu0gu; u0 Ku; u0Ur Br; u; u0 3:27where gu; u0 and Ku; u0 are bounded for u in the neighborhood of u0;Ku; u0 . 0 Br; u; u0 opi1uiu0i; UR is bounded and hascontinuous distribution function for each u in V. If (3.27) is satised we saythat a test is LBI for testing H0 against H1 if its rejection region is given byUr ! Cwhere the constant C depends on the level a of the test.EXERCISES1 Let fPu; u [ Vg, the family of distributions on (X; A), be such that each Pu isabsolutely continuous with respect to a s-nite measure m; i.e., if mA 0 forA [ A, then PuA 0. Let pu @Pu=@m and dene the measure mg1forg [ G, the group of transformations on X, bymg1A mg1A:Suppose that(a) m is absolutely continuous with respect to mg1for all g [ G;(b) pux is absolutely continuous in u for all x;(c) V is separable;(d) the subspaces VH0 and VH1 are invariant with respect to G. Then show thatsupVH1pux= supVH0puxis almost invariant with respect to G.2 Let X1; . . . ; Xn be a random sample of size n from a normal population withunknown mean m and variance s2. Find the uniformly most powerful invarianttest of H0 : s2, s20 (specied) against the alternatives s2. s20 with respect tothe group of transformations which transform Xi !Xic; 1 , c , 1;i 1; . . . ; n.3 Let X1; . . . ; Xn be a random sample of size n1 from a normal population withmean m and variance s21; and let Y1; . . . ; Yn2 be a random sample of size n2from another normal population with mean n and variance s22. LetMultivariate Distributions and Invariance 63(X1; . . . ; Xn1) be independent of (Y1; . . . ; Yn2). WriteXX 1n1n1i1Xi;S21 n1i1Xi XX2;YY 1n2n2i1Yi;S22 n2i1Yi YY2:The problem of testing H0 : s21=s22 l0 (specied) against the alternativesH1 : s21=s22. l0 remains invariant under the group of transformationsXX ! XX c1; YY ! YY c2; S21 !S21; S22 !S22;where 1 , c1; c2, 1 and also under the group of common scale changesXX !aXX; YY !aYY; S21 !a2S21; S22 !a2S22;where a . 0. A maximal invariant under these two groups of transformations isF S21n11= S22n21:Show that for testing H0 against H1 the test which rejects H0 whenever F ! Ca,where Ca is a constant such that PF ! Ca a when H0 is true, is theuniformly most powerful invariant. Is it uniformly most powerful unbiased fortesting H0 against H1?4 In exercise 3 assume that s21 s22. Let S2 S21S22.(a) The problem of testing H0 : n m 0 against the alternatives H1 :n m . 0 is invariant under the group of transformationsXX ! XX c; YY ! YY c; S2!S2;where 1 , c , 1, and also under the group of transformationsXX !aXX; YY !aYY; S2!a2S2;0 , a , 1. Find the uniformly most powerful invariant test with respect tothese transformations.64 Chapter 3(b) The problem of testing H0 : n m 0 against the alternatives H1 :n m =0 is invariant under the group of afne transformationsXi !aXib; Yj aYjb;a =0; 1 , b , 1; i 1; . . . ; n1; j 1; . . . ; n2. Find the uniformlymost powerful test of H0 against H1 with respect to this group oftransformations.5 (Linear hypotheses) Let Y1; . . . ; Yn be independently distributed normalrandom variables with a common variance s2and with meansEYi mi; i 1; . . . ; ss , n0; i s 1; . . . ; nand let d2ri1m2i =s2. Show that the test which rejectsH0 : m1 mr 0; r , s, wheneverW ri1 Y2ir =nis1 Y2in s ! k;where the constant k is determined so that the probability of rejection is awhenever H0 is true, is uniformly most powerful among all tests whose powerfunction depends only on d2.6 (General linear hypotheses) Let X1; . . . ; Xn be n independently distributednormal random variables with mean ji; i 1; . . . ; n and common variance s2.Assume that j j1; . . . ; jn lies in a linear subspace of PV of dimensions , n. Show that the problem of testing H0 : j [ Pv, PV can be reduced toExercise 5 by means of an orthogonal transformation. Find the test statistic W(of Exercise 5) in terms of X1; . . . ; Xn.7 (Analysis of variance, one-way classication) Let Yij; j 1; . . . ; ni;i 1; . . . ; k, be independently distributed normal random variables withmeans EYij mi and common variance s2. Let H0 : m1 mk. Identifythis as a problem of general linear hypotheses. Find the uniformly mostpowerful invariant test with respect to a suitable group of transformations.8 In Example 3.2.3 show that for testing H0 : m 0 against H1 : d2. 0,students test is minimax. Is it stringent for H0 against H1?REFERENCESAnderson, S. A. (1982). Distribution of maximal invariants using quotientmeasures. Ann. Statist. 10:955961.Bonder, J. V. (1976). Borel cross-section and maximal invariants. Ann. Statist.4:866877.Multivariate Distributions and Invariance 65Blackwell, D. (1956). On a class of probability spaces. In: Proc. Berkeley Symp.Math. Statist. Probability. 3rd. Univ. of California Press, Berkeley, California.Eaton, M. L. (1989). Group Invariance Applications in Statistics. Institute ofMathematical Statistics and American Statistical Association: USA.Ferguson, T. S. (1969). Mathematical Statistics. New York: Academic Press.Ghosh, J. K. (1967). Invariance in Testing and Estimation. Indian Statist. Inst.,Calcutta, Publ. No. SM67/2.Giri, N. (1961). On the Likelihood Ratio Tests of Some Multivariate Problems.Ph.D. thesis, Stanford Univ.Giri, N. (1964). On the likelihood ratio test of a normal multivariate testingproblem. Ann. Math. Statist. 35:181189.Giri, N. (1965). On the likelihood ratio test of a normal multivariate testingproblem II. Ann. Math. Statist. 36:10611065.Giri, N. (1968). Locally and asymptotically minimax tests of a multivariateproblem. Ann. Math. Statist. 39:171178.Giri, N. (1993). Introduction to Probability and Statistics, 2nd ed., Revised andExpanded. New York: Marcel Dekker.Giri, N. (1975). Invariance and Minimax Statistical Tests. India: The Univ. Pressof Canada and Hindusthan Publ. Corp.Giri, N. (1997). Group Invariance in Statistical Inference. Singapore: WorldScientic.Giri, N., Kiefer, J. (1964a). Local and asymptotic minimax properties ofmultivariate tests. Ann. Math. Statist. 35:2135.Giri, N., Kiefer, J. (1964b). Minimax character of R2-test in the simplest case.Ann. Math. Statist. 35:14751490.Giri, N., Kiefer, J., Stein, C. (1963). Minimax character of Hotellings T2-test inthe simplest case. Ann. Math. Statist. 34:15241535.Hall, W. J., Wijsman, R. A., Ghosh, J. K. (1965). The relationship betweensufciency and invariance with application in sequential analysis. Ann. Math.Statist. 36:575614.Halmos, P. R. (1958). Measure Theory. Princeton, NJ: Van Nostrand-Reinhold.Kiefer, J. (1957). Invariance sequential estimation and continuous timeprocesses. Ann. Math. Statist. 28:675699.66 Chapter 3Kiefer, J. (1958). On the nonrandomized optimality and randomizednonoptimality of symmetrical designs. Ann. Math. Statist. 29:675699.Kiefer, J. (1966). Multivariate optimality results. In: Krishnaiah, P. R., ed.Multivariate Analysis. Academic Press: New York.Koehn, U. (1970). Global cross-sections and densities of maximal invariants.Ann. Math. Statist. 41:20462056.Kolmogorov, A. N. (1950). Foundations of the Theory of Probability. Chelsea:New York.Lehmann, E. L. (1959). Testing Statistical Hypotheses. New York: Wiley.Lehmann, E. L. (1959a). Optimum invariant tests. Ann. Math. Statist. 30:881884.Lehmann, E. L., Stein, C. (1953). The admissibility of certain invariant statisticaltests involving a translation parameter. Ann. Math. Statist. 24:473479.Linnik, Ju, V., Pliss, V. A., Salaevskii, O. V. (1966). Sov. Math. Dokl. 7:719.Nachbin, L. (1965). The Haar Integral. Princeton, NJ: Van Nostrand-Reinhold.Pitman, E. J. G. (1939). Tests of hypotheses concerning location and scaleparameters. Biometrika 31:200215.Salaevskii, O. V. (1968). Minimax character of Hotellings T2-test. Sov. Math.Dokl. 9:733735.Schwartz, R. (1967). Local minimax tests. Ann. Math. Statist. 38:340360.Stein, C. (1956). Some Problems in Multivariate Analysis, Part I. StanfordUniversity Technical Report, No. 6, Stanford, Calif.Wijsman, R. A. (1967). Cross-sections of orbits and their application to densitiesof maximal invariants. In: Proc. Fifth Berk Symp. Math. Statist. Prob. Vol. 1,University of California Press, pp. 389400.Wijsman, R. A. (1967a). General proof of termination with probability one ofinvariant sequential probability ratio tests based on multivariate observations.Ann. Math. Statist. 38:824.Wijsman, R. A. (1985). Proper action in steps, with application to density ratiosof maximal invariants. Ann. Statist. 13:395402.Wijsman, R. A. (1990). Invariance Measures on Groups and Their Use inStatistics. USA: Institute of Mathematical Statistics.Multivariate Distributions and Invariance 674Properties of Multivariate Distributions4.0. INTRODUCTIONWe will rst dene the multivariate normal distribution in the classical way bymeans of its probability density function and study some of its basic properties.This denition does not include the cases in which the covariance matrix issingular and also the cases in which the dimension of the random vector iscountable or uncountable. We will then dene multivariate normal distribution inthe general way to include such cases. A number of characterizations of themultivariate normal distribution will also be given in order to enable the reader tostudy this distribution in Hilbert and Banach spaces.The complex multivariate normal distribution plays an important role indescribing the statistical variability of estimators and of functions of the elementsof a multiple stationary Gaussian time series. This distribution is also useful inanalyzing linear models with complex covariance structures which arise whenthey are invariant under cyclic groups. We treat it here along with some basicproperties.Multivariate normal distribution has many advantages from the theoreticalviewpoints. Most elegant statistical theories are centered around this distribution.However, in practice, it is hard to ascertain if a sample of observation is drawnfrom a multivariate normal population or not. Sometimes, it is advantageous toconsider a family of distributions having certain similar properties. The family ofelliptically symmetric distributions include, among others, the multivariatenormal, the compound multivariate normal, the multivariate t-distribution and69the multivariate Cauchy distribution. For all probability density functions in thisfamily the shapes of contours of equal densities are elliptical. We shall treat ithere along with some of its basic properties.Another deviation from the multivariate normal family is the family ofmultivariate exponential power distributions where the multivariate normaldistribution is enlarged through the introduction of an additional parameter u andthe deviation from the multivariate normal family is described in terms of u. Thisfamily (problem 25) includes the multivariate normal family (u 1),multivariate double exponential family (u 12) and the asymptotically uniformdistributions (u !1). The univariate case ( p 1) is often treated in Bayesianinference (Box and Tiao (1973)).4.1. MULTIVARIATE NORMAL DISTRIBUTION (CLASSICALAPPROACH)Denition 4.1.1. Multivariate normal distribution. A random vector X X1; . . . ; Xp0 taking values x x1; . . . ; xp0 in Ep(Euclidean space of dimensionp) is said to have a p-variate normal distribution if its probability density functioncan be written asfXx 12pp=2jSj12exp 12x m0S1x m ; 4:1where m m1; . . . ; mp0[ Epand S is a p p symmetric positive denitematrix.In what follows a random vector will always imply a real vector unless it isspecically stated otherwise.We show now that fXx is an honest probability density function of X. Since Sis positive denite, x m0S1x m ! 0 for all x [ Epand detS . 0. HencefXx ! 0 for all x [ Ep. Furthermore, since S is a p p positive denite matrixthere exists a p p nonsingular matrix C such that S CC0. Let y C1x. TheJacobian of the transformation x !y C1x is det C. Writingn n1; . . . ; np0 C1m, we obtainEp12pp=2det S12exp 12x m0S1x m dxEp12pp=2 exp 12y n0y n dypi11112p1=2 exp 12yini2 dyi 1:70 Chapter 4Theorem 4.1.1. If the random vector X has a multivariate normal distributionwith probability density function fXx, then the parameters m and S are given byEX m; EX mX m0 S.Proof. The random vector Y C1X, with S CC0, has probability densityfunctionfYy pi112p1=2 exp 12yini2 Thus EY EY1; . . . ; EYp0 n C1m,covY EY nY n0 I: 4:2From thisEC1X C1EX C1mEC1X C1mC1X C1m0 C1EX mX m0C10 I:HenceEX m; EX mX m0 S:Q.E.D.We will frequently write S asS s21 s12 s1ps21 s22 s2p......sp1 sp2 s2p with sij sjiThe fact that S is symmetric follows from the identityEX mX m00 EX mX m0.The term covariance matrix is used here instead of the matrix of variances andcovariances of the components.We will now prove some basic characteristic properties of multivariate normaldistributions in the following theorems.Theorem 4.1.2. If the covariance matrix of a normal random vector X X1; . . . ; Xp0 is a diagonal matrix, then the components of X are independentlydistributed normal variables.Properties of Multivariate Distributions 71Proof. LetS s21 0 00 s22 0......0 0 s2pThenx m0S1x m pi1ximisi 2; det S pi1s2iHencefXx pi112p1=2siexp 12ximisi 2 ;which implies that the components are independently distributed normal randomvariables with means mi and variance s2i . Q.E.D.It may be remarked that the converse of this theorem holds for any randomvector X. The theorem does not hold if X is not a normal vector. The followingtheorem is a generalization of the above theorem to two subvectors.Theorem 4.1.3. Let X X01; X020; X1 X1; . . . ; Xq0; X2 Xq1; . . . ; Xp0, let m be similarly partitioned as m m01; m020, and let S bepartitioned asS S11 S12S21 S22 where S11 is the upper left-hand corner submatrix of S of dimension q q. If Xhas normal distribution with means m and positive denite covariance matrix Sand S12 0, then X1; X2 are independently normally distributed with meansm1; m2 and covariance matrices S11; S22 respectively.Proof. Under the assumption that S12 0, we obtainx m0S1x m x1m10S111x1m1 x2m20S122x2m2; det S det S11det S22:72 Chapter 4HencefXx 12pq=2det S111=2 exp 12x1m10S111x1m1 12ppq=2det S221=2 exp 12x2m20S122x2m2 and the result follows. Q.E.D.This theorem can be easily extended to the case where X is partitioned intomore than two subvectors, to get the result that any two of these subvectors areindependent if and only if the covariance between them is zero. An importantreproductive property of the multivariate normal distribution is given in thefollowing theorem.Theorem 4.1.4. Let X X1; . . . ; Xp0 with values x in Epbe normallydistributed with mean m and positive denite covariance matrix S. Then therandom vector Y CX with values y Cx in Epwhere C is a nonsingular matrixof dimension p p has p-variate normal distribution with mean Cm andcovariance matrix CSC0.Proof. The Jacobian of the transformation x !y Cx is det C1. Hence theprobability density function of Y is given byfYy fxC1ydet C1 12pp=2det CSC01=2exp 12y Cm0CSC01y Cm Thus Y has p-variate normal distribution with mean Cm and positive denitecovariance matrix CSC0. Q.E.D.Theorem 4.1.5. Let X X01; X020 be distributed as Npm; S where X1; X2are as dened in Theorem 4.1.3. Then(a) X1; X2S21S111 X1 are independently normally distributed with meansm1; m2S21S111m1 and positive denite covariance matricesS11; S22:1 S22S21S111S12 respectively.(b) The marginal distribution of X1 is q-variate normal with mean m1 andcovariance matrix S11.(c) The condition distribution of X2 given X1 x1 is normal with meanm2S21S111x1m1 and covariance matrix S22:1.Properties of Multivariate Distributions 73Proof.(a) LetY Y1Y2 X1X2S21S111 X1 :ThenY I1 0S21S111 I2 X1X2 CXwhere I1 and I2 are identity matrices of dimensions q q and p q p q respectively andC I1 0S21S111 I2 :Obviously C is a nonsingular matrix. By Theorem 4.1.4 Y has p-variatenormal distribution with meanCm m1m2S21S111m1 and covariance matrixCSC0 S11 00 S22:1 :Hence by Theorem 4.1.3 we get the result.(b) It follows trivially from part (a).(c) The Jacobian of the inverse transformation Y CX is unity. From (a) theprobability density function of X can be written asfXx expf12x1m10S111x1m1g2pq=2det S111=2 12ppq=2det S22:11=2exp 12x2m2S21S111x1m10S122:1x2m2S21S111x1m14:374 Chapter 4Hence the results. Q.E.D.Thus it is interesting to note that if X has p-variate normal distribution, themarginal distribution of any subvector of X is also a multivariate normal and theconditional distribution of any subvector given the values of the remainingsubvector is also a multivariate normal.Example 4.1.1. Bivariate normal. LetS s21 rs1s2rs1s2 s22 with s21. 0; s22. 0; 1 , r , 1. Since det S s21s221 r2 . 0; S1existsand is given byS11s21rs1s2rs1s21s2211 r2 :Furthermore, for x x1; x20=0x0Sx s1x1rs2x221 r2s22x22. 0:Hence S is positive denite. With m m1; m20,x m0S1x m 11 r2x1m1s1 2 x2m2s2 22r x1m1s1 x2m2s2 :The probability density function of a bivariate normal random variable withvalues in E2is12ps1s21 r21=2exp 121 r2x1m1s1 2 x2m2s2 2 2r x1m1s1 x2m2s2 :The coefcient of correlation between X1 and X2 iscovX1; X2varX1varX21=2 r:Properties of Multivariate Distributions 75If r 0; X1; X2 are independently normally distributed with means m1; m2 andvariances s21; s22, respectively. If r . 0, then X1; X2 are positively related; and ifr , 0, then X1; X2 are negatively related.The marginal distributions of X1 and of X2 are both normal with means m1 andm2, and with variances s21 and s22, respectively. The conditional probabilitydensity of X2 given X1 x1 is a normal withEX2jX1 x1 m2r s2s1 x1m1; varX2jX1 x1 s221 r2:Figures 4.1 and 4.2 give the graphical presentation of the bivariate normaldistribution and its contours.We now give an example to show that the normality of marginal distributionsdoes not necessarily imply the multinormality of the joint distribution though theconverse is always true.Figure 4.1. Bivariate normal with mean 0 and S 1 1212 1 .76 Chapter 4Example 4.1.2. Letf x1; x2jr1 12p1 r211=2 exp 121 r21x21x222r1x1x2 f x1; x2jr2 12p1 r221=2 exp 121 r22x21x222r2x1x2 be two bivariate normal probability functions with 0 means, unit variances anddifferent correlation coefcients. Letf x1; x2 12f x1; x2jr1 12f x1; x2jr2:Obviously f x1; x2 is not a bivariate normal density function though themarginals of X1 and of X2 are both normals.Theorem 4.1.6. Let X X1; . . . ; Xp0 be normally distributed with mean mand positive denite covariance matrix S. The characteristic function of theFigure 4.2. Contours of bivariate normal in Figure 4.1.Properties of Multivariate Distributions 77random vector X is given byfXt Eeit0X exp it0m 12t0St 4:4wheret t1; . . . ; tp0[ Ep; i 11=2:Proof. Since S is positive denite, there exists a nonsingular matrix C such thatS CC0. Write y C1x; a a1; . . . ; ap0 C0t; q C1m q1; . . . ; qp0.ThenEeit0X Ep2pp=2exp ia0y 12 y q0y q dypj12p1=2exp iajyj12 yjqj2 dyjpj1exp iajqj12a2j exp ia0q 12a0a exp it0m 12t0St as the characteristic function of a univariate normal random variable isexpfitm 12t2s2g. Q.E.D.Since the characteristic function determines uniquely the distribution functionit follows from (4.4) that the p-variate normal distribution is completely speciedby its mean vector m and covariance matrix S. We shall therefore use the notationNpm; S for the density function of a p-variate normal random vector involvingparameter m; S whenever S is positive denite.In Theorem 4.1.4 we have shown that if C is a nonsingular matrix then CX is ap-variate normal whenever X is a p-variate normal. The following theorem willassert that this restriction on C is not essential.Theorem 4.1.7. Let X X1; . . . ; Xp0 be distributed as Npm; S and let Y AX where A is a matrix of dimension q p of rank qq , p. Then Y is distributedas NqAm; ASA0.78 Chapter 4Proof. Let C be a nonsingular matrix of dimension p p such thatC AB ;where B is a matrix of dimension p q p of rank p q, and let Z BX. Thenby Theorem 4.1.4.YZ has p-variate normal distribution with meanCm AmBm and covariance matrixCSC0 ASA0 ASB0BSA0 BSB0 :By Theorem 4.1.5(b) we get the result. Q.E.D.This theorem tells us that if X is distributed as Npm; S, then every linearcombination of X has a univariate normal distribution. We will now show that if,for a random vector X with mean m and covariance matrix S, every linearcombination of the components of X having a univariate normal distribution, thenX has a multivariate normal distribution.Theorem 4.1.8. Let X X1; . . . ; Xp0. If every linear combination of thecomponents of X is distributed as a univariate normal, then X is distributed as ap-variate normal.Proof. For any nonnull xed real p-vector L, let L0X have a univariate normalwith mean L0m and variance L0SL. Then for any real t the characteristic functionof L0X isft; L EeitL0X exp itL0m 12t2L0SL :Hencef1; L EeiL0X exp iL0m 12L0SL ;which as a function of L is the characteristic function of X. By the inversiontheorem of the characteristic function (see Giri (1993), or Giri (1974)) theprobability density function of X is Npm; S. Q.E.D.Properties of Multivariate Distributions 79Motivated by Theorem 4.1.7 and Theorem 4.1.8 we now give a more generaldenition of the multivariate normal distribution.Denition 4.1.2. Multivariate normal distribution. A p-variate random vectorX with values in Epis said to have a multivariate normal distribution if and only ifevery linear combination of the components of X has a univariate normaldistribution.When S is nonsingular, this denition is equivalent to that of the multivariatenormal distribution given in 4.1.1. If X has a multivariate normal distributionaccording to Denition 4.1.2, then each component Xi of X is distributedas univariate normal so that 1 , EXi , 1; varXi , 1, and hencecovXi; Xi s2i ; covXi; Xj sij. Then EX; covX exist and we denotethem by m; S respectively.In Denition 4.1.2 it is not necessary that S be positive denite; it can besemipositive denite also.Denition 4.1.2 can be extended to the denition of a normal probabilitymeasure on Hilbert and Banach spaces by demanding that the induceddistribution of every linear functional be univariate normal. The reader is referredto Frechet (1951) for further details. One other big advantage of Denition 4.1.2over Denition 4.1.1 is that certain results of univariate normal distribution canbe immediately generalized to the multivariate case. Readers may nd itinstructive to prove Theorems 4.1.14.1.8 by using Denition 4.1.2. As anillustration let us prove Theorem 4.1.3 and then Theorem 4.1.7.Proof of Theorem 4.1.3. For any nonzero real p-vector L l1; . . . ; lp0 thecharacteristic function of L0X isft; L exp itL0m 12t2L0SL : 4:5Write L L01; L020 where L1 l1; . . . ; lq0.ThenL0m L01m1L02m2; L0SL L01S11L1L02S22L2Henceft; L exp itL01m112t2L01S11L1 exp itL02m212t2L02S22L2 80 Chapter 4In other words the characteristic function of X is the product of the characteristicfunctions of X1 and X2 and each one is the characteristic function of amultivariate normal distribution. Hence Theorem 4.1.3 is proved.Proof of Theorem 4.1.7. Let Y AX. For any xed nonnull vector L,L0Y L0AX:By Denition 4.1.2 L0AX has univariate normal distribution with mean L0Am andvariance L0ASA0L. Since L is arbitrary, this implies that Y has q-variate normaldistribution with mean Am and covariance matrix ASA0. Q.E.D.Using Denition 4.1.2 we need to establish the existence of the probabilitydensity function of the multivariate normal distribution. Let us now examine thefollowing question: Does Denition 4.1.2 always guarantee the existence of theprobability density function? If not, under what conditions can we determineexplicitly the probability density function?Evidently Denition 4.1.2 does not restrict the covariance matrix to bepositive denite. If S is a nonnegative denite of rank q, then for any real nonnullvector L, L0SL can be written asL0SL a01L2 a0qL24:6where ai ai1; . . . ; aip0 i 1; . . . ; q are linearly independent vectors. Hencethe characteristic function of X can be written asexp iL0m 12qj1a0jL2 : 4:7Now expfiL0mg is the characteristic function of a p-dimensional random variableZ0 which assumes value m with probability one and expf12qj1a0jL2g is thecharacteristic function of a p-dimensional random variableZi ai1Ui; . . . ; aipUi0where U1; . . . ; Uq are independently, identically distributed (real) randomvariables with mean zero and variance unity.Theorem 4.1.9. The random vector X X1; . . . ; Xp0 has p-variate normaldistribution with mean m and with covariance matrix S of rank qq p if andonly ifX m aU; aa0 S;where a is a p q matrix of rank q and U U1; . . . ; Uq0 has q-variate normaldistribution with mean 0 and covariance matrix I (identity matrix).Properties of Multivariate Distributions 81Proof. Let X m aU, aa0 S, and U be normally distributed with mean 0and covariance matrix I. For any nonnull xed real p-vector L,L0X L0m L0aU:But L0aU has univariate normal distribution with mean zero and varianceL0aa0L. Hence L0X has univariate normal distribution with mean L0m and varianceL0aa0L. Since L is arbitrary, by Denition 4.1.2 X has p-variate normaldistribution with mean m and covariance matrix S aa0 of rank q.Conversely, if the rank of S is q and X has a p-variate normal distribution withmean m and covariance matrix S, then from (4.7) we can writeX Z0Z1 Zq m aU;satisfying the conditions of the theorem. Q.E.D.4.1.1. Some Characterizations of the Multivariate NormalDistributionWe give here only two characterizations of the multivariate normal distributionwhich are useful for our purpose. For other characterizations we refer to the bookby Kagan et al. (1972).Before we begin to discuss characterization results we need to state thefollowing results due to Cramer (1937) regarding univariate random variables.If the sum of two independent random variables X; Y is normally distributed,then each one is normally distributed. For a proof of this the reader is referred toCramer (1937). The following characterizations of the multivariate normaldistribution are due to Basu (1955).Theorem 4.1.10. If X; Y are two independent p-vectors and if X Y has a p-variate normal distribution, then both X; Y have p-variate normal distribution.Proof. Since X Y has a p-variate normal distribution, for any nonnull p-vector L; L0X Y L0X L0Y has univariate normal distribution. Since L0Xand L0Y are independent, by Cramers result, L0X; L0Y are both univariate normalrandom variables. This, by Denition 4.1.2, implies that both X; Y have p-variatenormal distribution. Q.E.D.82 Chapter 4Theorem 4.1.11. Let X1; . . . ; Xn be a set of mutually independent p-vectors andletX ni1aiXi; Y ni1biXiwhere a1; . . . ; an; b1; . . . ; bn are two sets of real constants.(a) If X1; . . . ; Xn are identically normally distributed p-vectors and ifni1 aibi 0, then X and Y are independent.(b) If X and Y are independently distributed, then each Xi for which aibi=0has p-variate normal distribution.Note: Part (b) of this theorem is a generalization of the Darmois-Skitovictheorem which states that if X1; . . . ; Xn are independently distributed randomvariables, then the independence ofni1 aiXi;ni1 biXi, implies that each Xi isnormally distributed provided aibi=0 (See Darmois (1953), Skitovic (1954), orBasu (1951)).Proof.(a) For any nonnull p-vector LL0X a1L0X1 anL0Xn:If X1; . . . Xn are independent and identically distributed normal randomvectors, then L0X1; . . . ; L0Xn are independently and identically distributednormal random variables and hence L0X has univariate normal distributionfor all L. This implies that X has a p-variate normal distribution. Similarly Yhas a p-variate normal distribution. Furthermore, the joint distribution ofX; Y is a 2p-variate normal. NowcovX; Y Siaibi covXi S 0 0:Thus X; Y are independent.(b) For any nonnull real p-vector LL0X ni1aiL0Xi; L0Y ni1biL0Xi:Since L0Xi are independent random variables, independence of L0X, L0Y andaibi=0 implies L0Xi has a univariate normal distribution. Since L isarbitrary, Xi has a p-variate normal distribution.Q.E.D.Properties of Multivariate Distributions 834.2. COMPLEX MULTIVARIATE NORMAL DISTRIBUTIONA complex random variable Z with values in C (eld of complex numbers) iswritten as Z X iY where i 1p ; X; Y are real random variables. Theexpected value of Z is dened byEZ EX iEY; 4:8assuming both EX and EY exist. The variance of Z is dened byvarZ EZ EZZ EZ*; varX varY 4:9where Z EZ* denote the adjoint of Z EZ, i.e. the conjugate andtranspose of Z EZ.Note that for 1-dimensional variables the transpose is superuous. It followsthat for a; b [ CvaraZ b EaZ EZaZ EZ* aa*varZ:The covariance of any two complex random variables Z1; Z2 is dened bycovZ1; Z2 EZ1EZ1Z2EZ2*: 4:10Theorem 4.2.1. Let Z1; . . . ; Zn be a sequence of n complex random variables.Then(a) covZ1; Z2 covZ2; Z1*.(b) For a1; . . . ; an[ C; b1; . . . ; bn[ C,covnj1ajZj;nj1bjZj nj1ajbbjvarZj 2j,kajbbkcovZj; Zk:84 Chapter 4Proof.(a) Let Zj XjiYj; j 1; . . . ; n, where X1; . . . ; Xn; Y1; . . . ; yn are realrandom variables.covZ1; Z2 covX1; X2 covY1; Y2icovY1; X2 covX1; Y2;covZ2; Z1 covX1; X2 covY1; Y2icovY1; X2 covX1; Y2:Hence the result.covjajZj;jbjZj EjajZjEZj jbjZjEZj *jajbj varZj 2j,kajbkcovZj; Zk:Q.E.D.A p-variate complex random vector with values in CpZ Z1; . . . ; Zp0;with Zj XjiYj, is a p-tuple of complex random variables Z1; . . . ; Zp. Theexpected value of Z isEZ EZ1; . . . ; EZp0: 4:11The complex covariance of Z is dened byS EZ EZZ EZ*: 4:12Since S* S, S is a Hermitian matrix.Denition 4.2.1. Let Z X iY [ Cpbe a complex vector of p-dimensionZ XY [ E2p: 4:13This representation denes an isomorphism between Cpand E2p.(b)Properties of Multivariate Distributions 85Theorem 4.2.2. Let Cp be the space of p p complex matrices and let C A iB [ Cp where A and B are p p real matrices. For Z [ CpCZ kClZ; 4:14wherekCl A BB A : 4:15Proof.CZ A iBX iY AX BY iBX AY AX BYBX AY kClZ:Q.E.D.Denition 4.2.2. A univariate complex normal random variable is a complexrandom variable Z X iY such that the distribution of Z XYis a bivariatenormal.The probability density function of Z can be written asfZz 1pvarZexp z az a*varZ 1pvarX varYexp x m2y n2varX varY where a m in EZ.Denition 4.2.3. A p-variate complex random vectorZ Z1; . . . ; Zp0;with Zj XjiYj is a p-tuple of complex normal random variables Z1; . . . ; Zpsuch that the real 2p-vector X1; . . . ; Xp; Y1; . . . ; Yp0 has a 2p-variate normaldistribution.Leta EZ EX iEY m in; S EZ aZ a*;where S [ Cp is a positive denite Hermitian matrix; a [ Cp; m; n [ Ep;X X1; . . . ; Xp [ Ep; Y Y1; . . . ; Yp0[ Ep. The joint probability density86 Chapter 4function of X; Y can be written asfZz 1ppdetSexpfz a*S1z ag 1ppdet2G 2D2D 2G 124:16exp x my n 0 2G 2D2D 2G 1x my n where S 2G i2D, G is a positive denite matrix and D D0 (skewsymmetric). Hence EX m; EY n and the covariance matrix ofXY isgiven byG DD G Thus if Z has the probability density function fZz given by (4.16), thenEZ m iq; covZ S.Example 4.2.1. Bivariate complex normal. HereZ Z1; Z20; Z1 X1iY1; Z2 X2iY2; EZ a a1; a20 m1; m20in1; n20:LetcovZj; Zk s2k if j k;ajk ibjksjsk if j =k:HenceS s21 a12ib12s1s2a12ib12s1s2 s22 ;det S s21s22a212b212s21s22;S1 11 a212b212s21s22 s22 a12ib12s1s2a12ib12s1s2 s22 :Properties of Multivariate Distributions 87ThusfZz 1p21 a212b212exp1 a212b212s21s221fs22z1a1*z1a1 s21z2a2*z a2z a22s1s2a12ib12z1a1*z2a2gThe numerator inside the braces can be expressed ass22x1m12y1n12 s21x2m22y2n22 4s1s2a12x1m1x2m2 y1n1y2n2b12x1m1y2m2 x2m2y1m1:The special case of the probability density function of the complex random vectorZ given in (4.16) with the added restrictionEZ aZ a0 0 4:17is of considerable interest in the literature. This condition implies that the real andimaginary parts of different components are pairwise independent and the realand the imaginary parts of the same components are independent with the samevariance.With the density function fZz in (4.16) one can obtain results analogous toTheorems 4.1.14.1.5 for the complex case. We shall prove below threetheorems which are analogous to Theorems 4.1.34.1.5.Theorem 4.2.3. Let Z Z1; . . . ; Zp0 with values in Cpbe distributed as thecomplex p-variate normal with mean a and Hermitian positive denitecovariance matrix S. Then CZ, where C is a complex nonsingular matrix ofdimension p p, has a complex p-variate normal distribution with mean Ca andHermitian positive denite covariance matrix CSC*.Proof.CZ AX BYBX AY A BB A XY kClZ88 Chapter 4Since Z is distributed asN2p a; G DD G ; A BB A Zis distributed as 2p-variate normal with meanAm BnBm An and 2p 2p covariance matrixAGA0BDA0ADB0BGB0 BGA0BDB0AGB0ADA0BGA0BDB0AGB0ADA0 AGA0BDA0ADB0BGB0 Hence CZ is distributed as p-variate complex normal with mean Ca and p pcomplex covariance matrix2AGA0BDA0BGB0ADB0i2BGA0ADA0AGB0BDB0 CSC*:Q.E.D.Theorem 4.2.4. Let Z Z01; Z020, where Z1 Z1; . . . ; Zq0; Z2 Zq1; . . . ; Zp0 be distributed as p-variate complex normal with mean a a01; a020 and positive denite Hermitian covariance matrix S and let S bepartitioned asS S11 S12S21 S22 where S11 is the upper left-hand corner submatrix of dimension q q. If S12 0,then Z1; Z2 are independently distributed complex normal vectors with meansa1; a2 and Hermitian covariance matrices S11; S22 respectively.Proof. Under the assumption that S12 0, we obtainz a*S1z a z1a1*S111z1a1z2a2*S122z2a2;det S det S11det S22:Properties of Multivariate Distributions 89HencefZz 1pqdet S11expfz1a1*S111z1a1g 1ppq det S22expfz2a2*S122z2a2gand the result follows. Q.E.D.Theorem 4.2.5. Let Z Z01; Z020, where Z1; Z2 are as dened in Theorem4.2.4.(a) Z1; Z2S21S111 Z1 are independently distributed complex normalrandom vectors with means a1; a2S21S111a1 and positive deniteHermitian covariance matrixes S11; S22:1 S22S21S111S12 respectively.(b) The marginal distribution of Z1 is a q-variate complex normal with meansa1 and positive denite Hermitian covariance matrix S11.(c) The conditional distribution of Z2 given Z1 z1 is complex normal withmean a2S21S111z1a1 and positive denite Hermitian covariancematrix S22:1.Proof. (a) LetU U1U2 Z1Z2S21S111 Z1 :ThenU I1 0S21S111 I2 Z1Z2 CZwhere I1 and I2 are identity matrices of dimensions q q and p q p qrespectively and C is a complex nonsingular matrix. By theorem 4.2.3 U has a p-variate complex normal distribution with meanCa a1a2S21S111a1 and (Hermitian) complex covariance matrixCSC* S11 00 S22:1 :By Theorem 4.2.4 we get the result.90 Chapter 4(b) and (c). They follow from part (a) above. Q.E.D.The characteristic function of Z is given byE expfiRt*Zg expfiRt*a t*Stg 4:18for t [ Cpand R denotes the real part of a complex number. As in the real casewe denote a p-variate complex normal with mean a and positive deniteHermitian matrix S by CNpa; S.From Theorem 4.2.3 we can dene a p-variate complex normal distribution inthe general case as follows.Denition 4.2.4. A complex random p-vector Z with values in Cphas acomplex normal distribution if, for each a [ Cp; a*Z has a univariate complexnormal distribution.4.3. SYMMETRIC DISTRIBUTION: ITS PROPERTIES ANDCHARACTERIZATIONSIn multivariate statistical analysis multivariate normal distribution plays a verydominant role. Many results relating to univariate normal statistical inferencehave been successfully extended to the multivariate normal distribution. Inpractice, the verication of the assumption that a given set of data arises from amultivariate normal population is cumbersome. A natural question thus ariseshow sensitive these results are to the assumption of multinormality. In recentyears one such investigation involves in considering a family of density functionshaving many similar properties as the multinormal. The family of ellipticallysymmetric distributions contains probability density functions whose contours ofequal probability have elliptical shapes. In recent years this family is becomingincreasingly popular because of its frequent use in ltering and stochasticcontrol (Chu (1973)), random signal input (McGraw and Wagner (1968)),stock market data analysis (Zellner (1976)) and because some optimum resultsof statistical inference in the multivariate normal preserves their properties for allmembers of the family. The family of spherically symmetric distributions is aspecial case of this family.They contain the multivariate student-t, compound (or scale mixed)multinormal, contaminated normal, multivariate normal with zero mean vectorand covariance matrix I among others.It is to be pointed out that these families do not possess all basic requirementsfor an ideal statistical inference. For example the sample observations are notindependent, in general, for all members of these families.Properties of Multivariate Distributions 914.3.1. Elliptically and Spherically Symmetric Distribution(Univariate)Denition 4.3.1.1. Elliptically symmetric distribution (univariate). A randomvector X X1; . . . ; Xp0 with values x in Rpis said to have a distributionbelonging to the family of elliptically symmetric distributions (univariate) withlocation parameter m m1; . . . ; mp0[ Rpand scale matrix S (symmetricpositive denite) if its probability density function (pdf ), if it exists, can beexpressed as a function of the quadratic form x m0S1x m and is given byfXx det S12qx m0S1x m; 4:19where q is a function on 0; 1 satisfyingRp q y0ydy 1 for y [ Rp.Denition 4.3.1.2. Spherically symmetric distribution (univariate). A randomvector X X1; . . . ; Xp0 is said to have a distribution belonging to the family ofspherically symmetric distributions if X and OX have the same distributions forall p p orthogonal matrices O.Let X X1; . . . ; Xp0 be a random vector having elliptically symmetric pdf(4.19) and let Y C1X m where C is a p p nonsingular matrix satisfyingS CC0 (by Theorem 1.5.5). The pdf of Y is given byfYy det S1=2qCy0S1Cy det C detC1SC101=2q y0C1SC101y 4:20 qy0yas the Jacobian of the transformation X !C1X m Y is detC.FurthermoreOy0Oy y0yfor all p p orthogonal matrix O and the Jacobian of the transformation Y !OYis unity. Hence fYy is the pdf of a spherically symmetric distribution.We denote the elliptically symmetric pdf (4.19) of a random vector X byEpm; S; q and the spherically symmetric pdf (4.20) of a random vector Y byEp0; I; q. When the mention of q is unnecessary we will omit q in the notations.4.3.2. Examples of Ep(m, S, q)Example 4.3.2.1. Multivariate normal Npm; S. The pdf of X X1; . . . ; Xp0isfXx 2pp2det S12etr 12x m0S1x m for x [ Rp. Here qz 2pp=2exp12z; z ! 0.92 Chapter 4Example 4.3.2.2. Multivariate student-t with m degrees of freedom. The pdf ofX X1; . . . ; Xp0 isfXx G12m pdet S12pm12pG12m1 1mx m0S1x m 12mp: 4:21Hereqz G12m ppm12pG12m1 zm 12mp:We will denote the pdf of the multivariate student-t with m degrees of freedomand with parameter m; S by tpm; S; m in order to distinguish it from themultivariate student-t based on spherically symmetric distribution with pdf givenbyfYy G12m ppm12pG12m1 1my0y 12mp: 4:22where: Y S12X m. Since the Jacobian of the transformation X !Y isdet S12the pdf of Y is given by (4.22).To prove fYy or fXx is a pdf we use the identity111 x2ndx 101 yny121dy101 un121u121duGn 12G12Gn 1 :Let A be a k p matrix of rank kk p and let C be a p p nonsingularmatrix such thatC AB where B is a p k p matrix of rank p k. ThenZ CX AXBX Properties of Multivariate Distributions 93is distributed as tpCm; CSC0; m andCm AmBm ; CSC0 ASA0 ASB0BSA0 BSB0 :Using problem 24 we get Y AX is distributed as tkAm; ASA0; m.Figures 4.3 and 4.4 give the graphical representation of the bivariate student-twith 2 degrees of freedom and its contour.Example 4.3.2.3. Scale mixed (compound) multivariate normal. Let X X1; . . . ; Xp0 be a random vector with pdffXx 102pz12 pdet S12exp 12x m0S1x mz1 dGzwhere Z is a positive random variable with distribution function G. TheFigure 4.3. Bivariate student-t with 2 degrees of freedom and m 0, S 1 1212 1 .94 Chapter 4multivariate t distribution given in (4.21) can be obtained from this by taking G tobe the inverse gamma, given by,dGzdz 1212mG12mz12m1expfm=2zg:4.3.3. Examples of Ep(0, I)Example 4.3.3.1. Contaminated normal. Let X X1; . . . ; Xp0. The pdf of acontaminated normal is given byfXx a2p12pz2112p exp x0x2z21 1 a2p12p z2212pexp x0x2z22 with 0 a 1; z2i . 0; i 1; 2.Figure 4.4. Contours of bivariate student-t in Figure 4.3.Properties of Multivariate Distributions 95Example 4.3.3.2. Multivariate student-t with m degrees of freedom. Its pdf isgiven by (4.22).4.3.4. Basic Properties of Ep(m, S, q) and Ep(0, I, q)Theorem 4.3.4.1. Let X X1; . . . ; Xp0 be distributed as Epm; S; q. Then(a) EX m for all q,(b) CovX EX mX m0 KqS where Kq is a positive constantdependinq on q,(c) the correlation matrix R Rij withRij CovXi; XjvarXivarXj124:23for all members of Epm; S; q are identical.Proof.(a) Let Z C1X m where C is a p p nonsingular matrix such thatS CC0 and let Y X m. Since the Jacobian of the transformationY !Z is det C and qZ0Z is an even function of Z we getEX m Rpx mdet S12qx m0S1x m dxRpydet S12qy0S1y dy CRpzqz0z dz 0:Hence EX m for all q.(b)EX mX m0 Rpyy0det S12qy0S10y dy CRpzz0qz0z dz C0:96 Chapter 4Using the fact that qz0z is an even function of z z1; . . . ; zp0 weconclude thatEzizj p1Kq; for all i j;0; for all i =j;where Kq is a positive constant depending on q. HenceRpzz0qzz0 dz p1KqI 4:24where I is the p p identity matrix. This implies thatKq Rptrzz0qzz0 dz: 4:25LetL trZZ0; ei Z2iL ; i 1; . . . ; p:We will prove in Theorem 6.12.1 that L is independent of (e1; . . . ; ep) andthe joint pdf of (e1; . . . ; ep) is Dirichlet D12; . . . ;12