192
Matrices LMNOPQ From Wikipedia, the free encyclopedia
Matrices LMNOPQFrom Wikipedia, the free encyclopedia
Contents
1 Laplacian matrix 11.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Incidence matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.5 Deformed Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.6 Symmetric normalized Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.7 Random walk normalized Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.7.1 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.8 Interpretation as the discrete Laplace operator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.8.1 Equilibrium Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.8.2 Example of the Operator on a Grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.9 Approximation to the negative continuous Laplacian . . . . . . . . . . . . . . . . . . . . . . . . . 81.10 In Directed Multigraphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.11 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.12 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Leslie matrix 92.1 Stable age structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.2 The random Leslie case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.4 Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3 Levinson recursion 123.1 Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.1.2 Introductory steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1.3 Obtaining the backward vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.1.4 Using the backward vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Block Levinson algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
i
ii CONTENTS
4 List of matrices 174.1 Matrices with explicitly constrained entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.1.1 Constant matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.2 Matrices with conditions on eigenvalues or eigenvectors . . . . . . . . . . . . . . . . . . . . . . . 184.3 Matrices satisfying conditions on products or inverses . . . . . . . . . . . . . . . . . . . . . . . . 184.4 Matrices with specific applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.5 Matrices used in statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184.6 Matrices used in graph theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.7 Matrices used in science and engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194.8 Other matrix-related terms and definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.9 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.10 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204.11 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5 Logical matrix 215.1 Matrix representation of a relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
5.1.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.2 Other examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215.3 Some properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225.7 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
6 M-matrix 246.1 Characterizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246.2 Equivalences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
7 Magic square 277.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
7.1.1 Lo Shu square (3×3 magic square) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.1.2 Persia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297.1.3 Arabia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307.1.4 India . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307.1.5 Europe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327.1.6 Albrecht Dürer's magic square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337.1.7 Sagrada Família magic square . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
7.2 Types of construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347.2.1 Method for constructing a magic square of order 3 . . . . . . . . . . . . . . . . . . . . . . 35
CONTENTS iii
7.2.2 Method for constructing a magic square of odd order . . . . . . . . . . . . . . . . . . . . 367.2.3 A method of constructing a magic square of doubly even order . . . . . . . . . . . . . . . 367.2.4 Medjig-method of constructing magic squares of even number of rows . . . . . . . . . . . 387.2.5 Construction of panmagic squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 387.2.6 Construction similar to the Kronecker Product . . . . . . . . . . . . . . . . . . . . . . . . 387.2.7 The construction of a magic square using genetic algorithms . . . . . . . . . . . . . . . . . 39
7.3 Generalizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.3.1 Extra constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.3.2 Different constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.3.3 Multiplicative magic squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.3.4 Multiplicative magic squares of complex numbers . . . . . . . . . . . . . . . . . . . . . . 397.3.5 Additive-multiplicative magic and semimagic squares . . . . . . . . . . . . . . . . . . . . 397.3.6 Other magic shapes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.3.7 Other component elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.3.8 Combined extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7.4 Related problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.4.1 Magic square of primes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.4.2 n-Queens problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.4.3 Enumeration of magic squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
7.5 Magic squares in popular culture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407.6 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437.9 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437.10 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
8 Main diagonal 478.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
9 Manin matrix 489.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
9.1.1 Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489.1.2 Formal definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 489.1.3 Ubiquity of 2 × 2 Manin matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499.1.4 Conceptual definition. Concept of “non-commutative symmetries” . . . . . . . . . . . . 50
9.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519.2.1 Elementary examples and properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519.2.2 Determinant = column-determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 519.2.3 Linear algebra theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
9.3 Examples and applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 529.3.1 Capelli matrix as Manin matrix, and center of U(gln) . . . . . . . . . . . . . . . . . . . . 52
iv CONTENTS
9.3.2 Loop algebra for gln, Langlands correspondence and Manin matrix . . . . . . . . . . . . . 539.3.3 Yangian type matrices as Manin matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
9.4 Further questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539.4.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
9.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
10 Matrix analysis 5510.1 Matrix spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5510.2 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5610.3 Eigenvalues and eigenvectors of matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
10.3.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5610.3.2 Perturbations of eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
10.4 Matrix similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5610.4.1 Unitary similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
10.5 Canonical forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.5.1 Row echelon form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.5.2 Jordan normal form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.5.3 Weyr canonical form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.5.4 Frobenius normal form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
10.6 Triangular factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.6.1 LU decomposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
10.7 Matrix norms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.7.1 Definition and axioms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5710.7.2 Frobenius norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
10.8 Positive definite and semidefinite matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5810.9 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
10.9.1 Functions of matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5810.9.2 Matrix-valued functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
10.10See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5810.10.1 Other branches of analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5810.10.2 Other concepts of linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5910.10.3 Types of matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5910.10.4 Matrix functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
10.11Footnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5910.12References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
10.12.1 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5910.12.2 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
11 Matrix chain multiplication 6011.1 A Dynamic Programming Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6011.2 More Efficient Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
11.2.1 Hu & Shing (1981) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
CONTENTS v
11.3 Generalizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6211.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
12 Matrix congruence 6512.1 Congruence over the reals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6512.2 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6512.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
13 Matrix consimilarity 6713.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
14 Matrix difference equation 6814.1 Non-homogeneous first-order matrix difference equations and the steady state . . . . . . . . . . . . 6814.2 Stability of the first-order case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6914.3 Solution of the first-order case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6914.4 Extracting the dynamics of a single scalar variable from a first-order matrix system . . . . . . . . . 6914.5 Solution and stability of higher-order cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6914.6 Nonlinear matrix difference equations: Riccati equations . . . . . . . . . . . . . . . . . . . . . . . 7014.7 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7114.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
15 Matrix equivalence 7215.1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7215.2 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
16 Matrix group 7316.1 Basic examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7316.2 Classical groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7316.3 Finite groups as matrix groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7316.4 Representation theory and character theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7416.5 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7416.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7416.7 Further reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7416.8 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
17 Matrix of ones 7517.1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7517.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
18 Matrix regularization 7718.1 Basic definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7718.2 General applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
18.2.1 Matrix completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7818.2.2 Multivariate regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
vi CONTENTS
18.2.3 Multi-task learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7818.3 Spectral regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7918.4 Structured sparsity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7918.5 Multiple kernel selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8018.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
19 Matrix representation 8219.1 Basic mathematical operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8219.2 Basics of 2D array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8219.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8319.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8319.5 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
20 Matrix similarity 8420.1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8420.2 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8520.3 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8520.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
21 Matrix splitting 8621.1 Regular splittings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8621.2 Matrix iterative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8721.3 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
21.3.1 Regular splitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8721.3.2 Jacobi method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8821.3.3 Gauss-Seidel method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8821.3.4 Successive over-relaxation method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
21.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8921.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8921.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
22 Metzler matrix 9122.1 Definition and terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9122.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9122.3 Relevant theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9222.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9222.5 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
23 Modal matrix 9323.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9323.2 Generalized modal matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
23.2.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9423.3 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
CONTENTS vii
23.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
24 Moment matrix 9624.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9624.2 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9624.3 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
25 Moore determinant of a Hermitian matrix 9725.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9725.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
26 Moore matrix 9826.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9826.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
27 Mueller calculus 10027.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10027.2 Mueller vs. Jones calculi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10027.3 Mueller matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10127.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10227.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
28 Network operator matrix 10328.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10328.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
29 Next-generation matrix 10629.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10729.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10729.3 Sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
30 Nilpotent matrix 10830.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10830.2 Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10930.3 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10930.4 Flag of subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11030.5 Additional properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11030.6 Generalizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11130.7 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11130.8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11130.9 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
31 Nonnegative matrix 11231.1 Inversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
viii CONTENTS
31.2 Specializations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11231.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11231.4 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
32 Normal matrix 11432.1 Special cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11432.2 Consequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11432.3 Equivalent definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11532.4 Analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11632.5 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11632.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
33 Orbital overlap 11733.1 Overlap matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11733.2 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11833.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
34 Orthogonal matrix 11934.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11934.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12034.3 Elementary constructions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
34.3.1 Lower dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12034.3.2 Higher dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12134.3.3 Primitives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
34.4 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12234.4.1 Matrix properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12234.4.2 Group properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12234.4.3 Canonical form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12334.4.4 Lie algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
34.5 Numerical linear algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12434.5.1 Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12434.5.2 Decompositions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12434.5.3 Randomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12534.5.4 Nearest orthogonal matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
34.6 Spin and pin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12634.7 Rectangular matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12634.8 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12634.9 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12734.10References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12734.11External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
35 Orthostochastic matrix 12835.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
CONTENTS ix
36 P-matrix 12936.1 Spectra of P -matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12936.2 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12936.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12936.4 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13036.5 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
37 Packed storage matrix 13137.1 Code examples (Fortran) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
38 Paley construction 13238.1 Quadratic character and Jacobsthal matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13238.2 Paley construction I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13238.3 Paley construction II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13338.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13338.5 The Hadamard conjecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13338.6 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13438.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
39 Parry–Sullivan invariant 13539.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13539.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13539.3 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
40 Pascal matrix 13640.1 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13640.2 Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13740.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13840.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13840.5 External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
41 Pauli matrices 13941.1 Algebraic properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
41.1.1 Eigenvectors and eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14041.1.2 Pauli vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14041.1.3 Commutation relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14141.1.4 Relation to dot and cross product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14141.1.5 Exponential of a Pauli vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14141.1.6 Completeness relation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14341.1.7 Relation with the permutation operator . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
41.2 SU(2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14441.2.1 SO(3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14441.2.2 Quaternions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
x CONTENTS
41.3 Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14541.3.1 Quantum mechanics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14541.3.2 Quantum information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
41.4 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14541.5 Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14641.6 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14641.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
42 Perfect matrix 14742.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
43 Permutation matrix 14843.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14943.2 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14943.3 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15043.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
43.4.1 Permutation of rows and columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15043.4.2 Permutation of rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
43.5 Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15143.6 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15143.7 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
44 Persymmetric matrix 15244.1 Definition 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15244.2 Definition 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15244.3 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15344.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
45 Polyconvex function 15545.1 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
46 Polynomial matrix 15646.1 Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15646.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
47 Positive-definite matrix 15747.1 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15747.2 Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15847.3 Characterizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15847.4 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15947.5 Simultaneous diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15947.6 Negative-definite, semidefinite and indefinite matrices . . . . . . . . . . . . . . . . . . . . . . . . 159
47.6.1 Negative-definite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16047.6.2 Positive-semidefinite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
CONTENTS xi
47.6.3 Negative-semidefinite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16047.6.4 Indefinite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
47.7 Further properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16047.8 Block matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16247.9 On the definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
47.9.1 Consistency between real and complex definitions . . . . . . . . . . . . . . . . . . . . . . 16247.9.2 Extension for non symmetric matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
47.10See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16347.11Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16347.12References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16347.13External links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164
48 Productive matrix 16548.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16548.2 Explicit definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16548.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16548.4 Properties*[2] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
48.4.1 Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16548.4.2 Transposition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
48.5 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16748.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
49 Pseudo-determinant 16849.1 Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16849.2 Definition of pseudo determinant using Vahlen Matrix . . . . . . . . . . . . . . . . . . . . . . . . 16849.3 Computation for positive semi-definite case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16849.4 Application in statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16849.5 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16949.6 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
50 Q-matrix 17050.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17050.2 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
51 Quaternionic matrix 17151.1 Matrix operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17151.2 Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17251.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17251.4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
52 Quincunx matrix 17352.1 See also . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17352.2 Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
xii CONTENTS
52.3 Text and image sources, contributors, and licenses . . . . . . . . . . . . . . . . . . . . . . . . . . 17452.3.1 Text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17452.3.2 Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17752.3.3 Content license . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179
Chapter 1
Laplacian matrix
In the mathematical field of graph theory, the Laplacian matrix, sometimes called admittance matrix, Kirchhoffmatrix or discrete Laplacian, is a matrix representation of a graph. Together with Kirchhoff's theorem, it can beused to calculate the number of spanning trees for a given graph. The Laplacian matrix can be used to find manyother properties of the graph. Cheeger's inequality from Riemannian geometry has a discrete analogue involving theLaplacian matrix; this is perhaps the most important theorem in spectral graph theory and one of the most usefulfacts in algorithmic applications. It approximates the sparsest cut of a graph through the second eigenvalue of itsLaplacian.
1.1 Definition
Given a simple graph G with n vertices, its Laplacian matrix Ln×n is defined as:*[1]
L = D −A,
where D is the degree matrix and A is the adjacency matrix of the graph. In the case of directed graphs, either theindegree or outdegree might be used, depending on the application.The elements of L are given by
Li,j :=
deg(vi) if i = j
−1 if i = j and vi is adjacent to vj0 otherwise
where deg(vi) is degree of the vertex i.The symmetric normalized Laplacian matrix is defined as:*[1]
Lsym := D−1/2LD−1/2 = I −D−1/2AD−1/2
The elements of Lsym are given by
Lsymi,j :=
1 if i = j and deg(vi) = 0
− 1√deg(vi) deg(vj)
if i = j and vi is adjacent to vj0 otherwise.
The random-walk normalized Laplacian matrix is defined as:
1
2 CHAPTER 1. LAPLACIAN MATRIX
Lrw := D−1L = I −D−1A
The elements of Lrw are given by
Lrwi,j :=
1 if i = j and deg(vi) = 0
− 1deg(vi) if i = j and vi is adjacent to vj
0 otherwise.
1.2 Example
Here is a simple example of a labeled graph and its Laplacian matrix.
1.3 Properties
For an (undirected) graph G and its Laplacian matrix L with eigenvalues λ0 ≤ λ1 ≤ · · · ≤ λn−1 :
• L is symmetric.
• L is positive-semidefinite (that is λi ≥ 0 for all i). This is verified in the incidence matrix section (below). Thiscan also be seen from the fact that the Laplacian is symmetric and diagonally dominant.
• L is an M-matrix (its off-diagonal entries are nonpositive, yet the real parts of its eigenvalues are nonnegative).
• Every row sum and column sum of L is zero. Indeed, in the sum, the degree of the vertex is summed with a"−1”for each neighbor.
• In consequence, λ0 = 0 , because the vector v0 = (1, 1, . . . , 1) satisfies Lv0 = 0.
• The number of times 0 appears as an eigenvalue in the Laplacian is the number of connected components inthe graph.
• The smallest non-zero eigenvalue of L is called the spectral gap.
• The second smallest eigenvalue of L is the algebraic connectivity (or Fiedler value) of G.
• The Laplacian is an operator on the n-dimensional vector space of functions f : V → R , where V is the vertexset of G, and n = |V|.
• When G is k-regular, the normalized Laplacian is: L = 1kL = I − 1
kA , where A is the adjacency matrix andI is an identity matrix.
• For a graph with multiple connected components, L is a block diagonal matrix, where each block is the respec-tive Laplacian matrix for each component, possibly after reordering the vertices (i.e. L is permutation-similarto a block diagonal matrix).
• Laplacian matrix is singular.
1.4 Incidence matrix
Define an |e| x |v| oriented incidence matrix M with element Mev for edge e (connecting vertex i and j, with i > j)and vertex v given by
Mev =
1, if v = i−1, if v = j0, otherwise.
1.5. DEFORMED LAPLACIAN 3
Then the Laplacian matrix L satisfies
L =MTM ,
where MT is the matrix transpose of M.Now consider an eigendecomposition of L , with unit-norm eigenvectors vi and corresponding eigenvalues λi :
λi = vTi Lvi= vTi MTMvi= (Mvi)T (Mvi).
Because λi can be written as the inner product of the vector Mvi with itself, this shows that λi ≥ 0 and so theeigenvalues of L are all non-negative.
1.5 Deformed Laplacian
The deformed Laplacian is commonly defined as
∆(s) = I − sA+ s2(D − I)
where I is the unit matrix, A is the adjacency matrix, and D is the degree matrix, and s is a (complex-valued) number.Note that the standard Laplacian is just ∆(1) .*[2]
1.6 Symmetric normalized Laplacian
The (symmetric) normalized Laplacian is defined as
Lsym := D−1/2LD−1/2 = I −D−1/2AD−1/2
where L is the (unnormalized) Laplacian, A is the adjacency matrix and D is the degree matrix. Since the degreematrix D is diagonal and positive, its reciprocal square root D−1/2 is just the diagonal matrix whose diagonal entriesare the reciprocals of the positive square roots of the diagonal entries of D. The symmetric normalized Laplacian isa symmetric matrix.One has: Lsym = SS∗ , where S is the matrix whose rows are indexed by the vertices and whose columns areindexed by the edges of G such that each column corresponding to an edge e = u, v has an entry 1√
duin the row
corresponding to u, an entry − 1√dv
in the row corresponding to v, and has 0 entries elsewhere. (Note: S∗ denotesthe transpose of S).All eigenvalues of the normalized Laplacian are real and non-negative. We can see this as follows. Since Lsym issymmetric, its eigenvalues are real. They are also non-negative: consider an eigenvector g of Lsym with eigenvalue λand suppose g = D1/2f . (We can consider g and f as real functions on the vertices v.) Then:
λ =⟨g, Lsymg⟩⟨g, g⟩
=⟨g,D−1/2LD−1/2g⟩
⟨g, g⟩=
⟨f, Lf⟩⟨D1/2f,D1/2f⟩
=
∑u∼v(f(u)− f(v))2∑
v f(v)2dv
> 0,
where we use the inner product ⟨f, g⟩ =∑v f(v)g(v) , a sum over all vertices v, and
∑u∼v denotes the sum over
all unordered pairs of adjacent vertices u,v. The quantity∑u,v(f(u) − f(v))2 is called the Dirichlet sum of f,
whereas the expression ⟨g,Lsymg⟩⟨g,g⟩ is called the Rayleigh quotient of g.
4 CHAPTER 1. LAPLACIAN MATRIX
Let 1 be the function which assumes the value 1 on each vertex. Then D1/21 is an eigenfunction of Lsym witheigenvalue 0.*[3]In fact, the eigenvalues of the normalized symmetric Laplacian satisfy 0 = μ0≤...≤ μn-1≤ 2. These eigenvalues (knownas the spectrum of the normalized Laplacian) relate well to other graph invariants for general graphs.*[4]
1.7 Random walk normalized Laplacian
The random walk normalized Laplacian is defined as
Lrw := D−1A
where A is the Adjacency matrix and D is the degree matrix. Since the degree matrix D is diagonal, its inverse D−1
is simply defined as a diagonal matrix, having diagonal entries which are the reciprocals of the corresponding positivediagonal entries of D. For the isolated vertices (those with degree 0), a common choice is to set the correspondingelement Lrw
i,i to 0. This convention results in a nice property that the multiplicity of the eigenvalue 0 is equal to thenumber of connected components in the graph. The matrix elements of Lrw are given by
Lrwi,j :=
1 if i = j and deg(vi) = 0
− 1deg(vi) if i = j and vi is adjacent to vj
0 otherwise.
The name of the random-walk normalized Laplacian comes from the fact that this matrix is simply the transitionmatrix of a random walker on the graph. For example let ei denote the i-th standard basis vector, then x = eiL
rw
is a probability vector representing the distribution of a random-walker's locations after taking a single step fromvertex i . i.e. xj = P(vi → vj) . More generally if the vector x is a probability distribution of the location of arandom-walker on the vertices of the graph then x′ = x(Lrw)t is the probability distribution of the walker after tsteps.One can check that
Lrw = D− 12 (I − Lsym)D
12
i.e., Lrw is similar to the normalized Laplacian Lsym . For this reason, even if Lrw is in general not hermitian, it hasreal eigenvalues. Indeed, its eigenvalues agree with those of Lsym (which is hermitian) up to a reflection about 1/2.In some of the literature, the matrix I −D−1A is also referred to as the random-walk Laplacian since its propertiesapproximate those of the standard discrete Laplacian from numerical analysis.
1.7.1 Graphs
As an aside about random walks on graphs, consider a simple undirected graph. Consider the probability that thewalker is at the vertex i at time t, given the probability distribution that he was at vertex j at time t-1 (assuming auniform chance of taking a step along any of the edges attached to a given vertex):
pi(t) =∑j
Aijdeg(vj)
pj(t− 1),
or in matrix-vector notation:
p(t) = AD−1p(t− 1).
(Equilibrium, which sets in as t→∞ , is defined by p = AD−1p .)
1.8. INTERPRETATION AS THE DISCRETE LAPLACE OPERATOR 5
We can rewrite this relation as
D− 12 p(t) =
[D− 1
2AD− 12
]D− 1
2 p(t− 1).
Areduced ≡ D− 12AD− 1
2 is a symmetric matrix called the reduced adjacency matrix. So, taking steps on thisrandom walk requires taking powers of Areduced , which is a simple operation because Areduced is symmetric.
1.8 Interpretation as the discrete Laplace operator
The Laplacian matrix can be interpreted as a matrix representation of a particular case of the discrete Laplace oper-ator. Such an interpretation allows one, e.g., to generalise the Laplacian matrix to the case of graphs with an infinitenumber of vertices and edges, leading to a Laplacian matrix of an infinite size.To expand upon this, we can “describe”the change of some element ϕi (with some constant k) as
dϕidt
= −k∑j
Aij(ϕi − ϕj)
= −kϕi∑j
Aij + k∑j
Aijϕj
= −kϕi deg(vi) + k∑j
Aijϕj
= −k∑j
(δij deg(vi)−Aij)ϕj
= −k∑j
(ℓij)ϕj .
In matrix-vector notation,
dϕ
dt= −k(D −A)ϕ
= −kLϕ,
which gives
dϕ
dt+ kLϕ = 0.
Notice that this equation takes the same form as the heat equation, where the matrix L is replacing the Laplacianoperator∇2 ; hence, the “graph Laplacian”.To find a solution to this differential equation, apply standard techniques for solving a first-order matrix differentialequation. That is, write ϕ as a linear combination of eigenvectors vi of L (so that Lvi = λivi ), with time-dependentϕ =
∑i
civi.
Plugging into the original expression (note that we will use the fact that because L is a symmetric matrix, its unit-normeigenvectors vi are orthogonal):
6 CHAPTER 1. LAPLACIAN MATRIX
d(∑i civi)dt
+ kL(∑i
civi) = 0
∑i
[dcidtvi + kciLvi
]=
∑i
[dcidtvi + kciλivi
]=
dcidt
+ kλici = 0,
whose solution is
ci(t) = ci(0) exp(−kλit).
As shown before, the eigenvalues λi of L are non-negative, showing that the solution to the diffusion equation ap-proaches an equilibrium, because it only exponentially decays or remains constant. This also shows that given λi andthe initial condition ci(0) , the solution at any time t can be found.*[5]To find ci(0) for each i in terms of the overall initial condition ϕ(0) , simply project ϕ(0) onto the unit-norm eigen-vectors vi ;ci(0) = ⟨ϕ(0), vi⟩ .In the case of undirected graphs, this works because L is symmetric, and by the spectral theorem, its eigenvectorsare all orthogonal. So the projection onto the eigenvectors of L is simply an orthogonal coordinate transformation ofthe initial condition to a set of coordinates which decay exponentially and independently of each other.
1.8.1 Equilibrium Behavior
To understand limt→∞ ϕ(t) , note that the only terms ci(t) = ci(0) exp(−kλit) that remain are those where λi = 0, since
limt→∞ exp(−kλit) =
0 if λi > 01 if λi = 0
In other words, the equilibrium state of the system is determined completely by the kernel of L . Since by definition,∑j Lij = 0 , the vector v1 of all ones is in the kernel. Note also that if there are k disjoint connected components
in the graph, then this vector of all ones can be split into the sum of k independent λ = 0 eigenvectors of ones andzeros, where each connected component corresponds to an eigenvector with ones at the elements in the connectedcomponent and zeros elsewhere.The consequence of this is that for a given initial condition c(0) for a graph with N verticeslimt→∞ ϕ(t) = ⟨c(0), v1⟩v1
wherev1 = 1√
N[1, 1, ..., 1]
For each element ϕj of ϕ , i.e. for each vertex j in the graph, it can be rewritten as
limt→∞ ϕj(t) =1N
∑Ni=1 ci(0) .
In other words, at steady state, the value of ϕ converges to the same value at each of the vertices of the graph, whichis the average of the initial values at all of the vertices. Since this is the solution to the heat diffusion equation, thismakes perfect sense intuitively. We expect that neighboring elements in the graph will exchange energy until thatenergy is spread out evenly throughout all of the elements that are connected to each other.
1.8.2 Example of the Operator on a Grid
This section shows an example of a function ϕ diffusing over time through a graph. The graph in this example isconstructed on a 2D discrete grid, with points on the grid connected to their eight neighbors. Three initial points are
1.8. INTERPRETATION AS THE DISCRETE LAPLACE OPERATOR 7
This GIF shows the progression of diffusion, as solved by the graph laplacian technique. A graph is constructed over a grid, whereeach pixel in the graph is connected to its 8 bordering pixels. Values in the image then diffuse smoothly to their neighbors over timevia these connections. This particular image starts off with three strong point values which spill over to their neighbors slowly. Thewhole system eventually settles out to the same value at equilibrium.
specified to have a positive value, while the rest of the values in the grid are zero. Over time, the exponential decayacts to distribute the values at these points evenly throughout the entire grid.The complete Matlab source code that was used to generate this animation is provided below. It shows the processof specifying initial conditions, projecting these initial conditions onto the eigenvalues of the Laplacian Matrix, andsimulating the exponential decay of these projected initial conditions.N = 20;%The number of pixels along a dimension of the image A = zeros(N, N);%The image Adj = zeros(N*N,N*N);%The adjacency matrix %Use 8 neighbors, and fill in the adjacency matrix dx = [−1, 0, 1, −1, 1, −1, 0, 1];dy = [−1, −1, −1, 0, 0, 1, 1, 1]; for x = 1:N for y = 1:N index = (x-1)*N + y; for ne = 1:length(dx) newx = x +dx(ne); newy = y + dy(ne); if newx > 0 && newx <= N && newy > 0 && newy <= N index2 = (newx-1)*N + newy;Adj(index, index2) = 1; end end end end %%%BELOW IS THE KEY CODE THAT COMPUTES THE SOLUTIONTO THE DIFFERENTIAL %%%EQUATION Deg = diag(sum(Adj, 2));%Compute the degree matrix L = Deg -Adj;%Compute the laplacian matrix in terms of the degree and adjacency matrices [V, D] = eig(L);%Compute theeigenvalues/vectors of the laplacian matrix D = diag(D); %Initial condition (place a few large positive values aroundand %make everything else zero) C0 = zeros(N, N); C0(2:5, 2:5) = 5; C0(10:15, 10:15) = 10; C0(2:5, 8:13) = 7;C0 = C0(:); C0V = V'*C0;%Transform the initial condition into the coordinate system %of the eigenvectors for t= 0:0.05:5 %Loop through times and decay each initial component Phi = C0V.*exp(-D*t);%Exponential decay foreach component Phi = V*Phi;%Transform from eigenvector coordinate system to original coordinate system Phi =reshape(Phi, N, N); %Display the results and write to GIF file imagesc(Phi); caxis([0, 10]); title(sprintf('Diffusion t= %3f', t)); frame = getframe(1); im = frame2im(frame); [imind, cm] = rgb2ind(im, 256); if t == 0 imwrite(imind,cm, 'out.gif', 'gif', 'Loopcount', inf, 'DelayTime', 0.1); else imwrite(imind, cm, 'out.gif', 'gif', 'WriteMode', 'append','DelayTime', 0.1); end end
8 CHAPTER 1. LAPLACIAN MATRIX
1.9 Approximation to the negative continuous Laplacian
The graph Laplacian matrix can be further viewed as a matrix form of an approximation to the (positive semi-definite)Laplacian operator obtained by the finite difference method.*[6] In this interpretation, every graph vertex is treatedas a grid point; the local connectivity of the vertex determines the finite difference approximation stencil at this gridpoint, the grid size is always one for every edge, and there are no constraints on any grid points, which correspondsto the case of the homogeneous Neumann boundary condition, i.e., free boundary.
1.10 In Directed Multigraphs
An analogue of the Laplacian matrix can be defined for directed multigraphs.*[7] In this case the Laplacian matrixL is defined as
L = D −A
where D is a diagonal matrix with Di,i equal to the outdegree of vertex i and A is a matix with Ai,j equal to the numberof edges from i to j (including loops).
1.11 See also• Stiffness matrix
• Resistance distance
1.12 References[1] Weisstein, Eric W., “Laplacian Matrix”, MathWorld.
[2]“The Deformed Consensus Protocol”, F. Morbidi, Automatica, vol. 49, n. 10, pp. 3049-3055, October 2013.
[3] Chung, Fan R.K. (1997). Spectral graph theory (Repr. with corr., 2. [pr.] ed.). Providence, RI: American Math. Soc.ISBN 0-8218-0315-8.
[4] Chung, Fan (1997) [1992]. Spectral Graph Theory. American Mathematical Society. ISBN 0821803158.
[5] Newman, Mark (2010). Networks: An Introduction. Oxford University Press. ISBN 0199206651.
[6] Smola, Alexander J.; Kondor, Risi (2003),“Kernels and regularization on graphs”, Learning Theory and Kernel Machines:16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August24-27, 2003, Proceedings, Lecture Notes in Computer Science 2777, Springer, pp. 144–158, doi:10.1007/978-3-540-45167-9_12.
[7] Chaiken, S. and Kleitman, D. (1978). “Matrix Tree Theorems”. Journal of Combinatorial Theory, Series A 24 (3): 377– 381. ISSN 0097-3165.
• T. Sunada, Discrete geometric analysis, Proceedings of Symposia in Pure Mathematics, (ed. by P. Exner, J. P.Keating, P. Kuchment, T. Sunada, A. Teplyaev), 77 (2008), 51-86.
• B. Bollobaás, Modern Graph Theory, Springer-Verlag (1998, corrected ed. 2013), ISBN 0-387-98488-7,Chapters II.3 (Vector Spaces and Matrices Associated with Graphs), VIII.2 (The Adjacency Matrix and theLaplacian), IX.2 (Electrical Networks and Random Walks).
Chapter 2
Leslie matrix
In applied mathematics, the Leslie matrix is a discrete, age-structured model of population growth that is verypopular in population ecology. It was invented by and named after Patrick H. Leslie. The Leslie matrix (also calledthe Leslie model) is one of the most well known ways to describe the growth of populations (and their projected agedistribution), in which a population is closed to migration, growing in an unlimited environment, and where only onesex, usually the female, is considered.The Leslie matrix is used in ecology to model the changes in a population of organisms over a period of time. In aLeslie model, the population is divided into groups based on age classes. A similar model which replaces age classeswith life stage is called a Lefkovitch matrix, whereby individuals can both remain in the same stage class or move onto the next one. At each time step, the population is represented by a vector with an element for each age class whereeach element indicates the number of individuals currently in that class.The Leslie matrix is a square matrix with the same number of rows and columns as the population vector has elements.The (i,j)th cell in the matrix indicates how many individuals will be in the age class i at the next time step for eachindividual in stage j. At each time step, the population vector is multiplied by the Leslie matrix to generate thepopulation vector for the following time step.To build a matrix, some information must be known from the population:
• nx , the count of individuals (n) of each age class x
• sx , the fraction of individuals that survives from age class x to age class x+1,
• fx , fecundity, the per capita average number of female offspring reaching n0 born from mother of the age classx. More precisely, it can be viewed as the number of offspring produced at the next age class bx+1 weightedby the probability of reaching the next age class. Therefore fx = sxbx+1.
From the observations that n0 at time t+1 is simply the sum of all offspring born from the previous time step and thatthe organisms surviving to time t+1 are the organisms at time t surviving at probability sx , one gets nx+1 = sxnx .This then motivates the following matrix representation:
n0n1...
nω−1
t+1
=
f0 f1 f2 . . . fω−2 fω−1
s0 0 0 . . . 0 00 s1 0 . . . 0 00 0 s2 . . . 0 0...
...... . . . ...
...0 0 0 . . . sω−2 0
n0n1...
nω−1
t
where ω is the maximum age attainable in the population.This can be written as:
nt+1 = Lnt
9
10 CHAPTER 2. LESLIE MATRIX
or:
nt = Ltn0
where nt is the population vector at time t andL is the Leslie matrix. The dominant eigenvalue ofL , denoted λ , givesthe population's asymptotic growth rate (growth rate at the stable age distribution). The corresponding eigenvectorprovides the stable age distribution, the proportion of individuals of each age within the population. Once the stableage distribution has been reached, a population undergoes exponential growth at rate λ .The characteristic polynomial of the matrix is given by the Euler–Lotka equation.The Leslie model is very similar to a discrete-time Markov chain. The main difference is that in a Markov model,one would have fx + sx = 1 for each x , while the Leslie model may have these sums greater or less than 1.
2.1 Stable age structure
This age-structured growth model suggests a steady-state, or stable, age-structure and growth rate. Regardless of theinitial population size, N0 , or age distribution, the population tends asymptotically to this age-structure and growthrate. It also returns to this state following perturbation. The Euler–Lotka equation provides a means of identifying theintrinsic growth rate. The stable age-structure is determined both by the growth rate and the survival function (i.e. theLeslie matrix). For example, a population with a large intrinsic growth rate will have a disproportionately “young”age-structure. A population with high mortality rates at all ages (i.e. low survival) will have a similar age-structure.Charlesworth (1980) provides further details on the rate and form of convergence to the stable age-structure.
2.2 The random Leslie case
To generalize the concept of the population growth rate when a Leslie matrix has random elements (correlated ornot), i.e., characterizing the disorder (uncertainties) in vital parameters, a perturbative formalism to deal with linearnon-negative random matrix difference equations have to be used. Then the non-trivial effective eigenvalue of whichdefines the long-time asymptotic dynamics of the mean-value population vector state, can be presented as the effectivegrowth rate. This effective eigenvalue and the associated mean value invariant vector state can be calculated from thesmallest positive root of a secular polynomial and the residue of the mean-value Green function. Analytical (exactand perturbative calculations) results can be presented for several models of disorder.
2.3 References• Caceres, M.O. and Caceres-Saez, I. 2011. Random Leslie Matrices in Population Dynamics, Journal of Math-
ematical Biology, Vol.63, N.3, 519-556; [DOI 10.1007/s00285-010-0378-0].
• Caceres, M.O. and Caceres-Saez, I. 2013. Calculating effective growth rate from random Leslie model: Appli-cation to incidental mortality analysis, Ecological Modelling, 251, 312-322; [DOI: 10.1016/j.ecolmodel.2012.12.021]
2.4 Sources• Krebs CJ (2001) Ecology: the experimental analysis of distribution and abundance (5th edition). San Francisco.
Benjamin Cummings.
• Charlesworth, B. (1980) Evolution in age-structured population. Cambridge. Cambridge University Press
• Leslie, P.H. (1945) “The use of matrices in certain population mathematics”. Biometrika, 33(3), 183–212.
• Leslie, P.H. (1948) “Some further notes on the use of matrices in population mathematics”. Biometrika,35(3–4), 213–245.
• Lotka, A.J. (1956) Elements of mathematical biology. New York. Dover Publications Inc.
2.4. SOURCES 11
• Kot, M. (2001) Elements of Mathematical Ecology, Cambridge. Cambridge University Press.
Chapter 3
Levinson recursion
Levinson recursion or Levinson–Durbin recursion is a procedure in linear algebra to recursively calculate thesolution to an equation involving a Toeplitz matrix. The algorithm runs in Θ(n2) time, which is a strong improvementover Gauss–Jordan elimination, which runs in Θ(n3).The Levinson–Durbin algorithm was proposed first by Norman Levinson in 1947, improved by James Durbin in1960, and subsequently improved to 4n2 and then 3n2 multiplications by W. F. Trench and S. Zohar, respectively.Other methods to process data include Schur decomposition and Cholesky decomposition. In comparison to these,Levinson recursion (particularly split Levinson recursion) tends to be faster computationally, but more sensitive tocomputational inaccuracies like round-off errors.The Bareiss algorithm for Toeplitz matrices (not to be confused with the general Bareiss algorithm) runs about asfast as Levinson recursion, but it uses O(n2) space, whereas Levinson recursion uses only O(n) space. The Bareissalgorithm, though, is numerically stable,*[1]*[2] whereas Levinson recursion is at best only weakly stable (i.e. itexhibits numerical stability for well-conditioned linear systems).*[3]Newer algorithms, called asymptotically fast or sometimes superfast Toeplitz algorithms, can solve in Θ(n log*pn)for various p (e.g. p = 2,*[4]*[5] p = 3 *[6]). Levinson recursion remains popular for several reasons; for one, it isrelatively easy to understand in comparison; for another, it can be faster than a superfast algorithm for small n (usuallyn < 256).*[7]
3.1 Derivation
3.1.1 Background
Matrix equations follow the form:
y =M x.
The Levinson–Durbin algorithm may be used for any such equation, as long as M is a known Toeplitz matrix with anonzero main diagonal. Here y is a known vector, and x is an unknown vector of numbers xi yet to be determined.For the sake of this article, ê i is a vector made up entirely of zeroes, except for its ith place, which holds the valueone. Its length will be implicitly determined by the surrounding context. The term N refers to the width of the matrixabove – M is an N×N matrix. Finally, in this article, superscripts refer to an inductive index, whereas subscriptsdenote indices. For example (and definition), in this article, the matrix T*n is an n×n matrix which copies the upperleft n×n block from M – that is, T*nij = M ij .T*n is also a Toeplitz matrix; meaning that it can be written as:
12
3.1. DERIVATION 13
Tn =
t0 t−1 t−2 . . . t−n+1
t1 t0 t−1 . . . t−n+2
t2 t1 t0 . . . t−n+3
......
... . . . ...tn−1 tn−2 tn−3 . . . t0
.
3.1.2 Introductory steps
The algorithm proceeds in two steps. In the first step, two sets of vectors, called the forward and backward vectors,are established. The forward vectors are used to help get the set of backward vectors; then they can be immediatelydiscarded. The backwards vectors are necessary for the second step, where they are used to build the solution desired.Levinson–Durbin recursion defines the n*th“forward vector”, denoted fn , as the vector of length n which satisfies:
Tnfn = e1.
The n*th “backward vector”bn is defined similarly; it is the vector of length n which satisfies:
Tnbn = en.
An important simplification can occur when M is a symmetric matrix; then the two vectors are related by b*ni =f*nn+1−i—that is, they are row-reversals of each other. This can save some extra computation in that special case.
3.1.3 Obtaining the backward vectors
Even if the matrix is not symmetric, then the n*th forward and backward vector may be found from the vectors oflength n − 1 as follows. First, the forward vector may be extended with a zero to obtain:
Tn[fn−1
0
]=
t−n+1
Tn−1 t−n+2
...tn−1 tn−2 . . . t0
fn−1
0
=
10...0ϵnf
.
In going from T*n−1 to T*n, the extra column added to the matrix does not perturb the solution when a zero is used toextend the forward vector. However, the extra row added to the matrix has perturbed the solution; and it has createdan unwanted error term εf which occurs in the last place. The above equation gives it the value of:
ϵnf =n−1∑i=1
Mni fn−1i =
n−1∑i=1
tn−i fn−1i .
This error will be returned to shortly and eliminated from the new forward vector; but first, the backwards vectormust be extended in a similar (albeit reversed) fashion. For the backwards vector,
Tn[
0
bn−1
]=
t0 . . . t−n+2 t−n+1
...tn−2 Tn−1
tn−1
0
bn−1
=
ϵnb0...01
.
14 CHAPTER 3. LEVINSON RECURSION
As before, the extra column added to the matrix does not perturb this new backwards vector; but the extra row does.Here we have another unwanted error εb with value:
ϵnb =n∑i=2
M1i bn−1i−1 =
n−1∑i=1
t−i bn−1i .
These two error terms can be used to eliminate each other. Using the linearity of matrices,
∀(α, β) T
αf0
+ β
0b
= α
10...0ϵf
+ β
ϵb0...01
.If α and β are chosen so that the right hand side yields ê1 or ên, then the quantity in the parentheses will fulfill thedefinition of the n*th forward or backward vector, respectively. With those alpha and beta chosen, the vector sum inthe parentheses is simple and yields the desired result.To find these coefficients, αnf , βnf are such that :
fn = αnf
[fn−1
0
]+ βnf
[0
bn−1
]and respectively αnb , βnb are such that :
bn = αnb
[fn−1
0
]+ βnb
[0
bn−1
].
By multiplying both previous equations by Tn one gets the following equation:
1 ϵnb0 0...
...0 0ϵnf 1
[αnf αnbβnf βnb
]=
1 00 0...
...0 00 1
.Now, all the zeroes in the middle of the two vectors above being disregarded and collapsed, only the following equationis left:
[1 ϵnbϵnf 1
][αnf αnbβnf βnb
]=
[1 00 1
].
With these solved for (by using the Cramer 2×2 matrix inverse formula), the new forward and backward vectors are:
fn =1
1− ϵnb ϵnf
[fn−1
0
]−
ϵnf1− ϵnb ϵnf
[0
bn−1
]
bn =1
1− ϵnb ϵnf
[0
bn−1
]− ϵnb
1− ϵnb ϵnf
[fn−1
0
].
Performing these vector summations, then, gives the n*th forward and backward vectors from the prior ones. All thatremains is to find the first of these vectors, and then some quick sums and multiplications give the remaining ones.The first forward and backward vectors are simply:
f1 = b1 =[
1M11
]=[1t0
].
3.2. BLOCK LEVINSON ALGORITHM 15
3.1.4 Using the backward vectors
The above steps give the N backward vectors for M. From there, a more arbitrary equation is:
y =M x.
The solution can be built in the same recursive way that the backwards vectors were built. Accordingly, x must begeneralized to a sequence xn , from which xN = x .The solution is then built recursively by noticing that if
Tn−1
xn−11
xn−12...
xn−1n−1
=
y1y2...
yn−1
.Then, extending with a zero again, and defining an error constant where necessary:
Tn
xn−11
xn−12...
xn−1n−1
0
=
y1y2...
yn−1
ϵn−1x
.
We can then use the n*th backward vector to eliminate the error term and replace it with the desired formula asfollows:
Tn
xn−11
xn−12...
xn−1n−1
0
+ (yn − ϵn−1x ) bn
=
y1y2...
yn−1
yn
.
Extending this method until n = N yields the solution x .In practice, these steps are often done concurrently with the rest of the procedure, but they form a coherent unit anddeserve to be treated as their own step.
3.2 Block Levinson algorithm
If M is not strictly Toeplitz, but block Toeplitz, the Levinson recursion can be derived in much the same way byregarding the block Toeplitz matrix as a Toeplitz matrix with matrix elements (Musicus 1988). Block Toeplitzmatrices arise naturally in signal processing algorithms when dealing with multiple signal streams (e.g., in MIMOsystems) or cyclo-stationary signals.
3.3 See also• Split Levinson recursion
• Linear prediction
• Autoregressive model
16 CHAPTER 3. LEVINSON RECURSION
3.4 Notes[1] Bojanczyk et al. (1995).
[2] Brent (1999).
[3] Krishna & Wang (1993).
[4] http://www.maths.anu.edu.au/~brent/pd/rpb143tr.pdf
[5] http://etd.gsu.edu/theses/available/etd-04182008-174330/unrestricted/kimitei_symon_k_200804.pdf
[6] http://web.archive.org/web/20070418074240/http://saaz.cs.gsu.edu/papers/sfast.pdf
[7] http://www.math.niu.edu/~ammar/papers/amgr88.pdf
3.5 References
Defining sources
• Levinson, N. (1947). “The Wiener RMS error criterion in filter design and prediction.”J. Math. Phys., v.25, pp. 261–278.
• Durbin, J. (1960). “The fitting of time series models.”Rev. Inst. Int. Stat., v. 28, pp. 233–243.
• Trench, W. F. (1964). “An algorithm for the inversion of finite Toeplitz matrices.”J. Soc. Indust. Appl.Math., v. 12, pp. 515–522.
• Musicus, B. R. (1988).“Levinson and Fast Choleski Algorithms for Toeplitz and Almost Toeplitz Matrices.”RLE TR No. 538, MIT.
• Delsarte, P. and Genin, Y. V. (1986).“The split Levinson algorithm.”IEEE Transactions on Acoustics, Speech,and Signal Processing, v. ASSP-34(3), pp. 470–478.
Further work
• Bojanczyk, A.W.; Brent, R.P.; De Hoog, F.R.; Sweet, D.R. (1995). “On the stability of the Bareiss andrelated Toeplitz factorization algorithms”. SIAM Journal on Matrix Analysis and Applications 16: 40–57.doi:10.1137/S0895479891221563.
• Brent R.P. (1999), “Stability of fast algorithms for structured linear systems”, Fast Reliable Algorithms forMatrices with Structure (editors—T. Kailath, A.H. Sayed), ch.4 (SIAM).
• Bunch, J. R. (1985). “Stability of methods for solving Toeplitz systems of equations.”SIAM J. Sci. Stat.Comput., v. 6, pp. 349–364.
• Krishna, H.; Wang, Y. (1993).“The Split Levinson Algorithm is weakly stable”. SIAM Journal on NumericalAnalysis 30 (5): 1498–1508. doi:10.1137/0730078.
Summaries
• Bäckström, T. (2004).“2.2. Levinson–Durbin Recursion.”Linear Predictive Modelling of Speech – Constraintsand Line Spectrum Pair Decomposition. Doctoral thesis. Report no. 71 / Helsinki University of Technology,Laboratory of Acoustics and Audio Signal Processing. Espoo, Finland.
• Claerbout, Jon F. (1976). “Chapter 7 – Waveform Applications of Least-Squares.”Fundamentals of Geo-physical Data Processing. Palo Alto: Blackwell Scientific Publications.
• Press, WH; Teukolsky, SA; Vetterling, WT; Flannery, BP (2007), “Section 2.8.2. Toeplitz Matrices”,Numerical Recipes: The Art of Scientific Computing (3rd ed.), New York: Cambridge University Press, ISBN978-0-521-88068-8
• Golub, G.H., and Loan, C.F. Van (1996).“Section 4.7 : Toeplitz and related Systems”Matrix Computations,Johns Hopkins University Press
Chapter 4
List of matrices
Several important classes of matrices are subsets of each other.
This page lists some important classes of matrices used in mathematics, science and engineering. A matrix (pluralmatrices, or less commonly matrixes) is a rectangular array of numbers called entries. Matrices have a long historyof both study and application, leading to diverse ways of classifying matrices. A first group is matrices satisfyingconcrete conditions of the entries, including constant matrices. An important example is the identity matrix given by
In =
1 0 · · · 00 1 · · · 0...
... . . . ...0 0 · · · 1
.Further ways of classifying matrices are according to their eigenvalues or by imposing conditions on the product ofthe matrix with other matrices. Finally, many domains, both in mathematics and other sciences including physics andchemistry have particular matrices that are applied chiefly in these areas.
17
18 CHAPTER 4. LIST OF MATRICES
4.1 Matrices with explicitly constrained entries
The following lists matrices whose entries are subject to certain conditions. Many of them apply to square matricesonly, that is matrices with the same number of columns and rows. The main diagonal of a square matrix is thediagonal joining the upper left corner and the lower right one or equivalently the entries ai,i. The other diagonal iscalled anti-diagonal (or counter-diagonal).
4.1.1 Constant matrices
The list below comprises matrices whose elements are constant for any given dimension (size) of matrix. The matrixentries will be denoted aij. The table below uses the Kronecker delta δij for two integers i and j which is 1 if i = j and0 else.
4.2 Matrices with conditions on eigenvalues or eigenvectors
4.3 Matrices satisfying conditions on products or inverses
A number of matrix-related notions is about properties of products or inverses of the given matrix. The matrixproduct of a m-by-n matrix A and a n-by-k matrix B is the m-by-k matrix C given by
(C)i,j =
n∑r=1
Ai,rBr,j .
This matrix product is denoted AB. Unlike the product of numbers, matrix products are not commutative, that isto say AB need not be equal to BA. A number of notions are concerned with the failure of this commutativity. Aninverse of square matrix A is a matrix B (necessarily of the same dimension as A) such that AB = I. Equivalently, BA= I. An inverse need not exist. If it exists, B is uniquely determined, and is also called the inverse of A, denoted A*−1.
4.4 Matrices with specific applications
• Derogatory matrix —a square n×n matrix whose minimal polynomial is of order less than n.
• Moment matrix —a symmetric matrix whose elements are the products of common row/column index depen-dent monomials.
• X-Y-Z matrix —a generalisation of the (rectangular) matrix to a cuboidal form (a 3-dimensional array ofentries).
4.5 Matrices used in statistics
The following matrices find their main application in statistics and probability theory.
• Bernoulli matrix —a square matrix with entries +1, −1, with equal probability of each.
• Centering matrix —a matrix which, when multiplied with a vector, has the same effect as subtracting the meanof the components of the vector from every component.
• Correlation matrix —a symmetric n×n matrix, formed by the pairwise correlation coefficients of severalrandom variables.
• Covariance matrix —a symmetric n×nmatrix, formed by the pairwise covariances of several random variables.Sometimes called a dispersion matrix.
4.6. MATRICES USED IN GRAPH THEORY 19
• Dispersion matrix —another name for a covariance matrix.
• Doubly stochastic matrix —a non-negative matrix such that each row and each column sums to 1 (thus thematrix is both left stochastic and right stochastic)
• Fisher information matrix —a matrix representing the variance of the partial derivative, with respect to aparameter, of the log of the likelihood function of a random variable.
• Hat matrix - a square matrix used in statistics to relate fitted values to observed values.
• Precision matrix —a symmetric n×n matrix, formed by inverting the covariance matrix. Also called the infor-mation matrix.
• Stochastic matrix —a non-negative matrix describing a stochastic process. The sum of entries of any row isone.
• Transition matrix —a matrix representing the probabilities of conditions changing from one state to another ina Markov chain
4.6 Matrices used in graph theory
The following matrices find their main application in graph and network theory.
• Adjacency matrix —a square matrix representing a graph, with aij non-zero if vertex i and vertex j are adjacent.
• Biadjacency matrix —a special class of adjacency matrix that describes adjacency in bipartite graphs.
• Degree matrix —a diagonal matrix defining the degree of each vertex in a graph.
• Edmonds matrix —a square matrix of a bipartite graph.
• Incidence matrix —a matrix representing a relationship between two classes of objects (usually vertices andedges in the context of graph theory).
• Laplacian matrix —a matrix equal to the degree matrix minus the adjacency matrix for a graph, used to findthe number of spanning trees in the graph.
• Seidel adjacency matrix —a matrix similar to the usual adjacency matrix but with −1 for adjacency; +1 fornonadjacency; 0 on the diagonal.
• Skew-adjacency matrix—an adjacency matrix in which each non-zero aij is 1 or −1, accordingly as the directioni → j matches or opposes that of an initially specified orientation.
• Tutte matrix —a generalisation of the Edmonds matrix for a balanced bipartite graph.
4.7 Matrices used in science and engineering• Cabibbo-Kobayashi-Maskawa matrix —a unitary matrix used in particle physics to describe the strength offlavour-changing weak decays.
• Density matrix —a matrix describing the statistical state of a quantum system. Hermitian, non-negative andwith trace 1.
• Fundamental matrix (computer vision) —a 3 × 3 matrix in computer vision that relates corresponding pointsin stereo images.
• Fuzzy associative matrix —a matrix in artificial intelligence, used in machine learning processes.
• Gamma matrices —4 × 4 matrices in quantum field theory.
• Gell-Mann matrices —a generalisation of the Pauli matrices; these matrices are one notable representation ofthe infinitesimal generators of the special unitary group SU(3).
20 CHAPTER 4. LIST OF MATRICES
• Hamiltonian matrix —a matrix used in a variety of fields, including quantum mechanics and linear-quadraticregulator (LQR) systems.
• Irregular matrix —a matrix used in computer science which has a varying number of elements in each row.
• Overlap matrix —a type of Gramian matrix, used in quantum chemistry to describe the inter-relationship of aset of basis vectors of a quantum system.
• S matrix —a matrix in quantum mechanics that connects asymptotic (infinite past and future) particle states.
• State transition matrix —Exponent of state matrix in control systems.
• Substitution matrix —a matrix from bioinformatics, which describes mutation rates of amino acid or DNAsequences.
• Z-matrix —a matrix in chemistry, representing a molecule in terms of its relative atomic geometry.
4.8 Other matrix-related terms and definitions• Jordan canonical form —an 'almost' diagonalised matrix, where the only non-zero elements appear on the lead
and super-diagonals.
• Linear independence —two or more vectors are linearly independent if there is no way to construct one fromlinear combinations of the others.
• Matrix exponential —defined by the exponential series.
• Matrix representation of conic sections
• Pseudoinverse —a generalization of the inverse matrix.
• Quaternionic matrix - matrix using quaternions as numbers
• Row echelon form —a matrix in this form is the result of applying the forward elimination procedure to amatrix (as used in Gaussian elimination).
• Wronskian —the determinant of a matrix of functions and their derivatives such that row n is the (n-1)*thderivative of row one.
4.9 See also• Perfect matrix
4.10 Notes[1] Hogben 2006, Ch. 31.3
4.11 References• Hogben, Leslie (2006), Handbook of Linear Algebra (Discrete Mathematics and Its Applications), Boca Raton:
Chapman & Hall/CRC, ISBN 978-1-58488-510-8
Chapter 5
Logical matrix
A logical matrix, binary matrix, relation matrix, Boolean matrix, or (0,1) matrix is a matrix with entries fromthe Boolean domain B = 0, 1. Such a matrix can be used to represent a binary relation between a pair of finite sets.
5.1 Matrix representation of a relation
If R is a binary relation between the finite indexed sets X and Y (so R ⊆ X×Y), then R can be represented by theadjacency matrix M whose row and column indices index the elements of X and Y, respectively, such that the entriesof M are defined by:
Mi,j =
1 (xi, yj) ∈ R0 (xi, yj) ∈ R
In order to designate the row and column numbers of the matrix, the sets X and Y are indexed with positive integers:i ranges from 1 to the cardinality (size) of X and j ranges from 1 to the cardinality of Y. See the entry on indexed setsfor more detail.
5.1.1 Example
The binary relation R on the set 1, 2, 3, 4 is defined so that aRb holds if and only if a divides b evenly, with noremainder. For example, 2R4 holds because 2 divides 4 without leaving a remainder, but 3R4 does not hold becausewhen 3 divides 4 there is a remainder of 1. The following set is the set of pairs for which the relation R holds.
(1, 1), (1, 2), (1, 3), (1, 4), (2, 2), (2, 4), (3, 3), (4, 4).
The corresponding representation as a Boolean matrix is:
1 1 1 10 1 0 10 0 1 00 0 0 1
.
5.2 Other examples
• A permutation matrix is a (0,1)-matrix, all of whose columns and rows each have exactly one nonzero element.
• A Costas array is a special case of a permutation matrix
21
22 CHAPTER 5. LOGICAL MATRIX
• An incidence matrix in combinatorics and finite geometry has ones to indicate incidence between points (orvertices) and lines of a geometry, blocks of a block design, or edges of a graph (mathematics)
• A design matrix in analysis of variance is a (0,1)-matrix with constant row sums.
• An adjacency matrix in graph theory is a matrix whose rows and columns represent the vertices and whoseentries represent the edges of the graph. The adjacency matrix of a simple, undirected graph is a binarysymmetric matrix with zero diagonal.
• The biadjacency matrix of a simple, undirected bipartite graph is a (0,1)-matrix, and any (0,1)-matrix arisesin this way.
• The prime factors of a list of m square-free, n-smooth numbers can be described as a m×π(n) (0,1)-matrix,where π is the prime-counting function and aij is 1 if and only if the jth prime divides the ith number. Thisrepresentation is useful in the quadratic sieve factoring algorithm.
• A bitmap image containing pixels in only two colors can be represented as a (0,1)-matrix in which the 0'srepresent pixels of one color and the 1's represent pixels of the other color.
• Using binary matrix to check the game rules in the game of Go
5.3 Some properties
The matrix representation of the equality relation on a finite set is an identity matrix, that is, one whose entries on thediagonal are all 1, while the others are all 0.If the Boolean domain is viewed as a semiring, where addition corresponds to logical OR and multiplication to logicalAND, the matrix representation of the composition of two relations is equal to the matrix product of the matrixrepresentations of these relation. This product can be computed in expected time O(n2).*[1]Frequently operations on binary matrices are defined in terms of modular arithmetic mod 2—that is, the elementsare treated as elements of the Galois field GF(2) = ℤ2. They arise in a variety of representations and have a numberof more restricted special forms. They are applied e.g. in XOR-satisfiability.The number of distinct m-by-n binary matrices is equal to 2*mn, and is thus finite.
5.4 See also• List of matrices
• Binatorix (a binary De Bruijn torus)
• Redheffer matrix
• Relation algebra
5.5 Notes[1] Patrick E. O'Neil, Elizabeth J. O'Neil (1973). “A Fast Expected Time Algorithm for Boolean Matrix Multiplication and
Transitive Closure” (PDF). Information and Control 22 (2): 132–138. doi:10.1016/s0019-9958(73)90228-3. —Thealgorithm relies on addition being idempotent, cf. p.134 (bottom).
5.6 References• Hogben, Leslie (2006), Handbook of Linear Algebra (Discrete Mathematics and Its Applications), Boca Raton:
Chapman & Hall/CRC, ISBN 978-1-58488-510-8, section 31.3, Binary Matrices
• Kim, Ki Hang, Boolean Matrix Theory and Applications, ISBN 0-8247-1788-0
5.7. EXTERNAL LINKS 23
5.7 External links• Hazewinkel, Michiel, ed. (2001), “Logical matrix”, Encyclopedia of Mathematics, Springer, ISBN 978-1-
55608-010-4
Chapter 6
M-matrix
In mathematics, especially linear algebra, an M-matrix is a Z-matrix with eigenvalues whose real parts are positive.M-matrices are also a subset of the class of P-matrices, and also of the class of inverse-positive matrices (i.e. matriceswith inverses belonging to the class of positive matrices).*[1] The name M-matrix was seemingly originally chosenby Alexander Ostrowski in reference to Hermann Minkowski, who proved that if a Z-matrix has all of its row sumspositive, then the determinant of that matrix is positive.*[2]
6.1 Characterizations
An M-matrix is commonly defined as follows:Definition: Let A be a n × n real Z-matrix. That is, A=(aij) where aij ≤ 0 for all i ≠ j, 1 ≤ i,j ≤ n. Then matrix A isalso an M-matrix if it can be expressed in the form A = sI - B, where B=(bij) with bij ≥ 0, for all 1 ≤ i,j ≤ n, where sis greater than the maximum of the moduli of the eigenvalues of B, and I is an identity matrix.For the non-singularity of A, according to Perron-Frobenius theorem, it must be the case that s > ρ(B). Also, fornon-singular M-matrix, the diagonal elements aii of A must be positive. Here we will further characterize only theclass of non-singular M-matrices.Many statements that are equivalent to this definition of non-singular M-matrices are known, and any one of thesestatement can serve as a starting definition of a non-singular M-matrix.*[3] For example, Plemmons lists 40 suchequivalences.*[4] These characterizations has been categorized by Plemmons in terms of their relations to the prop-erties of: (1) positivity of principal minors, (2) inverse-positivity and splittings, (3) stability, and (4) semipositivityand diagonal dominance. It makes sense to categorize the properties in this way because the statements within aparticular group are related to each other even when matrix A is an arbitrary matrix, and not necessarily a Z-matrix.Here we mention a few characterizations from each category.
6.2 Equivalences
Let A be a n × n real Z-matrix, then the following statements are equivalent to A being a non-singular M-matrix:Positivity of Principal Minors
• All the principal minors of A are positive. That is, the determinant of each submatrix of A obtained by deletinga set, possibly empty, of corresponding rows and columns of A is positive.
• A + D is nonsingular for each nonnegative diagonal matrix D.
• Every real eigenvalue of A is positive.
• All the leading principal minors of A are positive.
• There exist lower and upper triangular matrices L and U respectively, with positive diagonals, such that A =LU.
24
6.3. APPLICATIONS 25
Inverse-Positivity and Splittings
• A is inverse-positive. That is, A*−1 exists and A*−1 ≥ 0.
• A is monotone. That is, Ax ≥ 0 implies x ≥ 0.
• A has a convergent regular splitting. That is, A has a representation A = M - N, where M*−1 ≥ 0, N ≥ 0 withM*−1N convergent. That is, ρ(M*−1N) < 1.
• There exist inverse-positive matrices M1 and M2 with M1 ≤ A ≤ M2.
• Every regular splitting of A is convergent.
Stability
• There exists a positive diagonal matrix D such that AD + DA*T is positive definite.
• A is positive stable. That is, the real part of each eigenvalue of A is positive.
• There exists a symmetric positive definite matrix W such that AW + WA*T is positive definite.
• A + I is non-singular, and G = (A + I)*−1(A + I) is convergent.
• A + I is non-singular, and for G = (A + I)*−1(A + I) there exists a positive definite symmetric matrix W suchthat W - G*TWG is positive definite.
Semipositivity and Diagonal Dominance
• A is semi-positive. That is, there exists x > 0 with Ax > 0.
• There exists x ≥ 0 with Ax > 0.
• There exists a positive diagonal matrix D such that AD has all positive row sums.
• A has all positive diagonal elements, and there exists a positive diagonal matrix D such that AD is strictlydiagonally dominant.
• A has all positive diagonal elements, and there exists a positive diagonal matrix D such that D*−1AD is strictlydiagonally dominant.
6.3 Applications
The primary contributions to M-matrix theory has mainly come from mathematicians and economists. M-matricesare used in mathematics to establish bounds on eigenvalues and on the establishment of convergence criteria foriterative methods for the solution of large sparse systems of linear equations. M-matrices arise naturally in somediscretizations of differential operators, such as the Laplacian, and as such are well-studied in scientific computing.M-matrices also occur in the study of solutions to linear complementarity problem. Linear complementarity prob-lems arise in linear and quadratic programming, computational mechanics, and in the problem of finding equilibriumpoint of a bimatrix game. Lastly, M-matrices occur in the study of finite Markov chains in the field of probabilitytheory and operations research like queuing theory. Meanwhile, the economists have studied M-matrices in connec-tion with gross substitutability, stability of a general equilibrium and Leontief's input-output analysis in economicsystems. The condition of positivity of all principal minors is also known as the Hawkins–Simon condition in eco-nomic literature.*[5] In engineering, M-matrices also occur in the problems of feedback control in control theory andis related to Hurwitz matrix. In computational biology, M-matrices occur in the study of population dynamics.
26 CHAPTER 6. M-MATRIX
6.4 See also• If A is an M-matrix, then -A is a Metzler matrix.
• A symmetric M-matrix is sometimes called a Stieltjes matrix.
• Hurwitz matrix
• Z-matrix
• P-matrix
• Perron-Frobenius theorem
6.5 References[1] Fujimoto, Takao & Ranade, Ravindra (2004),“Two Characterizations of Inverse-Positive Matrices: The Hawkins-Simon
Condition and the Le Chatelier-Braun Principle” (PDF), Electronic Journal of Linear Algebra 11: 59–65.
[2] Bermon, Abraham & Plemmons, Robert J. (1994), Nonnegative Matrices in the Mathematical Sciences, Philadelphia: So-ciety for Industrial and Applied Mathematics, p. 134,161 (Thm. 2.3 and Note 6.1 of chapter 6), ISBN 0-89871-321-8.
[3] Fiedler, M; Ptak, V. (1962), “On matrices with non-positive off-diagonal elements and positive principal minors”,Czechoslovak Mathematical Journal 12 (3): 382–400.
[4] Plemmons, R.J. (1977),“M-Matrix Characterizations. I -- Nonsingular M-Matrices”, Linear Algebra and its Applications18 (2): 175–188, doi:10.1016/0024-3795(77)90073-8.
[5] Nikaido, H. (1970). Introduction to Sets and Mappings in Modern Economics. New York: Elsevier. pp. 13–19. ISBN0-444-10038-5.
Chapter 7
Magic square
In recreational mathematics, a magic square is an arrangement of distinct numbers (i.e. each number is used once),usually integers, in a square grid, where the numbers in each row, and in each column, and the numbers in the mainand secondary diagonals, all add up to the same number. A magic square has the same number of rows as it hascolumns, and in conventional math notation, "n" stands for the number of rows (and columns) it has. Thus, a magicsquare always contains n2 numbers, and its size (the number of rows [and columns] it has) is described as being“oforder n".*[1] A magic square that contains the integers from 1 to n2 is called a normal magic square. (The term
“magic square”is also sometimes used to refer to any of various types of word squares.)Normal magic squares of all sizes except 2 × 2 (that is, where n = 2) can be constructed. The 1 × 1 magic square, withonly one cell containing the number 1, is trivial. The smallest (and unique up to rotation and reflection) nontrivialcase, 3 × 3, is shown below.
Any magic square can be rotated and reflected to produce 8 trivially distinct squares. In magic square theory all of
27
28 CHAPTER 7. MAGIC SQUARE
these are generally deemed equivalent and the eight such squares are said to comprise a single equivalence class.*[2]The constant that is the sum of every row, column and diagonal is called the magic constant or magic sum, M. Everynormal magic square has a constant dependent on n, calculated by the formula M = [n(n2 + 1)] / 2. For normal magicsquares of order n = 3, 4, 5, 6, 7, and 8, the magic constants are, respectively: 15, 34, 65, 111, 175, and 260 (sequenceA006003 in the OEIS).Magic squares have a long history, dating back to 650 BC in China. At various times they have acquired magical ormythical significance, and have appeared as symbols in works of art. In modern times they have been generalizeda number of ways, including using extra or different constraints, multiplying instead of adding cells, using alternateshapes or more than two dimensions, and replacing numbers with shapes and addition with geometric operations.
7.1 History
Iron plate with an order 6 magic square in Arabic numbers from China, dating to the Yuan Dynasty (1271–1368).
Magic squares were known to Chinese mathematicians as early as 650 BC, and explicitly given since 570 AD,*[3]and to Islamic mathematicians possibly as early as the seventh century AD. The first magic squares of order 5 and6 appear in an encyclopedia from Baghdad circa 983, the Encyclopedia of the Brethren of Purity (Rasa'il Ihkwanal-Safa); simpler magic squares were known to several earlier Arab mathematicians.*[3] Some of these squares werelater used in conjunction with magic letters, as in (Shams Al-ma'arif), to assist Arab illusionists and magicians.*[4]
7.1. HISTORY 29
7.1.1 Lo Shu square (3×3 magic square)
Main article: Lo Shu Square
Chinese literature dating from as early as 650 BC tells the legend of Lo Shu or “scroll of the river Lo”.*[3] Earlyrecords are ambiguous references to a “river map”, but clearly refer to a magic square by 80 AD, and explicitlygive one since 570 AD.*[3] According to the legend, there was at one time in ancient China a huge flood. While thegreat king Yu (禹) was trying to channel the water out to sea, a turtle emerged from it with a curious figure / patternon its shell: a 3×3 grid in which circular dots of numbers were arranged, such that the sum of the numbers in eachrow, column and diagonal was the same: 15, which is also the number of days in each of the 24 cycles of the Chinesesolar year. According to the legend, thereafter people were able to use this pattern in a certain way to control theriver and protect themselves from floods.The Lo Shu Square, as the magic square on the turtle shell is called, is the unique normal magic square of order threein which 1 is at the bottom and 2 is in the upper right corner. Every normal magic square of order three is obtainedfrom the Lo Shu by rotation or reflection.The Square of Lo Shu is also referred to as the Magic Square of Saturn.
7.1.2 Persia
Original script from the Shams al-Ma'arif.
30 CHAPTER 7. MAGIC SQUARE
Printed version of the previous manuscript. Eastern Arabic numerals were used.
Although the early history of magic squares in Persia is not known, it has been suggested that they were knownin pre-Islamic times.*[5] It is clear, however, that the study of magic squares was common in medieval Islam inPersia, and it was thought to have begun after the introduction of chess into the region.*[6] The 10th-century Persianmathematician Buzjani, for example, left a manuscript that on page 33 contains a series of magic squares, filled bynumbers in arithmetic progression, in such a way that the sums of each row, column and diagonal are equal.*[7]
7.1.3 Arabia
Magic squares were known to Islamic mathematicians in Arabia as early as the seventh century. They may havelearned about them when the Arabs came into contact with Indian culture and learned Indian astronomy and mathe-matics – including other aspects of combinatorial mathematics. Alternatively, the idea may have come to them fromChina. The first magic squares of order 5 and 6 known to have been devised by Arab mathematicians appear inan encyclopedia from Baghdad circa 983, the Rasa'il Ikhwan al-Safa (the Encyclopedia of the Brethren of Purity);simpler magic squares were known to several earlier Arab mathematicians.*[3]The Arab mathematician Ahmad al-Buni, who worked on magic squares around 1250, attributed mystical propertiesto them, although no details of these supposed properties are known. There are also references to the use of magicsquares in astrological calculations, a practice that seems to have originated with the Arabs.*[3]
7.1.4 India
The 3×3 magic square has been a part of rituals in India since Vedic times, and still is today. The Ganesh yantra is a3×3 magic square. There is a well-known 10th-century 4×4 magic square on display in the Parshvanath Jain templein Khajuraho, India.*[8]This is known as the Chautisa Yantra. Each row, column, and diagonal, as well as each 2×2 sub-square, the corners ofeach 3×3 and 4×4 square, the corners of each 2x4 and 4x2 rectangle, and the offset diagonals (12+8+5+9, 1+11+16+6,2+12+15+5, 14+2+3+15 and 7+11+10+6, 12+2+5+15, 1+13+16+4) sum to 34.In this square, every second diagonal number adds to 17 (the same applies to offset diagonals). In addition to squaresand rectangles, there are eight trapeziums – two in one direction, and the others at a rotation of 90 degrees, such as(12, 1, 16, 5) and (13, 8, 9, 4).These characteristics (which identify it as one of the three 4x4 pandiagonal magic squares and as a most-perfect magic
7.1. HISTORY 31
32 CHAPTER 7. MAGIC SQUARE
square) mean that the rows or columns can be rotated and maintain the same characteristics - for example:The Kubera-Kolam, a magic square of order three, is commonly painted on floors in India. It is essentially the sameas the Lo Shu Square, but with 19 added to each number, giving a magic constant of 72.
7.1.5 Europe
In 1300, building on the work of the Arab Al-Buni, Greek Byzantine scholar Manuel Moschopoulos wrote a math-ematical treatise on the subject of magic squares, leaving out the mysticism of his predecessors.*[9] Moschopouloswas essentially unknown to the Latin west. He was not, either, the first Westerner to have written on magic squares.They appear in a Spanish manuscript written in the 1280s, presently in the Biblioteca Vaticana (cod. Reg. Lat.1283a) due to Alfonso X of Castille.*[10] In that text, each magic square is assigned to the respective planet, as inthe Islamic literature.*[11] Magic squares surface again in Italy in the 14th century, and specifically in Florence. Infact, a 6×6 and a 9×9 square are exhibited in a manuscript of the Trattato d'Abbaco (Treatise of the Abacus) byPaolo dell'Abbaco, aka Paolo Dagomari, a mathematician, astronomer and astrologer who was, among other things,in close contact with Jacopo Alighieri, a son of Dante. The squares can be seen on folios 20 and 21 of MS. 2433,at the Biblioteca Universitaria of Bologna. They also appear on folio 69rv of Plimpton 167, a manuscript copy ofthe Trattato dell'Abbaco from the 15th century in the Library of Columbia University.*[12] It is interesting to ob-serve that Paolo Dagomari, like Pacioli after him, refers to the squares as a useful basis for inventing mathematicalquestions and games, and does not mention any magical use. Incidentally, though, he also refers to them as beingrespectively the Sun's and the Moon's squares, and mentions that they enter astrological calculations that are not betterspecified. As said, the same point of view seems to motivate the fellow Florentine Luca Pacioli, who describes 3×3to 9×9 squares in his work De Viribus Quantitatis.*[13] Pacioli states: A lastronomia summamente hanno mostrato lisupremi di quella commo Ptolomeo, al bumasar ali, al fragano, Geber et gli altri tutti La forza et virtu de numeri eserlinecessaria (Masters of astronomy, such as Ptolemy, Albumasar, Alfraganus, Jabir and all the others, have shown thatthe force and the virtue of numbers are necessary to that science) and then goes on to describe the seven planetarysquares, with no mention of magical applications.Magic squares of order 3 through 9, assigned to the seven planets, and described as means to attract the influence ofplanets and their angels (or demons) during magical practices, can be found in several manuscripts all around Europestarting at least since the 15th century. Among the best known, the Liber de Angelis, a magical handbook writtenaround 1440, is included in Cambridge Univ. Lib. MS Dd.xi.45.*[14] The text of the Liber de Angelis is very close tothat ofDe septem quadraturis planetarum seu quadrati magici, another handbook of planetary image magic containedin the Codex 793 of the Biblioteka Jagiellońska (Ms BJ 793).*[15] The magical operations involve engraving theappropriate square on a plate made with the metal assigned to the corresponding planet,*[16] as well as performinga variety of rituals. For instance, the 3×3 square, that belongs to Saturn, has to be inscribed on a lead plate. It will,in particular, help women during a difficult childbirth.In 1514 Albrecht Dürer immortalizes a 4×4 square in his famous engraving “Melancholia I”.In about 1510 Heinrich Cornelius Agrippa wroteDe Occulta Philosophia, drawing on the Hermetic and magical worksof Marsilio Ficino and Pico della Mirandola. In its 1531 edition, he expounded on the magical virtues of the sevenmagical squares of orders 3 to 9, each associated with one of the astrological planets, much in the same way as theolder texts did. This book was very influential throughout Europe until the counter-reformation, and Agrippa's magicsquares, sometimes called kameas, continue to be used within modern ceremonial magic in much the same way ashe first prescribed.*[3]*[17]The most common use for these kameas is to provide a pattern upon which to construct the sigils of spirits, angels ordemons; the letters of the entity's name are converted into numbers, and lines are traced through the pattern that thesesuccessive numbers make on the kamea. In a magical context, the term magic square is also applied to a variety ofword squares or number squares found in magical grimoires, including some that do not follow any obvious pattern,and even those with differing numbers of rows and columns. They are generally intended for use as talismans. Forinstance the following squares are: The Sator square, one of the most famous magic squares found in a number ofgrimoires including the Key of Solomon; a square “to overcome envy”, from The Book of Power;*[18] and twosquares from The Book of the Sacred Magic of Abramelin the Mage, the first to cause the illusion of a superb palaceto appear, and the second to be worn on the head of a child during an angelic invocation:
7.1. HISTORY 33
This page from Athanasius Kircher's Oedipus Aegyptiacus (1653) belongs to a treatise on magic squares and shows the SigillumIovis associated with Jupiter
7.1.6 Albrecht Dürer's magic square
The order-4 magic square in Albrecht Dürer's engraving Melencolia I is believed to be the first seen in Europeanart. It is very similar to Yang Hui's square, which was created in China about 250 years before Dürer's time. The
34 CHAPTER 7. MAGIC SQUARE
1381432 26442022139 8 33 2745
3461540 9 34 28
13 31 7 2543193730 6 24491836125 234817 421129224716 411035 4
Hagiel = = 5; 3; 10; 1(10); 30(3)The derivation of the sigil of Hagiel, the planetary intelligence of Venus, drawn on the magic square of Venus. Each Hebrew letterprovides a numerical value, giving the vertices of the sigil.
sum 34 can be found in the rows, columns, diagonals, each of the quadrants, the center four squares, and the cornersquares(of the 4×4 as well as the four contained 3×3 grids). This sum can also be found in the four outer numbersclockwise from the corners (3+8+14+9) and likewise the four counter-clockwise (the locations of four queens in thetwo solutions of the 4 queens puzzle*[19]), the two sets of four symmetrical numbers (2+8+9+15 and 3+5+12+14),the sum of the middle two entries of the two outer columns and rows (5+9+8+12 and 3+2+15+14), and in four kiteor cross shaped quartets (3+5+11+15, 2+10+8+14, 3+9+7+15, and 2+6+12+14). The two numbers in the middleof the bottom row give the date of the engraving: 1514. The numbers 1 and 4 at either side of the date correspondto the letters 'A' and 'D' which are the initials of the artist.Dürer's magic square can also be extended to a magic cube.*[20]Dürer's magic square and his Melencolia I both also played large roles in Dan Brown's 2009 novel, The Lost Symbol.
7.1.7 Sagrada Família magic square
The Passion façade of the Sagrada Família church in Barcelona, conceptualized by Antoni Gaudí and designed bysculptor Josep Subirachs, features a 4×4 magic square:The magic constant of the square is 33, the age of Jesus at the time of the Passion. Structurally, it is very similar tothe Melancholia magic square, but it has had the numbers in four of the cells reduced by 1.While having the same pattern of summation, this is not a normal magic square as above, as two numbers (10 and14) are duplicated and two (12 and 16) are absent, failing the 1→n2 rule.Similarly to Dürer's magic square, the Sagrada Familia's magic square can also be extended to a magic cube.*[21]
7.2 Types of construction
There are many ways to construct magic squares, but the standard (and most simple) way is to follow certain config-urations/formulas which generate regular patterns. Magic squares exist for all values of n, with only one exception:it is impossible to construct a magic square of order 2. Magic squares can be classified into three types: odd, doublyeven (n divisible by four) and singly even (n even, but not divisible by four). Odd and doubly even magic squares areeasy to generate; the construction of singly even magic squares is more difficult but several methods exist, including
7.2. TYPES OF CONSTRUCTION 35
Detail of Melencolia I
the LUX method for magic squares (due to John Horton Conway) and the Strachey method for magic squares.Group theory was also used for constructing new magic squares of a given order from one of them.*[22]The numbers of different n×n magic squares for n from 1 to 5, not counting rotations and reflections are: 1, 0, 1, 880,275305224 (sequence A006052 in OEIS). The number for n = 6 has been estimated to be (0.17745 ± 0.00016) ×1020.*[23]*[24]
7.2.1 Method for constructing a magic square of order 3
In the 19th century, Édouard Lucas devised the general formula for order 3 magic squares. Consider the followingtable made up of positive integers a, b and c:These 9 numbers will be distinct positive integers forming a magic square so long as 0 < a < b < c - a and b ≠ 2a.Moreover, every 3 x 3 square of distinct positive integers is of this form.
36 CHAPTER 7. MAGIC SQUARE
A magic square on the Sagrada Família church façade
7.2.2 Method for constructing a magic square of odd order
See also: Siamese methodA method for constructing magic squares of odd order was published by the French diplomat de la Loubère in his
book, A new historical relation of the kingdom of Siam (Du Royaume de Siam, 1693), in the chapter entitled Theproblem of the magical square according to the Indians.*[25] The method operates as follows:The method prescribes starting in the central column of the first row with the number 1. After that, the fundamentalmovement for filling the squares is diagonally up and right, one step at a time. If a filled square is encountered, onemoves vertically down one square instead, then continues as before. When an “up and to the right”move wouldleave the square, it is wrapped around to the last row or first column, respectively.Starting from other squares rather than the central column of the first row is possible, but then only the row andcolumn sums will be identical and result in a magic sum, whereas the diagonal sums will differ. The result will thus bea semimagic square and not a true magic square. Moving in directions other than north east can also result in magicsquares.The following formulae help construct magic squares of odd orderExample:The "middle number" is always in the diagonal bottom left to top right.The "last number" is always opposite the number 1 in an outside column or row.
7.2.3 A method of constructing a magic square of doubly even order
Doubly even means that n is an even multiple of an even integer; or 4p (e.g. 4, 8, 12), where p is an integer.Generic pattern All the numbers are written in order from left to right across each row in turn, starting from the topleft hand corner. The resulting square is also known as a mystic square. Numbers are then either retained in the same
7.2. TYPES OF CONSTRUCTION 37
Yang Hui's construction method
place or interchanged with their diametrically opposite numbers in a certain regular pattern. In the magic square oforder four, the numbers in the four central squares and one square at each corner are retained in the same place andthe others are interchanged with their diametrically opposite numbers.A construction of a magic square of order 4 (This is reflection of Albrecht Dürer's square.) Go left to right throughthe square counting and filling in on the diagonals only. Then continue by going left to right from the top left of thetable and fill in counting down from 16 to 1. As shown below.An extension of the above example for Orders 8 and 12 First generate a “truth”table, where a '1' indicatesselecting from the square where the numbers are written in order 1 to n2 (left-to-right, top-to-bottom), and a '0'indicates selecting from the square where the numbers are written in reverse order n2 to 1. For M = 4, the “truth”table is as shown below, (third matrix from left.)Note that a) there are equal number of '1's and '0's; b) each row and each column are “palindromic"; c) the left-and right-halves are mirror images; and d) the top- and bottom-halves are mirror images (c & d imply b.) The truthtable can be denoted as (9, 6, 6, 9) for simplicity (1-nibble per row, 4 rows.) Similarly, for M=8, two choices for thetruth table are (A5, 5A, A5, 5A, 5A, A5, 5A, A5) or (99, 66, 66, 99, 99, 66, 66, 99) (2-nibbles per row, 8 rows.)For M=12, the truth table (E07, E07, E07, 1F8, 1F8, 1F8, 1F8, 1F8, 1F8, E07, E07, E07) yields a magic square(3-nibbles per row, 12 rows.) It is possible to count the number of choices one has based on the truth table, takingrotational symmetries into account.
38 CHAPTER 7. MAGIC SQUARE
7.2.4 Medjig-method of constructing magic squares of even number of rows
This method is based on a 2006 published mathematical game called medjig (author: Willem Barink, editor: Philos-Spiele). The pieces of the medjig puzzle are squares divided in four quadrants on which the numbers 0, 1, 2 and 3are dotted in all sequences. There are 18 squares, with each sequence occurring 3 times. The aim of the puzzle is totake 9 squares out of the collection and arrange them in a 3 × 3 “medjig-square”in such a way that each row andcolumn formed by the quadrants sums to 9, along with the two long diagonals.The medjig method of constructing a magic square of order 6 is as follows:
• Construct any 3 × 3 medjig-square (ignoring the original game's limit on the number of times that a givensequence is used).
• Take the 3 × 3 magic square and divide each of its squares into four quadrants.
• Fill these quadrants with the four numbers from 1 to 36 that equal the original number modulo 9, i.e. x+9ywhere x is the original number and y is a number from 0 to 3, following the pattern of the medjig-square.
Example:Similarly, for any larger integer N, a magic square of order 2N can be constructed from any N × N medjig-squarewith each row, column, and long diagonal summing to 3N, and any N × N magic square (using the four numbers from1 to 4N2 that equal the original number modulo N2).
7.2.5 Construction of panmagic squares
Any number p in the order-n square can be uniquely written in the form p = an + r, with r chosen from 1,...,n.Note that due to this restriction, a and r are not the usual quotient and remainder of dividing p by n. Consequently,the problem of constructing can be split in two problems easier to solve. So, construct two matching square gridsof order n satisfying panmagic properties, one for the a-numbers (0,..., n−1), and one for the r-numbers (1,...,n).This requires a lot of puzzling, but can be done. When successful, combine them into one panmagic square. Vanden Essen and many others supposed this was also the way Benjamin Franklin (1706–1790) constructed his famousFranklin squares. Three panmagic squares are shown below. The first two squares have been constructed April 2007by Barink, the third one is some years older, and comes from Donald Morris, who used, as he supposes, the Franklinway of construction.The order 8 square satisfies all panmagic properties, including the Franklin ones. It consists of 4 perfectly panmagic4×4 units. Note that both order 12 squares show the property that any row or column can be divided in three partshaving a sum of 290 (= 1/3 of the total sum of a row or column). This property compensates the absence of the morestandard panmagic Franklin property that any 1/2 row or column shows the sum of 1/2 of the total. For the rest theorder 12 squares differ a lot. The Barink 12×12 square is composed of 9 perfectly panmagic 4×4 units, moreoverany 4 consecutive numbers starting on any odd place in a row or column show a sum of 290. The Morris 12×12square lacks these properties, but on the contrary shows constant Franklin diagonals. For a better understandingof the constructing decompose the squares as described above, and see how it was done. And note the differencebetween the Barink constructions on the one hand, and the Morris/Franklin construction on the other hand.In the book Mathematics in the Time-Life Science Library Series, magic squares by Euler and Franklin are shown.Franklin designed this one so that any four-square subset (any four contiguous squares that form a larger square, orany four squares equidistant from the center) total 130. In Euler's square, the rows and columns each total 260, andhalfway they total 130 – and a chess knight, making its L-shaped moves on the square, can touch all 64 boxes inconsecutive numerical order.
7.2.6 Construction similar to the Kronecker Product
There is a method reminiscent of the Kronecker product of two matrices, that builds an nm × nm magic square froman n × n magic square and an m × m magic square.*[26]
7.3. GENERALIZATIONS 39
7.2.7 The construction of a magic square using genetic algorithms
A magic square can be constructed using genetic algorithms.*[27] In this process an initial population of squareswith random values is generated. The fitness scores of these individual squares are calculated based on the degreeof deviation in the sums of the rows, columns, and diagonals. The population of squares reproduce by exchangingvalues, together with some random mutations. Those squares with a higher fitness score are more likely to reproduce.The fitness scores of the next generation squares are calculated, and this process continues until a magic square isfound or a time limit is reached.
7.3 Generalizations
7.3.1 Extra constraints
Certain extra restrictions can be imposed on magic squares. If not only the main diagonals but also the brokendiagonals sum to the magic constant, the result is a panmagic square.If raising each number to the nth power yields another magic square, the result is a bimagic (n = 2), a trimagic (n =3), or, in general, a multimagic square.A magic square in which the number of letters in the name of each number in the square generates another magicsquare is called an alphamagic square.
7.3.2 Different constraints
Sometimes the rules for magic squares are relaxed, so that only the rows and columns but not necessarily the diagonalssum to the magic constant (this is usually called a semimagic square).In heterosquares and antimagic squares, the 2n + 2 sums must all be different.
7.3.3 Multiplicative magic squares
Instead of adding the numbers in each row, column and diagonal, one can apply some other operation. For example,a multiplicative magic square has a constant product of numbers. A multiplicative magic square can be derived froman additive magic square by raising 2 (or any other integer) to the power of each element, because the logarithm ofthe product of 2 numbers is the sum of logarithm of each. Alternatively, if any 3 numbers in a line are 2*a, 2*b and2*c, their product is 2*a+b+c, which is constant if a+b+c is constant, as they would be if a, b and c were taken fromordinary (additive) magic square.*[28] For example, the original Lo-Shu magic square becomes:Other examples of multiplicative magic squares include:
7.3.4 Multiplicative magic squares of complex numbers
Still using Ali Skalli's non iterative method, it is possible to produce an infinity of multiplicative magic squares ofcomplex numbers*[29] belonging to C set. On the example below, the real and imaginary parts are integer numbers,but they can also belong to the entire set of real numbers R . The product is: −352,507,340,640 − 400,599,719,520i.
7.3.5 Additive-multiplicative magic and semimagic squares
Additive-multiplicative magic squares and semimagic squares satisfy properties of both ordinary and multiplicativemagic squares and semimagic squares, respectively.*[30]It is unknown if any additive-multiplicative magic squares smaller than 8×8 exist, but it has been proven that no 3×3or 4×4 additive-multiplicative magic squares and no 3×3 additive-multiplicative semimagic squares exist.*[31]
40 CHAPTER 7. MAGIC SQUARE
7.3.6 Other magic shapes
Other shapes than squares can be considered. The general case is to consider a design with N parts to be magic if theN parts are labeled with the numbers 1 throughN and a number of identical sub-designs give the same sum. Examplesinclude magic dodecahedrons, magic triangles*[32] magic stars, and magic hexagons. Going up in dimension resultsin magic cubes and other magic hypercubes.Edward Shineman has developed yet another design in the shape of magic diamonds.Possible magic shapes are constrained by the number of equal-sized, equal-sum subsets of the chosen set of labels.For example, if one proposes to form a magic shape labeling the parts with 1, 2, 3, 4, the sub-designs will have tobe labeled with 1,4 and 2,3.*[32]
7.3.7 Other component elements
Magic squares may be constructed which contain geometric shapes instead of numbers. Such squares, known asgeometric magic squares, were invented and named by Lee Sallows in 2001.*[33]
7.3.8 Combined extensions
One can combine two or more of the above extensions, resulting in such objects as multiplicative multimagic hyper-cubes. Little seems to be known about this subject.
7.4 Related problems
Over the years, many mathematicians, including Euler, Cayley and Benjamin Franklin have worked on magic squares,and discovered fascinating relations.
7.4.1 Magic square of primes
Rudolf Ondrejka (1928–2001) discovered the following 3×3 magic square of primes, in this case nine Chen primes:The Green–Tao theorem implies that there are arbitrarily large magic squares consisting of primes.
7.4.2 n-Queens problem
In 1992, Demirörs, Rafraf, and Tanik published a method for converting some magic squares into n-queens solutions,and vice versa.*[34]
7.4.3 Enumeration of magic squares
As mentioned above, the set of normal squares of order three constitutes a single equivalence class-all equivalentto the Lo Shu square. Thus there is basically just one normal magic square of order 3. But the number of distinctnormal magic squares rapidly increases for higher orders.*[35] There are 880 distinct magic squares of order 4 and275,305,224 of order 5.*[36] The number of distinct normal squares is not yet known for any higher order.*[37]
7.5 Magic squares in popular culture
On October 9, 2014 the post office of Macao in the People's Republic of China issued a series of stamps basedon magic squares.*[38] The figure below shows the stamps featuring the six magic squares chosen to be in thiscollection.*[39]
7.6. SEE ALSO 41
7.6 See also
• Arithmetic sequence
• Combinatorial design
• Freudenthal magic square
• John R. Hendricks
• Hexagonal tortoise problem
• Latin square
• Magic circle
• Magic cube classes
• Magic series
• Most-perfect magic square
• Nasik magic hypercube
• Prime reciprocal magic square
• Room square
• Square matrices
• Sriramachakra
• Sudoku
• Unsolved problems in mathematics
• Vedic square
7.7 Notes[1] "Magic Square" by Onkar Singh, Wolfram Demonstrations Project.
[2] The lost theorem, by Lee Sallows The Mathematical Intelligencer, Fall 1997, Volume 19, Issue 4, pp 51-54, Jan 09, 2009
[3] Swaney, Mark. “Mark Swaney on the History of Magic Squares”. Archived from the original on 2004-08-07.
[4] The most famous Arabic book on magic, named “Shams Al-ma'arif (Arabic: المعارف شمس ,( كتاب for Ahmed bin AliAl-boni, who died about 1225 (622 AH). Reprinted in Beirut in 1985
[5] J. P. Hogendijk, A. I. Sabra, The Enterprise of Science in Islam: New Perspectives, Published by MIT Press, 2003, ISBN0-262-19482-1, p. xv.
[6] Helaine Selin, Ubiratan D'Ambrosio, Mathematics Across Cultures: The History of Non-Western Mathematics, Publishedby Springer, 2001, ISBN 1-4020-0260-2, p. 160.
[7] Sesiano, J., Abūal-Wafā\rasp's treatise on magic squares (French), Z. Gesch. Arab.-Islam. Wiss. 12 (1998), 121–244.
[8] Magic Squares and Cubes By William Symes Andrews, 1908, Open court publish company
[9] Manuel Moschopoulos – Mathematics and the Liberal Arts
[10] See Alfonso X el Sabio, Astromagia (Ms. Reg. lat. 1283a), a cura di A.D'Agostino, Napoli, Liguori, 1992
[11] Mars magic square appears in figure 1 of“Saturn and Melancholy: Studies in the History of Natural Philosophy, Religion,and Art”by Raymond Klibansky, Erwin Panofsky and Fritz Saxl, Basic Books (1964)
42 CHAPTER 7. MAGIC SQUARE
[12] In a 1981 article (“Zur Frühgeschichte der magischen Quadrate in Westeuropa”i.e. “Prehistory of Magic Squaresin Western Europe”, Sudhoffs Archiv Kiel (1981) vol. 65, pp. 313–338) German scholar Menso Folkerts lists severalmanuscripts in which the “Trattato d'Abbaco”by Dagomari contains the two magic square. Folkerts quotes a 1923article by Amedeo Agostini in the Bollettino dell'Unione Matematica Italiana: “A. Agostini in der Handschrift Bologna,Biblioteca Universitaria, Ms. 2433, f. 20v-21r; siehe Bollettino della Unione Matematica Italiana 2 (1923), 77f. Agostinibemerkte nicht, dass die Quadrate zur Abhandlung des Paolo dell’Abbaco gehören und auch in anderen Handschriftendieses Werks vorkommen, z. B. New York, Columbia University, Plimpton 167, f. 69rv; Paris, BN, ital. 946, f. 37v-38r;Florenz, Bibl. Naz., II. IX. 57, f. 86r, und Targioni 9, f. 77r; Florenz, Bibl. Riccard., Ms. 1169, f. 94-95.”
[13] This manuscript text (circa 1496–1508) is also at the Biblioteca Universitaria in Bologna. It can be seen in full at theaddress http://www.uriland.it/matematica/DeViribus/Presentazione.html
[14] See Juris Lidaka, The Book of Angels, Rings, Characters and Images of the Planets in Conjuring Spirits, C. Fangier ed.(Pennsylvania State University Press, 1994)
[15] Benedek Láng, Demons in Krakow, and Image Magic in a Magical Handbook, in Christian Demonology and PopularMythology, Gábor Klaniczay and Éva Pócs eds. (Central European University Press, 2006)
[16] According to the correspondence principle, each of the seven planets is associated to a given metal: lead to Saturn, iron toMars, gold to the Sun, etc.
[17] Drury, Nevill (1992). Dictionary of Mysticism and the Esoteric Traditions. Bridport, Dorset: Prism Press. ISBN 1-85327-075-X.
[18]“The Book of Power: Cabbalistic Secrets of Master Aptolcater, Mage of Adrianople”, transl. 1724. In Shah, Idries(1957). The Secret Lore of Magic. London: Frederick Muller Ltd.
[19] http://www.muljadi.org/MagicSquares.htm
[20] "Magic cube with Dürer's square" Ali Skalli's magic squares and magic cubes
[21] "Magic cube with Gaudi's square" Ali Skalli's magic squares and magic cubes
[22] Structure of Magic and Semi-Magic Squares, Methods and Tools for Enumeration
[23] Pinn K. and Wieczerkowski C., (1998)“Number of Magic Squares From Parallel Tempering Monte Carlo”, Int. J. Mod.Phys. C 9 541
[24] “Number of Magic Squares From Parallel Tempering Monte Carlo, arxiv.org, April 9, 1998. Retrieved November 2,2013.
[25] Mathematical Circles Squared By Phillip E. Johnson, Howard Whitley Eves, p.22
[26] Hartley, M. “Making Big Magic Squares”.
[27] Evolving a Magic Square using Genetic Algorithms
[28] Stifel, Michael (1544), Arithmetica integra (in Latin), pp. 29–30.
[29] "8x8 multiplicative magic square of complex numbers" Ali Skalli's magic squares and magic cubes
[30] “MULTIMAGIE.COM - Additive-Multiplicative magic squares, 8th and 9th-order”. Retrieved 26 August 2015.
[31] “MULTIMAGIE.COM - Smallest additive-multiplicative magic square”. Retrieved 26 August 2015.
[32] Magic Designs,Robert B. Ely III, Journal of Recreational Mathematics volume 1 number 1, January 1968
[33] Magic squares are given a whole new dimension, The Observer, April 3, 2011
[34] O. Demirörs, N. Rafraf, and M. M. Tanik. “Obtaining n-queens solutions from magic squares and constructing magicsquares from n-queens solutions”. Journal of Recreational Mathematics, 24:272–280, 1992
[35] How many magic squares are there? by Walter Trump, Nürnberg, January 11, 2001
[36] A006052 in the on-line encyclopedia of integer sequences
[37] Anything but square: from magic squares to Sudoku by Hardeep Aiden, Plus Magazine, March 1, 2006
[38] Macau Post Office web site
[39] Macau's magic square stamps just made philately even more nerdy The Guardian Science, November 3, 2014
7.8. REFERENCES 43
7.8 References• Weisstein, Eric W., “Magic Square”, MathWorld.
• Magic Squares at Convergence
• W. S. Andrews, Magic Squares and Cubes. (New York: Dover, 1960), originally printed in 1917
• John Lee Fults, Magic Squares. (La Salle, Illinois: Open Court, 1974).
• Cliff Pickover, The Zen of Magic Squares, Circles, and Stars (Princeton, New Jersey: Princeton UniversityPress)
• Leonhard Euler, On magic squares
• Mark Farrar, Magic Squares
• Asker Ali Abiyev, The Natural Code of Numbered Magic Squares (1996)
• William H. Benson and Oswald Jacoby,“New Recreations with Magic Squares”. (New York: Dover, 1976).
• A 'perfect' magic square presented as a magic trick (Online Generator – Magic Square 4×4 using Javascript)
• Magic Squares of Order 4,5,6, and some theory, hbmeyer.de
• Evolving a Magic Square using Genetic Algorithms, dcs.napier.ac.uk
• Magic squares and magic cubes: examples of magic squares and magic cubes built with Ali Skalli's non iterativemethod, sites.google.com
7.9 Further reading• Block, Seymour (2009). Before Sudoku: The World of Magic Squares. Oxford University Press. ISBN
0195367901.
• McCranie, Judson (November 1988). “Magic Squares of All Orders”. Mathematics Teacher: 674–78.
• Ollerenshaw, Kathleen; Bree, David (October 1998). Most perfect pandiagonal magic squares: their construc-tion and enumeration. The Institute of Mathematics and its Applications. ISBN 978-0905091068.
• Semanisinova, Ingrid; Trenkler, Marian (August 2007). “Discovering the Magic of Magic Squares”. Math-ematics Teacher 101 (1): 32–39.
• King, J. R. (1963). “Magic Square Numbers”.
7.10 External links• Eaves, Laurence (2009). “Magic Square”. Sixty Symbols. Brady Haran for the University of Nottingham.
• Magic square at DMOZ
44 CHAPTER 7. MAGIC SQUARE
Semimagic square
(Ordinary) magic square
Panmagic square
Complete magic square
Most-perfect magic square
*
Euler diagram of requirements of some types of 4×4 magic squares. Cells of the same colour sum to the magic constant.* In 4×4 most-perfect magic squares, any 2 cells that are 2 cells diagonally apart (including wraparound) sum to half the magicconstant, hence any 2 such pairs also sum to the magic constant.
7.10. EXTERNAL LINKS 45
A geometric magic square.
46 CHAPTER 7. MAGIC SQUARE
Macau stamps featuring magic squares
Chapter 8
Main diagonal
In linear algebra, the main diagonal (sometimes principal diagonal, primary diagonal, leading diagonal, ormajor diagonal) of a matrixA is the collection of entriesAi,j where i = j . The following three matrices have theirmain diagonals indicated by red 1's:
1 0 00 1 00 0 1
1 0 0 00 1 0 00 0 1 0
1 0 00 1 00 0 10 0 0
The antidiagonal (sometimes counterdiagonal, secondary diagonal, orminor diagonal) of a dimensionN squarematrix, B , is the collection of entries Bi,j such that i+ j = N + 1 . That is, it runs from the top right corner to thebottom left corner:
0 0 10 1 01 0 0
8.1 See also• Diagonal matrix
• Trace
• Anti-diagonal matrix
8.2 References• Weisstein, Eric W., “Main diagonal”, MathWorld.
47
Chapter 9
Manin matrix
In mathematics,Manin matrices, named after Yuri Manin who introduced them around 1987–88,*[1]*[2]*[3] are aclass of matrices with elements in a not-necessarily commutative ring, which in a certain sense behave like matriceswhose elements commute. In particular there is natural definition of the determinant for them and most linear algebratheorems like Cramer's rule, Cayley–Hamilton theorem, etc. hold true for them. Any matrix with commuting ele-ments is a Manin matrix. These matrices have applications in representation theory in particular to Capelli's identity,Yangian and quantum integrable systems.Manin matrices are particular examples of Manin's general construction of“non-commutative symmetries”which canbe applied to any algebra. From this point of view they are“non-commutative endomorphisms”of polynomial algebraC[x1, ...xn]. Taking (q)-(super)-commuting variables one will get (q)-(super)-analogs of Manin matrices, which areclosely related to quantum groups. Manin works were influenced by the quantum group theory. He discovered thatquantized algebra of functions Funq(GL) can be defined by the requirement that T and T*t are simultaneously q-Manin matrices. In that sense it should be stressed that (q)-Manin matrices are defined only by half of the relationsof related quantum group Funq(GL), and these relations are enough for many linear algebra theorems.
9.1 Definition
9.1.1 Context
Matrices with generic noncommutative elements do not admit a natural construction of the determinant with valuesin a ground ring and basic theorems of the linear algebra fail to hold true. There are several modifications of thedeterminant theory: Dieudonné determinant which takes values in the abelianization K**/[K**, K**] of the multi-plicative group K** of ground ring K; and theory of quasideterminants. But the analogy between these determinantsand commutative determinants is not complete. On the other hand, if one considers certain specific classes of ma-trices with non-commutative elements, then there are examples where one can define the determinant and provelinear algebra theorems which are very similar to their commutative analogs. Examples include: quantum groups andq-determinant; Capelli matrix and Capelli determinant; super-matrices and Berezinian.Manin matrices is a general and natural class of matrices with not-necessarily commutative elements which admitnatural definition of the determinant and generalizations of the linear algebra theorems.
9.1.2 Formal definition
An n by m matrix M with entries Mij over a ring R (not necessarily commutative) is a Manin matrix if all elementsin a given column commute and if for all i,j,k,l it holds that [M ij ,Mkl] = [Mkj ,M il]. Here [a,b] denotes (ab − ba) thecommutator of a and b.*[3]The definition can be better seen from the following formulas. A rectangular matrix M is called a Manin matrix iffor any 2×2 submatrix, consisting of rows i and k, and columns j and l:
48
9.1. DEFINITION 49
· · · · · · · · · · · · · · ·· · · Mij · · · Mil · · ·· · · · · · · · · · · · · · ·· · · Mkj · · · Mkl · · ·· · · · · · · · · · · · · · ·
=
· · · · · · · · · · · · · · ·· · · a · · · b · · ·· · · · · · · · · · · · · · ·· · · c · · · d · · ·· · · · · · · · · · · · · · ·
the following commutation relations hold
ac = ca, bd = db, commute) column same the in (entries
ad− da = cb− bc, relation) commutation (cross.
9.1.3 Ubiquity of 2 × 2 Manin matrices
Below are presented some examples of the appearance of the Manin property in various very simple and naturalquestions concerning 2×2 matrices. The general idea is the following: consider well-known facts of linear algebraand look how to relax the commutativity assumption for matrix elements such that the results will be preserved to betrue. The answer is: if and only if M is a Manin matrix.*[3] The proofs of all observations is direct 1 line check.
Consider a 2×2 matrix M =
(a bc d
).
Observation 1. Coaction on a plane.Consider the polynomial ring C[x1, x2], and assume that the matrix elements a, b, c, d commute with x1, x2. Definey1, y2 by
(y1y2
)=
(a bc d
)(x1x2
).
Then y1, y2 commute among themselves if and only if M is a Manin matrix.Proof:
[y1, y2] = [ax1 + bx2, cx1 + dx2] = [a, c]x21 + [b, d]x22 + ([a, d] + [b, c])x1x2.
Requiring this to be zero, we get Manin's relations.Observation 2. Coaction on a super-plane.Consider the Grassmann algebra C[ψ1, ψ2], and assume that the matrix elements a, b, c, d commute with ψ1, ψ2.Define φ1, φ2 by
(ϕ1, ϕ2
)=(ψ1, ψ2
)(a bc d
).
Then φ1, φ2 are Grassmann variables (i.e. anticommute among themselves and φᵢ2=0) if and only if M is a Maninmatrix.Observations 1,2 holds true for general n × m Manin matrices. They demonstrate original Manin's approach asdescribed below (one should thought of usual matrices as homomorphisms of polynomial rings, while Manin matricesare more general “non-commutative homomorphisms”). Pay attention that polynomial algebra generators arepresented as column vectors, while Grassmann algebra as row-vectors, the same can be generalized to arbitrary pairof Koszul dual algebras and associated general Manin matrices.Observation 3. Cramer's rule. The inverse matrix is given by the standard formula
M−1 =1
ad− cb
(d −b−c a
)
50 CHAPTER 9. MANIN MATRIX
if and only if M is a Manin matrix.Proof:
(d −b−c a
)(a bc d
)=
(da− bc db− bd−ca+ ac −cb+ ad
)= if only and ifMmatrix Manin a is =
(ad− cb 0
0 ad− cb
).
Observation 4. Cayley–Hamilton theorem. The equality
M2 − (a+ d)M + (ad− cb)12×2 = 0
holds if and only if M is a Manin matrix.Observation 5. Multiplicativity of determinants.det*column(MN) = det*column(M)det(N) holds true for all complex-valued matrices N if and only if M is a Maninmatrix.Where det*column of 2×2 matrix is defined as ad − cb, i.e. elements from first column (a,c) stands first in theproducts.
9.1.4 Conceptual definition. Concept of “non-commutative symmetries”
According to Yu. Manin's ideology one can associate to any algebra certain bialgebra of its “non-commutativesymmetries (i.e. endomorphisms)". More generally to a pair of algebras A, B one can associate its algebra of“non-commutative homomorphisms”between A and B. These ideas are naturally related with ideas of non-commutativegeometry. Manin matrices considered here are examples of this general construction applied to polynomial algebrasC[x1, ...xn].The realm of geometry concerns of spaces, while the realm of algebra respectively with algebras, the bridge betweenthe two realms is association to each space an algebra of functions on it, which is commutative algebra. Many conceptsof geometry can be respelled in the language of algebras and vice versa.The idea of symmetry G of space space V can be seen as action of G on V, i.e. existence of a map G× V -> V.This idea can be translated in the algebraic language as existence of homomorphism Fun(G) ⊗ Fun(V) <- Fun(V)(as usually maps between functions and spaces go in opposite directions). Also maps from a space to itself can becomposed (they form a semigroup), hence a dual object Fun(G) is a bialgebra.Finally one can take these two properties as basics and give purely algebraic definition of“symmetry”which can beapplied to an arbitrary algebra (non-necessarily commutative):Definition. Algebra of non-commutative symmetries (endomorphisms) of some algebra A is a bialgebra End(A),such that there exists homomorphisms called coaction:coaction : End(A)⊗A← A,
which is compatible with a comultiplication in a natural way. Finally End(A) is required to satisfy only the relationswhich come from the above, no other relations, i.e. it is universal coacting bialgebra for A.Coaction should be thought as dual to action G× V -> V, that is why it is called coaction. Compatibility of thecomultiplication map with the coaction map, is dual to g (h v) = (gh) v. One can easyly write this compatibility.Somewhat surprising fact is that this construction applied to the polynomial algebra C[x1, ..., xn] will give not the usualalgebra of matrices Matn (more precisely algebra of function on it), but much bigger non-commutative algebra ofManin matrices (more precisely algebra generated by elements Mij. More precisely the following simple propositionshold true.Proposition. Consider polynomial algebra Pol = C[x1, ..., xn] and matrix M with elements in some algebra EndPol.The elements yi =
∑kMik⊗xk ∈ EndPol⊗Pol commute among themselves if and only ifM is a Manin matrix.
Corollary. The map xi 7→ yi =∑kMik ⊗ xk is homomorphism from Pol to EndPol ⊗ Pol. It defines coaction.
Indeed to ensure that the map is homomorphism the only thing we need to check is that yi commute among themselves.Proposition. Define the comultiplication map by the formula ∆(Mij) =
∑lMil ⊗Mlj . Then it is coassociative
and is compatible with coaction on the polynomial algebra defined in the previous proposition.
9.2. PROPERTIES 51
The two propositions above imply that the algebra generated by elements of a Manin matrix is a bialgebra coacting onthe polynomial algebra. If one does not impose other relations ones get algebra of non-commutative endomorphismsof the polynomial algebra.
9.2 Properties
9.2.1 Elementary examples and properties
• Any matrix with commuting elements is a Manin matrix.
• Any matrix whose elements from different rows commute among themselves (such matrices sometimes calledCartier-Foata matrices) is a Manin matrix.
• Any submatrix of a Manin matrix is a Manin matrix.
• One can interchange rows and columns in a Manin matrix the result will also be a Manin matrix. One can addrow or column multiplied by the central element to another row or column and results will be Manin matrixagain. I.e. one can make elementary transformations with restriction that multiplier is central.
• Consider two Manin matrices M,N such that their all elements commute, then the sum M+N and the productMN will also be Manin matrices.
• If matrix M and simultaneously transpose matrix M*t are Manin matrices, then all elements of M commutewith each other.
• No-go facts: M*k is not a Manin matrix in general (except k=−1 discussed below); neither det(M), nor Tr(M)are central in the algebra generated by Mij in general (in that respect Manin matrices differs from quantumgroups); det(e*M) ≠ e*Tr(M); log(det(M)) ≠ Tr(log(M)).
• Consider polynomial algebra C[xij] and denote by ∂ij the operators of differentiation with respect to
xij, form matrices X, D with the corresponding elements. Also consider variable z and corresponding differentialoperator ∂z . The following gives an example of a Manin matrix which is important for Capelli identities:
(zId Dt
X ∂zId
).
One can replace X, D by any matrices whose elements satisfy the relation: Xij Dkl - Dkl Xij = δikδkl, same about z andits derivative.Calculating the determinant of this matrix in two ways: direct and via Schur complement formula essentially givesthe Capelli identity and its generalization (see section 4.3.1,*[4] based on*[5]).
9.2.2 Determinant = column-determinant
The determinant of a Manin matrix can be defined by the standard formula, with the prescription that elements fromthe first columns comes first in the product.
9.2.3 Linear algebra theorems
Many linear algebra statements hold for Manin matrices even when R is not commutative. In particular, the determinantcan be defined in the standard way using permutations and it satisfies a Cramer's rule.*[3] MacMahon Master theoremholds true for Manin matrices and actually for their generalizations (super), (q), etc. analogs.Proposition. Cramer's rule (See*[2] or section 4.1.*[3]) The inverse to a Manin matrix M can be defined by thestandard formula: M−1 = 1
detcol(M)Madj , where M*adj is adjugate matrix given by the standard formula - its (i,j)-th
element is the column-determinant of the (n − 1) × (n − 1) matrix that results from deleting row j and column i of Mand multiplication by (−1)*i+j.
52 CHAPTER 9. MANIN MATRIX
The only difference with commutative case is that one should pay attention that all determinants are calculated ascolumn-determinants and also adjugate matrix stands on the right, while commutative inverse to the determinant ofM stands on the left, i.e. due to non-commutativity the order is important.Proposition. Inverse is also Manin. (See section 4.3.*[3]) Assume a two-sided inverse to a Manin matrix M exists,then it will also be a Manin matrix. Moreover det(M*−1) = (det(M))*−1.This proposition is somewhat non-trivial, it implies the result by Enriquez-Rubtsov and Babelon-Talon in the theoryof quantum integrable systems (see section 4.2.1*[4]).Proposition. Cayley-Hamilton theorem (See section 7.1.*[3])
detcolumn(t−M)|right substitutet=M = 0, i.e.∑
i=0...n
(−1)iσiMn−i = 0.
Where σi are coefficients of the characteristic polynomial detcolumn(t−M) =∑i=0...n(−1)iσitn−i .
Proposition. Newton identities (See section 7.2.1.*[3]) ∀k ≥ 0 : −(−1)kkσk =∑i=0...k−1 σiTr(M
k−i)
Where σi are coefficients of the characteristic polynomial detcolumn(t − M) =∑i=0...n(−1)iσitn−i , and by
convention σi=0, for i>n, where n is size of matrix M.Proposition. Determinant via Schur complement (See section 5.2.*[3]) Assume block matrix below is a Maninmatrix and two-sided inverses M*−1, A*−1, D*−1 exist, then
detcolumn(A BC d
)= detcolumn(A)detcolumn(D − CA−1B) = detcolumn(D)detcolumn(A−BD−1C).
Moreover Schur complements (D − CA−1B), (A−BD−1C) are Manin matrices.Proposition. MacMahon Master theorem*[6]
9.3 Examples and applications
9.3.1 Capelli matrix as Manin matrix, and center of U(gln)
The Capelli identity from 19th century gives one of the first examples of determinants for matrices with non-commuting elements. Manin matrices give a new look on this classical subject. This example is related to Liealgebra gln and serves as a prototype for more complicated applications to loop Lie algebra for gln, Yangian andintegrable systems.Take Eij be matrices with 1 at position (i,j) and zeros everywhere else. Form a matrix E with elements Eij at position(i,j). It is matrix with elements in ring of matrices Matn. It is not Manin matrix however there are modificationswhich transform it to Manin matrix as described below.Introduce a formal variable z which commute with Eij, respectively d/dz is operator of differentiation in z. The onlything which will be used that commutator of these operators is equal to 1.Observation. The matrix d/dzId− E/z is a Manin matrix.Here Id is identity matrix.
2 × 2 example: M =
(d/dz − E11/z −E12/z−E21/z d/z − E22/z
).
It is instructive to check the column commutativity requirement: [d/dz − E11/z,−E21/z] = [d/dz,−E21/z] +[−E11/z,−E21/z] = E21/z
2 − E21/z2 = 0 .
Observation. The matrix exp(−d/dz)(Id+ E/z) is a Manin matrix.The only fact required from Eij for these observations is that they satisfy commutation relations [Eij, Ekl]= δjkEil -δliEkj. So observations holds true if Eij are generators of the universal enveloping algebra of Lie algebra gln, or itsimages in any representation. For example one can take
9.4. FURTHER QUESTIONS 53
Eij = xi∂
∂xj; Eij =
n∑a=1
xia∂
∂xja; Eij = ψi
∂
∂ψj.
Here ψ are Grassmann variables.Observation. zn−1detcol(d/dz − E/z) = detcol(zd/dz − E − diag(n− 1, n− 2, ..., 1, 0))
On the right hand side of this equality one recognizes the Capelli determinant (or more precisely the Capelli char-acteristic polynomial), while on the left hand side one has a Manin matrix with its natural determinant. So Maninmatrices gives new look on Capelli's determinant. Moreover Capelli identity and its generalization can be derivedby techniques of Manin matrices. Also it gives an easy way to prove that this expression belongs to the center ofthe universal enveloping algebra U(gln), which is far from being trivial. Indeed, it's enough to check invariance withrespect to action of the group GLn by conjugation. detcol(d/dz − gEg−1/z) = detcol(g(d/dz − E/z)g−1) =det(g)detcol(d/dz − E/z)det(g−1) = detcol(d/dz − E/z) . So the only property used here is that det(gM) =det(Mg) = det(M)det(g) which is true for any Manin matrix M and any matrix g with central (e.g. scalar) ele-ments.
9.3.2 Loop algebra for gln, Langlands correspondence and Manin matrix
9.3.3 Yangian type matrices as Manin matrices
Observation. Let T(z) be a generating matrix of the Yangian for gln. Then the matrix exp(-d/dz) T(z) is a Maninmatrix.The quantum determinant for Yangian can be defined as exp (n d/dz)det*column(exp(-d/dz) T(z)). Pay attention thatexp(-d/dz) can be cancelled, so the expression does not depend on it. So the determinant in Yangian theory has naturalinterpretation via Manin matrices.For the sake of quantum integrable systems it is important to construct commutative subalgebras in Yangian. It iswell known that in the classical limit expressions Tr(T*k(z)) generate Poisson commutative subalgebra. The correctquantization of these expressions has been first proposed by the use of Newton identities for Manin matrices:Proposition. Coefficients of Tr(T(z+k-1)T(z+k-2)...T(z)) for all k commute among themselves. They generate com-mutative subalgebra in Yangian. The same subalgebra as coefficients of the characteristic polynomial det*column(1-exp(-d/dz) T(z)) .(The subalgebra sometimes called Bethe subalgebra, since Bethe ansatz is a method to find its joint eigpairs.)
9.4 Further questions
9.4.1 History
Manin proposed general construction of“non-commutative symmetries”in,*[1] the particular case which is calledManin matrices is discussed in,*[2] where some basic properties were outlined. The main motivation of these workswas to give another look on quantum groups. Quantum matrices Funq(GLn) can be defined as such matrices that Tand simultaneously T*t are q-Manin matrices (i.e. are non-commutative symmetries of q-commuting polynomials xixj = q xj xi. After original Manin's works there were only a few papers on Manin matrices until 2003. But around andsome after this date Manin matrices appeared in several not quite related areas:*[6] obtained certain noncommutativegeneralization of the MacMahon master identity, which was used in knot theory; applications to quantum integrablesystems, Lie algebras has been found in;*[4] generalizations of the Capelli identity involving Manin matrices appearedin.*[7] Directions proposed in these papers has been further developed.
9.5 References[1] Manin, Yuri (1987),“Some remarks on Koszul algebras and quantum groups”, Ann. de l'Inst. Fourier 37 (4): 191–205,
doi:10.5802/aif.1117, Zbl 0625.58040
54 CHAPTER 9. MANIN MATRIX
[2] Manin, Y. (1988). “Quantum Groups and Non Commutative Geometry”. Université de Montréal, Centre de RecherchesMathématiques: 91 pages. ISBN 2-921120-00-3. Zbl 0724.17006.
[3] A. Chervov; G. Falqui; V. Rubtsov (2009).“Algebraic properties of Manin matrices I”. Advances in Applied Mathematics(Elsevier) 43 (3): 239–315. arXiv:0901.0235. doi:10.1016/j.aam.2009.02.003. ISSN 0196-8858. Zbl 1230.05043.
[4] A. Chervov; G. Falqui (2007). “Manin matrices and Talalaev's formula”. Journal of Physics A 41 (19): 239–315.arXiv:0711.2236. doi:10.1088/1751-8113/41/19/194006. Zbl 1151.81022.
[5] Mukhin, E.; Tarasov, V.; Varchenko, A. (2006), A generalization of the Capelli identity, arXiv:math.QA/0610799
[6] Garoufalidis, Stavros; Le, T. T. Q.; Zeilberger, Doron (2006),“The Quantum MacMahon Master Theorem”, Proc. Natl.Acad. Sci. U.S.A. 103 (38): 13928–13931, arXiv:math/0303319, doi:10.1073/pnas.0606003103
[7] Caracciolo, Sergio; Sportiello, Andrea; Sokal, Alan D. (2009),“Noncommutative determinants, Cauchy–Binet formulae,and Capelli-type identities. I. Generalizations of the Capelli and Turnbull identities” (RESEARCH PAPER), Electron. J.Comb. 16 (1, number R103): 43, arXiv:0809.3516, ISSN 1077-8926, Zbl 1192.15001
• V. Rubtsov; D. Talalaev; A. Silantiev (2009).“Manin Matrices, Quantum Elliptic Commutative Families andCharacteristic Polynomial of Elliptic Gaudin Model”. SIGMA. arXiv:0908.4064. doi:10.3842/SIGMA.2009.110.Zbl 1190.37079.
• Suemi Rodriguez-Romo, Earl Taft (2002).“Some quantum-like Hopf algebras which remain noncommutativewhen q = 1”. Lett. Math. Phys. 61: 4150. doi:10.1023/A:1020221319846.
• Suemi Rodriguez-Romo, Earl Taft (2005).“A left quantum group”. J. Algebra 286: 154 160. doi:10.1016/j.jalgebra.2005.01.002.
• S. Wang (1998). “Quantum symmetry groups of finite spaces”. Comm. Math. Phys. 195: 195–211.arXiv:math/9807091. doi:10.1007/s002200050385.
• Teodor Banica, Julien Bichon, Benoit Collins (2007). “Noncommutative harmonic analysis with applicationsto probability”. Quantum permutation groups: a survey. Banach Center Publ. 78 (Warsaw: Polish Acad. Sci.).pp. 13–34. arXiv:math/0612724.
• Matjaz Konvalinka. “A generalization of Foata’s fundamental transformation and its applications to theright-quantum algebra”. arXiv:math.CO/0703203.
• Matjaz Konvalinka (2007). “Non-commutative Sylvester’s determinantal identity”. Electron. J. Combin.14 (1). Research Paper 42, 29 pp. (electronic). arXiv:math.CO/0703213
Chapter 10
Matrix analysis
In mathematics, particularly in linear algebra and applications, matrix analysis is the study of matrices and theiralgebraic properties.*[1] Some particular topics out of many include; operations defined on matrices (such as matrixaddition, matrix multiplication and operations derived from these), functions of matrices (such as matrix exponen-tiation and matrix logarithm, and even sines and cosines etc. of matrices),*[2] and the eigenvalues of matrices(eigendecomposition of a matrix, eigenvalue perturbation theory).
10.1 Matrix spaces
The set of all m×n matrices over a number field F denoted in this article Mmn(F) form a vector space. Examples ofF include the set of integers ℤ, the real numbers ℝ, and set of complex numbers ℂ. The spaces Mmn(F) and Mpq(F)are different spaces if m and p are unequal, and if n and q are unequal; for instance M32(F) ≠ M23(F). Two m×nmatrices A and B in Mmn(F) can be added together to form another matrix in the space Mmn(F):
A,B ∈Mmn(F ) , A+ B ∈Mmn(F )
and multiplied by a α in F, to obtain another matrix in Mmn(F):
α ∈ F , αA ∈Mmn(F )
Combining these two properties, a linear combination of matricesA andB are inMmn(F) is another matrix inMmn(F):
αA+ βB ∈Mmn(F )
where α and β are numbers in F.Any matrix can be expressed as a linear combination of basis matrices, which play the role of the basis vectors forthe matrix space. For example, for the set of 2×2 matrices over the field of real numbers, M22(ℝ), one legitimatebasis set of matrices is:
(1 00 0
),
(0 10 0
),
(0 01 0
),
(0 00 1
),
because any 2×2 matrix can be expressed as:
(a bc d
)= a
(1 00 0
)+ b
(0 10 0
)+ c
(0 01 0
)+ d
(0 00 1
),
where a, b, c,d are all real numbers. This idea applies to other fields and matrices of higher dimensions.
55
56 CHAPTER 10. MATRIX ANALYSIS
10.2 Determinants
Main article: Determinant
The determinant of a square matrix is an important property. The determinant indicates if a matrix is invertible(i.e. the inverse of a matrix exists when the determinant is nonzero). Determinants are used for finding eigenvaluesof matrices (see below), and for solving a system of linear equations (see Cramer's rule).
10.3 Eigenvalues and eigenvectors of matrices
Main article: Eigenvalues and eigenvectors
10.3.1 Definitions
An n×n matrix A has eigenvectors x and eigenvalues λ defined by the relation:
Ax = λx
In words, the matrix multiplication of A followed by an eigenvector x (here an n-dimensional column matrix), is thesame as multiplying the eigenvector by the eigenvalue. For an n×n matrix, there are n eigenvalues. The eigenvaluesare the roots of the characteristic polynomial:
pA(λ) = det(A− λI) = 0
where I is the n×n identity matrix.Roots of polynomials, in this context the eigenvalues, can all be different, or some may be equal (in which case eigen-value has multiplicity, the number of times an eigenvalue occurs). After solving for the eigenvalues, the eigenvectorscorresponding to the eigenvalues can be found by the defining equation.
10.3.2 Perturbations of eigenvalues
Main article: Eigenvalue perturbation
10.4 Matrix similarity
Main articles: Matrix similarity and Change of basis
Two n×n matrices A and B are similar if they are related by a similarity transformation:
B = PAP−1
The matrix P is called a similarity matrix, and is necessarily invertible.
10.4.1 Unitary similarity
Main article: Unitary matrix
10.5. CANONICAL FORMS 57
10.5 Canonical forms
For other uses, see Canonical form.
10.5.1 Row echelon form
Main article: Row echelon form
10.5.2 Jordan normal form
Main article: Jordan normal form
10.5.3 Weyr canonical form
Main article: Weyr canonical form
10.5.4 Frobenius normal form
Main article: Frobenius normal form
10.6 Triangular factorization
10.6.1 LU decomposition
Main article: LU decomposition
LU decomposition splits a matrix into a matrix product of an upper triangular matrix and a lower triangle matrix.
10.7 Matrix norms
Main article: Matrix norm
Since matrices form vector spaces, one can form axioms (analogous to those of vectors) to define a “size”of aparticular matrix. The norm of a matrix is a positive real number.
10.7.1 Definition and axioms
For all matrices A and B in Mmn(F), and all numbers α in F, a matrix norm, delimited by double vertical bars || ... ||,fulfills:*[note 1]
• Nonnegative:
∥A∥ ≥ 0
58 CHAPTER 10. MATRIX ANALYSIS
with equality only for A = 0, the zero matrix.
• Scalar multiplication:
∥αA∥ = |α|∥A∥
• The triangular inequality:
∥A+ B∥ ≤ ∥A∥+ ∥B∥
10.7.2 Frobenius norm
The Frobenius norm is analogous to the dot product of Euclidean vectors; multiply matrix elements entry-wise, addup the results, then take the positive square root:
∥A∥ =√A : A =
√√√√ m∑i=1
n∑j=1
(Aij)2
It is defined for matrices of any dimension (i.e. no restriction to square matrices).
10.8 Positive definite and semidefinite matrices
Main article: Positive definite matrix
10.9 Functions
Main article: Function (mathematics)
Matrix elements are not restricted to constant numbers, they can be mathematical variables.
10.9.1 Functions of matrices
A functions of a matrix takes in a matrix, and return something else (a number, vector, matrix, etc...).
10.9.2 Matrix-valued functions
A matrix valued function takes in something (a number, vector, matrix, etc...) and returns a matrix.
10.10 See also
10.10.1 Other branches of analysis• Mathematical analysis• Tensor analysis• Matrix calculus• Numerical analysis
10.11. FOOTNOTES 59
10.10.2 Other concepts of linear algebra
• Tensor product
• Spectrum of an operator
• Matrix geometrical series
10.10.3 Types of matrix
• Orthogonal matrix, unitary matrix
• Symmetric matrix, antisymmetric matrix
• Stochastic matrix
10.10.4 Matrix functions
• Matrix polynomial
• Matrix exponential
10.11 Footnotes[1] Some authors, e.g. Horn and Johnson, use triple vertical bars instead of double: |||A|||.
10.12 References
10.12.1 Notes[1] R. A. Horn, C. R. Johnson (2012). Matrix Analysis (2nd ed.). Cambridge University Press. ISBN 052-183-940-8.
[2] N. J. Higham (2000). Functions of Matrices: Theory and Computation. SIAM. ISBN 089-871-777-9.
10.12.2 Further reading
• C. Meyer (2000). Matrix Analysis and Applied Linear Algebra Book and Solutions Manual. Matrix Analysisand Applied Linear Algebra 2. SIAM. ISBN 089-871-454-0.
• T. S. Shores (2007). Applied Linear Algebra and Matrix Analysis. Undergraduate Texts in Mathematics.Springer. ISBN 038-733-195-6.
• Rajendra Bhatia (1997). Matrix Analysis. Matrix Analysis Series 169. Springer. ISBN 038-794-846-5.
• Alan J. Laub (2012). Computational Matrix Analysis. SIAM. ISBN 161-197-221-3.
Chapter 11
Matrix chain multiplication
Matrix chain multiplication (or Matrix Chain Ordering Problem, MCOP) is an optimization problem that canbe solved using dynamic programming. Given a sequence of matrices, the goal is to find the most efficient way tomultiply these matrices. The problem is not actually to perform the multiplications, but merely to decide the sequenceof the matrix multiplications involved.We have many options because matrix multiplication is associative. In other words, no matter how we parenthesizethe product, the result obtained will remain the same. For example, if we had four matrices A, B, C, and D, we wouldhave:
((AB)C)D = ((A(BC))D) = (AB)(CD) = A((BC)D) = A(B(CD)).
However, the order in which we parenthesize the product affects the number of simple arithmetic operations neededto compute the product, or the efficiency. For example, suppose A is a 10 × 30 matrix, B is a 30 × 5 matrix, and C isa 5 × 60 matrix. Then,
(AB)C = (10×30×5) + (10×5×60) = 1500 + 3000 = 4500 operationsA(BC) = (30×5×60) + (10×30×60) = 9000 + 18000 = 27000 operations.
Clearly the first method is more efficient. With this information, the problem statement can be refined, how do wedetermine the optimal parenthesization of a product of n matrices? We could go through each possible parenthe-sization (brute force), requiring a run-time that is exponential in the number of matrices, which is very slow andimpractical for large n. A quicker solution to this problem can be achieved by breaking up the problem into a setof related subproblems. By solving subproblems one time and reusing these solutions, we can drastically reduce therun-time required. This concept is known as dynamic programming.
11.1 A Dynamic Programming Algorithm
To begin, let us assume that all we really want to know is the minimum cost, or minimum number of arithmeticoperations, needed to multiply out the matrices. If we are only multiplying two matrices, there is only one way tomultiply them, so the minimum cost is the cost of doing this. In general, we can find the minimum cost using thefollowing recursive algorithm:
• Take the sequence of matrices and separate it into two subsequences.
• Find the minimum cost of multiplying out each subsequence.
• Add these costs together, and add in the cost of multiplying the two result matrices.
• Do this for each possible position at which the sequence of matrices can be split, and take the minimum overall of them.
60
11.2. MORE EFFICIENT ALGORITHMS 61
For example, if we have four matrices ABCD, we compute the cost required to find each of (A)(BCD), (AB)(CD),and (ABC)(D), making recursive calls to find the minimum cost to compute ABC, AB, CD, and BCD. We then choosethe best one. Better still, this yields not only the minimum cost, but also demonstrates the best way of doing themultiplication: group it the way that yields the lowest total cost, and do the same for each factor.Unfortunately, if we implement this algorithm we discover that it is just as slow as the naive way of trying all permu-tations! What went wrong? The answer is that we're doing a lot of redundant work. For example, above we madea recursive call to find the best cost for computing both ABC and AB. But finding the best cost for computing ABCalso requires finding the best cost for AB. As the recursion grows deeper, more and more of this type of unnecessaryrepetition occurs.One simple solution is called memoization: each time we compute the minimum cost needed to multiply out aspecific subsequence, we save it. If we are ever asked to compute it again, we simply give the saved answer, and donot recompute it. Since there are about n2/2 different subsequences, where n is the number of matrices, the spacerequired to do this is reasonable. It can be shown that this simple trick brings the runtime down to O(n3) from O(2*n),which is more than efficient enough for real applications. This is top-down dynamic programming.From *[1] Pseudocode:// Matrix Ai has dimension p[i-1] x p[i] for i = 1..n MatrixChainOrder(int p[]) // length[p] = n + 1 n = p.length - 1;// m[i,j] = Minimum number of scalar multiplications (i.e., cost) // needed to compute the matrix A[i]A[i+1]...A[j]= A[i..j] // cost is zero when multiplying one matrix for (i = 1; i <= n; i++) m[i,i] = 0; for (L=2; L<=n; L++) // Lis chain length for (i=1; i<=n-L+1; i++) j = i+L-1; m[i,j] = MAXINT; for (k=i; k<=j-1; k++) // q = cost/scalarmultiplications q = m[i,k] + m[k+1,j] + p[i-1]*p[k]*p[j]; if (q < m[i,j]) m[i,j] = q; s[i,j]=k; // s[i,j] = Secondauxiliary table that stores k // k = Index that achieved optimal cost
• Note : The first index for p is 0 and the first index for m and s is 1
Another solution is to anticipate which costs we will need and precompute them. It works like this:
• For each k from 2 to n, the number of matrices:
• Compute the minimum costs of each subsequence of length k, using the costs already computed.
The code in java using zero based array indexes along with a convenience method for printing the solved order ofoperations:public class MatrixOrderOptimization protected int[][]m; protected int[][]s; public void matrixChainOrder(int[]p) int n = p.length - 1; m = new int[n][n]; s = new int[n][n]; for (int ii = 1; ii < n; ii++) for (int i = 0; i < n -ii; i++) int j = i + ii; m[i][j] = Integer.MAX_VALUE; for (int k = i; k < j; k++) int q = m[i][k] + m[k+1][j] +p[i]*p[k+1]*p[j+1]; if (q < m[i][j]) m[i][j] = q; s[i][j] = k; public void printOptimalParenthesizations() boolean[] inAResult = new boolean[s.length]; printOptimalParenthesizations(s, 0, s.length - 1, inAResult); voidprintOptimalParenthesizations(int[][]s, int i, int j, /* for pretty printing: */ boolean[] inAResult) if (i != j) print-OptimalParenthesizations(s, i, s[i][j], inAResult); printOptimalParenthesizations(s, s[i][j] + 1, j, inAResult); Stringistr = inAResult[i] ? "_result " : " "; String jstr = inAResult[j] ? "_result " : " "; System.out.println(" A_”+ i + istr+ "* A_”+ j + jstr); inAResult[i] = true; inAResult[j] = true;
At the end of this program, we have the minimum cost for the full sequence. Although, this algorithm requiresO(n3) time, this approach has practical advantages that it requires no recursion, no testing if a value has alreadybeen computed, and we can save space by throwing away some of the subresults that are no longer required. This isbottom-up dynamic programming: a second way by which this problem can be solved.
11.2 More Efficient Algorithms
There are algorithms that are more efficient than the O(n3) dynamic programming algorithm, though they are morecomplex.
62 CHAPTER 11. MATRIX CHAIN MULTIPLICATION
11.2.1 Hu & Shing (1981)
An algorithm published in 1981 by Hu and Shing achieves O(n log n) complexity.*[2]*[3]*[4] They showed how thematrix chain multiplication problem can be transformed (or reduced) into the problem of triangulation of a regularpolygon. The polygon is oriented such that there is a horizontal bottom side, called the base, which represents thefinal result. The other n sides of the polygon, in the clockwise direction, represent the matrices. The vertices on eachend of a side are the dimensions of the matrix represented by that side. With n matrices in the multiplication chainthere are n−1 binary operations and Cn−1 ways of placing parenthesizes, where Cn−1 is the (n−1)-th Catalan number.The algorithm exploits that there are also Cn−1 possible triangulations of a polygon with n+1 sides.This image illustrates possible triangulations of a regular hexagon. These correspond to the different ways that paren-theses can be placed to order the multiplications for a product of 5 matrices.
For the example below, there are four sides: A, B, C and the final result ABC. A is a 10×30 matrix, B is a 30×5matrix, C is a 5×60 matrix, and the final result is a 10×60 matrix. The regular polygon for this example is a 4-gon,i.e. a square:The matrix product AB is a 10x5 matrix and BC is a 30x60 matrix. The two possible triangulations in this exampleare:
• Polygon representation of (AB)C
• Polygon representation of A(BC)
The cost of a single triangle in terms of the number of multiplications needed is the product of its vertices. The totalcost of a particular triangulation of the polygon is the sum of the costs of all its triangles:
(AB)C: (10×30×5) + (10×5×60) = 1500 + 3000 = 4500 multiplicationsA(BC): (30×5×60) + (10×30×60) = 9000 + 18000 = 27000 multiplications
Hu & Shing developed an algorithm that finds an optimum solution for the minimum cost partition problem in O(nlog n) time.
11.3 Generalizations
The matrix chain multiplication problem generalizes to solving a more abstract problem: given a linear sequence ofobjects, an associative binary operation on those objects, and a way to compute the cost of performing that operationon any two given objects (as well as all partial results), compute the minimum cost way to group the objects to apply
11.4. REFERENCES 63
10
30
A
5B
60
C
ABC
the operation over the sequence.*[5] One somewhat contrived special case of this is string concatenation of a list ofstrings. In C, for example, the cost of concatenating two strings of length m and n using strcat is O(m + n), since weneed O(m) time to find the end of the first string and O(n) time to copy the second string onto the end of it. Usingthis cost function, we can write a dynamic programming algorithm to find the fastest way to concatenate a sequenceof strings. However, this optimization is rather useless because we can straightforwardly concatenate the strings intime proportional to the sum of their lengths. A similar problem exists for singly linked lists.Another generalization is to solve the problem when parallel processors are available. In this case, instead of addingthe costs of computing each factor of a matrix product, we take the maximum because we can do them simultaneously.This can drastically affect both the minimum cost and the final optimal grouping; more “balanced”groupings thatkeep all the processors busy are favored. There are even more sophisticated approaches.*[6]
11.4 References
[1] Cormen, Thomas H; Leiserson, Charles E; Rivest, Ronald L; Stein, Clifford (2001). “15.2: Matrix-chain multiplication”. Introduction to Algorithms. Second Edition. MIT Press and McGraw-Hill. pp. 331–338. ISBN 0-262-03293-7.
[2] Hu, TC; Shing, MT (1981). Computation of Matrix Chain Products, Part I, Part II (PDF) (Technical report). StanfordUniversity, Department of Computer Science. STAN-CS-TR-81-875.
64 CHAPTER 11. MATRIX CHAIN MULTIPLICATION
[3] Hu, TC; Shing, MT (1982).“Computation of Matrix Chain Products, Part I”(PDF). SIAM Journal on Computing (Societyfor Industrial and Applied Mathematics) 11 (2): 362–373. doi:10.1137/0211028. ISSN 0097-5397.
[4] Hu, TC; Shing, MT (1984). “Computation of Matrix Chain Products, Part II” (PDF). SIAM Journal on Computing(Society for Industrial and Applied Mathematics) 13 (2): 228–251. doi:10.1137/0213017. ISSN 0097-5397.
[5] G. Baumgartner, D. Bernholdt, D. Cociorva, R. Harrison, M. Nooijen, J. Ramanujam and P. Sadayappan. A Perfor-mance Optimization Framework for Compilation of Tensor Contraction Expressions into Parallel Programs. 7th Interna-tional Workshop on High-Level Parallel Programming Models and Supportive Environments (HIPS '02). Fort Lauderdale,Florida. 2002 available at http://citeseer.ist.psu.edu/610463.html and at http://www.csc.lsu.edu/~gb/TCE/Publications/OptFramework-HIPS02.pdf
[6] Heejo Lee, Jong Kim, Sungje Hong, and Sunggu Lee. Processor Allocation and Task Scheduling of Matrix Chain Productson Parallel Systems. IEEE Trans. on Parallel and Distributed Systems, Vol. 14, No. 4, pp. 394–407, Apr. 2003
Chapter 12
Matrix congruence
In mathematics, two matrices A and B over a field are called congruent if there exists an invertible matrix P over thesame field such that
P*TAP = B
where “T”denotes the matrix transpose. Matrix congruence is an equivalence relation.Matrix congruence arises when considering the effect of change of basis on the Gram matrix attached to a bilinearform or quadratic form on a finite-dimensional vector space: two matrices are congruent if and only if they representthe same bilinear form with respect to different bases.Note that Halmos*[1] defines congruence in terms of conjugate transpose (with respect to a complex inner productspace) rather than transpose, but this definition has not been adopted by most other authors.
12.1 Congruence over the reals
Sylvester's law of inertia states that two congruent symmetric matrices with real entries have the same numbersof positive, negative, and zero eigenvalues. That is, the number of eigenvalues of each sign is an invariant of theassociated quadratic form.*[2]
12.2 See also• Congruence relation
• Matrix similarity
• Matrix equivalence
12.3 References[1] Halmos, Paul R. (1958). Finite dimensional vector spaces. van Nostrand. p. 134.
[2] Sylvester, J J (1852). “A demonstration of the theorem that every homogeneous quadratic polynomial is reducible byreal orthogonal substitutions to the form of a sum of positive and negative squares” (PDF). Philosophical Magazine IV:138–142. Retrieved 2007-12-30.
• Gruenberg, K.W.; Weir, A.J. (1967). Linear geometry. van Nostrand. p. 80.
• Hadley, G. (1961). Linear algebra. Addison-Wesley. p. 253.
• Herstein, I.N. (1975). Topics in algebra. Wiley. p. 352. ISBN 0-471-02371-X.
65
66 CHAPTER 12. MATRIX CONGRUENCE
• Mirsky, L. (1990). An introduction to linear algebra. Dover Publications. p. 182. ISBN 0-486-66434-1.
• Marcus, Marvin; Minc, Henryk (1992). A survey of matrix theory and matrix inequalities. Dover Publications.p. 81. ISBN 0-486-67102-X.
• Norman, C.W. (1986). Undergraduate algebra. Oxford University Press. p. 354. ISBN 0-19-853248-2.
Chapter 13
Matrix consimilarity
In linear algebra, two n-by-n matrices A and B are called consimilar if
A = SBS−1
for some invertible n × n matrix S , where S denotes the elementwise complex conjugation. So for real matricessimilar by some real matrix S , consimilarity is the same as matrix similarity.Like ordinary similarity, consimilarity is an equivalence relation on the set of n× n matrices, and it is reasonable toask what properties it preserves.The theory of ordinary similarity arises as a result of studying linear transformations referred to different bases.Consimilarity arises as a result of studying antilinear transformations referred to different bases.A matrix is consimilar to itself, its complex conjugate, its transpose and its adjoint matrix. Every matrix is consimilarto a real matrix and to a Hermitian matrix. There is a standard form for the consimilarity class, analogous to theJordan normal form.
13.1 References• Hong, YooPyo; Horn, Roger A. (April 1988). “A canonical form for matrices under consimilarity”. LinearAlgebra and its Applications 102: 143–168. doi:10.1016/0024-3795(88)90324-2. Zbl 0657.15008.
• Horn, Roger A.; Johnson, Charles R. (1985). Matrix analysis. Cambridge: Cambridge University Press. ISBN0-521-38632-2. Zbl 0576.15001. (sections 4.5 and 4.6 discuss consimilarity)
67
Chapter 14
Matrix difference equation
Amatrix difference equation*[1]*[2] is a difference equation in which the value of a vector (or sometimes, a matrix)of variables at one point in time is related to its own value at one or more previous points in time, using matrices.Occasionally, the time-varying entity may itself be a matrix instead of a vector. The order of the equation is themaximum time gap between any two indicated values of the variable vector. For example,
xt = Axt−1 +Bxt−2
is an example of a second-order matrix difference equation, in which x is an n × 1 vector of variables and A and Bare n×n matrices. This equation is homogeneous because there is no vector constant term added to the end of theequation. The same equation might also be written as
xt+2 = Axt+1 +Bxt
or as
xn = Axn−1 +Bxn−2
The most commonly encountered matrix difference equations are first-order.
14.1 Non-homogeneous first-order matrix difference equations and thesteady state
An example of a non-homogeneous first-order matrix difference equation is
xt = Axt−1 + b
with additive constant vector b. The steady state of this system is a value x* of the vector x which, if reached, wouldnot be deviated from subsequently. x* is found by setting xt = xt−1 = x∗ in the difference equation and solving forx* to obtain
x∗ = [I −A]−1b
where I is the n×n identity matrix, and where it is assumed that [I − A] is invertible. Then the non-homogeneousequation can be rewritten in homogeneous form in terms of deviations from the steady state:
[xt − x∗] = A[xt−1 − x∗].
68
14.2. STABILITY OF THE FIRST-ORDER CASE 69
14.2 Stability of the first-order case
The first-order matrix difference equation [xt - x*] = A[xt−1-x*] is stable—that is, xt converges asymptotically to thesteady state x*—if and only if all eigenvalues of the transition matrix A (whether real or complex) have an absolutevalue which is less than 1.
14.3 Solution of the first-order case
Assume that the equation has been put in the homogeneous form yt = Ayt−1 . Then we can iterate and substituterepeatedly from the initial condition y0 , which is the initial value of the vector y and which must be known in orderto find the solution:
y1 = Ay0,
y2 = Ay1 = AAy0 = A2y0,
y3 = Ay2 = AA2y0 = A3y0,
and so forth.Assume A is diagonalizable. By induction, we obtain the solution in terms of t:
yt = Aty0 = PDtP−1y0,
where P is an n × n matrix whose columns are the eigenvectors of A (assuming the eigenvalues are all distinct) andD is an n × n diagonal matrix whose diagonal elements are the eigenvalues of A. This solution motivates the abovestability result: At shrinks to the zero matrix over time if and only if the eigenvalues of A are all less than unity inabsolute value.
14.4 Extracting the dynamics of a single scalar variable from a first-ordermatrix system
Starting from the n-dimensional system yt = Ayt−1, we can extract the dynamics of one of the state variables, sayy1. The above solution equation for yt shows that the solution for y1,t is in terms of the n eigenvalues of A. Thereforethe equation describing the evolution of y1 by itself must have a solution involving those same eigenvalues. Thisdescription intuitively motivates the equation of evolution of y1, which is
y1,t = a1y1,t−1 + a2y1,t−2 + · · ·+ any1,t−n
where the parameters ai are from the characteristic equation of the matrix A:
λn − a1λn−1 − a2λn−2 − · · · − anλ0 = 0.
Thus each individual scalar variable of an n-dimensional first-order linear system evolves according to a univariate n*thdegree difference equation, which has the same stability property (stable or unstable) as does the matrix differenceequation.
14.5 Solution and stability of higher-order cases
Matrix difference equations of higher order—that is, with a time lag longer than one period—can be solved, and theirstability analyzed, by converting them into first-order form using a block matrix. For example, suppose we have thesecond-order equation
70 CHAPTER 14. MATRIX DIFFERENCE EQUATION
xt = Axt−1 +Bxt−2
with the variable vector x being n×1 and A and B being n×n. This can be stacked in the form
(xtxt−1
)=
(A BI 0
)(xt−1
xt−2
),
where I is the n×n identity matrix and 0 is the n×n zero matrix. Then denoting the 2n×1 stacked vector of currentand once-lagged variables as zt and the 2n×2n block matrix as L, we have as before the solution
zt = Ltz0.
Also as before, this stacked equation and thus the original second-order equation are stable if and only if all eigenvaluesof the matrix L are smaller than unity in absolute value.
14.6 Nonlinear matrix difference equations: Riccati equations
In linear-quadratic-Gaussian control, there arises a nonlinear matrix equation for the evolution backwards throughtime of a current-and-future-cost matrix, denoted below as H. This equation is called a discrete dynamic Riccatiequation, and it arises when a variable vector evolving according to a linear matrix difference equation is to becontrolled by manipulating an exogenous vector in order to optimize a quadratic cost function. This Riccati equationassumes the following form or a similar form:
Ht−1 = K +A′HtA−A′HtC(C′HtC +R)−1C ′HtA,
where H, K, and A are n×n, C is n×k, R is k×k, n is the number of elements in the vector to be controlled, and k isthe number of elements in the control vector. The parameter matrices A and C are from the linear equation, and theparameter matrices K and R are from the quadratic cost function. See here for details.In general this equation cannot be solved analytically for Ht in terms of t ; rather, the sequence of values for Ht
is found by iterating the Riccati equation. However, it was shown in *[3] that this Riccati equation can be solvedanalytically if R is the zero matrix and n=k+1, by reducing it to a scalar rational difference equation; moreover, forany k and n if the transition matrix A is nonsingular then the Riccati equation can be solved analytically in terms ofthe eigenvalues of a matrix, although these may need to be found numerically.*[4]In most contexts the evolution of H backwards through time is stable, meaning that H converges to a particular fixedmatrix H* which may be irrational even if all the other matrices are rational. See also Stochastic control#Discretetime.A related Riccati equation*[5] is
Xt+1 = −(E +BXt)(C +AXt)−1
in which the matrices X, A, B, C, and E are all n×n. This equation can be solved explicitly. Suppose Xt = NtD−1t ,
which certainly holds for t=0 with N0 = X0 and with D0 equal to the identity matrix. Then using this in the differenceequation yields
Xt+1 = −(E +BNtD−1t )DtD
−1t (C +ANtD
−1t )−1
= −(EDt +BNt)[(C +ANtD−1t )Dt]
−1
= −(EDt +BNt)[CDt +ANt]−1
14.7. SEE ALSO 71
= Nt+1D−1t+1,
so by induction the form Xt = NtD−1t holds for all t. Then the evolution of N and D can be written as
(Nt+1
Dt+1
)=
(−B −EA C
)(NtDt
)≡ J
(NtDt
).
Thus
(NtDt
)= J t
(N0
D0
).
14.7 See also• Matrix differential equation
• Difference equation
• Dynamical system
• Matrix Riccati equation#Mathematical description of the problem and solution
14.8 References[1] Cull, Paul; Flahive, Mary; and Robson, Robbie. Difference Equations: From Rabbits to Chaos, Springer, 2005, chapter 7;
ISBN 0-387-23234-6.
[2] Chiang, Alpha C., Fundamental Methods of Mathematical Economics, third edition, McGraw-Hill, 1984: 608–612.
[3] Balvers, Ronald J., and Mitchell, Douglas W.,“Reducing the dimensionality of linear quadratic control problems,”Journalof Economic Dynamics and Control 31, 2007, 141–159.
[4] Vaughan, D. R., “A nonrecursive algebraic solution for the discrete Riccati equation,”IEEE Transactions on AutomaticControl 15, 1970, 597-599.
[5] Martin, C. F., and Ammar, G., “The geometry of the matrix Riccati equation and associated eigenvalue method,”inBittani, Laub, and Willems (eds.), The Riccati Equation, Springer-Verlag, 1991.
Chapter 15
Matrix equivalence
In linear algebra, two rectangular m-by-n matrices A and B are called equivalent if
B = Q−1AP
for some invertible n-by-n matrix P and some invertible m-by-m matrix Q. Equivalent matrices represent the samelinear transformation V → W under two different choices of a pair of bases of V and W, with P and Q being thechange of basis matrices in V and W respectively.The notion of equivalence should not be confused with that of similarity, which is only defined for square matrices,and is much more restrictive (similar matrices are certainly equivalent, but equivalent square matrices need not besimilar). That notion corresponds to matrices representing the same endomorphism V → V under two differentchoices of a single basis of V, used both for initial vectors and their images.
15.1 Properties
Matrix equivalence is an equivalence relation on the space of rectangular matrices.For two rectangular matrices of the same size, their equivalence can also be characterized by the following conditions
• The matrices can be transformed into one another by a combination of elementary row and column operations.
• Two matrices are equivalent if and only if they have the same rank.
15.2 See also• Matrix similarity
• Row equivalence
• Matrix congruence
72
Chapter 16
Matrix group
In mathematics, a matrix group is a group G consisting of invertible matrices over some field K, usually fixed inadvance, with operations of matrix multiplication and inversion. More generally, one can consider n × n matricesover a commutative ring R. (The size of the matrices is restricted to be finite, as any group can be represented as agroup of infinite matrices over any field.) A linear group is an abstract group that is isomorphic to a matrix groupover a field K, in other words, admitting a faithful, finite-dimensional representation over K.Any finite group is linear, because it can be realized by permutation matrices using Cayley's theorem. Among infinitegroups, linear groups form an interesting and tractable class. Examples of groups that are not linear include all
“sufficiently large”groups; for example, the infinite symmetric group of permutations of an infinite set.
16.1 Basic examples
The set MR(n,n) of n × n matrices over a commutative ring R is itself a ring under matrix addition and multiplication.The group of units of MR(n,n) is called the general linear group of n × n matrices over the ring R and is denotedGLn(R) or GL(n,R). All matrix groups are subgroups of some general linear group.
16.2 Classical groups
Main article: Classical group
Some particularly interesting matrix groups are the so-called classical groups. When the ring of coefficients of thematrix group is the real numbers, these groups are the classical Lie groups. When the underlying ring is a finite fieldthe classical groups are groups of Lie type. These groups play an important role in the classification of finite simplegroups.
16.3 Finite groups as matrix groups
Every finite group is isomorphic to some matrix group. This is similar to Cayley's theorem which states that everyfinite group is isomorphic to some permutation group. Since the isomorphism property is transitive one need onlyconsider how to form a matrix group from a permutation group.Let G be a permutation group on n points (Ω = 1,2,…,n) and let g1,...,gk be a generating set for G. The generallinear group GLn(C) of n×n matrices over the complex numbers acts naturally on the vector space C*n. Let B=b1,…,bn be the standard basis for C*n. For each gi let M i in GLn(C) be the matrix which sends each bj to bgi(j). Thatis, if the permutation gi sends the point j to k then M i sends the basis vector bj to bk. Let M be the subgroup ofGLn(C) generated by M1,…,Mk. The action of G on Ω is then precisely the same as the action of M on B. It canbe proved that the function taking each gi to M i extends to an isomorphism and thus every group is isomorphic to amatrix group.
73
74 CHAPTER 16. MATRIX GROUP
Note that the field (C in the above case) is irrelevant since M contains only elements with entries 0 or 1. One can justas easily perform the construction for an arbitrary field since the elements 0 and 1 exist in every field.As an example, let G = S3, the symmetric group on 3 points. Let g1 = (1,2,3) and g2 = (1,2). Then
M1 =
0 0 11 0 00 1 0
M2 =
0 1 01 0 00 0 1
M1b1 = b2, M1b2 = b3 and M1b3 = b1. Likewise, M2b1 = b2, M2b2 = b1 and M2b3 = b3.
16.4 Representation theory and character theory
Linear transformations and matrices are (generally speaking) well-understood objects in mathematics and have beenused extensively in the study of groups. In particular representation theory studies homomorphisms from a groupinto a matrix group and character theory studies homomorphisms from a group into a field given by the trace of arepresentation.
16.5 Examples• See table of Lie groups, list of finite simple groups, and list of simple Lie groups for many examples.
• See list of transitive finite linear groups.
• In 2000 a longstanding conjecture was resolved when it was shown that the braid groups Bn are linear for alln.*[1]
16.6 References• Brian C. Hall Lie Groups, Lie Algebras, and Representations: An Elementary Introduction, 1st edition, Springer,
2006. ISBN 0-387-40122-9
• Wulf Rossmann, Lie Groups: An Introduction Through Linear Groups (Oxford Graduate Texts in Mathematics),Oxford University Press ISBN 0-19-859683-9.
• La géométrie des groupes classiques, J. Dieudonné. Springer, 1955. ISBN 1-114-75188-X
• The classical groups, H. Weyl, ISBN 0-691-05756-7
[1] Stephen J. Bigelow (December 13, 2000),“Braid groups are linear”(PDF), Journal of the American Mathematical Society14 (2): 471–486
16.7 Further reading• Suprnenko, D.A. (1976). Matrix groups. Translations of mathematical monographs 45. American Mathemat-
ical Society. ISBN 0-8218-1595-4.
16.8 External links• Linear groups, Encyclopaedia of Mathematics
Chapter 17
Matrix of ones
In mathematics, a matrix of ones or all-ones matrix is a matrix where every element is equal to one.*[1] Examplesof standard notation are given below:
J2 =
(1 11 1
); J3 =
1 1 11 1 11 1 1
; J2,5 =
(1 1 1 1 11 1 1 1 1
).
Some sources call the all-ones matrix the unit matrix,*[2] but that term may also refer to the identity matrix, adifferent matrix.
17.1 Properties
For an n×n matrix of ones J, the following properties hold:
• The trace of J is n,*[3] and the determinant is 1 if n is 1, or 0 otherwise.
• The rank of J is 1 and the eigenvalues are n (once) and 0 (n−1 times).*[4]
• J is positive semi-definite matrix. This follows from the previous property.
• Jk = nk−1J, for k = 1, 2, . . . . *[5]
• The matrix 1nJ is idempotent. This is a simple corollary of the above.*[5]
• exp(J) = I + en−1n J, where exp(J) is the matrix exponential.
• J is the neutral element of the Hadamard product.*[6]
• IfA is the adjacency matrix of a n-vertex undirected graphG, and J is the all-ones matrix of the same dimension,then G is a regular graph if and only if AJ = JA.*[7]
17.2 References[1] Horn, Roger A.; Johnson, Charles R. (2012), “0.2.8 The all-ones matrix and vector”, Matrix Analysis, Cambridge
University Press, p. 8, ISBN 9780521839402.
[2] Weisstein, Eric W., “Unit Matrix”, MathWorld.
[3] Stanley, Richard P. (2013), Algebraic Combinatorics: Walks, Trees, Tableaux, and More, Springer, Lemma 1.4, p. 4, ISBN9781461469988.
[4] Stanley (2013); Horn & Johnson (2012), p. 65.
75
76 CHAPTER 17. MATRIX OF ONES
[5] Timm, Neil H. (2002), Applied Multivariate Analysis, Springer texts in statistics, Springer, p. 30, ISBN 9780387227719.
[6] Smith, Jonathan D. H. (2011), Introduction to Abstract Algebra, CRC Press, p. 77, ISBN 9781420063721.
[7] Godsil, Chris (1993), Algebraic Combinatorics, CRC Press, Lemma 4.1, p. 25, ISBN 9780412041310.
Chapter 18
Matrix regularization
In the field of statistical learning theory, matrix regularization generalizes notions of vector regularization to caseswhere the object to be learned is a matrix. The purpose of regularization is to enforce conditions, for example sparsityor smoothness, that can produce stable predictive functions. For example, in the more common vector framework,Tikhonov regularization optimizes over
minx∥Ax− y∥2 + λ∥x∥2
to find a vector, x , that is a stable solution to the regression problem. When the system is described by a matrixrather than a vector, this problem can be written as
minX∥AX − Y ∥2 + λ∥X∥2
where the vector norm enforcing a regularization penalty on x has been extended to a matrix norm on X .Matrix Regularization has applications in matrix completion, multivariate regression, and multi-task learning. Ideasof feature and group selection can also be extended to matrices, and these can be generalized to the nonparametriccase of multiple kernel learning.
18.1 Basic definition
Consider a matrix W to be learned from a set of examples, S = (Xti , y
ti) , where i goes from 1 to n , and t goes
from 1 to T . Let each input matrix Xi be ∈ RDT , and let W be of size D × T . A general model for the output ycan be posed as
yti = ⟨W,Xti ⟩F
where the inner product is the Frobenius inner product. For different applications the matrices Xi will have differentforms,*[1] but for each of these the optimization problem to infer W can be written as
minW∈H
E(W ) +R(W )
whereE defines the empirical error for a givenW , andR(W ) is a matrix regularization penalty. The functionR(W )is typically chosen to be convex, and is often selected to enforce sparsity (using ℓ1 -norms) and/or smoothness (usingℓ2 -norms). Finally, W is in the space of matrices,H , with Forbenius inner product,.
77
78 CHAPTER 18. MATRIX REGULARIZATION
18.2 General applications
18.2.1 Matrix completion
In the problem of matrix completion, the matrix Xti takes the form
Xti = et ⊗ e′i
where (et)t and (e′i)i are the canonical basis in RT and RD . In this case the role of the Frobenius inner product isto select individual elements, wti , from the matrix W . Thus, the output, y , is a sampling of entries from the matrixW .The problem of reconstructing W from a small set of sampled entries is possible only under certain restrictions onthe matrix, and these restrictions can be enforced by a regularization function. For example, it might be assumed thatW is low-rank, in which case the regularization penalty can take the form of a nuclear norm.*[2]
R(W ) = λ∥W∥∗ = λ∑|σi|
where σi , with i from 1 to minD,T , are the singular values of W .
18.2.2 Multivariate regression
Models used in multivariate regression are parameterized by a matrix of coefficients. In the Frobenius inner productabove, each matrix X is
Xti = et ⊗ xi
such that the output of the inner product is the dot product of one row of the input with one column of the coefficientmatrix. The familiar form of such models is
Y = XW + b
Many of the vector norms used in single variable regression can be extended to the multivariate case. One exampleis the squared Frobenius norm, which can be viewed as an ℓ2 -norm acting either entrywise, or on the singular valuesof the matrix:
R(W ) = λ∥W∥2F = λ∑∑
|wij |2 = λTr(W ∗W ) = λ∑
σ2i .
In the multivariate case the effect of regularizing with the Frobenius norm is the same as the vector case; very complexmodels will have larger norms, and, thus, will be penalized more.
18.2.3 Multi-task learning
The setup for multi-task learning is almost the same as the setup for multivariate regression. The primary differenceis that the input variables are also indexed by task (columns of Y ). The representation with the Frobenius innerproduct is then
Xti = et ⊗ xti.
The role of matrix regularization in this setting can be the same as in multivariate regression, but matrix norms canalso be used to couple learning problems across tasks. In particular, note that for the optimization problem
18.3. SPECTRAL REGULARIZATION 79
minW∥XW − Y ∥22 + λ∥W∥22
the solutions corresponding to each column of Y are decoupled. That is, the same solution can be found by solvingthe joint problem, or by solving an isolated regression problem for each column. The problems can be coupled byadding an additional regulatization penalty on the covariance of solutions
minW,Ω∥XW − Y ∥22 + λ1∥W∥22 + λ2 Tr(WTΩ−1W )
where Ω models the relationship between tasks. This scheme can be used to both enforce similarity of solutionsacross tasks, and to learn the specific structure of task similarity by alternating between optimizations of W and Ω.*[3] When the relationship between tasks is known to lie on a graph, the Laplacian matrix of the graph can be usedto couple the learning problems.
18.3 Spectral regularization
Regularization by spectral filtering has been used to find stable solutions to problems such as those discussed above byaddressing ill-posed matrix inversions (see for example Filter function for Tikhonov regularization). In many casesthe regularization function acts on the input (or kernel) to ensure a bounded inverse by eliminating small singularvalues, but it can also be useful to have spectral norms that act on the matrix that is to be learned.There are a number of matrix norms that act on the singular values of the matrix. Frequently used examples includethe Schatten p-norms, with p = 1 or 2. For example, matrix regularization with a Schatten 1-norm, also called thenuclear norm, can be used to enforce sparsity in the spectrum of a matrix. This has been used in the context ofmatrix completion when the matrix in question is believed to have a restricted rank.*[2] In this case the optimizationproblem becomes:
min ∥W∥∗ subject to Wi,j = Yij .
Spectral Regularization is also used to enforce a reduced rank coefficient matrix in multivariate regression.*[4] Inthis setting, a reduced rank coefficient matrix can be found by keeping just the top n singular values, but this can beextended to keep any reduced set of singular values and vectors.
18.4 Structured sparsity
Sparse optimization has become the focus of much research interest as a way to find solutions that depend on a smallnumber of variables (see e.g. the Lasso method). In principle, entry-wise sparsity can be enforced by penalizing theentry-wise ℓ0 -norm of the matrix, but the ℓ0 -norm is not convex. In practice this can be implemented by convexrelaxation to the ℓ1 -norm. While entry-wise regularization with an ℓ1 -norm will find solutions with a small numberof nonzero elements, applying an ℓ1 -norm to different groups of variables can enforce structure in the sparsity ofsolutions.*[5]The most straightforward example of structured sparsity uses the ℓp,q norm with p = 2 and q = 1 :
∥W∥2,1 =∑∥wi∥2.
For example, the ℓ2,1 norm is used in multi-task learning to group features across tasks, such that all the elements in agiven row of the coefficient matrix can be forced to zero as a group.*[6] The grouping effect is achieved by taking theℓ2 -norm of each row, and then taking the total penalty to be the sum of these row-wise norms. This regularizationresults in rows that will tend to be all zeros, or dense. The same type of regularization can be used to enforce sparsitycolumn-wise by taking the ℓ2 -norms of each column.More generally, the ℓ2,1 norm can be applied to arbitrary groups of variables:
80 CHAPTER 18. MATRIX REGULARIZATION
R(W ) = λG∑g
√√√√|Gg|∑j
|wjg|2 = λG∑g
∥wg∥g
where the index g is across groups of variables, and |Gg| indicates the cardinality of group g .Algorithms for solving these group sparsity problems extend the more well-known Lasso and group Lasso methodsby allowing overlapping groups, for example, and have been implemented via matching pursuit:*[7] and proximalgradient methods.*[8] By writing the proximal gradient with respect to a given coefficient, wig , it can be seen thatthis norm enforces a group-wise soft threshold*[1]
proxλ,Rg(wg)
i =
(wig − λ
wig∥wg∥g
)1∥wg∥g≥λ.
where 1∥wg∥g≥λ is the indicator function for group norms ≥ λ .Thus, using ℓ2,1 norms it is straightforward to enforce structure in the sparsity of a matrix either row-wise, column-wise, or in arbitrary blocks. By enforcing group norms on blocks in multivariate or multi-task regression, for example,it is possible to find groups of input and output variables, such that defined subsets of output variables (columns inthe matrix Y ) will depend on the same sparse set of input variables.
18.5 Multiple kernel selection
The ideas of structured sparsity and feature selection can be extended to the nonparametric case of multiple kernellearning.*[9] This can be useful when there are multiple types of input data (color and texture, for example) withdifferent appropriate kernels for each, or when the appropriate kernel is unknown. If there are two kernels, forexample, with feature maps A and B that lie in corresponding reproducing kernel Hilbert spaces HA,HB , then alarger space,HD , can be created as the sum of two spaces:
HD : f = h+ h′;h ∈ HA, h′ ∈ HB
assuming linear independence in A and B . In this case the ℓ2,1 -norm is again the sum of norms:
∥f∥HD,1 = ∥h∥HA + ∥h′∥HB
Thus, by choosing a matrix regularization function as this type of norm, it is possible to find a solution that is sparsein terms of which kernels are used, but dense in the coefficient of each used kernel. Multiple kernel learning can alsobe used as a form of nonlinear variable selection, or as a model aggregation technique (e.g. by taking the sum ofsquared norms and relaxing sparsity constraints). For example, each kernel can be taken to be the Gaussian kernelwith a different width.
18.6 References[1] Lorenzo Rosasco, Tomaso Poggio,“A Regularization Tour of Machine Learning—MIT-9.520 Lectures Notes”Manuscript,
Dec. 2014.
[2] Exact Matrix Completion via Convex Optimization by Candès, Emmanuel J. and Recht, Benjamin (2009) in Foundationsof Computational Mathematics, 9 (6). pp. 717–772. ISSN 1615-3375
[3] Zhang and Yeung. A Convex Formulation for Learning Task Relationships in Multi-Task Learning. Proceedings of theTwenty-Sixth Conference on Uncertainty in Artificial Intelligence (UAI2010)
[4] Alan Izenman. Reduced Rank Regression for the Multivariate Linear Model. Journal of Multivariate Analysis 5,248-264(1975)
18.6. REFERENCES 81
[5] Kakade, Shalev-Shwartz and Tewari. Regularization Techniques for Learning with Matrices. Journal of Machine LearningResearch 13 (2012) 1865-1890.
[6] A. Argyriou, T. Evgeniou, and M. Pontil. Convex multi-task feature learning. Machine Learning, 73(3):243–272, 2008.
[7] Huang, Zhang, and Metaxas. Learning with Structured Sparsity. Journal of Machine Learning Research 12 (2011) 3371-3412.
[8] Chen et. al. Smoothing Proximal Gradient Method for General Structured Sparse Regression. The Annals of AppliedStatistics, 2012, Vol. 6, No. 2, 719–752 DOI: 10.1214/11-AOAS514
[9] Sonnenburg, Ratsch, Schafer AND Scholkopf. Large Scale Multiple Kernel Learning. Journal of Machine LearningResearch 7 (2006) 1531–1565.
Chapter 19
Matrix representation
This article is about the layout of matrices in the memory of computers. For the representation of groups and algebrasby matrices in linear algebra, see representation theory.
Matrix representation is a method used by a computer language to store matrices of more than one dimension inmemory. Fortran and C use different schemes. Fortran uses“Column Major”, in which all the elements for a givencolumn are stored contiguously in memory. C uses “Row Major”, which stores all the elements for a given rowcontiguously in memory. LAPACK defines various matrix representations in memory. There is also Sparse matrixrepresentation and Morton-order matrix representation. According to the documentation, in LAPACK the unitarymatrix representation is optimized.*[1] Some languages such as Java store matrices using Iliffe vectors. These areparticularly useful for storing irregular matrices. Matrices are of primary importance in linear algebra.
19.1 Basic mathematical operations
Main article: Matrix (mathematics) § Basic operations
An m × n (read as m by n) order matrix is a set of numbers arranged in m rows and n columns. Matrices of the sameorder can be added by adding the corresponding elements. Two matrices can be multiplied, the condition being thatthe number of columns of the first matrix is equal to the number of rows of the second matrix. Hence, if an m × nmatrix is multiplied with an n × r matrix, then the resultant matrix will be of the order m × r.*[2]Operations like row operations or column operations can be performed on a matrix, using which we can obtain theinverse of a matrix. The inverse may be obtained by determining the adjoint as well.*[2] rows and columns are thedifferent classes of matrices
19.2 Basics of 2D array
The mathematical definition of a matrix finds applications in computing and database management, a basic startingpoint being the concept of arrays. A two-dimensional array can function exactly like a matrix. Two-dimensionalarrays can be visualized as a table consisting of rows and columns.
• int a[3][4], declares an integer array of 3 rows and 4 columns. Index of row will start from 0 and will go up to2.
• Similarly, index of column will start from 0 and will go up to 3.*[3]
This table shows arrangement of elements with their indices.Initializing Two-Dimensional arrays: Two-Dimensional arrays may be initialized by providing a list of initial values.int a[2][3] = 1,2,3,4,5,6, or int a[2][3] = 2,3,4,4,4,5;
82
19.3. SEE ALSO 83
Calculation of Address : An m x n matrix (a[1...m][1...n]) where the row index varies from 1 to m and column indexfrom 1 to n,aij denotes the number in the i*th row and the j*th column. In the computer memory, all elements arestored linearly using contiguous addresses. Therefore,in order to store a two-dimensional matrix a, two dimensionaladdress space must be mapped to one-dimensional address space. In the computer's memory matrices are stored ineither Row-major order or Column-major order form.
19.3 See also• Row- and column-major order
• Sparse matrix
• Skyline matrix
19.4 References[1] “Representation of Orthogonal or Unitary Matrices”. University of Texas at Austin. Retrieved 14 September 2011.
[2] Ramana, B.V (2008). Higher Engineering Mathematics. New Delhi: Tata Mcgraw-Hill. ISBN 978-0-07-063419-0.
[3] Balagurusamy, E (2006). Programming in ANSI C. New Delhi: Tata McGraw-Hill.
19.5 External links• a description of sparse matrices in R.
^1 R. LEHOUCQ, The computation of elementary unitary matrices, Computer Science Dept. Technical ReportCS-94-233, University of Tennessee, Knoxville, 1994. (LAPACK Working Note 72).
Chapter 20
Matrix similarity
For other uses, see Similarity (geometry), Similarity transformation, and Similarity (disambiguation).Not to be confused with similarity matrix.
In linear algebra, two n-by-n matrices A and B are called similar if
B = P−1AP
for some invertible n-by-n matrix P. Similar matrices represent the same linear operator under two different bases,with P being the change of basis matrix.*[1]*[2]A transformationA 7→ P−1AP is called a similarity transformation or conjugation of the matrixA. In the generallinear group, similarity is therefore the same as conjugacy, and similar matrices are also called conjugate; howeverin a given subgroup H of the general linear group, the notion of conjugacy may be more restrictive than similarity,since it requires that P can be chosen to lie in H.
20.1 Properties
Similarity is an equivalence relation on the space of square matrices.Similar matrices share any properties that are really properties of the represented linear operator:
• Rank
• Characteristic polynomial, and attributes that can be derived from it:
• Determinant• Trace• Eigenvalues, and their algebraic multiplicities
• Geometric multiplicities of eigenvalues (but not the eigenspaces, which are transformed according to the basechange matrix P used).
• Minimal polynomial
• Elementary divisors, which form a complete set of invariants for similarity
• Rational canonical form
Because of this, for a given matrix A, one is interested in finding a simple“normal form”B which is similar to A—the study of A then reduces to the study of the simpler matrix B. For example, A is called diagonalizable if it is similarto a diagonal matrix. Not all matrices are diagonalizable, but at least over the complex numbers (or any algebraicallyclosed field), every matrix is similar to a matrix in Jordan form. Neither of these forms is unique (diagonal entries
84
20.2. SEE ALSO 85
or Jordan blocks may be permuted) so they are not really normal forms; moreover their determination depends onbeing able to factor the minimal or characteristic polynomial of A (equivalently to find its eigenvalues). The rationalcanonical form does not have these drawbacks: it exists over any field, is truly unique, and it can be computed usingonly arithmetic operations in the field; A and B are similar if and only if they have the same rational canonical form.The rational canonical form is determined by the elementary divisors of A; these can be immediately read off froma matrix in Jordan form, but they can also be determined directly for any matrix by computing the Smith normalform, over the ring of polynomials, of the matrix (with polynomial entries) XIn − A (the same one whose determinantdefines the characteristic polynomial). Note that this Smith normal form is not a normal form of A itself; moreoverit is not similar to XIn − A either, but obtained from the latter by left and right multiplications by different invertiblematrices (with polynomial entries).Similarity of matrices does not depend on the base field: if L is a field containing K as a subfield, and A and B aretwo matrices over K, then A and B are similar as matrices over K if and only if they are similar as matrices over L.This is so because the rational canonical form over K is also the rational canonical form over L. This means that onemay use Jordan forms that only exist over a larger field to determine whether the given matrices are similar.In the definition of similarity, if the matrix P can be chosen to be a permutation matrix thenA and B are permutation-similar; if P can be chosen to be a unitary matrix then A and B are unitarily equivalent. The spectral theorem saysthat every normal matrix is unitarily equivalent to some diagonal matrix. Specht's theorem states that two matricesare unitarily equivalent if and only if they satisfy certain trace equalities.
20.2 See also• Matrix congruence
• Matrix equivalence
• Canonical forms
20.3 Notes[1] Beauregard & Fraleigh (1973, pp. 240-243)
[2] Bronson (1970, pp. 176-178)
20.4 References• Beauregard, Raymond A.; Fraleigh, John B. (1973), A First Course In Linear Algebra: with Optional Introduc-tion to Groups, Rings, and Fields, Boston: Houghton Mifflin Co., ISBN 0-395-14017-X
• Bronson, Richard (1970), Matrix Methods: An Introduction, New York: Academic Press, LCCN 70097490
• Horn and Johnson, Matrix Analysis, Cambridge University Press, 1985. ISBN 0-521-38632-2. (Similarity isdiscussed many places, starting at page 44.)
Chapter 21
Matrix splitting
In the mathematical discipline of numerical linear algebra, a matrix splitting is an expression which represents agiven matrix as a sum or difference of matrices. Many iterative methods (e.g., for systems of differential equations)depend upon the direct solution of matrix equations involving matrices more general than tridiagonal matrices. Thesematrix equations can often be solved directly and efficiently when written as a matrix splitting. The technique wasdevised by Richard S. Varga in 1960.*[1]
21.1 Regular splittings
We seek to solve the matrix equation
Ax = k, (1)
where A is a given n × n non-singular matrix, and k is a given column vector with n components. We split the matrixA into
A = B− C, (2)
where B and C are n × n matrices. If, for an arbitrary n × n matrix M, M has nonnegative entries, we write M ≥ 0.If M has only positive entries, we write M > 0. Similarly, if the matrix M1 − M2 has nonnegative entries, we writeM1 ≥ M2.Definition: A = B − C is a regular splitting of A if and only if B*−1 ≥ 0 and C ≥ 0.We assume that matrix equations of the form
Bx = g, (3)
where g is a given column vector, can be solved directly for the vector x. If (2) represents a regular splitting of A,then the iterative method
Bx(m+1) = Cx(m) + k, m = 0, 1, 2, . . . , (4)
where x*(0) is an arbitrary vector, can be carried out. Equivalently, we write (4) in the form
x(m+1) = B−1Cx(m) + B−1k, m = 0, 1, 2, . . . (5)
The matrix D = B*−1C has nonnegative entries if (2) represents a regular splitting of A.*[2]
86
21.2. MATRIX ITERATIVE METHODS 87
It can be shown that if A*−1 > 0, then ρ(D) < 1, where ρ(D) represents the spectral radius of D, and thus D is aconvergent matrix. As a consequence, the iterative method (5) is necessarily convergent.*[3]*[4]If, in addition, the splitting (2) is chosen so that the matrix B is a diagonal matrix (with the diagonal entries allnon-zero, since B must be invertible), then B can be inverted in linear time (see Time complexity).
21.2 Matrix iterative methods
Many iterative methods can be described as a matrix splitting. If the diagonal entries of the matrix A are all nonzero,and we express the matrix A as the matrix sum
A = D− U− L, (6)
where D is the diagonal part of A, and U and L are respectively strictly upper and lower triangular n × n matrices,then we have the following.The Jacobi method can be represented in matrix form as a splitting
x(m+1) = D−1(U+ L)x(m) + D−1k. (7) *[5]*[6]
The Gauss-Seidel method can be represented in matrix form as a splitting
x(m+1) = (D− L)−1Ux(m) + (D− L)−1k. (8) *[7]*[8]
The method of successive over-relaxation can be represented in matrix form as a splitting
x(m+1) = (D− ωL)−1[(1− ω)D+ ωU]x(m) + ω(D− ωL)−1k. (9) *[9]*[10]
21.3 Example
21.3.1 Regular splitting
In equation (1), let
A =
6 −2 −3−1 4 −2−3 −1 5
, k =
5−1210
. (10)
Let us apply the splitting (7) which is used in the Jacobi method: we split A in such a way that B consists of all ofthe diagonal elements of A, and C consists of all of the off-diagonal elements of A, negated. (Of course this is notthe only useful way to split a matrix into two matrices.) We have
B =
6 0 00 4 00 0 5
, C =
0 2 31 0 23 1 0
, (11)
A−1 =1
47
18 13 1611 21 1513 12 22
, B−1 =
16 0 0
0 14 0
0 0 15
,
D = B−1C =
0 13
12
14 0 1
235
15 0
, B−1k =
56
−32
.
88 CHAPTER 21. MATRIX SPLITTING
Since B*−1 ≥ 0 and C ≥ 0, the splitting (11) is a regular splitting. SinceA*−1 > 0, the spectral radius ρ(D) < 1. (Theapproximate eigenvalues of D are λi ≈ –0.4599820, –0.3397859, 0.7997679.) Hence, the matrix D is convergentand the method (5) necessarily converges for the problem (10). Note that the diagonal elements of A are all greaterthan zero, the off-diagonal elements of A are all less than zero and A is strictly diagonally dominant.*[11]The method (5) applied to the problem (10) then takes the form
x(m+1) =
0 13
12
14 0 1
235
15 0
x(m) +
56
−32
, m = 0, 1, 2, . . . (12)
The exact solution to equation (12) is
x =
2−13
. (13)
The first few iterates for equation (12) are listed in the table below, beginning with x*(0) = (0.0, 0.0, 0.0)*T. Fromthe table one can see that the method is evidently converging to the solution (13), albeit rather slowly.
21.3.2 Jacobi method
As stated above, the Jacobi method (7) is the same as the specific regular splitting (11) demonstrated above.
21.3.3 Gauss-Seidel method
Since the diagonal entries of the matrixA in problem (10) are all nonzero, we can express the matrixA as the splitting(6), where
D =
6 0 00 4 00 0 5
, U =
0 2 30 0 20 0 0
, L =
0 0 01 0 03 1 0
. (14)
We then have
(D− L)−1 = 1
120
20 0 05 30 013 6 24
,
(D− L)−1U =1
120
0 40 600 10 750 26 51
, (D− L)−1k =1
120
100−335233
.The Gauss-Seidel method (8) applied to the problem (10) takes the form
x(m+1) =1
120
0 40 600 10 750 26 51
x(m) +1
120
100−335233
, m = 0, 1, 2, . . . (15)
The first few iterates for equation (15) are listed in the table below, beginning with x*(0) = (0.0, 0.0, 0.0)*T. Fromthe table one can see that the method is evidently converging to the solution (13), somewhat faster than the Jacobimethod described above.
21.4. SEE ALSO 89
21.3.4 Successive over-relaxation method
Let ω = 1.1. Using the splitting (14) of the matrix A in problem (10) for the successive over-relaxation method, wehave
(D− ωL)−1 = 1
12
2 0 00.55 3 01.441 0.66 2.4
,
(D− ωL)−1[(1− ω)D+ ωU] = 1
12
−1.2 4.4 6.6−0.33 0.01 8.415−0.8646 2.9062 5.0073
,ω(D− ωL)−1k =
1
12
11−36.57525.6135
.The successive over-relaxation method (9) applied to the problem (10) takes the form
x(m+1) =1
12
−1.2 4.4 6.6−0.33 0.01 8.415−0.8646 2.9062 5.0073
x(m) +1
12
11−36.57525.6135
, m = 0, 1, 2, . . . (16)
The first few iterates for equation (16) are listed in the table below, beginning with x*(0) = (0.0, 0.0, 0.0)*T. Fromthe table one can see that the method is evidently converging to the solution (13), slightly faster than the Gauss-Seidelmethod described above.
21.4 See also
• Matrix decomposition
• M-matrix
• Stieltjes matrix
21.5 Notes[1] Varga (1960)
[2] Varga (1960, pp. 121–122)
[3] Varga (1960, pp. 122–123)
[4] Varga (1962, p. 89)
[5] Burden & Faires (1993, p. 408)
[6] Varga (1962, p. 88)
[7] Burden & Faires (1993, p. 411)
[8] Varga (1962, p. 88)
[9] Burden & Faires (1993, p. 416)
[10] Varga (1962, p. 88)
[11] Burden & Faires (1993, p. 371)
90 CHAPTER 21. MATRIX SPLITTING
21.6 References• Burden, Richard L.; Faires, J. Douglas (1993), Numerical Analysis (5th ed.), Boston: Prindle, Weber and
Schmidt, ISBN 0-534-93219-3.
• Varga, Richard S. (1960).“Factorization and Normalized Iterative Methods”. In Langer, Rudolph E. Bound-ary Problems in Differential Equations. Madison: University of Wisconsin Press. pp. 121–142. LCCN 60-60003.
• Varga, Richard S. (1962), Matrix Iterative Analysis, New Jersey: Prentice-Hall, LCCN 62-21277.
Chapter 22
Metzler matrix
In mathematics, a Metzler matrix is a matrix in which all the off-diagonal components are nonnegative (equal to orgreater than zero)
∀i =j xij ≥ 0.
It is named after the American economist Lloyd Metzler.Metzler matrices appear in stability analysis of time delayed differential equations and positive linear dynamicalsystems. Their properties can be derived by applying the properties of nonnegative matrices to matrices of the formM + aI where M is a Metzler matrix.
22.1 Definition and terminology
In mathematics, especially linear algebra, a matrix is calledMetzler, quasipositive (or quasi-positive) or essentiallynonnegative if all of its elements are non-negative except for those on the main diagonal, which are unconstrained.That is, a Metzler matrix is any matrix A which satisfies
A = (aij); aij ≥ 0, i = j.
Metzler matrices are also sometimes referred to as Z(−) -matrices, as a Z-matrix is equivalent to a negated quasi-positive matrix.
• Nonnegative matrices
• Positive matrix
• Delay differential equation
• M-matrix
• P-matrix
• Z-matrix
• Stochastic matrix
22.2 Properties
The exponential of a Metzler (or quasipositive) matrix is a nonnegative matrix because of the corresponding prop-erty for the exponential of a nonnegative matrix. This is natural, once one observes that the generator matrices
91
92 CHAPTER 22. METZLER MATRIX
of continuous-time finite-state Markov processes are always Metzler matrices, and that probability distributions arealways non-negative.A Metzler matrix has an eigenvector in the nonnegative orthant because of the corresponding property for nonnegativematrices.
22.3 Relevant theorems• Perron–Frobenius theorem
22.4 See also• Nonnegative matrices
• Delay differential equation
• M-matrix
• P-matrix
• Z-matrix
• Stochastic matrix
22.5 Bibliography• Berman, Abraham; Plemmons, Robert J. (1994). Nonnegative Matrices in the Mathematical Sciences. SIAM.
ISBN 0-89871-321-8.
• Farina, Lorenzo; Rinaldi, Sergio (2000). Positive Linear Systems: Theory and Applications. New York: WileyInterscience.
• Berman, Abraham; Neumann, Michael; Stern, Ronald (1989). Nonnegative Matrices in Dynamical Systems.Pure and Applied Mathematics. New York: Wiley Interscience.
• Kaczorek, Tadeusz (2002). Positive 1D and 2D Systems. London: Springer.
• Luenberger, David (1979). Introduction to Dynamic Systems: Theory, Modes & Applications. John Wiley &Sons.
• Kemp, Murray C.; Kimura, Yoshio (1978). Introduction to Mathematical Economics. New York: Springer.pp. 102–114. ISBN 0-387-90304-6.
Chapter 23
Modal matrix
In linear algebra, themodal matrix is used in the diagonalization process involving eigenvalues and eigenvectors.*[1]Specifically the modal matrix M for the matrix A is the n × n matrix formed with the eigenvectors of A as columnsin M . It is utilized in the similarity transformation
D =M−1AM,
where D is an n × n diagonal matrix with the eigenvalues of A on the main diagonal of D and zeros elsewhere. Thematrix D is called the spectral matrix for A . The eigenvalues must appear left to right, top to bottom in the sameorder as their corresponding eigenvectors are arranged left to right in M .*[2]
23.1 Example
The matrix
A =
3 2 02 0 01 0 2
has eigenvalues and corresponding eigenvectors
λ1 = −1, b1 = (−3, 6, 1) ,
λ2 = 2, b2 = (0, 0, 1) ,
λ3 = 4, b3 = (2, 1, 1) .
A diagonal matrix D , similar to A is
D =
−1 0 00 2 00 0 4
.One possible choice for an invertible matrix M such that D =M−1AM, is
M =
−3 0 26 0 11 1 1
. *[3]
Note that since eigenvectors themselves are not unique, and since the columns of bothM andDmay be interchanged,it follows that both M and D are not unique.*[4]
93
94 CHAPTER 23. MODAL MATRIX
23.2 Generalized modal matrix
Let A be an n × n matrix. A generalized modal matrixM for A is an n × n matrix whose columns, considered asvectors, form a canonical basis for A and appear in M according to the following rules:
• All Jordan chains consisting of one vector (that is, one vector in length) appear in the first columns of M .• All vectors of one chain appear together in adjacent columns of M .• Each chain appears inM in order of increasing rank (that is, the generalized eigenvector of rank 1 appears be-
fore the generalized eigenvector of rank 2 of the same chain, which appears before the generalized eigenvectorof rank 3 of the same chain, etc.).*[5]
One can show that
where J is a matrix in Jordan normal form. By premultiplying by M−1 , we obtain
Note that when computing these matrices, equation (1) is the easiest of the two equations to verify, since it does notrequire inverting a matrix.*[6]
23.2.1 Example
This example illustrates a generalized modal matrix with four Jordan chains. Unfortunately, it is a little difficult toconstruct an interesting example of low order.*[7] The matrix
A =
−1 0 −1 1 1 3 00 1 0 0 0 0 02 1 2 −1 −1 −6 0−2 0 −1 2 1 3 00 0 0 0 1 0 00 0 0 0 0 1 0−1 −1 0 1 2 4 1
has a single eigenvalue λ1 = 1 with algebraic multiplicity µ1 = 7 . A canonical basis for A will consist of onelinearly independent generalized eigenvector of rank 3 (generalized eigenvector rank; see generalized eigenvector),two of rank 2 and four of rank 1; or equivalently, one chain of three vectors x3, x2, x1 , one chain of two vectorsy2, y1 , and two chains of one vector z1 , w1 .An “almost diagonal”matrix J in Jordan normal form, similar to A is obtained as follows:
M =(z1 w1 x1 x2 x3 y1 y2
)=
0 1 −1 0 0 −2 10 3 0 0 1 0 0−1 1 1 1 0 2 0−2 0 −1 0 0 −2 01 0 0 0 0 0 00 1 0 0 0 0 00 0 0 −1 0 −1 0
,
J =
1 0 0 0 0 0 00 1 0 0 0 0 00 0 1 1 0 0 00 0 0 1 1 0 00 0 0 0 1 0 00 0 0 0 0 1 10 0 0 0 0 0 1
,
23.3. NOTES 95
whereM is a generalized modal matrix forA , the columns ofM are a canonical basis forA , andAM =MJ .*[8]Note that since generalized eigenvectors themselves are not unique, and since some of the columns of both M and Jmay be interchanged, it follows that both M and J are not unique.*[9]
23.3 Notes[1] Bronson (1970, pp. 179-183)
[2] Bronson (1970, p. 181)
[3] Beauregard & Fraleigh (1973, pp. 271,272)
[4] Bronson (1970, p. 181)
[5] Bronson (1970, p. 205)
[6] Bronson (1970, pp. 206-207)
[7] Nering (1970, pp. 122,123)
[8] Bronson (1970, pp. 208,209)
[9] Bronson (1970, p. 206)
23.4 References• Beauregard, Raymond A.; Fraleigh, John B. (1973), A First Course In Linear Algebra: with Optional Introduc-tion to Groups, Rings, and Fields, Boston: Houghton Mifflin Co., ISBN 0-395-14017-X
• Bronson, Richard (1970), Matrix Methods: An Introduction, New York: Academic Press, LCCN 70097490
• Nering, Evar D. (1970), Linear Algebra and Matrix Theory (2nd ed.), New York: Wiley, LCCN 76091646
Chapter 24
Moment matrix
In mathematics, a moment matrix is a special symmetric square matrix whose rows and columns are indexed bymonomials. The entries of the matrix depend on the product of the indexing monomials only (cf. Hankel matrices.)Moment matrices play an important role in polynomial optimization, since positive semidefinite moment matricescorrespond to polynomials which are sums of squares.
24.1 Definition
24.2 See also
24.3 External links• Hazewinkel, Michiel, ed. (2001), “Moment matrix”, Encyclopedia of Mathematics, Springer, ISBN 978-1-
55608-010-4
96
Chapter 25
Moore determinant of a Hermitian matrix
Not to be confused with Moore determinant over a finite field.
In mathematics, the Moore determinant is a determinant defined for Hermitian matrices over a quaternion algebra,introduced by Moore (1922).
25.1 See also• Dieudonné determinant
25.2 References• Moore, E. H. (1922),“On the determinant of an hermitian matrix with quaternionic elements. Definition and
elementary properties with applications.”, Bulletin of the American Mathematical Society 28 (4): 161–162,doi:10.1090/S0002-9904-1922-03536-7, ISSN 0002-9904
97
Chapter 26
Moore matrix
In linear algebra, a Moore matrix, introduced by E. H. Moore (1896), is a matrix defined over a finite field. Whenit is a square matrix its determinant is called a Moore determinant (this is unrelated to the Moore determinant ofa quaternionic Hermitian matrix). The Moore matrix has successive powers of the Frobenius automorphism appliedto the first column, so it is an m × n matrix
M =
α1 αq1 . . . αq
n−1
1
α2 αq2 . . . αqn−1
2
α3 αq3 . . . αqn−1
3...
... . . . ...αm αqm . . . αq
n−1
m
or
Mi,j = αqj−1
i
for all indices i and j. (Some authors use the transpose of the above matrix.)The Moore determinant of a square Moore matrix (so m = n) can be expressed as:
det(V ) =∏c(c1α1 + · · ·+ cnαn) ,
where c runs over a complete set of direction vectors, made specific by having the last non-zero entry equal to 1, i.e.
det(V ) =∏
1≤i≤n
∏c1,...,ci−1
(c1α1 + · · ·+ ci−1αi−1 + αi) .
In particular the Moore determinant vanishes if and only if the elements in the left hand column are linearly dependentover the finite field of order q. So it is analogous to the Wronskian of several functions.Dickson used the Moore determinant in finding the modular invariants of the general linear group over a finite field.
26.1 See also• Alternant matrix
• Vandermonde determinant
• List of matrices
98
26.2. REFERENCES 99
26.2 References• Dickson, Leonard Eugene (1958) [1901], Magnus, Wilhelm, ed., Linear groups: With an exposition of theGalois field theory, Dover Phoenix editions, New York: Dover Publications, ISBN 978-0-486-49548-4, MR0104735
• David Goss (1996). Basic Structures of Function Field Arithmetic. Springer Verlag. ISBN 3-540-63541-6.Chapter 1.
• Moore, E. H. (1896), “A two-fold generalization of Fermat's theorem.”, American M. S. Bull. 2: 189–199,doi:10.1090/S0002-9904-1896-00337-2, JFM 27.0139.05
Chapter 27
Mueller calculus
Mueller calculus is a matrix method for manipulating Stokes vectors, which represent the polarization of light. Itwas developed in 1943 by Hans Mueller. In this technique, the effect of a particular optical element is representedby a Mueller matrix—a 4×4 matrix that is an overlapping generalization of the Jones matrix.
27.1 Introduction
Disregarding coherent wave superposition, any fully polarized, partially polarized, or unpolarized state of light canbe represented by a Stokes vector ( S ); and any optical element can be represented by a Mueller matrix (M).If a beam of light is initially in the state Si and then passes through an optical element M and comes out in a state So, then it is written
So = MSi .
If a beam of light passes through optical element M1 followed by M2 then M3 it is written
So = M3
(M2(M1Si)
)given that matrix multiplication is associative it can be written
So = M3M2M1Si .
Matrix multiplication is not commutative, so in general
M3M2M1Si = M1M2M3Si .
27.2 Mueller vs. Jones calculi
With disregard for coherence, light which is unpolarized or partially polarized must be treated using the Muellercalculus, while fully polarized light can be treated with either the Mueller calculus or the simpler Jones calculus.Many problems involving coherent light (such as from a laser) must be treated with Jones calculus, however, because itworks directly with the electric field of the light rather than with its intensity or power, and thereby retains informationabout the phase of the waves.More specifically, the following can be said about Mueller matrices and Jones matrices:*[1]
100
27.3. MUELLER MATRICES 101
Stokes vectors and Mueller matrices operate on intensities and their differences, i.e. incoherentsuperpositions of light; they are not adequate to describe neither interference nor diffraction effects.
...Any Jones matrix [J] can be transformed into the corresponding Mueller–Jones matrix, M, using the
following relation:
M = A(J⊗ J∗)A−1
where * indicates the complex conjugate [sic], [A is:]
A =
1 0 0 11 0 0 −10 1 1 00 i −i 0
and ⊗ is the tensor (Kronecker) product.
...While the Jones matrix has eight independent parameters [two Cartesian or polar components for
each of the four complex values in the 2-by-2 matrix], the absolute phase information is lost in the[equation above], leading to only seven independent matrix elements for a Mueller matrix derived froma Jones matrix.
27.3 Mueller matrices
Below are listed the Mueller matrices for some ideal common optical elements:
1
2
1 1 0 01 1 0 00 0 0 00 0 0 0
1
2
1 −1 0 0−1 1 0 00 0 0 00 0 0 0
1
2
1 0 1 00 0 0 01 0 1 00 0 0 0
1
2
1 0 −1 00 0 0 0−1 0 1 00 0 0 0
1 0 0 00 1 0 00 0 0 −10 0 1 0
Quarter wave plate (fast-axis vertical)
1 0 0 00 1 0 00 0 0 10 0 −1 0
Quarter wave plate (fast-axis horizontal)
102 CHAPTER 27. MUELLER CALCULUS
1 0 0 00 1 0 00 0 −1 00 0 0 −1
Half wave plate (fast-axis vertical)
1
4
1 0 0 00 1 0 00 0 1 00 0 0 1
27.4 See also• Stokes parameters
• Jones calculus
• Polarization (waves)
27.5 References[1] Savenkov, S. N. (2009). “Jones and Mueller matrices: Structure, symmetry relations and information content”. Light
Scattering Reviews 4. p. 71. doi:10.1007/978-3-540-74276-0_3. ISBN 978-3-540-74275-3.
• E. Collett, Field Guide to Polarization, SPIE Field Guides vol. FG05, SPIE (2005). ISBN 0-8194-5868-6.
• E. Hecht, Optics, 2nd ed., Addison-Wesley (1987). ISBN 0-201-11609-X.
• del Toro Iniesta, Jose Carlos (2003). Introduction to Spectropolarimetry. Cambridge, UK: Cambridge Univer-sity Press. p. 227. ISBN 978-0-521-81827-8.
Chapter 28
Network operator matrix
Network operator matrix (NOM) is a representation of mathematical expressions in computer memory.NOM is a new approach to the problem of automatic search of mathematical equations. The researcher defines thesets of operations, variables and parameters.*[1] The computer program generates a number of mathematical equa-tions that satisfy given restrictions. Then the optimization algorithm finds the structure of appropriate mathematicalexpression and its parameters.Network operator is a directed graph that corresponds to some mathematical expressions. Every source nodes of thegraph are variables or constants of mathematical expression, inner nodes correspond to binary operations and edgescorrespond to unary operations. The calculation’s result of mathematical expression is kept in the last sink node.*[2]
28.1 Example
Consider the following mathematical expression
The graph for Expression (1), is presented in Fig. 1.On the edges we place unary operationsρ1(z) = z ;ρ3(z) = −z ;
ρ6(n) =
ε−1, ifz > ln εez, otherwise
;
ρ12(z) = sin z ;In the inner and sink nodes we place binary operationsχ0(z
′, z′′) = z′ + z′′ ;χ1(z
′, z′′) = z′ × z′′ ;Expression 1 can be presented in the PC memory as a NOM
Ψ =
0 0 0 1 1 0 0 120 0 0 0 1 0 0 00 0 0 0 0 3 0 00 0 0 0 0 0 0 10 0 0 0 1 0 1 00 0 0 0 0 0 6 00 0 0 0 0 0 1 10 0 0 0 0 0 0 0
Any mathematical expression can be presented as a network operator matrix.*[3]*[4]*[5]*[6]To calculate the mathematical expression by the network operator matrix the node vector is used.
103
104 CHAPTER 28. NETWORK OPERATOR MATRIX
Fig. 1. Graph for Expression 1
Each component of node vector corresponds to the nodes of the network operator graph. Initially each component isequal to the argument for the given node or the unit element of binary operation.For addition and multiplication the number of operation equals its unit element. For addition it is 0, for multiplicationit is 1. The node vector for the given examplez = [x1 q1 x2 0 1 0 1 0]T
The calculation by matrix Ψ is performed for nonzero nondaigonal elements ψ(i,j) = 0 byzj = χψj,j
(zj , ρψi,j(zi))
Follow the rows of the matrix.For the Row 1 we have ψ1,4 = 1 , that is i = 1 , j = 4 , ψj,j = ψ4,4 = 0 .Take arguments from z = [x1 q1 x2 0 1 0 1 0]T
z1 = x1 , z4 = 0 thenz4 = χψ4,4
(z4, ρψ1,4(z1)) = χ0(z4, ρ1(z1)) = 0 + x1 = x1
Further in Row 1 we have ψ1,5 = 1
28.2. REFERENCES 105
z5 = χψ5,5(z5, ρψ1,5(z1)) = χ1(z5, ρ1(z1)) = 1× x1 = x1ψ1,8 = 12z8 = χψ8,8(z8, ρψ1,8(z1)) = χ0(z8, ρ12(z1)) = 0 + sinx1 = sinx1 .As a result after Row 1 we have z = [x1 q1 x2 x1 x1 0 1 sinx1]T .For the Row 2 we have ψ2,5 = 1z5 = χψ5,5(z5, ρψ2,5(z2)) = χ1(z5, ρ1(z1)) = x1q1then follow the rowsψ3,6 = 3z6 = χψ6,6(z6, ρψ3,6(z3)) = χ0(z6, ρ3(z3)) = 0 + (−x2) = −x2ψ4,8 = 1z8 = χψ8,8(z8, ρψ4,8(z4)) = χ0(z8, ρ1(z4)) = sinx1 + x1
ψ5,7 = 1z7 = χψ7,7(z7, ρψ5,7(z5)) = χ1(z7, ρ1(z5)) = 1x1q1 = x1q1
ψ6,7 = 6z7 = χψ7,7(z7, ρψ6,7(z6)) = χ1(z7, ρ6(z6)) = x1q1e
−x2
ψ7,8 = 1z8 = χψ8,8(z8, ρψ7,8(z7)) = χ0(z8, ρ1(z7)) = sinx1 + x1 + x1q1e
−x2 .
28.2 References[1] Diveyev A.I., Sofronova E.A. Application of network operator method for synthesis of optimal structure and parameters
of automatic control system. Proceedings of 17-th IFAC World Congress, Seoul, Korea 05 – 12 July 2008
[2] “Network Operator”. Network Operator, Inc. Retrieved 24 January 2012.
[3] A.I. Diveev, E.A. Sofronova The synthesis of optimal control system by the network operator method IFAC Workshop onControl Applications and Optimization, CAO’09, 6–8 May, University of Jyväskylä, Agora, Finland.
[4] Diveev A.I., Sofronova E.A. Numerical method of network operator for multi-objective synthesis of optimal control system,Proceedings of 7-th International Conference on Control andAutomation (ICCA’09), Christchurch, New Zealand, December9–11, 2009.
[5] Diveev A.I. A multiobjective synthesis of optimal control system by the network operator method. Proceedings of inter-national conference «Optimization and applications» (OPTIMA 2009), Petrovac, Montenegro, September 21–25, 2009
[6] “A.I. Diveev, E.A. Sofronova The Network Operator Method for Search of the Most Suitable Mathematical equation //Bio-inspired computational algorithms and their applications.”. Soft_NOM, Inc. Retrieved 7 March 2012.
Chapter 29
Next-generation matrix
In epidemiology, the next-generation matrix is a method used to derive the basic reproduction number, for acompartmental model of the spread of infectious diseases. This method is given by Diekmann et al. (1990)*[1] andvan den Driessche and Watmough (2002).*[2] To calculate the basic reproduction number by using a next-generationmatrix, the whole population is divided into n compartments in which there are m < n infected compartments. Letxi, i = 1, 2, 3, . . . ,m be the numbers of infected individuals in the ith infected compartment at time t. Now, theepidemic model is
dxi
dt = Fi(x)− Vi(x) , where Vi(x) = [V −i (x)− V +
i (x)]
In the above equations, Fi(x) represents the rate of appearance of new infections in compartment i . V +i represents
the rate of transfer of individuals into compartment i by all other means, and V −i (x) represents the rate of transfer
of individuals out of compartment i . The above model can also be written as
dxidt = F (x)− V (x)
where
F (x) =(F1(x), F2(x), . . . , Fn(x)
)Tand
V (x) =(V1(x), V2(x), . . . , Vn(x)
)T.
Let x0 be the disease-free equilibrium. The values of the Jacobian matrices F (x) and V (x) are:
DF (x0) =
(F 00 0
)and
DV (x0) =
(V 0J3 J4
)respectively.Here, F and V are m × m matrices, defined as F = ∂Fi
∂xj(x0) and V = ∂Vi
∂xj(x0) .
Now, the matrix FV −1 is known as the next-generation matrix. The largest eigenvalue or spectral radius of FV −1
is the basic reproduction number of the model.
106
29.1. SEE ALSO 107
29.1 See also• Mathematical modelling of infectious disease
29.2 References[1] Diekmann, O.; Heesterbeek, J. A. P.; Metz, J. A. J. (1990). “On the definition and the computation of the basic repro-
duction ratio R0 in models for infectious diseases in heterogeneous populations”. Journal of Mathematical Biology 28 (4):365–382. doi:10.1007/BF00178324. PMID 2117040.
[2] Van den Driessche, P.; Watmough, J. (2002).“Reproduction numbers and sub-threshold endemic equilibria for compart-mental models of disease transmission”. Mathematical Biosciences 180 (1–2): 29–48. doi:10.1016/S0025-5564(02)00108-6. PMID 12387915.
29.3 Sources• Ma, Zhien; Li, Jia (2009). Dynamical Modeling and analysis of Epidemics. World Scientific. ISBN 978-981-
279-749-0. OCLC 225820441.
• Diekmann, O.; Heesterbeek, J. A. P. (2000). Mathematical Epidemiology of Infectious Disease. John Wiley &Son.
• Hefferenan, J. M.; Smith, R. J.; Wahl, L. M. (2005). “Prospective on the basic reproductive ratio”. J. R.Soc. Interface.
Chapter 30
Nilpotent matrix
In linear algebra, a nilpotent matrix is a square matrix N such that
Nk = 0
for some positive integer k. The smallest such k is sometimes called the degree or index of N.*[1]More generally, a nilpotent transformation is a linear transformation L of a vector space such that L*k = 0 for somepositive integer k (and thus, L*j = 0 for all j ≥ k).*[2]*[3]*[4] Both of these concepts are special cases of a moregeneral concept of nilpotence that applies to elements of rings.
30.1 Examples
The matrix
M =
[0 10 0
]is nilpotent, since M2 = 0. More generally, any triangular matrix with 0s along the main diagonal is nilpotent. Forexample, the matrix
N =
0 2 1 60 0 1 20 0 0 30 0 0 0
is nilpotent, with
N2 =
0 0 2 70 0 0 30 0 0 00 0 0 0
; N3 =
0 0 0 60 0 0 00 0 0 00 0 0 0
; N4 =
0 0 0 00 0 0 00 0 0 00 0 0 0
.Though the examples above have a large number of zero entries, a typical nilpotent matrix does not. For example,the matrix
N =
5 −3 215 −9 610 −6 4
squares to zero, though the matrix has no zero entries.
108
30.2. CHARACTERIZATION 109
30.2 Characterization
For an n × n square matrix N with real (or complex) entries, the following are equivalent:
• N is nilpotent.
• The minimal polynomial for N is x*k for some positive integer k ≤ n.
• The characteristic polynomial for N is x*n.
• The only eigenvalue for N is 0.*[5]
• tr(N*k) = 0 for all k > 0.
The last theorem holds true for matrices over any field of characteristic 0 or sufficiently large characteristic. (cf.Newton's identities)This theorem has several consequences, including:
• The degree of an n × n nilpotent matrix is always less than or equal to n. For example, every 2 × 2 nilpotentmatrix squares to zero.
• The determinant and trace of a nilpotent matrix are always zero. Consequently, a nilpotent matrix cannot beinvertible.
• The only nilpotent diagonalizable matrix is the zero matrix.
30.3 Classification
Consider the n × n shift matrix:
S =
0 1 0 . . . 00 0 1 . . . 0...
...... . . . ...
0 0 0 . . . 10 0 0 . . . 0
.
This matrix has 1s along the superdiagonal and 0s everywhere else. As a linear transformation, the shift matrix“shifts”the components of a vector one position to the left, with a zero appearing in the last position:
S(x1, x2, . . . , xn) = (x2, . . . , xn, 0).*[6]
This matrix is nilpotent with degree n, and is the “canonical”nilpotent matrix.Specifically, if N is any nilpotent matrix, then N is similar to a block diagonal matrix of the form
S1 0 . . . 00 S2 . . . 0...
... . . . ...0 0 . . . Sr
where each of the blocks S1, S2, ..., Sr is a shift matrix (possibly of different sizes). This form is a special case of theJordan canonical form for matrices.*[7]*[8]For example, any nonzero 2 × 2 nilpotent matrix is similar to the matrix
110 CHAPTER 30. NILPOTENT MATRIX
[0 10 0
].
That is, if N is any nonzero 2 × 2 nilpotent matrix, then there exists a basis b1, b2 such that Nb1 = 0 and Nb2 = b1.This classification theorem holds for matrices over any field. (It is not necessary for the field to be algebraicallyclosed.)
30.4 Flag of subspaces
A nilpotent transformation L on R*n naturally determines a flag of subspaces
0 ⊂ kerL ⊂ kerL2 ⊂ . . . ⊂ kerLq−1 ⊂ kerLq = Rn
and a signature
0 = n0 < n1 < n2 < . . . < nq−1 < nq = n, ni = dim kerLi.
The signature characterizes L up to an invertible linear transformation. Furthermore, it satisfies the inequalities
nj+1 − nj ≤ nj − nj−1, for all j = 1, . . . , q − 1.
Conversely, any sequence of natural numbers satisfying these inequalities is the signature of a nilpotent transforma-tion.
30.5 Additional properties• If N is nilpotent, then I + N is invertible, where I is the n × n identity matrix. The inverse is given by
(I +N)−1 = I −N +N2 −N3 + · · · ,
where only finitely many terms of this sum are nonzero.
• If N is nilpotent, then
det(I +N) = 1,
where I denotes the n × n identity matrix. Conversely, if A is a matrix and
det(I + tA) = 1
for all values of t, then A is nilpotent. In fact, since p(t) = det(I + tA)− 1 is a polynomial of degree n, it suffices to have this hold for n+ 1 distinct values of t .
• Every singular matrix can be written as a product of nilpotent matrices.*[9]
• A nilpotent matrix is a special case of a convergent matrix.
30.6. GENERALIZATIONS 111
30.6 Generalizations
A linear operator T is locally nilpotent if for every vector v, there exists a k such that
T k(v) = 0.
For operators on a finite-dimensional vector space, local nilpotence is equivalent to nilpotence.
30.7 Notes[1] Herstein (1964, p. 250)
[2] Beauregard & Fraleigh (1973, p. 312)
[3] Herstein (1964, p. 224)
[4] Nering (1970, p. 274)
[5] Herstein (1964, p. 248)
[6] Beauregard & Fraleigh (1973, p. 312)
[7] Beauregard & Fraleigh (1973, pp. 312,313)
[8] Herstein (1964, p. 250)
[9] R. Sullivan, Products of nilpotent matrices, Linear and Multilinear Algebra, Vol. 56, No. 3
30.8 References• Beauregard, Raymond A.; Fraleigh, John B. (1973), A First Course In Linear Algebra: with Optional Introduc-tion to Groups, Rings, and Fields, Boston: Houghton Mifflin Co., ISBN 0-395-14017-X
• Herstein, I. N. (1964), Topics In Algebra, Waltham: Blaisdell Publishing Company, ISBN 978-1114541016
• Nering, Evar D. (1970), Linear Algebra and Matrix Theory (2nd ed.), New York: Wiley, LCCN 76091646
30.9 External links• Nilpotent matrix and nilpotent transformation on PlanetMath.
Chapter 31
Nonnegative matrix
Not to be confused with Totally positive matrix and Positive-definite matrix.
In mathematics, a nonnegative matrix is a matrix in which all the elements are equal to or greater than zero
X ≥ 0, ∀i, j xij ≥ 0.
A positive matrix is a matrix in which all the elements are greater than zero. The set of positive matrices is a subsetof all non-negative matrices. While such matrices are commonly found, the term is only occasionally used due to thepossible confusion with positive-definite matrices, which are different.A rectangular non-negative matrix can be approximated by a decomposition with two other non-negative matricesvia non-negative matrix factorization.A positive matrix is not the same as a positive-definite matrix. A matrix that is both non-negative and positivesemidefinite is called a doubly non-negative matrix.Eigenvalues and eigenvectors of square positive matrices are described by the Perron–Frobenius theorem.
31.1 Inversion
The inverse of any non-singular M-matrix is a non-negative matrix. If the non-singular M-matrix is also symmetricthen it is called a Stieltjes matrix.The inverse of a non-negative matrix is usually not non-negative. The exception is the non-negative monomial matri-ces: a non-negative matrix has non-negative inverse if and only if it is a (non-negative) monomial matrix. Note thatthus the inverse of a positive matrix is not positive or even non-negative, as positive matrices are not monomial, fordimension n > 1.
31.2 Specializations
There are a number of groups of matrices that form specializations of non-negative matrices, e.g. stochastic matrix;doubly stochastic matrix; symmetric non-negative matrix.
31.3 See also
• Metzler matrix
112
31.4. BIBLIOGRAPHY 113
31.4 Bibliography1. Abraham Berman, Robert J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, 1994, SIAM.
ISBN 0-89871-321-8.
2. A. Berman and R. J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, Academic Press, 1979(chapter 2), ISBN 0-12-092250-9
3. R.A. Horn and C.R. Johnson, Matrix Analysis, Cambridge University Press, 1990 (chapter 8).
4. Krasnosel'skii, M. A. (1964). Positive Solutions of Operator Equations. Groningen: P.Noordhoff Ltd. pp. 381pp.
5. Krasnosel'skii, M. A.; Lifshits, Je.A.; Sobolev, A.V. (1990). Positive Linear Systems: The method of positiveoperators. Sigma Series in Applied Mathematics 5. Berlin: Helderman Verlag. pp. 354 pp.
6. Henryk Minc, Nonnegative matrices, John Wiley&Sons, New York, 1988, ISBN 0-471-83966-3
7. Seneta, E. Non-negative matrices and Markov chains. 2nd rev. ed., 1981, XVI, 288 p., Softcover SpringerSeries in Statistics. (Originally published by Allen & Unwin Ltd., London, 1973) ISBN 978-0-387-29765-1
8. Richard S. Varga 2002 Matrix Iterative Analysis, Second ed. (of 1962 Prentice Hall edition), Springer-Verlag.
Chapter 32
Normal matrix
In mathematics, a complex square matrix A is normal if
A∗A = AA∗
where A*∗ is the conjugate transpose of A. That is, a matrix is normal if it commutes with its conjugate transpose.A real square matrix A satisfies A*∗ = A*T, and is therefore normal if A*TA = AA*T.Normality is a convenient test for diagonalizability: a matrix is normal if and only if it is unitarily similar to a diagonalmatrix, and therefore any matrix A satisfying the equation A*∗A = AA*∗ is diagonalizable.The concept of normal matrices can be extended to normal operators on infinite dimensional Hilbert spaces and tonormal elements in C*-algebras. As in the matrix case, normality means commutativity is preserved, to the extentpossible, in the noncommutative setting. This makes normal operators, and normal elements of C*-algebras, moreamenable to analysis.
32.1 Special cases
Among complex matrices, all unitary, Hermitian, and skew-Hermitian matrices are normal. Likewise, among realmatrices, all orthogonal, symmetric, and skew-symmetric matrices are normal. However, it is not the case that allnormal matrices are either unitary or (skew-)Hermitian. For example,
A =
1 1 00 1 11 0 1
is neither unitary, Hermitian, nor skew-Hermitian, yet it is normal because
AA∗ =
2 1 11 2 11 1 2
= A∗A.
32.2 Consequences
Proposition. A normal triangular matrix is diagonal.
Let A be a normal upper triangular matrix. Since (A*∗A)ii = (AA*∗)ii, one has ⟨eᵢ, A*Aeᵢ⟩= ⟨eᵢ, AA*eᵢ⟩ i.e. the firstrow must have the same norm as the first column:
114
32.3. EQUIVALENT DEFINITIONS 115
∥Ae1∥2 = ∥A∗e1∥2 .
The first entry of row 1 and column 1 are the same, and the rest of column 1 is zero. This implies the first row mustbe zero for entries 2 through n. Continuing this argument for row–column pairs 2 through n shows A is diagonal.The concept of normality is important because normal matrices are precisely those to which the spectral theoremapplies:
Proposition. A matrix A is normal if and only if there exists a diagonal matrix Λ and a unitary matrixU such that A = UΛU *∗.
The diagonal entries of Λ are the eigenvalues of A, and the columns of U are the eigenvectors of A. The matchingeigenvalues in Λ come in the same order as the eigenvectors are ordered as columns of U.Another way of stating the spectral theorem is to say that normal matrices are precisely those matrices that can berepresented by a diagonal matrix with respect to a properly chosen orthonormal basis of C*n. Phrased differently:a matrix is normal if and only if its eigenspaces span C*n and are pairwise orthogonal with respect to the standardinner product of C*n.The spectral theorem for normal matrices is a special case of the more general Schur decomposition which holds forall square matrices. Let A be a square matrix. Then by Schur decomposition it is unitary similar to an upper-triangularmatrix, say, B. If A is normal, so is B. But then B must be diagonal, for, as noted above, a normal upper-triangularmatrix is diagonal.The spectral theorem permits the classification of normal matrices in terms of their spectra, for example:
Proposition. A normal matrix is unitary if and only if its spectrum is contained in the unit circle of thecomplex plane.
Proposition. A normal matrix is self-adjoint if and only if its spectrum is contained in R.
In general, the sum or product of two normal matrices need not be normal. However, the following holds:
Proposition. If A and B are normal withAB = BA, then bothAB andA + B are also normal. Furthermorethere exists a unitary matrix U such that UAU *∗ and UBU *∗ are diagonal matrices. In other words Aand B are simultaneously diagonalizable.
In this special case, the columns ofU *∗ are eigenvectors of both A and B and form an orthonormal basis inC*n. Thisfollows by combining the theorems that, over an algebraically closed field, commuting matrices are simultaneouslytriangularizable and a normal matrix is diagonalizable – the added result is that these can both be done simultaneously.
32.3 Equivalent definitions
It is possible to give a fairly long list of equivalent definitions of a normal matrix. Let A be a n × n complex matrix.Then the following are equivalent:
1. A is normal.
2. A is diagonalizable by a unitary matrix.
3. The entire space is spanned by some orthonormal set of eigenvectors of A.
4. ||Ax|| = ||A*∗x|| for every x.
5. The Frobenius norm of A can be computed by the eigenvalues of A: tr(A∗A) =∑j |λj |
2 .
6. The Hermitian part 1/2(A + A*∗) and skew-Hermitian part 1/2(A − A*∗) of A commute.
116 CHAPTER 32. NORMAL MATRIX
7. A*∗ is a polynomial (of degree ≤ n − 1) in A.*[1]
8. A*∗ = AU for some unitary matrix U.*[2]
9. U and P commute, where we have the polar decomposition A = UP with a unitary matrix U and some positivesemidefinite matrix P.
10. A commutes with some normal matrix N with distinct eigenvalues.
11. σi = |λi| for all 1 ≤ i ≤ n where A has singular values σ1 ≥ ... ≥ σn and eigenvalues |λ1| ≥ ... ≥ |λn|.*[3]
12. The operator norm of a normal matrix A equals the numerical and spectral radii of A. (This fact generalizesto normal operators.) Explicitly, this means:
sup∥x∥=1
∥Ax∥ = sup∥x∥=1
|⟨Ax, x⟩| = max |λ| : λ ∈ σ(A)
Some but not all of the above generalize to normal operators on infinite-dimensional Hilbert spaces. For example, abounded operator satisfying (9) is only quasinormal.
32.4 Analogy
It is occasionally useful (but sometimes misleading) to think of the relationships of different kinds of normal matricesas analogous to the relationships between different kinds of complex numbers:
• Invertible matrices are analogous to non-zero complex numbers
• The conjugate transpose is analogous to the complex conjugate
• Unitary matrices are analogous to complex numbers on the unit circle
• Hermitian matrices are analogous to real numbers
• Hermitian positive definite matrices are analogous to positive real numbers
• Skew Hermitian matrices are analogous to purely imaginary numbers
As a special case, the complex numbers may be embedded in the normal 2 × 2 real matrices by the mapping
a+ bi 7→(a b−b a
),
which preserves addition and multiplication. It is easy to check that this embedding respects all of the above analogies.
32.5 Notes[1] Proof: When A is normal, use Lagrange's interpolation formula to construct a polynomial P such that λj = P(λj), where λj
are the eigenvalues of A.
[2] Horn, pp. 109
[3] Horn, Roger A.; Johnson, Charles R. (1991). Topics in Matrix Analysis. Cambridge University Press. p. 157. ISBN978-0-521-30587-7.
32.6 References• Horn, Roger A.; Johnson, Charles R. (1985), Matrix Analysis, Cambridge University Press, ISBN 978-0-521-
38632-6.
Chapter 33
Orbital overlap
In chemical bonds, an orbital overlap is the concentration of orbitals on adjacent atoms in the same regions of space.Orbital overlap can lead to bond formation. The importance of orbital overlap was emphasized by Linus Paulingto explain the molecular bond angles observed through experimentation and is the basis for the concept of orbitalhybridisation. Since s orbitals are spherical (and have no directionality) and p orbitals are oriented 90° to each other,a theory was needed to explain why molecules such as methane (CH4) had observed bond angles of 109.5°.*[1]Pauling proposed that s and p orbitals on the carbon atom can combine to form hybrids (sp3 in the case of methane)which are directed toward the hydrogen atoms. The carbon hybrid orbitals have greater overlap with the hydrogenorbitals, and can therefore form stronger C–H bonds.*[2]A quantitative measure of the overlap of two atomic orbitals on atoms A and B is their overlap integral, defined as
SAB =
∫Ψ∗
AΨB dV,
where the integration extends over all space. The star on the first orbital wavefunction indicates the complex conjugateof the function, which in general may be complex-valued.
33.1 Overlap matrix
The overlap matrix is a square matrix, used in quantum chemistry to describe the inter-relationship of a set of basisvectors of a quantum system, such as an atomic orbital basis set used in molecular electronic structure calculations.In particular, if the vectors are orthogonal to one another, the overlap matrix will be diagonal. In addition, if the basisvectors form an orthonormal set, the overlap matrix will be the identity matrix. The overlap matrix is always n×n,where n is the number of basis functions used. It is a kind of Gramian matrix.In general, each overlap matrix element is defined as an overlap integral:
Sjk = ⟨bj |bk⟩ =∫
Ψ∗jΨk dτ
where
|bj⟩ is the j-th basis ket (vector), and
Ψj is the j-th wavefunction, defined as : Ψj(x) = ⟨x|bj⟩ .
In particular, if the set is normalized (though not necessarily orthogonal) then the diagonal elements will be identically1 and the magnitude of the off-diagonal elements less than or equal to one with equality if and only if there is lineardependence in the basis set as per the Cauchy–Schwarz inequality. Moreover, the matrix is always positive definite;that is to say, the eigenvalues are all strictly positive.
117
118 CHAPTER 33. ORBITAL OVERLAP
33.2 See also• Roothaan equations
• Hartree–Fock method
• Pi bond
• Sigma bond
33.3 References[1] Anslyn, Eric V./Dougherty, Dennis A. (2006). Modern Physical Organic Chemistry. University Science Books.
[2] Pauling, Linus. (1960). The Nature Of The Chemical Bond. Cornell University Press.
Quantum Chemistry: Fifth Edition, Ira N. Levine, 2000
Chapter 34
Orthogonal matrix
In linear algebra, an orthogonal matrix is a square matrix with real entries whose columns and rows are orthogonalunit vectors (i.e., orthonormal vectors), i.e.
QTQ = QQT = I,
where I is the identity matrix.This leads to the equivalent characterization: a matrix Q is orthogonal if its transpose is equal to its inverse:
QT = Q−1,
An orthogonal matrix Q is necessarily invertible (with inverse Q*−1 = Q*T), unitary (Q*−1 = Q*∗) and thereforenormal (Q*∗Q = QQ*∗) in the reals. The determinant of any orthogonal matrix is either +1 or −1. As a linear trans-formation, an orthogonal matrix preserves the dot product of vectors, and therefore acts as an isometry of Euclideanspace, such as a rotation or reflection. In other words, it is a unitary transformation.The set of n × n orthogonal matrices forms a group O(n), known as the orthogonal group. The subgroup SO(n)consisting of orthogonal matrices with determinant +1 is called the special orthogonal group, and each of its elementsis a special orthogonal matrix. As a linear transformation, every special orthogonal matrix acts as a rotation.The complex analogue of an orthogonal matrix is a unitary matrix.
34.1 Overview
An orthogonal matrix is the real specialization of a unitary matrix, and thus always a normal matrix. Althoughwe consider only real matrices here, the definition can be used for matrices with entries from any field. However,orthogonal matrices arise naturally from dot products, and for matrices of complex numbers that leads instead to theunitary requirement. Orthogonal matrices preserve the dot product,*[1] so, for vectors u, v in an n-dimensional realEuclidean space
u · v = (Qu) · (Qv)
where Q is an orthogonal matrix. To see the inner product connection, consider a vector v in an n-dimensionalreal Euclidean space. Written with respect to an orthonormal basis, the squared length of v is v*Tv. If a lineartransformation, in matrix form Qv, preserves vector lengths, then
vTv = (Qv)T(Qv) = vTQTQv.
Thus finite-dimensional linear isometries—rotations, reflections, and their combinations—produce orthogonal ma-trices. The converse is also true: orthogonal matrices imply orthogonal transformations. However, linear algebra
119
120 CHAPTER 34. ORTHOGONAL MATRIX
includes orthogonal transformations between spaces which may be neither finite-dimensional nor of the same dimen-sion, and these have no orthogonal matrix equivalent.Orthogonal matrices are important for a number of reasons, both theoretical and practical. The n×n orthogonal ma-trices form a group under matrix multiplication, the orthogonal group denoted by O(n), which—with its subgroups—is widely used in mathematics and the physical sciences. For example, the point group of a molecule is a sub-group of O(3). Because floating point versions of orthogonal matrices have advantageous properties, they are keyto many algorithms in numerical linear algebra, such as QR decomposition. As another example, with appropriatenormalization the discrete cosine transform (used in MP3 compression) is represented by an orthogonal matrix.
34.2 Examples
Below are a few examples of small orthogonal matrices and possible interpretations.
•[1 00 1
](transformation identity)
An instance of a 2×2 rotation matrix:
• R(16.26) =[
cos θ − sin θsin θ cos θ
]=
[0.96 −0.280.28 0.96
]( by rotation16.26)
•[1 00 −1
]( across reflectionx-axis)
•
0 −0.80 −0.600.80 −0.36 0.480.60 0.48 −0.64
(rotoinversion:axis(0,−3/5, 4/5), angle 90
)
•
0 0 0 10 0 1 01 0 0 00 1 0 0
(axes coordinate of permutation)
34.3 Elementary constructions
34.3.1 Lower dimensions
The simplest orthogonal matrices are the 1×1 matrices [1] and [−1] which we can interpret as the identity and areflection of the real line across the origin.The 2 × 2 matrices have the form
[p tq u
],
which orthogonality demands satisfy the three equations
1 = p2 + t2,
1 = q2 + u2,
0 = pq + tu.
In consideration of the first equation, without loss of generality let p = cos θ, q = sin θ; then either t = −q, u = p or t =q, u = −p. We can interpret the first case as a rotation by θ (where θ = 0 is the identity), and the second as a reflectionacross a line at an angle of θ/2.
34.3. ELEMENTARY CONSTRUCTIONS 121
[cos θ − sin θsin θ cos θ
](rotation),
[cos θ sin θsin θ − cos θ
](reflection)
The special case of the reflection matrix with θ = 90° generates a reflection about the line at 45° given by y = x andtherefore exchanges x and y; it is a permutation matrix, with a single 1 in each column and row (and otherwise 0):
[0 11 0
].
The identity is also a permutation matrix.A reflection is its own inverse, which implies that a reflection matrix is symmetric (equal to its transpose) as well asorthogonal. The product of two rotation matrices is a rotation matrix, and the product of two reflection matrices isalso a rotation matrix.
34.3.2 Higher dimensions
Regardless of the dimension, it is always possible to classify orthogonal matrices as purely rotational or not, but for 3× 3 matrices and larger the non-rotational matrices can be more complicated than reflections. For example,
−1 0 00 −1 00 0 −1
and
0 −1 01 0 00 0 −1
represent an inversion through the origin and a rotoinversion about the z axis.
cos(α) cos(γ)− sin(α) sin(β) sin(γ) − sin(α) cos(β) − cos(α) sin(γ)− sin(α) sin(β) cos(γ)cos(α) sin(β) sin(γ) + sin(α) cos(γ) cos(α) cos(β) cos(α) sin(β) cos(γ)− sin(α) sin(γ)
cos(β) sin(γ) − sin(β) cos(β) cos(γ)
Rotations become more complicated in higher dimensions; they can no longer be completely characterized by oneangle, and may affect more than one planar subspace. It is common to describe a 3 × 3 rotation matrix in terms ofan axis and angle, but this only works in three dimensions. Above three dimensions two or more angles are needed,each associated with a plane of rotation.However, we have elementary building blocks for permutations, reflections, and rotations that apply in general.
34.3.3 Primitives
The most elementary permutation is a transposition, obtained from the identity matrix by exchanging two rows. Anyn×n permutation matrix can be constructed as a product of no more than n − 1 transpositions.A Householder reflection is constructed from a non-null vector v as
Q = I − 2vvT
vTv .
Here the numerator is a symmetric matrix while the denominator is a number, the squared magnitude of v. This isa reflection in the hyperplane perpendicular to v (negating any vector component parallel to v). If v is a unit vector,then Q = I − 2vv*T suffices. A Householder reflection is typically used to simultaneously zero the lower part of acolumn. Any orthogonal matrix of size n × n can be constructed as a product of at most n such reflections.A Givens rotation acts on a two-dimensional (planar) subspace spanned by two coordinate axes, rotating by a chosenangle. It is typically used to zero a single subdiagonal entry. Any rotation matrix of size n×n can be constructed as aproduct of at most n(n − 1)/2 such rotations. In the case of 3 × 3 matrices, three such rotations suffice; and by fixing
122 CHAPTER 34. ORTHOGONAL MATRIX
the sequence we can thus describe all 3 × 3 rotation matrices (though not uniquely) in terms of the three angles used,often called Euler angles.A Jacobi rotation has the same form as a Givens rotation, but is used to zero both off-diagonal entries of a 2 × 2symmetric submatrix.
34.4 Properties
34.4.1 Matrix properties
A real square matrix is orthogonal if and only if its columns form an orthonormal basis of the Euclidean space R*nwith the ordinary Euclidean dot product, which is the case if and only if its rows form an orthonormal basis of R*n.It might be tempting to suppose a matrix with orthogonal (not orthonormal) columns would be called an orthogonalmatrix, but such matrices have no special interest and no special name; they only satisfyM*TM =D, withD a diagonalmatrix.The determinant of any orthogonal matrix is +1 or −1. This follows from basic facts about determinants, as follows:
1 = det(I) = det(QTQ) = det(QT) det(Q) = (det(Q))2.
The converse is not true; having a determinant of ±1 is no guarantee of orthogonality, even with orthogonal columns,as shown by the following counterexample.
[2 00 1
2
]With permutation matrices the determinant matches the signature, being +1 or −1 as the parity of the permutation iseven or odd, for the determinant is an alternating function of the rows.Stronger than the determinant restriction is the fact that an orthogonal matrix can always be diagonalized over thecomplex numbers to exhibit a full set of eigenvalues, all of which must have (complex) modulus 1.
34.4.2 Group properties
The inverse of every orthogonal matrix is again orthogonal, as is the matrix product of two orthogonal matrices. Infact, the set of all n × n orthogonal matrices satisfies all the axioms of a group. It is a compact Lie group of dimensionn(n − 1)/2, called the orthogonal group and denoted by O(n).The orthogonal matrices whose determinant is +1 form a path-connected normal subgroup of O(n) of index 2, thespecial orthogonal group SO(n) of rotations. The quotient group O(n)/SO(n) is isomorphic to O(1), with the projectionmap choosing [+1] or [−1] according to the determinant. Orthogonal matrices with determinant −1 do not includethe identity, and so do not form a subgroup but only a coset; it is also (separately) connected. Thus each orthogonalgroup falls into two pieces; and because the projection map splits, O(n) is a semidirect product of SO(n) by O(1). Inpractical terms, a comparable statement is that any orthogonal matrix can be produced by taking a rotation matrixand possibly negating one of its columns, as we saw with 2×2 matrices. If n is odd, then the semidirect product is infact a direct product, and any orthogonal matrix can be produced by taking a rotation matrix and possibly negatingall of its columns. This follows from the property of determinants that negating a column negates the determinant,and thus negating an odd (but not even) number of columns negates the determinant.Now consider (n + 1) × (n + 1) orthogonal matrices with bottom right entry equal to 1. The remainder of the lastcolumn (and last row) must be zeros, and the product of any two such matrices has the same form. The rest of thematrix is an n × n orthogonal matrix; thus O(n) is a subgroup of O(n + 1) (and of all higher groups).
0
O(n)...0
0 · · · 0 1
34.4. PROPERTIES 123
Since an elementary reflection in the form of a Householder matrix can reduce any orthogonal matrix to this con-strained form, a series of such reflections can bring any orthogonal matrix to the identity; thus an orthogonal groupis a reflection group. The last column can be fixed to any unit vector, and each choice gives a different copy of O(n)in O(n + 1); in this way O(n + 1) is a bundle over the unit sphere S*n with fiber O(n).Similarly, SO(n) is a subgroup of SO(n + 1); and any special orthogonal matrix can be generated by Givens planerotations using an analogous procedure. The bundle structure persists: SO(n) SO(n + 1) → S*n. A single rotationcan produce a zero in the first row of the last column, and series of n−1 rotations will zero all but the last row of thelast column of an n × n rotation matrix. Since the planes are fixed, each rotation has only one degree of freedom, itsangle. By induction, SO(n) therefore has
(n− 1) + (n− 2) + · · ·+ 1 = n(n− 1)/2
degrees of freedom, and so does O(n).Permutation matrices are simpler still; they form, not a Lie group, but only a finite group, the order n! symmetricgroup Sn. By the same kind of argument, Sn is a subgroup of Sn+1. The even permutations produce the subgroup ofpermutation matrices of determinant +1, the order n!/2 alternating group.
34.4.3 Canonical form
More broadly, the effect of any orthogonal matrix separates into independent actions on orthogonal two-dimensionalsubspaces. That is, if Q is special orthogonal then one can always find an orthogonal matrix P, a (rotational) changeof basis, that brings Q into block diagonal form:
P TQP =
R1
. . .Rk
(neven ), P TQP =
R1
. . .Rk
1
(nodd ).
where the matricesR1, ..., Rk are 2 × 2 rotation matrices, and with the remaining entries zero. Exceptionally, a rotationblock may be diagonal, ±I. Thus, negating one column if necessary, and noting that a 2 × 2 reflection diagonalizes toa +1 and −1, any orthogonal matrix can be brought to the form
P TQP =
R1
. . .Rk
0
0
±1. . .
±1
,
The matrices R1, ..., Rk give conjugate pairs of eigenvalues lying on the unit circle in the complex plane; so thisdecomposition confirms that all eigenvalues have absolute value 1. If n is odd, there is at least one real eigenvalue,+1 or −1; for a 3 × 3 rotation, the eigenvector associated with +1 is the rotation axis.
34.4.4 Lie algebra
Suppose the entries of Q are differentiable functions of t, and that t = 0 gives Q = I. Differentiating the orthogonalitycondition
QTQ = I
yields
124 CHAPTER 34. ORTHOGONAL MATRIX
QTQ+QTQ = 0
Evaluation at t = 0 (Q = I) then implies
QT = −Q.
In Lie group terms, this means that the Lie algebra of an orthogonal matrix group consists of skew-symmetric matrices.Going the other direction, the matrix exponential of any skew-symmetric matrix is an orthogonal matrix (in fact,special orthogonal).For example, the three-dimensional object physics calls angular velocity is a differential rotation, thus a vector inthe Lie algebra so(3) tangent to SO(3). Given ω = (xθ, yθ, zθ), with v = (x, y, z) being a unit vector, the correctskew-symmetric matrix form of ω is
Ω =
0 −zθ yθzθ 0 −xθ−yθ xθ 0
.The exponential of this is the orthogonal matrix for rotation around axis v by angle θ; setting c = cos θ/2, s = sin θ/2,
exp(Ω) =
1− 2s2 + 2x2s2 2xys2 − 2zsc 2xzs2 + 2ysc2xys2 + 2zsc 1− 2s2 + 2y2s2 2yzs2 − 2xsc2xzs2 − 2ysc 2yzs2 + 2xsc 1− 2s2 + 2z2s2
.34.5 Numerical linear algebra
34.5.1 Benefits
Numerical analysis takes advantage of many of the properties of orthogonal matrices for numerical linear algebra, andthey arise naturally. For example, it is often desirable to compute an orthonormal basis for a space, or an orthogonalchange of bases; both take the form of orthogonal matrices. Having determinant ±1 and all eigenvalues of magnitude1 is of great benefit for numeric stability. One implication is that the condition number is 1 (which is the minimum),so errors are not magnified when multiplying with an orthogonal matrix. Many algorithms use orthogonal matriceslike Householder reflections and Givens rotations for this reason. It is also helpful that, not only is an orthogonalmatrix invertible, but its inverse is available essentially free, by exchanging indices.Permutations are essential to the success of many algorithms, including the workhorse Gaussian elimination withpartial pivoting (where permutations do the pivoting). However, they rarely appear explicitly as matrices; their specialform allows more efficient representation, such as a list of n indices.Likewise, algorithms using Householder and Givens matrices typically use specialized methods of multiplication andstorage. For example, a Givens rotation affects only two rows of a matrix it multiplies, changing a full multiplicationof order n3 to a much more efficient order n. When uses of these reflections and rotations introduce zeros in a matrix,the space vacated is enough to store sufficient data to reproduce the transform, and to do so robustly. (FollowingStewart (1976), we do not store a rotation angle, which is both expensive and badly behaved.)
34.5.2 Decompositions
A number of important matrix decompositions (Golub & Van Loan 1996) involve orthogonal matrices, includingespecially:
QR decomposition M = QR, Q orthogonal, R upper triangularSingular value decomposition M = UΣV*T, U and V orthogonal, Σ non-negative diagonalEigendecomposition of a symmetric matrix (decomposition according to the spectral theorem)
S = QΛQ*T, S symmetric, Q orthogonal, Λ diagonalPolar decomposition M = QS, Q orthogonal, S symmetric non-negative definite
34.5. NUMERICAL LINEAR ALGEBRA 125
Examples
Consider an overdetermined system of linear equations, as might occur with repeated measurements of a physicalphenomenon to compensate for experimental errors. Write Ax = b, where A is m × n, m > n. A QR decompositionreduces A to upper triangular R. For example, if A is 5 × 3 then R has the form
R =
⋆ ⋆ ⋆0 ⋆ ⋆0 0 ⋆0 0 00 0 0
.The linear least squares problem is to find the x that minimizes ‖Ax − b‖, which is equivalent to projecting b to thesubspace spanned by the columns of A. Assuming the columns of A (and hence R) are independent, the projectionsolution is found from A*TAx = A*Tb. Now A*TA is square (n × n) and invertible, and also equal to R*TR. But thelower rows of zeros in R are superfluous in the product, which is thus already in lower-triangular upper-triangularfactored form, as in Gaussian elimination (Cholesky decomposition). Here orthogonality is important not only forreducing A*TA = (R*TQ*T)QR to R*TR, but also for allowing solution without magnifying numerical problems.In the case of a linear system which is underdetermined, or an otherwise non-invertible matrix, singular value de-composition (SVD) is equally useful. With A factored as UΣV*T, a satisfactory solution uses the Moore-Penrosepseudoinverse, VΣ*+U*T, where Σ*+ merely replaces each non-zero diagonal entry with its reciprocal. Set x toVΣ*+U*Tb.The case of a square invertible matrix also holds interest. Suppose, for example, that A is a 3 × 3 rotation matrixwhich has been computed as the composition of numerous twists and turns. Floating point does not match themathematical ideal of real numbers, so A has gradually lost its true orthogonality. A Gram-Schmidt process couldorthogonalize the columns, but it is not the most reliable, nor the most efficient, nor the most invariant method. Thepolar decomposition factors a matrix into a pair, one of which is the unique closest orthogonal matrix to the givenmatrix, or one of the closest if the given matrix is singular. (Closeness can be measured by any matrix norm invariantunder an orthogonal change of basis, such as the spectral norm or the Frobenius norm.) For a near-orthogonal matrix,rapid convergence to the orthogonal factor can be achieved by a "Newton's method" approach due to Higham (1986)(1990), repeatedly averaging the matrix with its inverse transpose. Dubrulle (1994) has published an acceleratedmethod with a convenient convergence test.For example, consider a non-orthogonal matrix for which the simple averaging algorithm takes seven steps
[3 17 5
]→[1.8125 0.06253.4375 2.6875
]→ · · · →
[0.8 −0.60.6 0.8
]and which acceleration trims to two steps (with γ = 0.353553, 0.565685).
[3 17 5
]→[1.41421 −1.060661.06066 1.41421
]→[0.8 −0.60.6 0.8
]Gram-Schmidt yields an inferior solution, shown by a Frobenius distance of 8.28659 instead of the minimum 8.12404.
[3 17 5
]→[0.393919 −0.9191450.919145 0.393919
]
34.5.3 Randomization
Some numerical applications, such as Monte Carlo methods and exploration of high-dimensional data spaces, requiregeneration of uniformly distributed random orthogonal matrices. In this context, “uniform”is defined in terms ofHaar measure, which essentially requires that the distribution not change if multiplied by any freely chosen orthogonalmatrix. Orthogonalizing matrices with independent uniformly distributed random entries does not result in uniformly
126 CHAPTER 34. ORTHOGONAL MATRIX
distributed orthogonal matrices, but the QR decomposition of independent normally distributed random entries does,as long as the diagonal of R contains only positive entries. Stewart (1980) replaced this with a more efficient ideathat Diaconis & Shahshahani (1987) later generalized as the “subgroup algorithm”(in which form it works just aswell for permutations and rotations). To generate an (n + 1) × (n + 1) orthogonal matrix, take an n × n one and auniformly distributed unit vector of dimension n + 1. Construct a Householder reflection from the vector, then applyit to the smaller matrix (embedded in the larger size with a 1 at the bottom right corner).
34.5.4 Nearest orthogonal matrix
The problem of finding the orthogonal matrix Q nearest a given matrix M is related to the Orthogonal Procrustesproblem. There are several different ways to get the unique solution, the simplest of which is taking the singularvalue decomposition of M and replacing the singular values with ones. Another method expresses the R explicitlybut requires the use of a matrix square root:*[2]
Q =M(MTM)−12
This may be combined with the Babylonian method for extracting the square root of a matrix to give a recurrencewhich converges to an orthogonal matrix quadratically:
Qn+1 = 2M(Q−1n M +MTQn)
−1
where Q0 =M . These iterations are stable provided the condition number of M is less than three.*[3]
34.6 Spin and pin
A subtle technical problem afflicts some uses of orthogonal matrices. Not only are the group components withdeterminant +1 and −1 not connected to each other, even the +1 component, SO(n), is not simply connected (exceptfor SO(1), which is trivial). Thus it is sometimes advantageous, or even necessary, to work with a covering group ofSO(n), the spin group, Spin(n). Likewise, O(n) has covering groups, the pin groups, Pin(n). For n > 2, Spin(n) issimply connected and thus the universal covering group for SO(n). By far the most famous example of a spin groupis Spin(3), which is nothing but SU(2), or the group of unit quaternions.The Pin and Spin groups are found within Clifford algebras, which themselves can be built from orthogonal matrices.
34.7 Rectangular matrices
If Q is not a square matrix, then the conditions Q*TQ = I and QQ*T = I are not equivalent. The condition Q*TQ =I says that the columns of Q are orthonormal. This can only happen if Q is an m × n matrix with n ≤ m. Similarly,QQ*T = I says that the rows of Q are orthonormal, which requires n ≥ m.There is no standard terminology for these matrices. They are sometimes called“orthonormal matrices”, sometimes
“orthogonal matrices”, and sometimes simply “matrices with orthonormal rows/columns”.
34.8 See also• Orthogonal group
• Rotation (mathematics)
• Skew-symmetric matrix, a matrix whose transpose is its negative
• Symplectic matrix
• Unitary matrix
34.9. NOTES 127
34.9 Notes[1] “Paul's online math notes”, Paul Dawkins, Lamar University, 2008. Theorem 3(c)
[2] “Finding the Nearest Orthonormal Matrix”, Berthold K. P. Horn, MIT.
[3] “Newton's Method for the Matrix Square Root”, Nicholas J. Higham, Mathematics of Computation, Volume 46, Number174, 1986.
34.10 References• Diaconis, Persi; Shahshahani, Mehrdad (1987), “The subgroup algorithm for generating uniform random
variables”, Prob. In Eng. And Info. Sci. 1: 15–32, doi:10.1017/S0269964800000255, ISSN 0269-9648
• Dubrulle, Augustin A. (1999), “An Optimum Iteration for the Matrix Polar Decomposition”, Elect. Trans.Num. Anal. 8: 21–25
• Golub, Gene H.; Van Loan, Charles F. (1996), Matrix Computations (3/e ed.), Baltimore: Johns HopkinsUniversity Press, ISBN 978-0-8018-5414-9
• Higham, Nicholas (1986), “Computing the Polar Decomposition—with Applications”, SIAM Journal onScientific and Statistical Computing 7 (4): 1160–1174, doi:10.1137/0907079, ISSN 0196-5204
• Higham, Nicholas; Schreiber, Robert (July 1990),“Fast polar decomposition of an arbitrary matrix”, SIAMJournal on Scientific and Statistical Computing 11 (4): 648–655, doi:10.1137/0911038, ISSN 0196-5204
• Stewart, G. W. (1976), “The Economical Storage of Plane Rotations”, Numerische Mathematik 25 (2):137–138, doi:10.1007/BF01462266, ISSN 0029-599X
• Stewart, G. W. (1980), “The Efficient Generation of Random Orthogonal Matrices with an Application toCondition Estimators”, SIAM J. Numer. Anal. 17 (3): 403–409, doi:10.1137/0717034, ISSN 0036-1429
34.11 External links• Hazewinkel, Michiel, ed. (2001),“Orthogonal matrix”, Encyclopedia of Mathematics, Springer, ISBN 978-
1-55608-010-4
• Tutorial and Interactive Program on Orthogonal Matrix
Chapter 35
Orthostochastic matrix
In mathematics, an orthostochastic matrix is a doubly stochastic matrix whose entries are the squares of the absolutevalues of the entries of some orthogonal matrix.The detailed definition is as follows. A square matrix B of size n is doubly stochastic (or bistochastic) if all its rowsand columns sum to 1 and all its entries are nonnegative real numbers, each of whose rows and columns sums to 1. Itis orthostochastic if there exists an orthogonal matrix O such that
Bij = O2ij for i, j = 1, . . . , n.
All 2-by-2 doubly stochastic matrices are orthostochastic (and also unistochastic) since for any
B =
[a 1− a
1− a a
]we find the corresponding orthogonal matrix
O =
[cosϕ sinϕ− sinϕ cosϕ
],
with cos2 ϕ = a, such that Bij = O2ij .
For larger n the sets of bistochastic matrices includes the set of unistochastic matrices, which includes the set oforthostochastic matrices and these inclusion relations are proper.
35.1 References• Brualdi, Richard A. (2006). Combinatorial matrix classes. Encyclopedia of Mathematics and Its Applications108. Cambridge: Cambridge University Press. ISBN 0-521-86565-4. Zbl 1106.05001.
128
Chapter 36
P-matrix
In mathematics, a P -matrix is a complex square matrix with every principal minor > 0. A closely related class isthat of P0 -matrices, which are the closure of the class of P -matrices, with every principal minor ≥ 0.
36.1 Spectra of P -matrices
By a theorem of Kellogg,*[1]*[2] the eigenvalues of P - and P0 - matrices are bounded away from a wedge about thenegative real axis as follows:
If u1, ..., un are the eigenvalues of an n -dimensional P -matrix, then
|arg(ui)| < π − π
n, i = 1, ..., n
If u1, ..., un , ui = 0 , i = 1, ..., n are the eigenvalues of an n -dimensional P0 -matrix, then
|arg(ui)| ≤ π −π
n, i = 1, ..., n
36.2 Remarks
The class of nonsingular M-matrices is a subset of the class of P -matrices. More precisely, all matrices that are bothP -matrices and Z-matrices are nonsingular M -matrices. The class of sufficient matrices is another generalizationof P -matrices.*[3]The linear complementarity problem LCP (M, q) has a unique solution for every vector q if and only if M is a P-matrix.*[4]If the Jacobian of a function is a P -matrix, then the function is injective on any rectangular region of Rn .*[5]A related class of interest, particularly with reference to stability, is that of P (−) -matrices, sometimes also referredto as N −P -matrices. A matrix A is a P (−) -matrix if and only if (−A) is a P -matrix (similarly for P0 -matrices).Since σ(A) = −σ(−A) , the eigenvalues of these matrices are bounded away from the positive real axis.
36.3 See also
• Q-matrix
• Z-matrix (mathematics)
• M-matrix
• Perron–Frobenius theorem
129
130 CHAPTER 36. P-MATRIX
• Hurwitz matrix
• Linear complementarity problem
36.4 Notes[1] Kellogg, R. B. (April 1972). “On complex eigenvalues ofM andP matrices”. Numerische Mathematik 19 (2): 170–175.
doi:10.1007/BF01402527.
[2] Fang, Li (July 1989). “On the spectra of P- and P0-matrices”. Linear Algebra and its Applications 119: 1–25.doi:10.1016/0024-3795(89)90065-7.
[3] Csizmadia, Zsolt; Illés, Tibor (2006).“New criss-cross type algorithms for linear complementarity problems with sufficientmatrices”(pdf). Optimization Methods and Software 21 (2): 247–266. doi:10.1080/10556780500095009. MR 2195759.
[4] Murty, Katta G. (January 1972). “On the number of solutions to the complementarity problem and spanning propertiesof complementary cones”. Linear Algebra and its Applications 5 (1): 65–108. doi:10.1016/0024-3795(72)90019-5.
[5] Gale, David; Nikaido, Hukukane (10 December 2013). “The Jacobian matrix and global univalence of mappings”.Mathematische Annalen 159 (2): 81–93. doi:10.1007/BF01360282.
36.5 References• Csizmadia, Zsolt; Illés, Tibor (2006). “New criss-cross type algorithms for linear complementarity problems
with sufficient matrices”(pdf). OptimizationMethods and Software 21 (2): 247–266. doi:10.1080/10556780500095009.MR 2195759.
• David Gale and Hukukane Nikaido, The Jacobian matrix and global univalence of mappings, Math. Ann.159:81-93 (1965) doi:10.1007/BF01360282
• Li Fang, On the Spectra of P - and P0 -Matrices, Linear Algebra and its Applications 119:1-25 (1989)
• R. B. Kellogg, On complex eigenvalues of M and P matrices, Numer. Math. 19:170-175 (1972)
Chapter 37
Packed storage matrix
A packed storage matrix, also known as packed matrix, is a term used in programming for representing anm×nmatrix. It is a more compact way than an m-by-n rectangular array by exploiting a special structure of the matrix.Typical examples of matrices that can take advantage of packed storage include:
• symmetric or hermitian matrix
• Triangular matrix
• Banded matrix.
37.1 Code examples (Fortran)
Both of the following storage schemes are used extensively in BLAS and LAPACK.An example of packed storage for hermitian matrix:complex:: A(n,n) ! a hermitian matrix complex:: AP(n*(n+1)/2) ! packed storage for A ! the lower triangle of Ais stored column-by-column in AP. ! unpacking the matrix AP to A do j=1,n k = j*(j-1)/2 A(1:j,j) = AP(1+k:j+k)A(j,1:j-1) = conjg(AP(1+k:j-1+k)) end doAn example of packed storage for banded matrix:real:: A(m,n) ! a banded matrix with kl subdiagonals and ku superdiagonals real:: AP(-kl:ku,n) ! packed storage forA ! the band of A is stored column-by-column in AP. Some elements of AP are unused. ! unpacking the matrix APto A do j=1,n forall(i=max(1,j-kl):min(m,j+ku)) A(i,j) = AP(i-j,j) end do print *,AP(0,:) ! the diagonal
131
Chapter 38
Paley construction
In mathematics, the Paley construction is a method for constructing Hadamard matrices using finite fields. Theconstruction was described in 1933 by the English mathematician Raymond Paley.The Paley construction uses quadratic residues in a finite field GF(q) where q is a power of an odd prime number.There are two versions of the construction depending on whether q is congruent to 1 or 3 (mod 4).
38.1 Quadratic character and Jacobsthal matrix
The quadratic character χ(a) indicates whether the given finite field element a is a perfect square. Specifically, χ(0)= 0, χ(a) = 1 if a = b2 for some non-zero finite field element b, and χ(a) = −1 if a is not the square of any finite fieldelement. For example, in GF(7) the non-zero squares are 1 = 12 = 62, 4 = 22 = 52, and 2 = 32 = 42. Hence χ(0) = 0,χ(1) = χ(2) = χ(4) = 1, and χ(3) = χ(5) = χ(6) = −1.The Jacobsthal matrix Q for GF(q) is the q×q matrix with rows and columns indexed by finite field elements such thatthe entry in row a and column b is χ(a − b). For example, in GF(7), if the rows and columns of the Jacobsthal matrixare indexed by the field elements 0, 1, 2, 3, 4, 5, 6, then
Q =
0 −1 −1 1 −1 1 11 0 −1 −1 1 −1 11 1 0 −1 −1 1 −1−1 1 1 0 −1 −1 11 −1 1 1 0 −1 −1−1 1 −1 1 1 0 −1−1 −1 1 −1 1 1 0
.
The Jacobsthal matrix has the properties QQ*T = qI − J and QJ = JQ = 0 where I is the q×q identity matrix and J isthe q×q all-1 matrix. If q is congruent to 1 (mod 4) then −1 is a square in GF(q) which implies that Q is a symmetricmatrix. If q is congruent to 3 (mod 4) then −1 is not a square, and Q is a skew-symmetric matrix. When q is a primenumber, Q is a circulant matrix. That is, each row is obtained from the row above by cyclic permutation.
38.2 Paley construction I
If q is congruent to 3 (mod 4) then
H = I +
[0 jT
−j Q
]is a Hadamard matrix of size q + 1. Here j is the all-1 column vector of length q and I is the (q+1)×(q+1) identitymatrix. The matrix H is a skew Hadamard matrix, which means it satisfies H+H*T = 2I.
132
38.3. PALEY CONSTRUCTION II 133
38.3 Paley construction II
If q is congruent to 1 (mod 4) then the matrix obtained by replacing all 0 entries in
[0 jT
j Q
]with the matrix
[1 −1−1 −1
]and all entries ±1 with the matrix
±[1 11 −1
]is a Hadamard matrix of size 2(q + 1). It is a symmetric Hadamard matrix.
38.4 Examples
Applying Paley Construction I to the Jacobsthal matrix for GF(7), one produces the 8×8 Hadamard matrix,11111111 −1-−1-11 −11-−1-1 −111-−1- -−111-−1 −1-111-- -−1-111- --−1-111.For an example of the Paley II construction when q is a prime power rather than a prime number, consider GF(9).This is an extension field of GF(3) obtained by adjoining a root of an irreducible quadratic. Different irreduciblequadratics produce equivalent fields. Choosing x2+x−1 and letting a be a root of this polynomial, the nine elementsof GF(9) may be written 0, 1, −1, a, a+1, a−1, −a, −a+1, −a−1. The non-zero squares are 1 = (±1)2, −a+1 = (±a)2,a−1 = (±(a+1))2, and −1 = (±(a−1))2. The Jacobsthal matrix is
Q =
0 1 1 −1 −1 1 −1 1 −11 0 1 1 −1 −1 −1 −1 11 1 0 −1 1 −1 1 −1 −1−1 1 −1 0 1 1 −1 −1 1−1 −1 1 1 0 1 1 −1 −11 −1 −1 1 1 0 −1 1 −1−1 −1 1 −1 1 −1 0 1 11 −1 −1 −1 −1 1 1 0 1−1 1 −1 1 −1 −1 1 1 0
.
It is a symmetric matrix consisting of nine 3×3 circulant blocks. Paley Construction II produces the symmetric 20×20Hadamard matrix,1- 111111 111111 111111 -- 1-1-1- 1-1-1- 1-1-1- 11 1-1111 ---−11 -−11-- 1- -−1-1- −1-11- −11-−1 11 111-11 11---- ---−11 1- 1--−1- 1-−1-1 −1-11- 11 11111- -−11-- 11---- 1- 1-1--- −11-−1 1-−1-1 11 -−11-- 1-1111---−11 1- −11-−1 -−1-1- −1-11- 11 ---−11 111-11 11---- 1- −1-11- 1--−1- 1-−1-1 11 11---- 11111- -−11-- 1-1-−1-1 1-1--- −11-−1 11 ---−11 -−11-- 1-1111 1- −1-11- −11-−1 -−1-1- 11 11---- ---−11 111-11 1- 1-−1-1−1-11- 1--−1- 11 -−11-- 11---- 11111- 1- −11-−1 1-−1-1 1-1--−.
38.5 The Hadamard conjecture
The size of a Hadamard matrix must be 1, 2, or a multiple of 4. The Kronecker product of two Hadamard matricesof sizes m and n is an Hadamard matrix of size mn. By forming Kronecker products of matrices from the Paleyconstruction and the 2×2 matrix,
134 CHAPTER 38. PALEY CONSTRUCTION
H2 =
[1 11 −1
],
Hadamard matrices of every allowed size up to 100 except for 92 are produced. In his 1933 paper, Paley says “Itseems probable that, wheneverm is divisible by 4, it is possible to construct an orthogonal matrix of orderm composedof ±1, but the general theorem has every appearance of difficulty.”This appears to be the first published statementof the Hadamard conjecture. A matrix of size 92 was eventually constructed by Baumert, Golomb, and Hall, using aconstruction due to Williamson combined with a computer search. Currently, Hadamard matrices have been shownto exist for all m≡ 0 mod 4 for m < 668.
38.6 See also• Paley biplane
• Paley graph
38.7 References• Paley, R.E.A.C. (1933). “On orthogonal matrices”. Journal of Mathematics and Physics 12: 311–320.
• L. D. Baumert; S. W. Golomb; M. Hall Jr (1962). “Discovery of an Hadamard matrix of order 92”. Bull.Amer. Math. Soc. 68 (3): 237–238. doi:10.1090/S0002-9904-1962-10761-7.
• F.J. MacWilliams; N.J.A. Sloane (1977). The Theory of Error-Correcting Codes. North-Holland. pp. 47, 56.ISBN 0-444-85193-3.
Chapter 39
Parry–Sullivan invariant
In mathematics, the Parry–Sullivan invariant (or Parry–Sullivan number) is a numerical quantity of interest inthe study of incidence matrices in graph theory, and of certain one-dimensional dynamical systems. It provides apartial classification of non-trivial irreducible incidence matrices.It is named after the English mathematician Bill Parry and the American mathematician Dennis Sullivan, who intro-duced the invariant in a joint paper published in the journal Topology in 1975.
39.1 Definition
Let A be an n × n incidence matrix. Then the Parry–Sullivan number of A is defined to be
PS(A) = det(I −A),
where I denotes the n × n identity matrix.
39.2 Properties
It can be shown that, for nontrivial irreducible incidence matrices, flow equivalence is completely determined by theParry–Sullivan number and the Bowen–Franks group.
39.3 References• Parry, W., & Sullivan, D. (1975). “A topological invariant of flows on 1-dimensional spaces”. Topology 14
(4): 297–299. doi:10.1016/0040-9383(75)90012-9.
135
Chapter 40
Pascal matrix
In mathematics, particularly matrix theory and combinatorics, the Pascal matrix is an infinite matrix containing thebinomial coefficients as its elements. There are three ways to achieve this: as either an upper-triangular matrix, alower-triangular matrix, or a symmetric matrix. The 5×5 truncations of these are shown below.
Upper triangular: U5 =
1 1 1 1 10 1 2 3 40 0 1 3 60 0 0 1 40 0 0 0 1
; lower triangular: L5 =
1 0 0 0 01 1 0 0 01 2 1 0 01 3 3 1 01 4 6 4 1
; symmetric: S5 =
1 1 1 1 11 2 3 4 51 3 6 10 151 4 10 20 351 5 15 35 70
.These matrices have the pleasing relationship Sn = LnUn. From this it is easily seen that all three matrices havedeterminant 1, as the determinant of a triangular matrix is simply the product of its diagonal elements, which are all1 for both Ln and Un. In other words, matrices Sn, Ln, and Un are unimodular, with Ln and Un having trace n.The elements of the symmetric Pascal matrix are the binomial coefficients, i.e.
Sij =
(n
r
)=
n!
r!(n− r)!, where n = i+ j, r = i.
In other words,
Sij = i+jCi =(i+ j)!
(i)!(j)!.
Thus the trace of Sn is given by
tr(Sn) =n∑i=1
[2(i− 1)]!
[(i− 1)!]2=n−1∑k=0
(2k)!
(k!)2
with the first few terms given by the sequence 1, 3, 9, 29, 99, 351, 1275, …(sequence A006134 in OEIS).
40.1 Construction
The Pascal matrix can actually be constructed by taking the matrix exponential of a special subdiagonal or superdiagonalmatrix. The example below constructs a 7-by-7 Pascal matrix, but the method works for any desired n×n Pascal ma-trices. (Note that dots in the following matrices represent zero elements.)
136
40.2. VARIANTS 137
L7 = exp
. . . . . . .1 . . . . . .. 2 . . . . .. . 3 . . . .. . . 4 . . .. . . . 5 . .. . . . . 6 .
=
1 . . . . . .1 1 . . . . .1 2 1 . . . .1 3 3 1 . . .1 4 6 4 1 . .1 5 10 10 5 1 .1 6 15 20 15 6 1
;
U7 = exp
. 1 . . . . .. . 2 . . . .. . . 3 . . .. . . . 4 . .. . . . . 5 .. . . . . . 6. . . . . . .
=
1 1 1 1 1 1 1. 1 2 3 4 5 6. . 1 3 6 10 15. . . 1 4 10 20. . . . 1 5 15. . . . . 1 6. . . . . . 1
;
∴ S7 = exp
. . . . . . .1 . . . . . .. 2 . . . . .. . 3 . . . .. . . 4 . . .. . . . 5 . .. . . . . 6 .
exp
. 1 . . . . .. . 2 . . . .. . . 3 . . .. . . . 4 . .. . . . . 5 .. . . . . . 6. . . . . . .
=
1 1 1 1 1 1 11 2 3 4 5 6 71 3 6 10 15 21 281 4 10 20 35 56 841 5 15 35 70 126 2101 6 21 56 126 252 4621 7 28 84 210 462 924
.It is important to note that one cannot simply assume exp(A)exp(B) = exp(A + B), for A and B n×n matrices. Suchan identity only holds when AB = BA (i.e. when the matrices A and B commute). In the construction of symmetricPascal matrices like that above, the sub- and superdiagonal matrices do not commute, so the (perhaps) temptingsimplification involving the addition of the matrices cannot be made.A useful property of the sub- and superdiagonal matrices used in the construction is that both are nilpotent; that is,when raised to a sufficiently high integer power, they degenerate into the zero matrix. (See shift matrix for furtherdetails.) As the n×n generalised shift matrices we are using become zero when raised to power n, when calculatingthe matrix exponential we need only consider the first n + 1 terms of the infinite series to obtain an exact result.
40.2 Variants
Interesting variants can be obtained by obvious modification of the matrix-logarithm PL7 and then application of thematrix exponential.The first example below uses the squares of the values of the log-matrix and constructs a 7-by-7“Laguerre"- matrix(or matrix of coefficients of Laguerre polynomials
LAG7 = exp
. . . . . . .1 . . . . . .. 4 . . . . .. . 9 . . . .. . . 16 . . .. . . . 25 . .. . . . . 36 .
=
1 . . . . . .1 1 . . . . .2 4 1 . . . .6 18 9 1 . . .24 96 72 16 1 . .120 600 600 200 25 1 .720 4320 5400 2400 450 36 1
;
The Laguerre-matrix is actually used with some other scaling and/or the scheme of alternating signs. (Literatureabout generalizations to higher powers is not found yet)The second example below uses the products v(v + 1) of the values of the log-matrix and constructs a 7-by-7“Lah"-matrix (or matrix of coefficients of Lah numbers)
LAH7 = exp
. . . . . . .2 . . . . . .. 6 . . . . .. . 12 . . . .. . . 20 . . .. . . . 30 . .. . . . . 42 .
=
1 . . . . . . .2 1 . . . . . .6 6 1 . . . . .24 36 12 1 . . . .120 240 120 20 1 . . .720 1800 1200 300 30 1 . .5040 15120 12600 4200 630 42 1 .40320 141120 141120 58800 11760 1176 56 1
;
Using v(v − 1) instead provides a diagonal shifting to bottom-right.The third example below uses the square of the original PL7-matrix, divided by 2, in other words: the first-order bino-mials (binomial(k, 2) ) in the second subdiagonal and constructs a matrix, which occurs in context of the derivativesand integrals of the Gaussian error function:
GS7 = exp
. . . . . . .. . . . . . .1 . . . . . .. 3 . . . . .. . 6 . . . .. . . 10 . . .. . . . 15 . .
=
1 . . . . . .. 1 . . . . .1 . 1 . . . .. 3 . 1 . . .3 . 6 . 1 . .. 15 . 10 . 1 .15 . 45 . 15 . 1
;
138 CHAPTER 40. PASCAL MATRIX
If this matrix is inverted (using, for instance, the negative matrix-logarithm), then this matrix has alternating signsand gives the coefficients of the derivatives (and by extension) the integrals of the Gauss' error-function . (Literatureabout generalizations to higher powers is not found yet.)Another variant can be obtained by extending the original matrix to negative values:
exp
. . . . . . . . . . . .−5 . . . . . . . . . . .. −4 . . . . . . . . . .. . −3 . . . . . . . . .. . . −2 . . . . . . . .. . . . −1 . . . . . . .. . . . . 0 . . . . . .. . . . . . 1 . . . . .. . . . . . . 2 . . . .. . . . . . . . 3 . . .. . . . . . . . . 4 . .. . . . . . . . . . 5 .
=
1 . . . . . . . . . . .−5 1 . . . . . . . . . .10 −4 1 . . . . . . . . .−10 6 −3 1 . . . . . . . .5 −4 3 −2 1 . . . . . . .−1 1 −1 1 −1 1 . . . . . .. . . . . 0 1 . . . . .. . . . . . 1 1 . . . .. . . . . . 1 2 1 . . .. . . . . . 1 3 3 1 . .. . . . . . 1 4 6 4 1 .. . . . . . 1 5 10 10 5 1
.
40.3 See also• Pascal's triangle
• LU decomposition
40.4 References• G. S. Call and D. J. Velleman, “Pascal's matrices”, American Mathematical Monthly, volume 100, (April
1993) pages 372–376
• Edelman, Alan; Strang, Gilbert (March 2004), “Pascal Matrices” (PDF), American Mathematical Monthly111 (3): 361–385, doi:10.2307/4145127
40.5 External links• G. Helms Pascalmatrix in a project of compilation of facts about binomial&related matrices
• G. Helms Gauss-matrix
• Weisstein, Eric W. Gaussian-function
• Weisstein, Eric W. Erf-function
• Weisstein, Eric W. “Hermite Polynomial.”Hermite-polynomials
• Endl, Kurt "Über eine ausgezeichnete Eigenschaft der Koeffizientenmatrizen des Laguerreschen und des Her-miteschen Polynomsystems”. In: PERIODICAL VOLUME 65 Mathematische Zeitschrift Kurt Endl
•“Coefficients of unitary Hermite polynomials Hen(x)" in the Online Encyclopedia of Integer SequencesA066325 (Related to Gauss-matrix).
Chapter 41
Pauli matrices
In mathematical physics and mathematics, the Pauli matrices are a set of three 2 × 2 complex matrices which areHermitian and unitary.*[1] Usually indicated by the Greek letter sigma (σ), they are occasionally denoted by tau (τ)when used in connection with isospin symmetries. They are
σ1 = σx =
(0 11 0
)σ2 = σy =
(0 −ii 0
)σ3 = σz =
(1 00 −1
).
These matrices are named after the physicist Wolfgang Pauli. In quantum mechanics, they occur in the Pauli equationwhich takes into account the interaction of the spin of a particle with an external electromagnetic field.Each Pauli matrix is Hermitian, and together with the identity matrix I (sometimes considered as the zeroth Paulimatrix σ0), the Pauli matrices (multiplied by real coefficients) span the full vector space of 2 × 2 Hermitian matrices.In the language of quantum mechanics, Hermitian matrices are observables, so the Pauli matrices span the spaceof observables of the 2-dimensional complex Hilbert space. In the context of Pauli's work, σk is the observablecorresponding to spin along the kth coordinate axis in three-dimensional Euclidean space ℝ3.The Pauli matrices (after multiplication by i to make them anti-Hermitian), also generate transformations in the senseof Lie algebras: the matrices iσ1, iσ2, iσ3 form a basis for su(2), which exponentiates to the special unitary groupSU(2). The algebra generated by the three matrices σ1, σ2, σ3 is isomorphic to the Clifford algebra of ℝ3, called thealgebra of physical space.
41.1 Algebraic properties
All three of the Pauli matrices can be compacted into a single expression:
σa =
(δa3 δa1 − iδa2
δa1 + iδa2 −δa3
)where i = √−1 is the imaginary unit, and δab is the Kronecker delta, which equals +1 if a = b and 0 otherwise. Thisexpression is useful for“selecting”any one of the matrices numerically by substituting values of a = 1, 2, 3, in turnuseful when any of the matrices (but no particular one) is to be used in algebraic manipulations.The matrices are involutory:
σ21 = σ2
2 = σ23 = −iσ1σ2σ3 =
(1 00 1
)= I
139
140 CHAPTER 41. PAULI MATRICES
where I is the identity matrix.
• The determinants and traces of the Pauli matrices are:
detσi = −1,Trσi = 0.
From above we can deduce that the eigenvalues of each σi are ±1.
• Together with the 2 × 2 identity matrix I (sometimes written as σ0), the Pauli matrices form an orthogonalbasis, in the sense of Hilbert–Schmidt, for the real Hilbert space of 2 × 2 complex Hermitian matrices, or thecomplex Hilbert space of all 2 × 2 matrices.
41.1.1 Eigenvectors and eigenvalues
Each of the (Hermitian) Pauli matrices has two eigenvalues, +1 and −1. The corresponding normalized eigenvectorsare:
ψx+ =1√2
(11
), ψx− =
1√2
(1−1
),
ψy+ =1√2
(1i
), ψy− =
1√2
(1−i
),
ψz+ =
(10
), ψz− =
(01
).
41.1.2 Pauli vector
The Pauli vector is defined by*[nb 1]
σ = σ1x+ σ2y + σ3z
and provides a mapping mechanism from a vector basis to a Pauli matrix basis*[2] as follows,
a · σ = (aixi) · (σj xj)= aiσj xi · xj= aiσjδij
= aiσi =
(a3 a1 − ia2
a1 + ia2 −a3
)using the summation convention. Further,
det a · σ = −a · a = −|a|2,
and also (see completeness, below)
1
2tr[(a · σ)σ] = a.
41.1. ALGEBRAIC PROPERTIES 141
41.1.3 Commutation relations
The Pauli matrices obey the following commutation relations:
[σa, σb] = 2iεabc σc ,
and anticommutation relations:
σa, σb = 2δab I.
where εabc is the Levi-Civita symbol, Einstein summation notation is used, δab is the Kronecker delta, and I is the 2× 2 identity matrix.For example,
[σ1, σ2] = 2iσ3
[σ2, σ3] = 2iσ1
[σ3, σ1] = 2iσ2
[σ1, σ1] = 0
σ1, σ1 = 2I
σ1, σ2 = 0 .
41.1.4 Relation to dot and cross product
Pauli vectors elegantly map these commutation and anticommutation relations to corresponding vector products.Adding the commutator to the anticommutator gives
[σa, σb] + σa, σb = (σaσb − σbσa) + (σaσb + σbσa)
2i∑c
εabc σc + 2δabI = 2σaσb
so that, cancelling the factors of 2,
Contracting each side of the equation with components of two 3-vectors ap and bq (which commute with the Paulimatrices, i.e., apσq = σqap) for each matrix σq and vector component ap (and likewise with bq), and relabeling indicesa, b, c → p, q, r, to prevent notational conflicts, yields
apbqσpσq = apbq
(i∑r
εpqr σr + δpqI
)apσpbqσq = i
∑r
εpqr apbqσr + apbqδpqI .
Finally, translating the index notation for the dot product and cross product results in
41.1.5 Exponential of a Pauli vector
For
142 CHAPTER 41. PAULI MATRICES
a = an, |n| = 1,
one has, for even powers,
(n · σ)2n = I
which can be shown first for the n = 1 case using the anticommutation relations.Thus, for odd powers,
(n · σ)2n+1 = n · σ .
Matrix exponentiating, and using the Taylor series for sine and cosine,
eia(n·σ) =∞∑n=0
in [a(n · σ)]n
n!
=
∞∑n=0
(−1)n(an · σ)2n
(2n)!+ i
∞∑n=0
(−1)n(an · σ)2n+1
(2n+ 1)!
= I∞∑n=0
(−1)na2n
(2n)!+ i(n · σ)
∞∑n=0
(−1)na2n+1
(2n+ 1)!
and, in the last line, the first sum is the cosine, while the second sum is the sine; so, finally,which is analogous to Euler's formula. Note
det[ia(n · σ)] = a2
while the determinant of the exponential itself is just 1, which makes it the generic group element of SU(2).A more abstract version of formula (2) for a general 2 × 2 matrix can be found in the article on matrix exponentials.A general version of (2) for an analytic (at a and −a) function is provided by application of Sylvester's formula,*[3]
f(a(n · σ)) = If(a) + f(−a)
2+ i(n · σ)f(a)− f(−a)
2.
The group composition law of SU(2)
A straightforward application of formula (2) provides a parameterization of the composition law of the group SU(2).*[nb2] One may directly solve for c in
eia(n·σ)eib(m·σ) = I(cos a cos b− n · m sin a sin b) + i(n sin a cos b+ m sin b cos a− n× m sin a sin b) · σ= I cos c+ i(k · σ) sin c
= eic(k·σ),
which specifies the generic group multiplication, where, manifestly,
cos c = cos a cos b− n · m sin a sin b ,
the spherical law of cosines. Given c, then,
41.1. ALGEBRAIC PROPERTIES 143
k =1
sin c (n sin a cos b+ m sin b cos a− n× m sin a sin b) .
Consequently, the composite rotation parameters in this group element (a closed form of the respective BCH expan-sion in this case) simply amount to*[4]
eick·σ = exp(ic
sin c (n sin a cos b+ m sin b cos a− n× m sin a sin b) · σ).
(Of course, when n is parallel to m, so is k, and c = a + b.)The fact that any 2 × 2 complex Hermitian matrices can be expressed in terms of the identity matrix and the Paulimatrices also leads to the Bloch sphere representation of 2 × 2 mixed states' density matrix, (2 × 2 positive semidef-inite matrices with trace 1). This can be seen by simply first writing an arbitrary Hermitian matrix as a real linearcombination of σ0, σ1, σ2, σ3 as above, and then imposing the positive-semidefinite and trace 1 conditions.See also: Rotation formalisms in three dimensions § Rodrigues parameters and Gibbs representation
41.1.6 Completeness relation
An alternative notation that is commonly used for the Pauli matrices is to write the vector index i in the superscript,and the matrix indices as subscripts, so that the element in row α and column β of the i-th Pauli matrix is σ *iαβ.In this notation, the completeness relation for the Pauli matrices can be written
σαβ · σγδ ≡3∑i=1
σiαβσiγδ = 2δαδδβγ − δαβδγδ.
Proof
The fact that the Pauli matrices, along with the identity matrix I, form an orthogonal basis for the complex Hilbertspace of all 2 × 2 matrices means that we can express any matrix M as
M = cI +∑i
aiσi
where c is a complex number, and a is a 3-component complex vector. It is straightforward to show, using theproperties listed above, that
trσiσj = 2δij
where “tr”denotes the trace, and hence that c = 12 trM and ai = 1
2 trσiM .This therefore gives
2M = ItrM +∑i
σitrσiM
which can be rewritten in terms of matrix indices as
2Mαβ = δαβMγγ +∑i
σiαβσiγδMδγ
144 CHAPTER 41. PAULI MATRICES
where summation is implied over the repeated indices γ and δ. Since this is true for any choice of the matrix M, thecompleteness relation follows as stated above.As noted above, it is common to denote the 2 × 2 unit matrix by σ0, so σ0αβ = δαβ. The completeness relation cantherefore alternatively be expressed as
3∑i=0
σiαβσiγδ = 2δαδδβγ
41.1.7 Relation with the permutation operator
Let Pij be the transposition (also known as a permutation) between two spins σi and σj living in the tensor productspace ℂ2 ⊗ ℂ2,
Pij |σiσj⟩ = |σjσi⟩ .
This operator can also be written more explicitly as Dirac's spin exchange operator,
Pij =12 (σi · σj + 1) .
Its eigenvalues are therefore*[5] 1 or −1. It may thus be utilized as an interaction term in a Hamiltonian, splitting theenergy eigenvalues of its symmetric versus antisymmetric eigenstates.
41.2 SU(2)
The group SU(2) is the Lie group of unitary 2×2 matrices with unit determinant; its Lie algebra is the set of all 2×2anti-Hermitian matrices with trace 0. Direct calculation, as above, shows that the Lie algebra su2 is the 3-dimensionalreal algebra spanned by the set iσj. In compact notation,
su(2) = spaniσ1, iσ2, iσ3.
As a result, each iσj can be seen as an infinitesimal generator of SU(2). The elements of SU(2) are exponentialsof linear combinations of these three generators, and multiply as indicated above in discussing the Pauli vector.Although this suffices to generate SU(2), it is not a proper representation of su(2), as the Pauli eigenvalues are scaledunconventionally. The conventional normalization is λ = 1/2, so that
su(2) = spaniσ12,iσ22,iσ32
.
As SU(2) is a compact group, its Cartan decomposition is trivial.
41.2.1 SO(3)
The Lie algebra su(2) is isomorphic to the Lie algebra so(3), which corresponds to the Lie group SO(3), the group ofrotations in three-dimensional space. In other words, one can say that the iσj are a realization (and, in fact, the lowest-dimensional realization) of infinitesimal rotations in three-dimensional space. However, even though su(2) and so(3)are isomorphic as Lie algebras, SU(2) and SO(3) are not isomorphic as Lie groups. SU(2) is actually a double coverof SO(3), meaning that there is a two-to-one group homomorphism from SU(2) to SO(3), see relationship betweenSO(3) and SU(2).
41.3. PHYSICS 145
41.2.2 Quaternions
Main article: versor
The real linear span of I, iσ1, iσ2, iσ3 is isomorphic to the real algebra of quaternions ℍ. The isomorphism fromℍ to this set is given by the following map (notice the reversed signs for the Pauli matrices):
1 7→ I, i 7→ −iσ1, j 7→ −iσ2, k 7→ −iσ3.
Alternatively, the isomorphism can be achieved by a map using the Pauli matrices in reversed order,*[6]
1 7→ I, i 7→ iσ3, j 7→ iσ2, k 7→ iσ1.
As the quaternions of unit norm is group-isomorphic to SU(2), this gives yet another way of describing SU(2) viathe Pauli matrices. The two-to-one homomorphism from SU(2) to SO(3) can also be explicitly given in terms of thePauli matrices in this formulation.Quaternions form a division algebra—every non-zero element has an inverse—whereas Pauli matrices do not. Fora quaternionic version of the algebra generated by Pauli matrices see biquaternions, which is a venerable algebra ofeight real dimensions.
41.3 Physics
41.3.1 Quantum mechanics
In quantum mechanics, each Pauli matrix is related to an angular momentum operator that corresponds to an observabledescribing the spin of a spin ½ particle, in each of the three spatial directions. As an immediate consequence of theCartan decomposition mentioned above, iσj are the generators of a projective representation (spin representation)of the rotation group SO(3) acting on non-relativistic particles with spin ½. The states of the particles are representedas two-component spinors. In the same way, the Pauli matrices are related to the isospin operatorAn interesting property of spin ½ particles is that they must be rotated by an angle of 4π in order to return to theiroriginal configuration. This is due to the two-to-one correspondence between SU(2) and SO(3) mentioned above,and the fact that, although one visualizes spin up/down as the north/south pole on the 2-sphere S 2, they are actuallyrepresented by orthogonal vectors in the two dimensional complex Hilbert space.For a spin ½ particle, the spin operator is given by J=ħ/2σ, the fundamental representation of SU(2). By takingKronecker products of this representation with itself repeatedly, one may construct all higher irreducible representa-tions. That is, the resulting spin operators for higher spin systems in three spatial dimensions, for arbitrarily large j,can be calculated using this spin operator and ladder operators. They can be found in Rotation group SO(3)#A noteon representations. The analog formula to the above generalization of Euler's formula for Pauli matrices, the groupelement in terms of spin matrices, is tractable, but less simple.*[7]Also useful in the quantum mechanics of multiparticle systems, the general Pauli group Gn is defined to consist of alln-fold tensor products of Pauli matrices.
41.3.2 Quantum information• In quantum information, single-qubit quantum gates are 2 × 2 unitary matrices. The Pauli matrices are some of
the most important single-qubit operations. In that context, the Cartan decomposition given above is called theZ–Y decomposition of a single-qubit gate. Choosing a different Cartan pair gives a similar X–Y decompositionof a single-qubit gate.
41.4 See also• For higher spin generalizations of the Pauli matrices, see spin (physics) § Higher spins
146 CHAPTER 41. PAULI MATRICES
• Gamma matrices
• Angular momentum
• Gell-Mann matrices
• Poincaré group
• Generalizations of Pauli matrices
• Bloch sphere
• Euler's four-square identity
41.5 Remarks[1] The Pauli vector is a formal device. It may be thought of as an element of M2(ℂ) ⊗ ℝ3, where the tensor product space is
endowed with a mapping ⋅: ℝ3 × M2(ℂ) ⊗ ℝ3 → M2(ℂ).
[2] N.B. The relation among a, b, c, n, m, k derived here in the 2 × 2 representation holds for all representations of SU(2),being a group identity.
41.6 Notes[1] “Pauli matrices”. Planetmath website. 28 March 2008. Retrieved 28 May 2013.
[2] See the spinor map.
[3] Nielsen, Michael A.; Chuang, Isaac L. (2000). Quantum Computation and Quantum Information. Cambridge, UK: Cam-bridge University Press. ISBN 978-0-521-63235-5. OCLC 43641333.
[4] cf. J W Gibbs (1884). Elements of Vector Analysis, New Haven, 1884, p. 67
[5] Explicitly, in the convention of “right-space matrices into elements of left-space matrices”, it is(
1 0 0 00 0 1 00 1 0 00 0 0 1
).
[6] Nakahara, Mikio (2003). Geometry, topology, and physics (2nd ed.). CRC Press. ISBN 978-0-7503-0606-5, pp. xxii.
[7] Curtright, T L; Fairlie, D B; Zachos, C K (2014).“A compact formula for rotations as spin matrix polynomials”. SIGMA10: 084. doi:10.3842/SIGMA.2014.084.
41.7 References• Liboff, Richard L. (2002). Introductory Quantum Mechanics. Addison-Wesley. ISBN 0-8053-8714-5.
• Schiff, Leonard I. (1968). Quantum Mechanics. McGraw-Hill. ISBN 978-0070552876.
• Leonhardt, Ulf (2010). Essential Quantum Optics. Cambridge University Press. ISBN 0-521-14505-8.
Chapter 42
Perfect matrix
In mathematics, a perfect matrix is an m-by-n binary matrix that has no possible k-by-k submatrix K that satisfiesthe following conditions:*[1]
• k > 3
• the row and column sums of K are each equal to b, where b ≥ 2
• there exists no row of the (m − k)-by-k submatrix formed by the rows not included inK with a row sum greaterthan b.
The following is an example of a K submatrix where k = 5 and b = 2:
1 1 0 0 00 1 1 0 00 0 1 1 00 0 0 1 11 0 0 0 1
.
42.1 References[1] D. M. Ryan, B. A. Foster, An Integer Programming Approach to Scheduling, p.274, University of Auckland, 1981.
147
Chapter 43
Permutation matrix
Matrices describing the permutations of 3 elementsThe product of two permutation matrices is a permutation matrix as well.These are the positions of the six matrices:(They are also permutation matrices.)
148
43.1. DEFINITION 149
In mathematics, in matrix theory, a permutation matrix is a square binary matrix that has exactly one entry of 1 ineach row and each column and 0s elsewhere. Each such matrix represents a specific permutation of m elements and,when used to multiply another matrix, can produce that permutation in the rows or columns of the other matrix.
43.1 Definition
Given a permutation π of m elements,
π : 1, . . . ,m → 1, . . . ,m
given in two-line form by
(1 2 · · · m
π(1) π(2) · · · π(m)
),
its permutation matrix acting on m-dimensional column vectors is the m × m matrix Pπ whose entries are all 0 exceptthat in row i, the entry π(i) equals 1. We may write
Pπ =
eπ(1)eπ(2)
...eπ(m)
,where ej denotes a row vector of length m with 1 in the jth position and 0 in every other position.*[1]
43.2 Properties
Given two permutations π and σ of m elements, the corresponding permutation matrices Pπ and Pσ which act oncolumn vectors can be composed
PσPπ = Pσ π.
However, note that if the corresponding permutation matrices are defined to act on row vectors, i.e., [Pπ]ij = δiπ(j)where δ is the Kronecker delta, the rule in reversed
PσPπ = Pπ σ.
As permutation matrices are orthogonal matrices (i.e., PπPTπ = I ), the inverse matrix exists and can be written as
P−1π = Pπ−1 = PTπ .
Multiplying Pπ times a column vector g will permute the rows of the vector:
Pπg =
eπ(1)eπ(2)
...eπ(n)
g1g2...gn
=
gπ(1)gπ(2)
...gπ(n)
.
150 CHAPTER 43. PERMUTATION MATRIX
Multiplying a row vector h times Pπ will permute the columns of the vector by the inverse of Pπ :
hPπ =[h1 h2 . . . hn
]eπ(1)eπ(2)
...eπ(n)
=[hπ−1(1) hπ−1(2) . . . hπ−1(n)
]
Again it can be checked that (hPσ)Pπ = hPπσ .
43.3 Notes
Let Sn denote the symmetric group, or group of permutations, on 1,2,...,n. Since there are n! permutations, thereare n! permutation matrices. By the formulas above, the n × n permutation matrices form a group under matrixmultiplication with the identity matrix as the identity element.If (1) denotes the identity permutation, then P(1) is the identity matrix.One can view the permutation matrix of a permutation σ as the permutation σ of the rows of the identity matrix I, oras the permutation σ*−1 of the columns of I.A permutation matrix is a doubly stochastic matrix. The Birkhoff–von Neumann theorem says that every doublystochastic real matrix is a convex combination of permutation matrices of the same order and the permutation matricesare precisely the extreme points of the set of doubly stochastic matrices. That is, the Birkhoff polytope, the set ofdoubly stochastic matrices, is the convex hull of the set of permutation matrices.*[2]The product PM, premultiplying a matrix M by a permutation matrix P, permutes the rows of M; row i moves to rowπ(i). Likewise, MP permutes the columns of M.The map Sn → A ⊂ GL(n, Z2) is a faithful representation. Thus, |A| = n!.The trace of a permutation matrix is the number of fixed points of the permutation. If the permutation has fixedpoints, so it can be written in cycle form as π = (a1)(a2)...(ak)σ where σ has no fixed points, then ea1 ,ea2 ,...,eak areeigenvectors of the permutation matrix.From group theory we know that any permutation may be written as a product of transpositions. Therefore, anypermutation matrix P factors as a product of row-interchanging elementary matrices, each having determinant −1.Thus the determinant of a permutation matrix P is just the signature of the corresponding permutation.
43.4 Examples
43.4.1 Permutation of rows and columns
When a permutation matrix P is multiplied with a matrix M from the left PM it will permute the rows of M (here theelements of a column vector),when P is multiplied with M from the right MP it will permute the columns of M (here the elements of a row vector):Permutations of rows and columns are for example reflections (see below) and cyclic permutations (see cyclic per-mutation matrix).
43.4.2 Permutation of rows
The permutation matrix Pπ corresponding to the permutation : π =
(1 2 3 4 51 4 2 5 3
), is
Pπ =
eπ(1)eπ(2)eπ(3)eπ(4)eπ(5)
=
e1e4e2e5e3
=
1 0 0 0 00 0 0 1 00 1 0 0 00 0 0 0 10 0 1 0 0
.
43.5. EXPLANATION 151
Given a vector g,
Pπg =
eπ(1)eπ(2)eπ(3)eπ(4)eπ(5)
g1g2g3g4g5
=
g1g4g2g5g3
.
43.5 Explanation
A permutation matrix will always be in the form
ea1ea2
...eaj
where eai represents the ith basis vector (as a row) for R*j, and where
[1 2 . . . ja1 a2 . . . aj
]is the permutation form of the permutation matrix.Now, in performing matrix multiplication, one essentially forms the dot product of each row of the first matrix witheach column of the second. In this instance, we will be forming the dot product of each row of this matrix with thevector of elements we want to permute. That is, for example, v= (g0,...,g5)*T,
eai·v=gai
So, the product of the permutation matrix with the vector v above, will be a vector in the form (ga1 , ga2 , ..., gaj ), andthat this then is a permutation of v since we have said that the permutation form is
(1 2 . . . ja1 a2 . . . aj
).
So, permutation matrices do indeed permute the order of elements in vectors multiplied with them.
43.6 See also• Alternating sign matrix
• Generalized permutation matrix
43.7 References[1] Brualdi (2006) p.2
[2] Brualdi (2006) p.19
• Brualdi, Richard A. (2006). Combinatorial matrix classes. Encyclopedia of Mathematics and Its Applications108. Cambridge: Cambridge University Press. ISBN 0-521-86565-4. Zbl 1106.05001.
Chapter 44
Persymmetric matrix
In mathematics, persymmetric matrix may refer to:
1. a square matrix which is symmetric in the northeast-to-southwest diagonal; or
2. a square matrix such that the values on each line perpendicular to the main diagonal are the same for a givenline.
The first definition is the most common in the recent literature. The designation "Hankel matrix" is often used formatrices satisfying the property in the second definition.
44.1 Definition 1
Let A = (ai j) be an n × n matrix. The first definition of persymmetric requires that
aij = an−j+1,n−i+1 for all i, j.*[1]
For example, 5-by-5 persymmetric matrices are of the form
A =
a11 a12 a13 a14 a15a21 a22 a23 a24 a14a31 a32 a33 a23 a13a41 a42 a32 a22 a12a51 a41 a31 a21 a11
.
This can be equivalently expressed as AJ = JA*T where J is the exchange matrix.A symmetric matrix is a matrix whose values are symmetric in the northwest-to-southeast diagonal. If a symmetricmatrix is rotated by 90°, it becomes a persymmetric matrix. Symmetric persymmetric matrices are sometimes calledbisymmetric matrices.
44.2 Definition 2
For more details on this topic, see Hankel matrix.
The second definition is due to Thomas Muir.*[2] It says that the square matrixA = (aij) is persymmetric if aij dependsonly on i + j. Persymmetric matrices in this sense, or Hankel matrices as they are often called, are of the form
152
44.3. SEE ALSO 153
Symmetry pattern of a persymmetric 5×5 matrix
A =
r1 r2 r3 · · · rnr2 r3 r4 · · · rn+1
r3 r4 r5 · · · rn+2
......
... . . . ...rn rn+1 rn+2 · · · r2n−1
.
A persymmetric determinant is the determinant of a persymmetric matrix.*[2]A matrix for which the values on each line parallel to the main diagonal are constant, is called a Toeplitz matrix.
44.3 See also
• Centrosymmetric matrix
154 CHAPTER 44. PERSYMMETRIC MATRIX
44.4 References[1] Golub, Gene H.; Van Loan, Charles F. (1996), Matrix Computations (3rd ed.), Baltimore: Johns Hopkins, ISBN 978-0-
8018-5414-9. See page 193.
[2] Muir, Thomas (1960), Treatise on the Theory of Determinants, Dover Press, p. 419
Chapter 45
Polyconvex function
In mathematics, the notion of polyconvexity is a generalization of the notion of convexity for functions defined onspaces of matrices. Let Mm×n(K) denote the space of all m × n matrices over the field K, which may be either the realnumbers R, or the complex numbers C. A function f : Mm×n(K) → R ∪ ±∞ is said to be polyconvex if
A 7→ f(A)
can be written as a convex function of the p × p subdeterminants of A, for 1 ≤ p ≤ minm, n.Polyconvexity is a weaker property than convexity. For example, the function f given by
f(A) =
1
det(A) , det(A) > 0;
+∞, det(A) ≤ 0;
is polyconvex but not convex.
45.1 References• Renardy, Michael and Rogers, Robert C. (2004). An introduction to partial differential equations. Texts in
Applied Mathematics 13 (Second ed.). New York: Springer-Verlag. p. 353. ISBN 0-387-00444-0. (Definition10.25)
155
Chapter 46
Polynomial matrix
Not to be confused with matrix polynomial.
In mathematics, a polynomial matrix or sometimes matrix polynomial is a matrix whose elements are univariateor multivariate polynomials. A λ-matrix is a matrix whose elements are polynomials in λ.A univariate polynomial matrix P of degree p is defined as:
P =
p∑n=0
A(n)xn = A(0) +A(1)x+A(2)x2 + · · ·+A(p)xp
where A(i) denotes a matrix of constant coefficients, and A(p) is non-zero. Thus a polynomial matrix is the matrix-equivalent of a polynomial, with each element of the matrix satisfying the definition of a polynomial of degree p.An example 3×3 polynomial matrix, degree 2:
P =
1 x2 x0 2x 2
3x+ 2 x2 − 1 0
=
1 0 00 0 22 −1 0
+
0 0 10 2 03 0 0
x+
0 1 00 0 00 1 0
x2.We can express this by saying that for a ring R, the rings Mn(R[X]) and (Mn(R))[X] are isomorphic.
46.1 Properties• A polynomial matrix over a field with determinant equal to a non-zero element of that field is called unimodular,
and has an inverse that is also a polynomial matrix. Note that the only scalar unimodular polynomials arepolynomials of degree 0 – nonzero constants, because an inverse of an arbitrary polynomial of higher degreeis a rational function.
• The roots of a polynomial matrix over the complex numbers are the points in the complex plane where thematrix loses rank.
Note that polynomial matrices are not to be confused with monomial matrices, which are simply matrices with exactlyone non-zero entry in each row and column.If by λ we denote any element of the field over which we constructed the matrix, by I the identity matrix, and we letA be a polynomial matrix, then the matrix λI − A is the characteristic matrix of the matrix A. Its determinant, |λI− A| is the characteristic polynomial of the matrix A.
46.2 References• E.V.Krishnamurthy, Error-free Polynomial Matrix computations, Springer Verlag, New York, 1985
156
Chapter 47
Positive-definite matrix
Not to be confused with Positive matrix and Totally positive matrix.
In linear algebra, a symmetric n × n real matrix M is said to be positive definite if zTMz is positive for everynon-zero column vector z of n real numbers. Here zT denotes the transpose of z .More generally, an n × n Hermitian matrix M is said to be positive definite if z∗Mz is real and positive for allnon-zero column vectors z of n complex numbers. Here z∗ denotes the conjugate transpose of z .The negative definite, positive semi-definite, and negative semi-definite matrices are defined in the same way,except that the expression zTMz or z∗Mz is required to be always negative, non-negative, and non-positive, respec-tively.Positive definite matrices are closely related to positive-definite symmetric bilinear forms (or sesquilinear forms inthe complex case), and to inner products of vector spaces.*[1]Some authors use more general definitions of“positive definite”that include some non-symmetric real matrices, ornon-Hermitian complex ones.
47.1 Examples
• The identity matrix I =
[1 00 1
]is positive definite. Seen as a real matrix, it is symmetric, and, for any non-zero
column vector z with real entries a and b, one has zTIz =[a b
][1 00 1
][ab
]= a2 + b2 . Seen as a complex
matrix, for any non-zero column vector zwith complex entries a and b one has z∗Iz =[a∗ b∗
][1 00 1
][ab
]=
a∗a+ b∗b = |a|2 + |b|2 . Either way, the result is positive since z is not the zero vector (that is, at least one ofa and b is not zero).
• The real symmetric matrix
M =
2 −1 0−1 2 −10 −1 2
is positive definite since for any non-zero column vector z with entries a, b and c, we have
zTMz = (zTM)z =[(2a− b) (−a+ 2b− c) (−b+ 2c)
]abc
= 2a2 − 2ab+ 2b2 − 2bc+ 2c2
= a2 + (a− b)2 + (b− c)2 + c2
157
158 CHAPTER 47. POSITIVE-DEFINITE MATRIX
This result is a sum of squares, and therefore non-negative; and is zero only if a = b = c = 0, that is, whenz is zero.
• The real symmetric matrix
N =
[1 22 1
]
is not positive definite. If z is the vector[−11
], one has zTNz =
[−1 1
][1 22 1
][−11
]=[−1 1
][ 1−1
]=
−2 > 0.
• For any real non-singular matrix A , the product ATA is a positive definite matrix. A simple proof is that forany non-zero vector z , the condition zTATAz = ∥Az∥22 > 0, since the non-singularity of matrix A meansthat Az = 0.
The examples M and N above show that a matrix in which some elements are negative may still be positive-definite,and conversely a matrix whose entries are all positive may not be positive definite.
47.2 Connections
The general purely quadratic real function f(z) on n real variables z1, ..., zn can always be written as z*TMz wherez is the column vector with those variables, and M is a symmetric real matrix. Therefore, the matrix being positivedefinite means that f has a unique minimum (zero) when z is zero, and is strictly positive for any other z.More generally, a twice-differentiable real function f on n real variables has an isolated local minimum at argumentsz1, ..., zn if its gradient is zero and its Hessian (the matrix of all second derivatives) is positive definite at that point.Similar statements can be made for negative definite and semi-definite matrices.In statistics, the covariance matrix of a multivariate probability distribution is always positive semi-definite; and it ispositive definite unless one variable is an exact linear combination of the others. Conversely, every positive semi-definite matrix is the covariance matrix of some multivariate distribution.
47.3 Characterizations
Let M be an n × n Hermitian matrix. The following properties are equivalent to M being positive definite:
1. All its eigenvalues are positive. Let P*−1DP be an eigendecomposition of M, where P is a unitary complexmatrix whose rows comprise an orthonormal basis of eigenvectors of M, and D is a real diagonal matrix whosemain diagonal contains the corresponding eigenvalues. The matrix M may be regarded as a diagonal matrix Dthat has been re-expressed in coordinates of the basis P. In particular, the one-to-one change of variable y =Pz shows that z*Mz is real and positive for any complex vector z if and only if y*Dy is real and positive for anyy; in other words, if D is positive definite. For a diagonal matrix, this is true only if each element of the maindiagonal—that is, every eigenvalue of M—is positive. Since the spectral theorem guarantees all eigenvalues ofa Hermitian matrix to be real, the positivity of eigenvalues can be checked using Descartes' rule of alternatingsigns when the characteristic polynomial of a real, symmetric matrix M is available.
2. The associated sesquilinear form is an inner product. The sesquilinear form defined by M is the function⟨·, ·⟩ fromC*n ×C*n toC such that ⟨x, y⟩ := y∗Mx for all x and y inC*n, where y** is the complex conjugateof y. For any complex matrixM, this form is linear in each argument separately. Therefore the form is an innerproduct on C*n if and only if ⟨z, z⟩ is real and positive for all nonzero z; that is if and only if M is positivedefinite. (In fact, every inner product on C*n arises in this fashion from a Hermitian positive definite matrix.)
3. It is the Gram matrix of linearly independent vectors. Let x1, . . . , xn be a list of n linearly independentvectors of some complex vector space with an inner product ⟨·, ·⟩ . It can be verified that the Gram matrix
47.4. QUADRATIC FORMS 159
M of those vectors, defined by Mij = ⟨xi, xj⟩ , is always positive definite. Conversely, if M is positivedefinite, it has an eigendecomposition P*−1DP where P is unitary, D diagonal, and all diagonal elements Dii= λi of D are real and positive. Let E be the real diagonal matrix with entries Eii =
√λi so E2 = D ; then
P−1DP = P ∗DP = P ∗EEP = (EP )∗EP. Now we let x1, . . . , xn be the columns of EP. These vectorsare linearly independent, and by the above M is their Gram matrix, under the standard inner product of C*n,namely ⟨xi, xj⟩ = xTi xj
4. Its leading principalminors are all positive. The kth leading principal minor of a matrixM is the determinantof its upper-left k by k sub-matrix. It turns out that a matrix is positive definite if and only if all these determi-nants are positive. This condition is known as Sylvester's criterion, and provides an efficient test of positive-definiteness of a symmetric real matrix. Namely, the matrix is reduced to an upper triangular matrix by usingelementary row operations, as in the first part of the Gaussian elimination method, taking care to preserve thesign of its determinant during pivoting process. Since the kth leading principal minor of a triangular matrixis the product of its diagonal elements up to row k, Sylvester's criterion is equivalent to checking whether itsdiagonal elements are all positive. This condition can be checked each time a new row k of the triangularmatrix is obtained.
5. It has a unique Cholesky decomposition. The matrix M is positive definite if and only if there exists aunique lower triangular matrix L, with real and strictly positive diagonal elements, such that M = LL*. Thisfactorization is called the Cholesky decomposition of M.
47.4 Quadratic forms
The (purely) quadratic form associated with a real matrix M is the function Q : R*n → R such that Q(x) = x*TMx forall x. It turns out that the matrix M is positive definite if and only if it is symmetric and its quadratic form is a strictlyconvex function.More generally, any quadratic function from R*n to R can be written as x*TMx + x*Tb + c where M is a symmetricn × n matrix, b is a real n-vector, and c a real constant. This quadratic function is strictly convex when M is positivedefinite, and hence has a unique finite global minimum, if and only if M is positive definite. For this reason, positivedefinite matrices play an important role in optimization problems.
47.5 Simultaneous diagonalization
A symmetric matrix and another symmetric and positive-definite matrix can be simultaneously diagonalized, althoughnot necessarily via a similarity transformation. This result does not extend to the case of three or more matrices. Inthis section we write for the real case. Extension to the complex case is immediate.Let M be a symmetric and N a symmetric and positive-definite matrix. Write the generalized eigenvalue equationas (M−λN)x = 0 where we impose that x be normalized, i.e. x*TNx = 1. Now we use Cholesky decomposition towrite the inverse of N as Q*TQ. Multiplying by Q and Q*T, we get Q(M−λN)Q*Tx = 0, which can be rewritten as(QMQ*T)y = λy where y*Ty = 1. Manipulation now yields MX = NXΛ where X is a matrix having as columns thegeneralized eigenvectors and Λ is a diagonal matrix with the generalized eigenvalues. Now premultiplication withX*T gives the final result: X*TMX = Λ and X*TNX = I, but note that this is no longer an orthogonal diagonalization.Note that this result does not contradict what is said on simultaneous diagonalization in the article Diagonalizablematrix, which refers to simultaneous diagonalization by a similarity transformation. Our result here is more akin to asimultaneous diagonalization of two quadratic forms, and is useful for optimization of one form under conditions onthe other. For this result see Horn&Johnson, 1985, page 218 and following.
47.6 Negative-definite, semidefinite and indefinite matrices
A Hermitian matrix is negative-definite, negative-semidefinite, or positive-semidefinite if and only if all of its eigenvaluesare negative, non-positive, or non-negative, respectively.
160 CHAPTER 47. POSITIVE-DEFINITE MATRIX
47.6.1 Negative-definite
The n × n Hermitian matrix M is said to be negative-definite if
x∗Mx < 0
for all non-zero x in C*n (or, all non-zero x in R*n for the real matrix), where x* is the conjugate transpose of x.A matrix is negative definite if its k-th order leading principal minor is negative when k is odd, and positive when kis even.
47.6.2 Positive-semidefinite
M is called positive-semidefinite (or sometimes nonnegative-definite) if
x∗Mx ≥ 0
for all x in C*n (or, all x in R*n for the real matrix).A matrix M is positive-semidefinite if and only if it arises as the Gram matrix of some set of vectors. In contrast tothe positive-definite case, these vectors need not be linearly independent.For any matrix A, the matrix A*A is positive semidefinite, and rank(A) = rank(A*A). Conversely, any Hermitianpositive semi-definite matrix M can be written as M = LL*, where L is lower triangular; this is the Cholesky decom-position. If M is not positive definite, then some of the diagonal elements of L may be zero.A Hermitian matrix is positive semidefinite if and only if all of its principal minors are nonnegative. It is howevernot enough to consider the leading principal minors only, as is checked on the diagonal matrix with entries 0 and−1.
47.6.3 Negative-semidefinite
It is called negative-semidefinite if
x∗Mx ≤ 0
for all x in C*n (or, all x in R*n for the real matrix).
47.6.4 Indefinite
A Hermitian matrix which is neither positive definite, negative definite, positive-semidefinite, nor negative-semidefiniteis called indefinite. Indefinite matrices are also characterized by having both positive and negative eigenvalues.
47.7 Further properties
If M is a Hermitian positive-semidefinite matrix, one sometimes writes M ≥ 0 and if M is positive-definite one writesM > 0.*[2] The notion comes from functional analysis where positive-semidefinite matrices define positive operators.For arbitrary square matrices M, N we write M ≥ N if M − N ≥ 0; i.e., M − N is positive semi-definite. This definesa partial ordering on the set of all square matrices. One can similarly define a strict partial ordering M > N.
1. Every positive definite matrix is invertible and its inverse is also positive definite.*[3] If M ≥ N > 0 then N*−1≥ M*−1 > 0, and √M > √N > 0.*[4] Moreover, by the min-max theorem, the kth largest eigenvalue of M isgreater than the kth largest eigenvalue of N
47.7. FURTHER PROPERTIES 161
2. If M is positive definite and r > 0 is a real number, then rM is positive definite.*[5] If M and N are positivedefinite, then the sum M + N*[5] and the products MNM and NMN are also positive definite. If MN = NM,then MN is also positive definite.
3. Every principal submatrix of a positive definite matrix is positive definite.
4. Q*T M Q is positive-semidefinite. If Q is invertible, then Q*T M Q is positive definite. Note that Q*−1 M Qneed not be positive definite.
5. The determinant of M is bounded by the product of its diagonal elements.
6. The diagonal entries mii are real and non-negative. As a consequence the trace, tr(M) ≥ 0. Furthermore,*[6]since every principal sub matrix (in particular, 2-by-2) is positive definite,
|mij | ≤√miimjj ≤
mii +mjj
2
and thus
max |mij | ≤ max |mii|
7. A matrix M is positive semi-definite if and only if there is a positive semi-definite matrix B with B2 = M. Thismatrix B is unique,*[7] is called the square root of M, and is denoted with B = M*1/2 (the square root B is notto be confused with the matrix L in the Cholesky factorization M = LL*, which is also sometimes called thesquare root of M). If M > N > 0 then M*1/2 > N*1/2 > 0.
8. If M is a symmetric matrix of the form mij = m(i−j), and the strict inequality holds
∑j =0|m(j)| < m(0)
then M is strictly positive definite.
9. Let M > 0 and N Hermitian. If MN + NM ≥ 0 (resp., MN + NM > 0) then N ≥ 0 (resp., N > 0).
10. If M > 0 is real, then there is a δ > 0 such that M > δI, where I is the identity matrix.
11. If Mk denotes the leading k by k minor, det(Mk)/ det(Mk−1) is the kth pivot during LU decomposition.
12. The set of positive semidefinite symmetric matrices is convex. That is, if M and N are positive semidefinite,then for any α between 0 and 1, αM + (1−α)N is also positive semidefinite. For any vector x:
x⊤(αM + (1− α)N)x = αx⊤Mx+ (1− α)x⊤Nx ≥ 0.
This property guarantees that semidefinite programming problems converge to a globally optimalsolution.
13. If M,N ≥ 0, although MN is not necessary positive-semidefinite, the Kronecker product M ⊗ N ≥ 0, theHadamard product M N ≥ 0 (this result is often called the Schur product theorem).,*[8] and the Frobeniusproduct M : N ≥ 0 (Lancaster-Tismenetsky, The Theory of Matrices, p. 218).
14. Regarding the Hadamard product of two positive-semidefinite matrices M = (mij) ≥ 0, N ≥ 0, there are twonotable inequalities:
• Oppenheim's inequality: det(M N) ≥ det(N)∏imii.
*[9]• det(M N) ≥ det(M) det(N).*[10]
162 CHAPTER 47. POSITIVE-DEFINITE MATRIX
47.8 Block matrices
A positive 2n × 2n matrix may also be defined by blocks:
M =
[A BC D
]where each block is n × n. By applying the positivity condition, it immediately follows that A and D are hermitian,and C = B*.We have that z*Mz ≥ 0 for all complex z, and in particular for z = ( v, 0)*T. Then
[v∗ 0
][ A BB∗ D
][v0
]= v∗Av ≥ 0.
A similar argument can be applied to D, and thus we conclude that both A and D must be positive definite matrices,as well.Converse results can be proved with stronger conditions on the blocks, for instance using the Schur complement.
47.9 On the definition
47.9.1 Consistency between real and complex definitions
Since every real matrix is also a complex matrix, the definitions of“positive definite”for the two classes must agree.For complex matrices, the most common definition says that "M is positive definite if and only if z*Mz is real andpositive for all non-zero complex column vectors z". This condition implies that M is Hermitian, that is, its transposeis equal to its conjugate. To see this, consider the matrices A = (M+M*)/2 and B = (M−M*)/(2i), so that M = A+iBand z*Mz = z*Az + iz*Bz. The matrices A and B are Hermitian, therefore z*Az and z*Bz are individually real. Ifz*Mz is real, then z*Bz must be zero for all z. Then B is the zero matrix and M = A, proving that M is Hermitian.By this definition, a positive definite real matrix M is Hermitian, hence symmetric; and z*TMz is positive for allnon-zero real column vectors z. However the last condition alone is not sufficient for M to be positive definite. Forexample, if
M =
[1 1−1 1
],
then for any real vector z with entries a and b we have z*TMz = (a−b)a + (a+b)b = a2 + b2, which is always positiveif z is not zero. However, if z is the complex vector with entries 1 and i, one gets
z*Mz = [1, −i]M[1, i]*T = [1+i, 1−i][1, i]*T = 2 + 2i,
which is not real. Therefore, M is not positive definite.On the other hand, for a symmetric real matrix M, the condition "z*TMz > 0 for all nonzero real vectors z" does implythat M is positive definite in the complex sense.
47.9.2 Extension for non symmetric matrices
Some authors choose to say that a complex matrix M is positive definite if Re(z*Mz) > 0 for all non-zero complexvectors z, where Re(c) denotes the real part of a complex number c.*[11] This weaker definition encompasses somenon-Hermitian complex matrices, including some non-symmetric real ones, such as
[1 1−1 1
].
Indeed, with this definition, a real matrix is positive definite if and only if z*TMz > 0 for all nonzero real vectors z,even if M is not symmetric.
47.10. SEE ALSO 163
In general, we have Re(z*Mz) > 0 for all complex nonzero vectors z if and only if the Hermitian part (M + M*)/2 ofM is positive definite in the narrower sense. Similarly, we have x*TMx > 0 for all real nonzero vectors x if and onlyif the symmetric part (M + M*T)/2 of M is positive definite in the narrower sense.In summary, the distinguishing feature between the real and complex case is that, a bounded positive operator on acomplex Hilbert space is necessarily Hermitian, or self adjoint. The general claim can be argued using the polarizationidentity. That is no longer true in the real case.
47.10 See also
• Cholesky decomposition
• Covariance matrix
• M-matrix
• Positive-definite function
• Positive-definite kernel
• Schur complement
• Square root of a matrix
• Sylvester's criterion
47.11 Notes
[1] Stewart, J. (1976). Positive definite functions and generalizations, an historical survey. Rocky Mountain J. Math, 6(3).
[2] This may be confusing, as sometimes nonnegative matrices are also denoted in this way. A common alternative notation isM ⪰ 0 and M ≻ 0 for positive semidefinite and positive definite matrices, respectively.
[3] Horn & Johnson (1985), p. 397
[4] Horn & Johnson (1985), Corollary 7.7.4(a)
[5] Horn & Johnson (1985), Observation 7.1.3
[6] Horn & Johnson (1985), p. 398
[7] Horn & Johnson (1985), Theorem 7.2.6 with k = 2
[8] Horn & Johnson (1985), Theorem 7.5.3
[9] Horn & Johnson (1985), Theorem 7.8.6
[10] (Styan 1973)
[11] Weisstein, Eric W. Positive Definite Matrix. From MathWorld--A Wolfram Web Resource. Accessed on 2012-07-26
47.12 References
• Horn, Roger A.; Johnson, Charles R. (1990). Matrix Analysis. Cambridge University Press. ISBN 978-0-521-38632-6..
• Bhatia, Rajendra (2007). Positive definite matrices. Princeton Series in Applied Mathematics. ISBN 978-0-691-12918-1.
164 CHAPTER 47. POSITIVE-DEFINITE MATRIX
47.13 External links• Hazewinkel, Michiel, ed. (2001), “Positive-definite form”, Encyclopedia of Mathematics, Springer, ISBN
978-1-55608-010-4
• Wolfram MathWorld: Positive Definite Matrix
Chapter 48
Productive matrix
In linear algebra, a square nonnegative matrix A of order n is said to be productive, or to be a Leontief matrix, ifthere exists a n× 1 nonnegative column matrix P such as P −AP is a positive matrix.
48.1 History
The concept of productive matrix was developed by the economist Wassily Leontief (Nobel Prize in Economics in1973) in order to model and analyze the relations between the different sectors of an economy.*[1] The interdepen-dency linkages between the latter can be examined by the input-output model with empirical data.
48.2 Explicit definition
The matrix A ∈ Mn(R) is productive if and only if A ⩾ 0 and ∃P ∈ Mn,1(R), P > 0 such as P −AP > 0 .
48.3 Examples
The matrix A =
0 1 00 1/2 1/21/4 1/2 0
is productive.
∀a ∈ R+ , the matrix A =
(0 a0 0
)is productive since P =
(a+ 11
)verifies the inequalities of definition.
48.4 Properties*[2]
48.4.1 Characterization
Theorem A nonnegative matrix A ∈ Mn(R) is productive if and only if In − A is invertible with a nonnegativeinverse.DemonstrationDirect involvement :
U ∈ Mn,1(R), P > 0
P = (In −A)−1U
P −AP = (In −A)P = (In −A)(In −A)−1U = U
165
166 CHAPTER 48. PRODUCTIVE MATRIX
P −AP > 0
A
Reciprocal involvement :
We shall proceed by reductio ad absurdum.Let us assume ∃P > 0 such as V = P −AP > 0 & In −A is singular.The endomorphism canonically associated with In−A can not be injective by singularity of the matrix.Thus ∃Z ∈ Mn,1(R) non zero such as (In −A)Z = 0 .The matrix−Z verifies the same properties as Z , therefore we can choose Z as an element of the kernelwith at least one positive entry;Hence c = supi∈[|1,n|]
zipi
is nonnegative and reached with at least one value k ∈ [|1, n|] .By definition of V and of Z , we can infer that:
cvk = c(pk −n∑i=1
akipi) = cpk −n∑i=1
akicpi
cpk = zk =n∑i=1
akizi
Thus cvk =∑ni=1 aki(zj − cpj) ≤ 0 .
Yet we know that c > 0 and that vk > 0 .Therefore there is a contradiction, ipso facto In −A is necessarily invertible.Now let us assume In −A is invertible but with at least one negative entry in its inverse.Hence ∃X ∈ Mn,1(R), X ⩾ 0 such as there is at least one negative entry in Y = (In −A)−1X .Then c = supi∈[|1,n|]−
yipi
is positive and reached with at least one value k ∈ [|1, n|] .By definition of V and of X , we can infer that:
cvk = c(pk −n∑i=1
akipi) = −yk −n∑i=1
akicpi
xk = yk −n∑i=1
akiyi
cvk + xk = −n∑i=1
aki(cpi + yi)
Thus xk ≤ −cvk < 0 since ∀i ∈ [|1, n|], aki ⩾ 0, cpi + yi ⩾ 0 .Yet we know that X ⩾ 0 .Therefore ther is a contradiction, ipso facto (In −A)−1 is necessarily nonnegative.
48.4.2 Transposition
Proposition The transpose of a productive matrix is productive.Demonstration
A ∈ Mn(R)
(In −A)−1
((In −tA))−1 = (t(In −A))−1 =t ((In −A)−1)
(In −tA)tA
48.5. APPLICATION 167
48.5 Application
Main article: Input-output analysis
With a matrix approach of the input-output model, the consumption matrix is productive if it is economically viableand if the latter and the demand vector are nonnegative.
48.6 References[1] Kim Minju, Leontief Input-Output Model (Application of Linear Algebra to Economics)
[2] (fr)Philippe Michel, “9.2 Matrices productives”, Cours de Mathématiques pour Economistes, Édition Economica, 1984
Chapter 49
Pseudo-determinant
In linear algebra and statistics, the pseudo-determinant*[1] is the product of all non-zero eigenvalues of a squarematrix. It coincides with the regular determinant when the matrix is non-singular.
49.1 Definition
The pseudo-determinant of a square n-by-n matrix A may be defined as:
|A|+ = limα→0
|A+ αI|αn−rank(A)
where |A| denotes the usual determinant, I denotes the identity matrix and rank(A) denotes the rank of A.
49.2 Definition of pseudo determinant using Vahlen Matrix
The Vahlen matrix of a conformal transformation, the Möbius transformation (i.e. (ax+b)(cx+d)−1 for a, b, c, d ∈G(p, q) )) is defined as [f ] =::
[a bc d
]. By the pseudo determinant of the Vahlen matrix for the conformal trans-
formation, we mean
pdet ::
[a bc d
]= ad† − bc†
If pdet[f ] > 0 , the transformation is sense-preserving (rotation) whereas if the pdet[f ] < 0 , the transformation issense-preserving (reflection).
49.3 Computation for positive semi-definite case
IfA is positive semi-definite, then the singular values and eigenvalues ofA coincide. In this case, if the singular valuedecomposition (SVD) is available, then |A|+ may be computed as the product of the non-zero singular values. If allsingular values are zero, then the pseudo-determinant is 1.
49.4 Application in statistics
If a statistical procedure ordinarily compares distributions in terms of the determinants of variance-covariance ma-trices then, in the case of singular matrices, this comparison can be undertaken by using a combination of the ranksof the matrices and their pseudo-determinants, with the matrix of higher rank being counted as “largest”and thepseudo-determinants only being used if the ranks are equal.*[2] Thus pseudo-determinants are sometime presentedin the outputs of statistical programs in cases where covariance matrices are singular.*[3]
168
49.5. SEE ALSO 169
49.5 See also• Matrix determinant
• Moore–Penrose pseudoinverse, which can also be obtained in terms of the non-zero singular values.
49.6 References[1] Minka, T.P. (2001). “Inferring a Gaussian Distribution”. PDF
[2] SAS documentation on “Robust Distance”
[3] Bohling, Geoffrey C. (1997)“GSLIB-style programs for discriminant analysis and regionalized classification”, Computers& Geosciences, 23 (7), 739–761 doi:10.1016/S0098-3004(97)00050-2
Chapter 50
Q-matrix
This article is about the notion used in the context of linear complementarity problems. For the one used in thecontext of Markov-chain, see Continuous-time Markov chain.
In mathematics, a Q-matrix is a square matrix whose associated linear complementarity problem LCP(M,q) has asolution for every vector q.
50.1 See also• P-matrix
50.2 References• Murty, Katta G. (January 1972). “On the number of solutions to the complementarity problem and spanning
properties of complementary cones”. Linear Algebra and its Applications 5 (1): 65–108. doi:10.1016/0024-3795(72)90019-5.
• Aganagic, Muhamed; Cottle, Richard W. (December 1979). “A note on Q-matrices”. Mathematical Pro-gramming 16 (1): 374–377. doi:10.1007/BF01582122.
• Pang, Jong-Shi (December 1979).“On Q-matrices”. Mathematical Programming 17 (1): 243–247. doi:10.1007/BF01588247.
• Danao, R. A. (November 1994). “Q-matrices and boundedness of solutions to linear complementarity prob-lems”. Journal of Optimization Theory and Applications 83 (2): 321–332. doi:10.1007/bf02190060.
170
Chapter 51
Quaternionic matrix
A quaternionic matrix is a matrix whose elements are quaternions.
51.1 Matrix operations
The quaternions form a noncommutative ring, and therefore addition and multiplication can be defined for quater-nionic matrices as for matrices over any ring.Addition. The sum of two quaternionic matrices A and B is defined in the usual way by element-wise addition:
(A+B)ij = Aij +Bij .
Multiplication. The product of two quaternionic matrices A and B also follows the usual definition for matrixmultiplication. For it to be defined, the number of columns of A must equal the number of rows of B. Then the entryin the ith row and jth column of the product is the dot product of the ith row of the first matrix with the jth columnof the second matrix. Specifically:
(AB)ij =∑s
AisBsj .
For example, for
U =
(u11 u12u21 u22
), V =
(v11 v12v21 v22
),
the product is
UV =
(u11v11 + u12v21 u11v12 + u12v22u21v11 + u22v21 u21v12 + u22v22
).
Since quaternionic multiplication is noncommutative, care must be taken to preserve the order of the factors whencomputing the product of matrices.The identity for this multiplication is, as expected, the diagonal matrix I = diag(1, 1, ... , 1). Multiplication followsthe usual laws of associativity and distributivity. The trace of a matrix is defined as the sum of the diagonal elements,but in general
trace(AB) = trace(BA).
171
172 CHAPTER 51. QUATERNIONIC MATRIX
Left scalar multiplication is defined by
(cA)ij = cAij , (Ac)ij = Aijc.
Again, since multiplication is not commutative some care must be taken in the order of the factors.*[1]
51.2 Determinants
There is no natural way to define a determinant for (square) quaternionic matrices so that the values of the determinantare quaternions.*[2] Complex valued determinants can be defined however.*[3] The quaternion a + bi + cj + dk canbe represented as the 2×2 complex matrix
[a+ bi c+ di−c+ di a− bi
].
This defines a map Ψmn from the m by n quaternionic matrices to the 2m by 2n complex matrices by replacing eachentry in the quaternionic matrix by its 2 by 2 complex representation. The complex valued determinant of a squarequaternionic matrix A is then defined as det(Ψ(A)). Many of the usual laws for determinants hold; in particular, an nby n matrix is invertible if and only if its determinant is nonzero.
51.3 Applications
Quaternionic matrices are used in quantum mechanics*[4] and in the treatment of multibody problems.*[5]
51.4 References[1] Tapp, Kristopher (2005). Matrix groups for undergraduates. AMS Bookstore. pp. 11 ff. ISBN 0-8218-3785-0.
[2] Helmer Aslaksen (1996).“Quaternionic determinants”. TheMathematical Intelligencer 18 (3): 57–65. doi:10.1007/BF03024312.
[3] E. Study (1920).“Zur Theorie der linearen Gleichungen”. ActaMathematica (in German) 42 (1): 1–61. doi:10.1007/BF02404401.
[4] N. Rösch (1983). “Time-reversal symmetry, Kramers' degeneracy and the algebraic eigenvalue problem”. ChemicalPhysics 80 (1–2): 1–5. doi:10.1016/0301-0104(83)85163-5.
[5] Klaus Gürlebeck; Wolfgang Sprössig (1997). “Quaternionic matrices”. Quaternionic and Clifford calculus for physicistsand engineers. Wiley. pp. 32–34. ISBN 978-0-471-96200-7.
Chapter 52
Quincunx matrix
In mathematics, the matrix
(1 −11 1
)is sometimes called the quincunx matrix. It is a 2×2 Hadamard matrix, and its rows form the basis of a diagonalsquare lattice consisting of the integer points whose coordinates both have the same parity; this lattice is a two-dimensional analogue of the three-dimensional body-centered cubic lattice.*[1]
52.1 See also• Quincunx
52.2 Notes[1] Van De Ville, D.; Blu, T.; Unser, M. (2005), “On the multidimensional extension of the quincunx subsampling matrix”
(PDF), IEEE Signal Processing Letters 12 (2): 112–115, doi:10.1109/LSP.2004.839697.
173
174 CHAPTER 52. QUINCUNX MATRIX
52.3 Text and image sources, contributors, and licenses
52.3.1 Text• Laplacian matrix Source: https://en.wikipedia.org/wiki/Laplacian_matrix?oldid=680676899 Contributors: SimonP, Tomo, Michael
Hardy, Meekohi, Doradus, Zero0000, MathMartin, David Gerard, Giftlite, BenFrantzDale, Dratman, Andreas Kaufmann, Jérôme, Linas,Jfr26, Snichols15, Arbor, Mathbot, Maxal, YurikBot, Michael Slone, SmackBot, Taxipom, Juffi, Georgevulov, Hiiiiiiiiiiiiiiiiiiiii, Gensdei,Amit Moscovich, Thijs!bot, David Eppstein, Robin S, Quantling, Epistemenical, TXiKiBoT, Mild Bill Hiccup, Yifeng zhou, Mleconte,IMneme, Addbot, Gtgith, Luckas-bot, Yobot, AnomieBOT, Megatang, Aminrahimian, Daniel.noland, Pnzrusher, Delio.mugnolo, 2an-drewknyazev, Dhirsbrunner, Bazuz, Helpful Pixie Bot, Svebert, Dlituiev, BattyBot, Daniel.Soudry, Altroware, Boxseat, More than onehalf, Coal scuttle, Ctralie, Ako90, Mcallara, Mattatk and Anonymous: 47
• Lesliematrix Source: https://en.wikipedia.org/wiki/Leslie_matrix?oldid=630666968Contributors: Michael Hardy, Giftlite, PhDP, Jmep-pley, Mennonot, Pak21, Alansohn, Boothy443, JYOuyang, Gbadgaiy, Closedmouth, Andy M. Wang, Shrumster, Qmwne235, Tawker-bot2, Thijs!bot, Mack2, Panoply77, Mausy5043, Troutinthemilk, Flyer22, Paolo.dL, ClueBot, Mild Bill Hiccup, Aandrewmmacdonald,WikHead, Addbot, Firl21, Yobot, Amirobot, DemocraticLuntz, Omnipaedista, Jpanwill, HRoestBot, Rushbugled13, Napi1phi, ClueBotNG, Jumanbar, Brian Tomasik, ModalPeak, MasterTriangle12, Manuel-osvaldo-caceres and Anonymous: 28
• Levinson recursion Source: https://en.wikipedia.org/wiki/Levinson_recursion?oldid=649769803 Contributors: Brion VIBBER, Tback-str, Amillar, Edward, Michael Hardy, Ronz, Charles Matthews, Giftlite, Finn-Zoltan, GeoGreg, Bender235, Keenan Pepper, Jheald,Drostie, Rjwilmsi, JungleKid, Whaa?, Lunch, Ortylp, Ignacioerrico, Tekhnofiend, MaxSem, John, Alex Selby, CapitalR, Icep, Drizzd~enwiki,GuidoGer, TheSeven, Paulginz, Lightbot, Yobot, Omnipaedista, EmausBot, ServiceAT, Poppopsun, Mark viking, Monkbot and Anony-mous: 18
• List of matrices Source: https://en.wikipedia.org/wiki/List_of_matrices?oldid=637322527 Contributors: AxelBoldt, Bryan Derksen,Tarquin, Tbackstr, Edward, Michael Hardy, Wshun, Tregoweth, Ahoerstemeier, Cyp, Poor Yorick, Charles Matthews, Timwi, Dys-prosia, Jitse Niesen, Johannes Hüsing, Aleph4, Altenmann, Kuszi, MathMartin, Bkell, Giftlite, Fropuff, Simon Lacoste-Julien, Jeff-BobFrank, C17GMaster, Mh, Rich Farmbrough, ZeroOne, Kipton, CanisRufus, Rgdboer, Jérôme, ABCD, RJFJR, Oleg Alexandrov,Julien Tuerlinckx, Vatter, Mathbot, Tardis, Wavelength, KSmrq, Welsh, Sangwine, Nahaj, Paul D. Anderson, Lunch, SmackBot, Tim-Bentley, Hongooi, Jon Awbrey, Dmh~enwiki, Syrcatbot, 16@r, TNeloms, Myasuda, WISo, Konradek, LachlanA, Harish victory, Vanish2,Jakob.scholbach, Sullivan.t.j, David Eppstein, ANONYMOUS COWARD0xC0DE, Adavidb, Nigholith, TomyDuby, Peskydan, Hasel-don, Kyap, Cuzkatzimhut, Ocolon, Wolfrock, Arcfrk, Petergans, Neparis, AlphaPyro, Kero 925, Mr. Granger, Cliff, Wikidsp, Katanada,Qwfp, Addbot, Jncraton, Breggen, Legobot, Luckas-bot, Titi2~enwiki, Omnipaedista, EmausBot, OZH, Ebrambot, Mxctor, Anita5192,Frietjes, BG19bot, Saranavan2013 and Anonymous: 37
• Logical matrix Source: https://en.wikipedia.org/wiki/Logical_matrix?oldid=660454704 Contributors: AugPi, Carlossuarez46, Paul Au-gust, El C, Oleg Alexandrov, Jeffrey O. Gustafson, RxS, Rjwilmsi, DoubleBlue, Nihiltres, TeaDrinker, BOT-Superzerocool, Wknight94,Closedmouth, SmackBot, InverseHypercube, C.Fred, Aksi great, Octahedron80, MaxSem, Jon Awbrey, Lambiam, JzG, Slakr, Mets501,Happy-melon, CBM, Михајло Анђелковић, Jheiv, Hut 8.5, Brusegadi, Catgut, David Eppstein, Brigit Zilwaukee, Yolanda Zilwaukee,Policron, Cerberus0, TXiKiBoT, Seb26, ClueBot, Cliff, Blanchardb, RABBU, REBBU, DEBBU, DABBUØ, BABBUØ, RABBUØ,Wolf of the Steppes, REBBUØ, Doubtentry, DEBBUØ, Education Is The Basis Of Law And Order, -Midorihana-, Bare In Mind, Pre-veiling Opinion Of Dominant Opinion Group, Buchanan's Navy Sec, Overstay, Marsboat, Unco Guid, Poke Salat Annie, Flower MoundBelle, Mrs. Lovett's Meat Puppets, Addbot, Breggen, Floquenbeam, Erik9bot, FrescoBot, Kimmy007, EmausBot, Tijfo098, Masssly,Deyvid Setti, Helpful Pixie Bot, Jochen Burghardt, Zeiimer and Anonymous: 12
• M-matrix Source: https://en.wikipedia.org/wiki/M-matrix?oldid=668153435Contributors: Giftlite, Bkonrad, Rpchase, Bender235, Rjwilmsi,Michael Slone, Gareth Jones, SmackBot, Jfgrcar, CBM, Wikid77, Geekdog, Hardmath, Magioladitis, Leyo, Peskydan, Mangledorf,Niceguyedc, MystBot, Addbot, Luckas-bot, CharlesMGraham, DrilBot, ZéroBot, Helpful Pixie Bot, Ess292, Manoguru, Qetuth, DarkSilver Crow, Siberian Patriot and Anonymous: 7
• Magic square Source: https://en.wikipedia.org/wiki/Magic_square?oldid=679669984Contributors: AxelBoldt, GayCommunist, Dan~enwiki,Mav, Bryan Derksen, Zundark, Tarquin, XJaM, Toby Bartels, PierreAbbat, Chas zzz brown, Michael Hardy, Wshun, Dominus, SGBai-ley, Alfio, Jpatokal, Snoyes, TraxPlayer, Jacquerie27, Lee M, Schneelocke, Ideyal, Charles Matthews, RickK, Dysprosia, KRS, Pop-ulus, Morn, AnonMoos, BenRG, Robbot, Lowellian, Sverdrup, Henrygb, Robinh, Kairos, Lzur, Cordell, Jooler, ManuelGR, Giftlite,Abigail-II, MSGJ, Herbee, Leflyman, Fleminra, Mike40033, Jason Quinn, Fak119, Peter Ellis, Barneyboo, The Inedible Bulk, Muker-jee, Jacob grace, Ary29, Sam Hocevar, B.d.mills, Rich Farmbrough, Paul August, Andrejj, Zaslav, Elwikipedista~enwiki, PittBill, El C,Bobo192, Phidauex, ריינהארט ,לערי MPerel, Haham hanuka, Knucmo2, Alansohn, Eric Kvaalen, DreamGuy, Here, HenkvD, Gpvos,MIT Trekkie, Umapathy, Kay Dekker, Oleg Alexandrov, Stephen, Reinoutr, Harvey Heinz, LOL, JeremyA, Firien, GregorB, Gisling, Pfal-stad, Marudubshinki, Mandarax, Sukolsak, Kinu, MarSch, Venullian, Ddawson, NeonMerlin, Bubba73, SLi, Mathbot, Nihiltres, Sanbeg,Chobot, Peterl, Pinecar, YurikBot, Wavelength, Splintercellguy, Sceptre, Deeptrivia, Pigman, DanMS, Wimt, NawlinWiki, Trovatore,Kdkeller, Asarelah, IainDavidson, Ott2, Tuckerresearch, Open2universe, Arthur Rubin, Reyk, Huss4in, Pred, JDspeeder1, Cmglee, Attil-ios, KnightRider~enwiki, SmackBot, Elonka, Kellen, Melchoir, Unyoyega, KocjoBot~enwiki, Jagged 85, Pieleric, Bluebot, PrimeHunter,Mgccl, Ikiroid, Nbarth, Colonies Chris, Rlevse, Emurphy42, Newmanbe, DéRahier, Radagast83, Matt Whyndham, Fuzzypeg, Quatloo,Alokchakrabarti, Yonderboy~enwiki, OliverTwist, Vanished user 9i39j3, Aelfgifu, Breno, PseudoSudo, Ckatz, Nbatra, Dr Greg, Deceglie,Beetstra, Iridescent, Newone, Kmbush40, CRGreathouse, Vrnataraj, Myasuda, Simeon, Karen Solvig, Vanished user vjhsduheuiui4t5hjri,Tkynerd, Xtv, DJBullfish, Epbr123, Headbomb, Marek69, Missvain, Fridayda13~enwiki, AntiVandalBot, Bm gub, Autocracy, Knottjl,Dsp13, Hut 8.5, Jimmy, LittleOldMe, Moruya, Jakob.scholbach, Soulbot, Email4mobile, Mwalimu59, David Eppstein, Aziz1005, Kope,Patstuart, Afil, MartinBot, Rettetast, Dorvaq, R'n'B, S.dedalus, J.delanoy, Paul1953h, Peter Chastain, Maurice Carbonaro, DarkFalls, Kr-ishnachandranvn, El monty, NewEnglandYankee, Bonadea, Pleasantville, Jeff G., JohnBlackburne, TylerJarHead, Dougie monty, PhilipTrueman, JuneGloom07, Pittaxx, Pivari, Briandavies80, Leafyplant, Ripepette, Aljawad, Steinhb, Markfarrar, PericlesofAthens, Do-gah, BotMultichill, Arda Xi, Dilettante books, Rziff, Mygerardromance, Nic bor, Cjhenrich, Nitrotex, Jacobantony, Khubaibahmad,Elassint, ClueBot, The Thing That Should Not Be, MikeVitale, J8079s, Arunsingh16, DragonBot, Dagordon01, Razorflame, Schreiber-Bike, 4drenalin3, BOTarate, Chakreshsinghai, Thingg, NJGW, SoxBot III, XLinkBot, Good Olfactory, Addbot, Jagan83, DOI bot,11montel, BabelStone, MrOllie, NailPuppy, Favonian, Texasjrvt, Ehrenkater, Benji104, Legobot, Yobot, Fraggle81, Pcap, ,Feldkurat Katz, Bingul, Piano non troppo, Rar, Materialscientist, Citation bot, Mnshahri, Nishantjr, Crenshaw1234, Srich32977, Xasha-iar, Amaury, IShadowed, Shadowjams, FrescoBot, Stpasta, Machine Elf 1735, VeeNeel, DrilBot, Pinethicket, Jschnur, RedBot, Wabul,
52.3. TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES 175
Foobarnix, , Tstormcandy, Reach Out to the Truth, Oppizippy, Scribbleink, Calcyman, EmausBot, Wolowelsky, Rarevogel, PaulMartyn-Smith, TuHan-Bot, Dcirovic, L Kensington, Donner60, TYelliot, 28bot, Xanchester, ClueBot NG, Wcherowi, Raghith, Satel-lizer, Baseball Watcher, Frietjes, Knechtconnect, John Adastra, Widr, Phoenixred, MerlIwBot, Helpful Pixie Bot, Snaevar-bot, Frze,PowerChaos, T.N. Mahesh, Jozefgajdos, CitationCleanerBot, Ccrags, Brad7777, Ahmedronaldo321, Justincheng12345-bot, Anuraa-grox, Honwer, Knecht03, ChrisGualtieri, JYBot, Caroline1981, Hmainsbot1, Jatinag22, Numbermaniac, Redalert2fan, C5st4wr6ch,Epicgenius, Jaq2013, Harlem Baker Hughes, Matty.007, Sherwin35, Tudor987, Antonio Pomares Olivares, S Harry White, Monkbot,Nitinvikky139, Vieque, Kavenesh, Fomima, Eurodyne, KasparBot, Abacus hm and Anonymous: 357
• Main diagonal Source: https://en.wikipedia.org/wiki/Main_diagonal?oldid=663536166 Contributors: AxelBoldt, Tarquin, Stan Shebs,Andres, Giftlite, GdB, Oleg Alexandrov, Obersachse, Margosbot~enwiki, RexNL, YurikBot, Laurentius, Zwobot, Gelingvistoj, Sum-mentier, Pwaldron, Thijs!bot, AntiVandalBot, Salgueiro~enwiki, JoergenB, James mcl, Rettetast, AlleborgoBot, Wahrmund, Marc vanLeeuwen, Addbot, Götz, Lendtuffz, ArthurBot, Erik9bot, Scientific29, ChuispastonBot, Vkpd11 and Anonymous: 16
• Maninmatrix Source: https://en.wikipedia.org/wiki/Manin_matrix?oldid=671238368Contributors: Michael Hardy, Giftlite, Bender235,Rjwilmsi, CmdrObot, Myasuda, Magioladitis, Jessicapierce, Leyo, JL-Bot, SchreiberBike, Pichpich, Quebec99, FrescoBot, AlexanderChervov, John of Reading, Khazar2, Spectral sequence, Anrnusna and Anonymous: 1
• Matrix analysis Source: https://en.wikipedia.org/wiki/Matrix_analysis?oldid=675931332Contributors: Michael Hardy, Jbergquist, DavidEppstein, Maschen, Solomon7968, ChrisGualtieri, Blurred Lines and Airwoz
• Matrix chainmultiplication Source: https://en.wikipedia.org/wiki/Matrix_chain_multiplication?oldid=667666914Contributors: MichaelHardy, Dominus, Charles Matthews, Dcoetzee, Jogloran, Giftlite, Real NC, Aranel, Oleg Alexandrov, Qwertyus, BenPoweski, Rjwilmsi,Cedar101, SmackBot, Chris the speller, Kostmo, MichaelBillington, Chrylis, Vina-iwbot~enwiki, Bleargh, Screw3d, CmdrObot, Cy-debot, Ike-bana, Asmeurer, Email4mobile, David Eppstein, Gwern, M simin, CrossWinds, Ged.R, Nicgarner, Caltas, Sioffe, Jonlan-drum, ClueBot, Cp111, Max613, Kletos, Addbot, Yoenit, Doniago, ,ماني Dodek, Yobot, Gms, Citation bot, PabloCastellano, MastiBot,R.J.C. van Haaften, Sihag.deepak, Polariseke, RjwilmsiBot, StitchProgramming, LokeshRavindranathan, Hiteshgupta88, Monkbot, Lee-grc, R.J.C.vanHaaften and Anonymous: 51
• Matrix congruence Source: https://en.wikipedia.org/wiki/Matrix_congruence?oldid=616802034 Contributors: Jitse Niesen, Giftlite,Rgdboer, Longhair, Oleg Alexandrov, Bluemoose, Michael Slone, Grafen, SmackBot, Where, Cronholm144, Nijdam, Vanish2, DavidEppstein, TXiKiBoT, Neparis, SieBot, Alexbot, DumZiBoT, Addbot, DOI bot, Luckas-bot, Citation bot, Xqbot, DrilBot, ZéroBot,Qetuth, Monkbot and Anonymous: 4
• Matrix consimilarity Source: https://en.wikipedia.org/wiki/Matrix_consimilarity?oldid=530697024Contributors: Michael Hardy, Rgdboer,Kjetil1001, MuffledThud, Deltahedron and Anonymous: 1
• Matrix difference equation Source: https://en.wikipedia.org/wiki/Matrix_difference_equation?oldid=662692832Contributors: MichaelHardy, Andreas Kaufmann, BD2412, Muhandes, Yobot, FrescoBot, Tom.Reding, Duoduoduo, Loraof and Anonymous: 4
• Matrix equivalence Source: https://en.wikipedia.org/wiki/Matrix_equivalence?oldid=578933366Contributors: Charbal, Rgdboer, Dzordzm,Jim.belk, Nijdam, CmdrObot, Peskydan, Daniel5Ko, Ivan Štambuk, Marc van Leeuwen, Addbot, Luckas-bot, Rekaikko and Anonymous:3
• Matrix group Source: https://en.wikipedia.org/wiki/Matrix_group?oldid=619568031 Contributors: AxelBoldt, Michael Hardy, CharlesMatthews, Giftlite, Zaslav, Ikh, Salix alba, R.e.b., KSmrq, SmackBot, Vanished User 0001, TooMuchMath, RHB, CBM, Keyi, Ndbrian1,Arcfrk, Malcolmxl5, JackSchmidt, XLinkBot, Addbot, ArthurBot, AbigailAbernathy, Anterior1, 3children, MatrixHugh, RjwilmsiBot,NarrabundahMan, Anita5192, Boodlepounce, Doubleducks and Anonymous: 7
• Matrix of ones Source: https://en.wikipedia.org/wiki/Matrix_of_ones?oldid=682022672 Contributors: Jitse Niesen, Brona, Kahkonen,Jérôme, BD2412, HappyCamper, SmackBot, Melchoir, Octahedron80, Zemyla, CRGreathouse, Cydebot, Vanish2, David Eppstein,P.wormer, Peskydan, Haseldon, Calliopejen1, Addbot, Maniulo, ArthurBot, Qetuth, Erikjm and Anonymous: 3
• Matrix regularization Source: https://en.wikipedia.org/wiki/Matrix_regularization?oldid=656820923Contributors: Michael Hardy, Su-perHamster, Yobot, Mark viking, Cleary83 and Anonymous: 1
• Matrix representation Source: https://en.wikipedia.org/wiki/Matrix_representation?oldid=637229161 Contributors: Michael Hardy,Mdupont, Charles Matthews, Dcoetzee, Jnc, BenFrantzDale, Beland, Andreas Kaufmann, Corti, Wikicaz, Natalya, Kerowyn, Wavelength,Gaius Cornelius, Gadget850, SmackBot, Chris the speller, Bluebot, CBM, Mblumber, Alaibot, Big Bird, MikeLynch, RainbowCrane,Magioladitis, Email4mobile, Gwern, Pomte, Geometry guy, Ddunn09, Addbot, Jesse V., Frietjes, BG19bot, Supriyaa, Pinglekp andAnonymous: 4
• Matrix similarity Source: https://en.wikipedia.org/wiki/Matrix_similarity?oldid=667375995 Contributors: TakuyaMurata, Cyp, JitseNiesen, Shizhao, Aleph4, Robbot, Giftlite, Rgdboer, Longhair, Falcorian, LOL, Dzordzm, RussBot, Mhardcastle, SmackBot, Bluebot,Kjetil1001, Jim.belk, Nijdam, 16@r, Andreas Rejbrand, Vaughan Pratt, Myasuda, Salgueiro~enwiki, GromXXVII, Romney, VolkovBot,TXiKiBoT, A4bot, AlleborgoBot, SieBot, Ivan Štambuk, Aboluay, Mgrfan, Niceguyedc, Humanengr, Marc van Leeuwen, MystBot, Ad-dbot, Luckas-bot, Ht686rg90, Sz-iwbot, Flewis, Xqbot, Raffamaiden, FrescoBot, X7q, Sławomir Biały, EmausBot, Cong Qiao, ZéroBot,D.Lazard, Maschen, Anita5192, Prof McCarthy, Vrajbabu and Anonymous: 16
• Matrix splitting Source: https://en.wikipedia.org/wiki/Matrix_splitting?oldid=623104194 Contributors: Jitse Niesen, Jérôme, Yobot,Anita5192, Batard0, ChrisGualtieri and Anonymous: 2
• Metzlermatrix Source: https://en.wikipedia.org/wiki/Metzler_matrix?oldid=649152044Contributors: Michael Hardy, Giftlite, Rpchase,Rich Farmbrough, Bender235, Oleg Alexandrov, Benja, JoshuaZ, Geekdog, El floz, Skimnc, R'n'B, Peskydan, Squids and Chips, Addbot,Luckas-bot, DrilBot, Zfeinst, Mhiji, Qetuth and Anonymous: 4
• Modal matrix Source: https://en.wikipedia.org/wiki/Modal_matrix?oldid=667553438 Contributors: Michael Hardy, Giftlite, Dragon-flySixtyseven, Rgdboer, Alaibot, Leolaursen, Inwind, Mboard182, Anita5192, ClueBot NG and Anonymous: 1
• Moment matrix Source: https://en.wikipedia.org/wiki/Moment_matrix?oldid=591232888 Contributors: Charles Matthews, Rich Farm-brough, Melaen, SDC, BD2412, Hardybosse, Bluebot, MaxSem, CBM, Peskydan, Policron, Yobot, AnomieBOT, Qetuth and Anonymous:1
• Moore determinant of aHermitianmatrix Source: https://en.wikipedia.org/wiki/Moore_determinant_of_a_Hermitian_matrix?oldid=440415251 Contributors: Michael Hardy and R.e.b.
176 CHAPTER 52. QUINCUNX MATRIX
• Moore matrix Source: https://en.wikipedia.org/wiki/Moore_matrix?oldid=581835886 Contributors: Michael Hardy, Giftlite, Bkkbrad,R.e.b., Headbomb, Vanish2, Sphilbrick, Addbot, GrouchoBot, Luizpuodzius, Qetuth, Mark viking and Anonymous: 3
• Mueller calculus Source: https://en.wikipedia.org/wiki/Mueller_calculus?oldid=678366272 Contributors: Giftlite, Jason Quinn, Lucky6.9, Stephan Leeds, Hailey C. Shannon, Srleffler, KristinLee, Chobot, Salsb, Jpbowen, Rodney meyer, Cydebot, Thijs!bot, R'n'B,ToePeu.bot, Tizeff, Addbot, Fgnievinski, Tassedethe, Luckas-bot, WikitanvirBot, Helpful Pixie Bot, BG19bot, Dexbot and Anonymous:10
• Network operator matrix Source: https://en.wikipedia.org/wiki/Network_operator_matrix?oldid=601853216 Contributors: MichaelHardy, Sarahj2107, Ktr101, Mabdul, Yobot, Bluerasberry, Edgars2007, Dcshank, Jamietw, ChzzBot IV, ArticlesForCreationBot, SoftNOM, DoctorKubla and Khazar2
• Next-generationmatrix Source: https://en.wikipedia.org/wiki/Next-generation_matrix?oldid=680905867Contributors: Michael Hardy,Jitse Niesen, Bender235, Rjwilmsi, Nihiltres, Optimale, David Eppstein, Mild Bill Hiccup, MenoBot II, Asukite, Dexbot, Mark viking,Sandeepiitm and Anonymous: 1
• Nilpotent matrix Source: https://en.wikipedia.org/wiki/Nilpotent_matrix?oldid=671135412 Contributors: Tbackstr, Michael Hardy,TakuyaMurata, Charles Matthews, Jitse Niesen, Josh Cherry, Giftlite, Fropuff, Rgdboer, Awhan, Squizzz~enwiki, Forderud, Oleg Alexan-drov, LOL, Mathbot, YurikBot, Cbogart2, Pred, SmackBot, BiT, Rwilsker, Tamfang, Jim.belk, 16@r, Nakazanie, Eulerianpath, Kon-radek, Haseldon, Warut, Dogah, This, that and the other, JackSchmidt, Addbot, Jasper Deng, PV=nRT, KamikazeBot, 白駒, Point-set topologist, Pinethicket, RedBot, RobinK, Katovatzschyn, EmausBot, Anita5192, Movses-bot, Friendlywaves, Marcusdavidwebb andAnonymous: 36
• Nonnegative matrix Source: https://en.wikipedia.org/wiki/Nonnegative_matrix?oldid=655569916 Contributors: Fnielsen, Gandalf61,Rpchase, Bender235, Uffish, Oleg Alexandrov, Natalya, Bjones, Algebraist, Mcld, Nbarth, Dr.enh, Magioladitis, Peskydan, Ndick-son, Addbot, Lightbot, AnomieBOT, ArthurBot, Kiefer.Wolfowitz, Duoduoduo, Redav, Zfeinst, Helpful Pixie Bot, Qetuth, Airwoz andAnonymous: 4
• Normal matrix Source: https://en.wikipedia.org/wiki/Normal_matrix?oldid=680362789 Contributors: AxelBoldt, Zundark, Tarquin,Tbackstr, Michael Hardy, SGBailey, TakuyaMurata, Loisel, Looxix~enwiki, Dysprosia, Jitse Niesen, Robbot, Giftlite, BenFrantzDale,Zumbo, Macrakis, Paul August, Pt, Rgdboer, LOL, Eclecticos, Flamingspinach, Chobot, YurikBot, Victordk13, Banus, Lunch, TomLougheed, Octahedron80, Nbarth, Robofish, Michael Kinyon, Ylloh, WeggeBot, Myasuda, Mct mht, Andri Egilsson, Picus viridis,Wootery, Charibdis, Martynas Patasius, Inquam, Kyap, Neparis, SieBot, Paolo.dL, DragonBot, Alexbot, Qwfp, Addbot, Luckas-bot,GilCahana, Xqbot, John of Reading, Styrofoams, Splibubay, Quondum, MerlIwBot, Helpful Pixie Bot, Undiskedste, Zoydb, Andorxor,Batnotation, YiFeiBot, Canto55, SoSivr, The Quixotic Potato and Anonymous: 20
• Orbital overlap Source: https://en.wikipedia.org/wiki/Orbital_overlap?oldid=680846691 Contributors: Michael Hardy, Bearcat, Bhny,Bduke, Gogo Dodo, Christian75, Dirac66, Robert Skyhawk, Addbot, AnomieBOT, ZéroBot, Molestash, Cpando11, ChrisGualtieri,DoctorKubla, Raziarehman, Toorpu santhosh and Anonymous: 3
• Orthogonalmatrix Source: https://en.wikipedia.org/wiki/Orthogonal_matrix?oldid=678471608Contributors: AxelBoldt, Tarquin, Patrick,Michael Hardy, Tim Starling, TakuyaMurata, Stevenj, Zhaoway~enwiki, Charles Matthews, Jitse Niesen, Robbot, 1984, Kaol, Math-Martin, Tosha, Giftlite, Fropuff, Dratman, Macrakis, Chris Howard, Paul August, Rgdboer, Echuck215, PAR, Oleg Alexandrov, LOL,BD2412, HappyCamper, Mathbot, Kcarnold, YurikBot, Wavelength, Vecter, KSmrq, Gaius Cornelius, NawlinWiki, Crasshopper, Lightcurrent, Danielx, Pred, SmackBot, Haymaker, BiT, Moocowpong1, Oli Filth, Silly rabbit, Kostmo, Rludlow, SundarBot, Jim.belk, Nij-dam, Yoderj, Nialsh, Ariel Pontes, Krasnoludek, Tawkerbot2, Myasuda, Mct mht, Countchoc, OrenBochman, JEBrown87544, Ben pcc,Salgueiro~enwiki, Erxnmedia, Coffee2theorems, Jakob.scholbach, David Eppstein, User A1, Pkrecker, JoergenB, Tercer, Shinigami Josh,Kesal, Policron, Burkhard.Plache, Kyap, Ezzaldeen, JohnBlackburne, TXiKiBoT, Vladsinger, Simogasp, AlleborgoBot, YonaBot, DaJoe, Paolo.dL, Blacklemon67, Rinconsoleao, Alexbot, Bender2k14, SchreiberBike, Humanengr, XLinkBot, Addbot, Roentgenium111,DOI bot, EconoPhysicist, Legobot, Luckas-bot, Yobot, Calle, 9258fahsflkh917fas, Citation bot, LilHelpa, The suffocated, LucienBOT,Grinevitski, Citation bot 1, NA Correct, Tkuvho, Tal physdancer, Hill2690, Bluefist, Diannaa, Alph Bot, WikitanvirBot, Quondum, Zueig-nung, ChrisGualtieri, Illia Connell, Latrace, Toussapace, Tzvy, Anas satti404, Kshithappens, Kfitzell29, JOEBLOGGES and Anonymous:72
• Orthostochasticmatrix Source: https://en.wikipedia.org/wiki/Orthostochastic_matrix?oldid=650448238Contributors: TedPavlic, Karolz,Addbot, Luckas-bot, Yobot, Spectral sequence and Anonymous: 3
• P-matrix Source: https://en.wikipedia.org/wiki/P-matrix?oldid=680799828 Contributors: Rgdboer, Grafen, Bduke, Wikid77, Geekdog,Peskydan, Addbot, Luckas-bot, Yobot, DrilBot, Kiefer.Wolfowitz, Trappist the monk, ZéroBot, Jean-Charles.Gilbert, Manoguru, Qetuth,Saung Tadashi, Mark viking and Anonymous: 3
• Packed storagematrix Source: https://en.wikipedia.org/wiki/Packed_storage_matrix?oldid=549101766Contributors: Mdupont, CharlesMatthews, Giftlite, Andreas Kaufmann, Corti, Oleg Alexandrov, Bobrayner, SmackBot, Ck lostsword, Voceditenore, The Giant Puffin,Highegg, Steel1943, Slysplace, Addbot, J04n, Erik9bot, Computergeekx and Raviraj V. Sindhani
• Paley construction Source: https://en.wikipedia.org/wiki/Paley_construction?oldid=671548349 Contributors: Edward, Michael Hardy,Giftlite, Pmanderson, Will Orrick, Melchoir, Hmains, Nbarth, Vanish2, Sullivan.t.j, David Eppstein, Citation bot, Citation bot 1, Trappistthe monk, Yunfeng.Hu90, Monkbot and Anonymous: 2
• Parry–Sullivan invariant Source: https://en.wikipedia.org/wiki/Parry%E2%80%93Sullivan_invariant?oldid=551304320 Contributors:Michael Hardy, Charles Matthews, Andreas Kaufmann, Tony1, SmackBot, Sullivan.t.j, DOI bot, Lightbot, Yobot and Citation bot 1
• Pascal matrix Source: https://en.wikipedia.org/wiki/Pascal_matrix?oldid=612079759 Contributors: Michael Hardy, Giftlite, Nzseries1,Chris Howard, Goochelaar, Rjwilmsi, RDBury, Nbarth, Druseltal2005, David Eppstein, R'n'B, Peskydan, Haseldon, Gfis, Addbot, Calle,LilHelpa, R. J. Mathar, Jc86035, Tutulive, Billevans271, Monkbot and Anonymous: 3
• Pauli matrices Source: https://en.wikipedia.org/wiki/Pauli_matrices?oldid=682285192 Contributors: AxelBoldt, CYD, Josh Grosse,XJaM, Toby Bartels, Roadrunner, N8chz, Michael Hardy, TakuyaMurata, Stevenj, AugPi, Charles Matthews, Dysprosia, Reina riemann,Mattblack82, Anthony, Giftlite, Gro-Tsen, JeffBobFrank, Jcobb, Dan Gardner, Karol Langner, Pgabolde, MuDavid, Rgdboer, Susvolans,Spoon!, Ibfw, BernardH, Jheald, Begemotv2718~enwiki, Linas, LOL, Isnow, Torquil~enwiki, Graham87, Ae77, HappyCamper, Maro-zols, FlaBot, Mathbot, ChongDae, Kri, Lababidi, Bgwhite, YurikBot, Gene.arboit, JabberWok, Archelon, Vanished user 1029384756,That Guy, From That Show!, Schizobullet, SmackBot, Stepa, Moocowpong1, Georgelulu, Wiki me, Atou~enwiki, CmdrObot, Mct mht,
52.3. TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES 177
Dr.enh, Michael C Price, Quibik, Omicronpersei8, Ehque, Headbomb, Orionus, Gnixon, Darklilac, JAnDbot, Magioladitis, Baccyak4H,Sullivan.t.j, David Eppstein, Stevvers, Lnemzer, Sweetser, Fagairolles 34, STBotD, DorganBot, SoCalSuperEagle, Cuzkatzimhut, Alnok-taBOT, Philip Trueman, YohanN7, Wing gundam, Peeter.joot, StewartMH, Vaurynovich, Warbler271, Razimantv, ChandlerMapBot,Bender2k14, Count Truthstein, Addbot, Matěj Grabovský, SPat, Zorrobot, Luckas-bot, Yobot, Jagamba, Citation bot, Bci2, B wik2051,GrouchoBot, Abyrvalg, ThibautLienart, ChrisKuklewicz, Howard McCay, I dream of horses, RedBot, Samnotwil, Netheril96, ZéroBot,Nomen4Omen, Quondum, Maschen, Isocliff, Episcophagus, Helpful Pixie Bot, BG19bot, Twilightrook, Dwightboone, Moishethepig,Fylbecatulous, Dexbot, Mark viking, Thiagomureebe, Friedlicherkoenig, Prokaryotes, Headdoggy64, Octowalrus, Anitasv and Anony-mous: 96
• Perfect matrix Source: https://en.wikipedia.org/wiki/Perfect_matrix?oldid=546029006 Contributors: Michael Hardy, Giftlite, EmilJ,Ozzah, Good Olfactory, Addbot, Yobot, Fly by Night, Qetuth and Anonymous: 1
• Permutationmatrix Source: https://en.wikipedia.org/wiki/Permutation_matrix?oldid=681797665Contributors: Michael Hardy, Wshun,Pcb21, Cyp, Poor Yorick, Charles Matthews, Dysprosia, Jitse Niesen, Wik, MathMartin, DHN, Giftlite, Dratman, Falcon Kirtaran, Jpp,Tagishsimon, Chris Howard, Zaslav, Gauge, Purplefeltangel, Reinyday, Cburnett, Aitter, Oleg Alexandrov, Thruston, Magister Mathemat-icae, FlaBot, Vatter, Gaius Cornelius, Airbete~enwiki, DHN-bot~enwiki, Wdvorak, Riteshsood, Michael Ross, JRSpriggs, AbsolutDan,Stebulus, Harrigan, Goldencako, Thijs!bot, West Brom 4ever, Olenielsen, Albmont, Jackbaird, David Eppstein, R'n'B, CommonsDelinker,Zhouf12, DorganBot, VolkovBot, Camrn86, LokiClock, Adamsandberg, Justin W Smith, Gvw007, Tim32, Watchduck, Pixelator30,DumZiBoT, Marc van Leeuwen, Addbot, Cbauckhage, LaaknorBot, SpBot, Arbitrarily0, Kewpie doll517, Leycec, Legobot, Luckas-bot, Yobot, Xqbot, Ladzin, FrescoBot, LucienBOT, Slawekb, A.A.Graff, Magara adami, Boriaj, Spectral sequence, NovemberMars andAnonymous: 32
• Persymmetric matrix Source: https://en.wikipedia.org/wiki/Persymmetric_matrix?oldid=676607624 Contributors: Charles Matthews,Jitse Niesen, Army1987, N3vln, Retired username, Myasuda, Luksuh, Addbot, Yobot, ArthurBot, LucienBOT and AManWithNoPlan
• Polyconvex function Source: https://en.wikipedia.org/wiki/Polyconvex_function?oldid=681305551Contributors: Oleg Alexandrov, Sul-livan.t.j, PV=nRT, Yobot, Materialscientist, Bigweeboy, BattyBot, Das O2 and Anonymous: 2
• Polynomial matrix Source: https://en.wikipedia.org/wiki/Polynomial_matrix?oldid=674280903 Contributors: Michael Hardy, Stevenj,Fredrik, Giftlite, BenFrantzDale, Arthur Rubin, SmackBot, Octahedron80, COMPFUNK2, CmdrObot, CBM, Thijs!bot, Vanish2, Pesky-dan, LenTheWhiteCat, Hurak, Addbot, LaaknorBot, 虞海, Strawberrysunday, ArthurBot, Chenopodiaceous, E.V.Krishnamurthy, Qe-tuth, Mark viking and Anonymous: 6
• Positive-definitematrix Source: https://en.wikipedia.org/wiki/Positive-definite_matrix?oldid=682160983Contributors: AxelBoldt, Shd~enwiki,Torfason, Michael Hardy, Wshun, Cyp, Stevenj, Charles Matthews, Dcoetzee, Jitse Niesen, Phys, Josh Cherry, MathMartin, Elusus, To-bias Bergemann, Giftlite, Fropuff, Jorge Stolfi, TedPavlic, Mattrix, Bender235, Floorsheim, Pt, El C, O18, Hesperian, Blahma, PAR,Sean3000, Cburnett, Jheald, Forderud, Simetrical, Eclecticos, Btyner, Sjö, Strait, Kevmitch, FlaBot, Don Gosiewski, Sodin, Chobot, Alge-braist, YurikBot, Wavelength, Syth, Bruguiea, Crasshopper, Eli Osherovich, Lunch, SmackBot, Maksim-e~enwiki, Eskimbot, Cabe6403,Njerseyguy, Drewnoakes, Nbarth, Svein Olav Nyberg, Kjetil1001, Lambiam, Tim bates, Breno, Lilily, CRGreathouse, Myasuda, Mctmht, Thijs!bot, Lfscheidegger, LachlanA, Ben pcc, JAnDbot, MER-C, Wootery, Stangaa, Magioladitis, JamesBWatson, Cyktsui, Mets-Bot, Americanhero, Tercer, Mythealias, Leyo, Maurice Carbonaro, Nathanshao, Policron, Ratfox, DavidIMcIntosh, Tomtheebomb,NathanHagen, PaulTanenbaum, Kkilger, Philmac, Daviddoria, AlleborgoBot, Hsbhat, PeterBFZ, Andrés Catalán, Yahastu, Sharov, Skip-pydo, Jdgilbey, Wcy~enwiki, Bender2k14, Muhandes, Bluemaster, Qwfp, Dingenis, Gjnaasaa, Job Inkop~enwiki, Tayste, Addbot, Cst17,Dr. Universe, PV=nRT, Luckas-bot, Yobot, Nghtwlkr, Nimrody, Legendre17, AnomieBOT, Joule36e5, Materialscientist, ArthurBot,Bdmy, Raffamaiden, Mardebikas, Pupdike, Sławomir Biały, Vineethkuruvath, Haein45, Avarela1965, MastiBot, Dzlot, Dividingbyzero-fordummies, Toolnut, Pfm77, Begoon, Duoduoduo, Suffusion of Yellow, Xnn, Wyverald, EmausBot, Wisapi, GoingBatty, Felix Hoff-mann, Chaohuang, O'society, Vilietha, Wayne Slam, Zaran, Joao Meidanis, Іванко1, Maschen, Est nomis, Fioravante Patrone, JoelB. Lewis, Helpful Pixie Bot, Lubdone, BG19bot, Dvomedo, Solomon7968, Intervallic, Manoguru, ChrisGualtieri, YFdyh-bot, PrestonKemeny, Davidcy123, Egorlarionov, Frytvm, Digiuno, Webclient101, Limit-theorem, HEKrogstad, GoplaWHya, The Disambiguator,Brownerthanu, Srinivas tudelft, Rangdor, H1729 and Anonymous: 154
• Productivematrix Source: https://en.wikipedia.org/wiki/Productive_matrix?oldid=673475112Contributors: Yobot, BG19bot, LaPlaieEtLe-Couteau and Anonymous: 1
• Pseudo-determinant Source: https://en.wikipedia.org/wiki/Pseudo-determinant?oldid=635471908 Contributors: Entropeneur, Mya-suda, Melcombe, Oldlaptop321, Stpasha, Quondum, Lynskyder, Brirush and Anonymous: 5
• Q-matrix Source: https://en.wikipedia.org/wiki/Q-matrix?oldid=681367203 Contributors: Michael Hardy, Centrx, Reetep, SmackBot,Insp nf, EmausBot, Saung Tadashi and Kebabpizza
• Quaternionicmatrix Source: https://en.wikipedia.org/wiki/Quaternionic_matrix?oldid=637776749 Contributors: Bearcat, Giftlite, PaulD. Anderson, RDBury, Lambiam, CRGreathouse, JohnBlackburne, Stephen J. Brooks, Cirt, Muhandes, Addbot, ArthurBot, Titi2~enwiki,Miyagawa, ZéroBot and Monkbot
• Quincunx matrix Source: https://en.wikipedia.org/wiki/Quincunx_matrix?oldid=535750850 Contributors: Michael Hardy, Stevenj,Giftlite, Archelon, David Eppstein, Thinking of England and Qetuth
52.3.2 Images• File:2152085cab.png Source: https://upload.wikimedia.org/wikipedia/commons/b/bd/2152085cab.png License: CC BY-SA 3.0 Con-tributors: Own work Original artist: RainerTypke
• File:4x4_magic_square_hierarchy.svg Source: https://upload.wikimedia.org/wikipedia/commons/f/f6/4x4_magic_square_hierarchy.svg License: CC BY-SA 3.0 Contributors: Own work Original artist: Cmglee
• File:6n-graf.svg Source: https://upload.wikimedia.org/wikipedia/commons/5/5b/6n-graf.svgLicense: Public domainContributors: Image:6n-graf.png simlar input data Original artist: User:AzaToth
• File:Albrecht_Dürer_-_Melencolia_I_(detail).jpg Source: https://upload.wikimedia.org/wikipedia/commons/7/7e/Albrecht_D%C3%BCrer_-_Melencolia_I_%28detail%29.jpg License: Public domain Contributors: Unknown Original artist: Albrecht Dürer
178 CHAPTER 52. QUINCUNX MATRIX
• File:Catalan-Hexagons-example.svg Source: https://upload.wikimedia.org/wikipedia/commons/a/a8/Catalan-Hexagons-example.svgLicense: Public domain Contributors: http://en.wikipedia.org/wiki/File:Catalan-Hexagons-example.svg Original artist: Dmharvey
• File:Commons-logo.svg Source: https://upload.wikimedia.org/wikipedia/en/4/4a/Commons-logo.svg License: ? Contributors: ? Origi-nal artist: ?
• File:Geomagic_square_-_Diamonds.jpg Source: https://upload.wikimedia.org/wikipedia/commons/2/2a/Geomagic_square_-_Diamonds.jpg License: CC BY-SA 3.0 Contributors: www.geomagicsquares.com/gallery.php? Original artist: Lee Sallows
• File:Graph1_for_mathematical_expression1_reduced.JPG Source: https://upload.wikimedia.org/wikipedia/commons/0/04/Graph1_for_mathematical_expression1_reduced.JPGLicense: Public domainContributors: http://www.network-operator.com/datas/eng/Description.pdf page 4 Original artist: Soft NOM
• File:Graph_Laplacian_Diffusion_Example.gif Source: https://upload.wikimedia.org/wikipedia/commons/e/e1/Graph_Laplacian_Diffusion_Example.gif License: CC BY-SA 3.0 Contributors: Own work Original artist: Ctralie
• File:Hagiel_sigil_derivation.svg Source: https://upload.wikimedia.org/wikipedia/en/d/db/Hagiel_sigil_derivation.svgLicense: PDCon-tributors: ? Original artist: ?
• File:Lebesgue_Icon.svg Source: https://upload.wikimedia.org/wikipedia/commons/c/c9/Lebesgue_Icon.svg License: Public domainContributors: w:Image:Lebesgue_Icon.svg Original artist: w:User:James pic
• File:Macau_stamps_featuring_6_magic_squares.jpg Source: https://upload.wikimedia.org/wikipedia/commons/7/74/Macau_stamps_featuring_6_magic_squares.jpg License: Public domain Contributors: http://www.leesallows.com/files/zes_postzegels.jpg Original artist:Lee Sallows
• File:Magic_square_4x4_Shams_Al-maarif2_(Arabic_magics).jpg Source: https://upload.wikimedia.org/wikipedia/en/3/3f/Magic_square_4x4_Shams_Al-maarif2_%28Arabic_magics%29.jpg License: Fair use Contributors:A very old script of Shams Al-ma'arif, dates back to the 1200s (600 AH). Original artist: ?
• File:Magic_square_4x4_Shams_Al-maarif3_(Arabic_magics).jpg Source: https://upload.wikimedia.org/wikipedia/en/e/eb/Magic_square_4x4_Shams_Al-maarif3_%28Arabic_magics%29.jpg License: Fair use Contributors:A very old script of Shams Al-ma'arif, dates back to the 1200s (600 AH). Original artist: ?
• File:Magicsquareexample.svg Source: https://upload.wikimedia.org/wikipedia/commons/e/e4/Magicsquareexample.svgLicense: Pub-lic domain Contributors: Own work Original artist: User:Phidauex
• File:Matrix_chain_multiplication_polygon_example.svg Source: https://upload.wikimedia.org/wikipedia/commons/e/ea/Matrix_chain_multiplication_polygon_example.svg License: CC BY-SA 4.0 Contributors: Own work Original artist: R.J.C. van Haaften
• File:Matrix_symmetry_qtl2.svg Source: https://upload.wikimedia.org/wikipedia/commons/2/22/Matrix_symmetry_qtl2.svg License:CC BY-SA 3.0 Contributors: Own work Original artist: Quartl
• File:Ms_sf_2.jpg Source: https://upload.wikimedia.org/wikipedia/commons/a/a6/Ms_sf_2.jpg License: CC-BY-SA-3.0 Contributors: ?Original artist: ?
• File:Nuvola_apps_edu_mathematics_blue-p.svg Source: https://upload.wikimedia.org/wikipedia/commons/3/3e/Nuvola_apps_edu_mathematics_blue-p.svg License: GPL Contributors: Derivative work from Image:Nuvola apps edu mathematics.png and Image:Nuvolaapps edu mathematics-p.svg Original artist: David Vignoni (original icon); Flamurai (SVG convertion); bayo (color)
• File:OEISicon_light.svg Source: https://upload.wikimedia.org/wikipedia/commons/d/d8/OEISicon_light.svg License: Public domainContributors: Own work Original artist: Watchduck (a.k.a. Tilman Piesk)
• File:Permutation_matrix;_P\__xunadd_text_character:nN\textasciicircum^T_*_column.svg Source: https://upload.wikimedia.org/wikipedia/commons/8/8f/Permutation_matrix%3B_P%5ET_%2A_column.svgLicense: Public domainContributors: Own workOrig-inal artist: Watchduck (a.k.a. Tilman Piesk)
• File:Permutation_matrix;_P_*_column.svg Source: https://upload.wikimedia.org/wikipedia/commons/2/2d/Permutation_matrix%3B_P_%2A_column.svg License: Public domain Contributors: Own work Original artist: Watchduck (a.k.a. Tilman Piesk)
• File:Permutation_matrix;_row_*_P.svg Source: https://upload.wikimedia.org/wikipedia/commons/a/a3/Permutation_matrix%3B_row_%2A_P.svg License: Public domain Contributors: Own work Original artist: Watchduck (a.k.a. Tilman Piesk)
• File:Permutation_matrix;_row_*_P\__xunadd_text_character:nN\textasciicircum^T.svg Source: https://upload.wikimedia.org/wikipedia/commons/4/4c/Permutation_matrix%3B_row_%2A_P%5ET.svg License: Public domain Contributors: Own work Origi-nal artist: Watchduck (a.k.a. Tilman Piesk)
• File:Question_book-new.svg Source: https://upload.wikimedia.org/wikipedia/en/9/99/Question_book-new.svg License: Cc-by-sa-3.0Contributors:Created from scratch in Adobe Illustrator. Based on Image:Question book.png created by User:Equazcion Original artist:Tkgd2007
• File:Rubik's_cube_v3.svg Source: https://upload.wikimedia.org/wikipedia/commons/b/b6/Rubik%27s_cube_v3.svg License: CC-BY-SA-3.0 Contributors: Image:Rubik's cube v2.svg Original artist: User:Booyabazooka, User:Meph666 modified by User:Niabot
• File:Sigillum_Iovis.jpg Source: https://upload.wikimedia.org/wikipedia/commons/0/0b/Sigillum_Iovis.jpgLicense: Public domainCon-tributors: Scan of “Oedipus Aegyptiacus”Original artist: Work by Athanasius Kircher, scan by Feldkurat Katz
• File:Symmetric_group_3;_Cayley_table;_matrices.svg Source: https://upload.wikimedia.org/wikipedia/commons/e/e0/Symmetric_group_3%3B_Cayley_table%3B_matrices.svg License: Public domain Contributors: ? Original artist: ?
• File:Symmetric_group_3;_Cayley_table;_positions.svg Source: https://upload.wikimedia.org/wikipedia/commons/c/cb/Symmetric_group_3%3B_Cayley_table%3B_positions.svg License: Public domain Contributors: Own work Original artist: Lipedia
• File:Taxonomy_of_Complex_Matrices.svg Source: https://upload.wikimedia.org/wikipedia/commons/d/d1/Taxonomy_of_Complex_Matrices.svg License: CC BY-SA 3.0 Contributors: own Original artist: Jérôme
52.3. TEXT AND IMAGE SOURCES, CONTRIBUTORS, AND LICENSES 179
• File:Text_document_with_red_question_mark.svg Source: https://upload.wikimedia.org/wikipedia/commons/a/a4/Text_document_with_red_question_mark.svg License: Public domain Contributors: Created by bdesham with Inkscape; based upon Text-x-generic.svgfrom the Tango project. Original artist: Benjamin D. Esham (bdesham)
• File:Wiki_letter_w.svg Source: https://upload.wikimedia.org/wikipedia/en/6/6c/Wiki_letter_w.svg License: Cc-by-sa-3.0 Contributors:? Original artist: ?
• File:Wiki_letter_w_cropped.svg Source: https://upload.wikimedia.org/wikipedia/commons/1/1c/Wiki_letter_w_cropped.svg License:CC-BY-SA-3.0 Contributors:
• Wiki_letter_w.svg Original artist: Wiki_letter_w.svg: Jarkko Piiroinen• File:Wikisource-logo.svg Source: https://upload.wikimedia.org/wikipedia/commons/4/4c/Wikisource-logo.svg License: CC BY-SA
3.0 Contributors: Rei-artur Original artist: Nicholas Moreau• File:Yanghui_magic_square.GIF Source: https://upload.wikimedia.org/wikipedia/commons/a/ab/Yanghui_magic_square.GIFLicense:
CC BY-SA 3.0 Contributors: Own work Original artist: Gisling• File:Yuan_dynasty_iron_magic_square.jpg Source: https://upload.wikimedia.org/wikipedia/commons/2/2d/Yuan_dynasty_iron_magic_
square.jpg License: CC BY-SA 3.0 Contributors: Own work Original artist: BabelStone
52.3.3 Content license• Creative Commons Attribution-Share Alike 3.0
