Lecture 15, 16: Diagonalization

• Suppose 𝐴 is a 𝑛 × 𝑛 matrix and has 𝑛 linearly independent eigenvectors 𝒑1…𝒑𝑛, put them into columns of eigenvector matrix 𝑃.

• Put eigenvalues into a diagonal matrix 𝐷. 𝐷𝑖𝑖 has the eigenvalue corresponding to the eigenvector in the 𝑖-th column of 𝑃.

𝐴𝑃 = 𝐴 𝒑1 … 𝒑𝑛 = 𝜆1𝒑1 … 𝜆𝑛𝒑𝑛 = 𝑃𝐷.

• From above equation it is implied:• 𝑃−1𝐴𝑃 = 𝐷 or 𝑃𝐷𝑃−1 = 𝐴• The square matrix 𝐴 is diagonalizable if it has 𝑛 distinct eigenvalues or if its 𝑛 eigenvectors are linearly independent.

1

Motivation: Eigenvalues and Eigenvectors are easy to compute for diagonal matrices. Hence, we would like (if possible) to convert matrix A into a diagonal matrix.

Powers of A

• Suppose 𝐴 is factored to 𝑃𝐷𝑃−1.

• To compute 𝐴2 we multiply its factored equivalent:

• 𝐴2 = 𝑃𝐷𝑃−1 𝑃𝐷𝑃−1 = 𝑃𝐷2𝑃−1.

• We can imply that 𝐴𝑘 = 𝑃𝐷𝑘𝑃−1

• 𝐷𝑘 is the diagonal matrix 𝐷 where its entries are to the power of 𝑘, and is very easy to calculate.

Powers of A

Steps for Matrix Diagonalization

• Diagonalize the following matrix:

𝐴 =1 3 3−3 −5 −33 3 1

• Find the eigenvalues 𝜆1, 𝜆2, 𝜆3.

• Find three linearly independent eigenvectors of A.

• Construct P = [v1, v2, v3]

• Construct D.

• Check 𝐴𝑃 = 𝑃𝐷 𝑎𝑛𝑑 𝐴 = 𝑃𝐷𝑃−1

Not all Matrices are Diagonalizable

Application: Steady State

• Suppose there is a system such that 𝒖𝑘+1 = 𝐴𝒖𝑘, where 𝑢 is the state vector and its subscripts denote its order in a time series.

• If 𝒖0 is the initial condition of the system at beginning, we can find 𝒖𝑘, the state at the time 𝑛, using 𝒖𝑘 = 𝐴𝑘𝒖0.

• We know that 𝐴𝑘 = 𝑋𝐷𝑘𝑋−1.

• So if 𝑛 → ∞:• If 𝜆𝑖 < 1 then 𝜆𝑖

𝑘 → 0.• If 𝜆𝑖 = 1 then 𝜆𝑖

𝑘 = 1.• If 𝜆𝑖 > 1 then 𝜆𝑖

𝑘 → ∞.

• So, stable systems 𝐴 should not have eigenvalues 𝜆𝑖 > 1 .

Application: Steady State

• Example: Suppose 10% of residents of the city A move to city B in a year. During the same time 5% of the city B residents move to city A. If initial residents of cities A and B are 10,000 and 100,000 respectively, what will happen in a long run ?

• We formulate the system as 𝒖 =𝑟𝑒𝑠𝑖𝑑𝑒𝑛𝑡𝑠 𝑜𝑓 𝑐𝑖𝑡𝑦 𝐴𝑟𝑒𝑠𝑖𝑑𝑒𝑛𝑡𝑠 𝑜𝑓 𝑐𝑖𝑡𝑦 𝐵

• If no other thing affect the population of the cities A and B, the matrix 𝐴 is:

• 𝒖1 = 𝐴𝒖0 → 𝒖1 =.9 .05.1 .95

𝒖0

• Eigenvalues of 𝐴, are 𝜆1 = 0.85 and 𝜆2 = 1.

• Eigenvectors of 𝐴 are 𝒙1 =−11

, 𝒙2 =−1−2

Application: Steady State

• Example Cont’d:• In a long run 𝒖𝑘 = 𝐴𝑘𝒖0

• 𝐴𝑘 = 𝑋𝐷𝑘𝑋−1 =−1 −11 −2

0.85 00 1

𝑘 −2

3

1

3

−1

3−

1

3

• If 𝑘 → ∞ then 𝜆1𝑘 → 0.

• Only the 𝜆2 will be effective in 𝐴𝑘 when 𝑘 → ∞

• lim𝑘→∞

𝐴𝑘 =−1 −11 −2

0 00 1

−2

3

1

3

−1

3−

1

3

=

1

3

1

32

3

2

3

• 𝒖∞ = 𝐴∞𝒖0 =

1

3

1

32

3

2

3

10,000100,000

~36,66773,333

Application: Steady State

• Example Cont’d:• Another approach:• Recall that if 𝒚 = 𝛼1𝒙1 + 𝛼2𝒙2 +⋯+ 𝛼𝑛𝒙𝑛 then 𝐴𝒚 = 𝛼1𝜆1𝒙1 + 𝛼2𝜆2𝒙2 +⋯+ 𝛼𝑛𝜆𝑛𝒙𝑛

• Therefore, 𝐴𝑘𝒚 = 𝛼1𝜆1𝑘𝒙1 + 𝛼2𝜆2

𝑘𝒙2 +⋯+ 𝛼𝑛𝜆𝑛𝑘𝒙𝑛

•10,000100,000

~−1 −11 −2

26667−36667

• 𝐴𝑘10,000100,000

= 26667(0.85)𝑘−11

− 36667(1)𝑘−1−2

• 𝐴∞10,000100,000

= −36667−1−2

~36,66773,333

Application: Discrete Dynamic Systems

Application: Discrete Dynamic Systems

Application: Fibonacci Series

• A Fibonacci series: 1, 1, 2, 3, 5, 8, 13 ……

• How to find the 100th element x100 quickly?

Application: Fibonacci Series

Application: Fibonacci Series

Diagonalization of Symmetric Matrices

• Symmetric matrices always have real eigenvalues and are diagonalizable.

• What is special about 𝐴𝑥 = 𝜆𝑥, when 𝐴 is symmetric (𝐴 = 𝐴𝑇)?

• 𝐴 = 𝐴𝑇 → 𝑃𝐷𝑃−1 = (𝑃𝐷𝑃−1)𝑇= (𝑃−1)𝑇𝐷𝑃𝑇

• Above equation is correct only if 𝑃−1 = 𝑃𝑇, and this means 𝑃 is orthonormal.

• Therefore, every symmetric matrix 𝐴 = 𝑄𝐷𝑄𝑇, where 𝑄 is the normalized eigenvector matrix, and 𝐷 is the eigenvalue diagonal matrix.

• 𝑄 is an orthonormal matrix and 𝑄−1 = 𝑄𝑇.

Diagonalization of Symmetric Matrices

Example: re-write matrix 𝐴 =1 22 −2

, as factorization 𝐴 = 𝑄𝐷𝑄𝑇.

• 𝐴 − 𝜆𝐼 =1 − 𝜆 22 −2 − 𝜆

• 1 − 𝜆 −2 − 𝜆 − 4 = 0 → 𝜆2 + 𝜆 − 6 = 0 → 𝜆 + 3 𝜆 − 2 = 0; 𝜆1 = −3 and 𝜆2 = 2

𝐴 − 𝜆1𝐼 𝒙1 = 0 →4 22 1

= 𝟎 → 𝒙1 =1−2

𝐴 − 𝜆2𝐼 𝒙2 = 0 →−1 22 −4

= 𝟎 → 𝒙2 =2−1

• 𝒙1 and 𝒙2 are orthogonal, but not orthonormal. If we normalize them, they will be orthonormal.

• 𝑄 =1

5

1 2−2 −1

, 𝐷 =−3 00 2

Diagonalization of Symmetric Matrices

Example: Diagonalize

Diagonalization of Symmetric Matrices

Spectrum Theorem

• If A is a n x n symmetric matrix

• All eigenvalues of A are real.

• A has exactly n real eigenvalues (counting for multiplicity). But this does not mean they are distinct.

• The geometric multiplicity of 𝜆 = dimension of Null(A - 𝜆I) = the algebraic multiplicity of 𝜆.

• The eigenspaces are mutually orthogonal.

• If 𝜆1 ≠ 𝜆2 are two distinct eigenvalues, then their corresponding eigenvectors v1 and v2 are orthogonal.

Proof