Eigenvalues and Eigenvectors - Hong Kong Baptist …felix_kwok/math2207/Ch5.pdf · ·...

Eigenvalues and EigenvectorsChapter 5

MATH 2207 – Linear Algebra

§5.1 Eigenvectors and Eigenvalues› We saw in Chapter 4 how choosing a non-standard basis can make a linear transformation T(x) = Ax look much simpler.

› In many (but not all) cases, a basis B can be chosen so that 𝑇𝑇 𝐵𝐵 is diagonal. These basis vectors are called eigenvectors of A, and the diagonal entries of 𝑇𝑇 𝐵𝐵 are called eigenvalues of A.• Eigen = “proper”, “characteristic” in German

› Eigenvalues are fundamental in mathematics, physics, computer science and engineering, because it simplifies analysis tremendously.

› We will only cover Sections 5.1-5.3. More details and applications of eigenvalues can be found in later courses (MATH 3405, 3407, 3605, 4615, …)

CHAPTER 5: EIGENVALUES AND EIGENVECTORS 2

› Example:• Consider the Fibonacci sequence, which starts with

𝑓𝑓0 = 0, 𝑓𝑓1 = 1and the next in the sequence is generated by adding the last two values: 𝑓𝑓𝑘𝑘 = 𝑓𝑓𝑘𝑘−1 + 𝑓𝑓𝑘𝑘−2

• The fk satisfy the vector equation𝑓𝑓𝑘𝑘𝑓𝑓𝑘𝑘−1

= 1 11 0=𝐴𝐴

𝑓𝑓𝑘𝑘−1𝑓𝑓𝑘𝑘−2

= 𝐴𝐴2 𝑓𝑓𝑘𝑘−2𝑓𝑓𝑘𝑘−3

= ⋯ = 𝐴𝐴𝑘𝑘−1 𝑓𝑓1𝑓𝑓0

• How to get an explicit formula for fk?• If we can find an invertible matrix P such that A = PDP-1, with

𝐷𝐷 = 𝜆𝜆1 00 𝜆𝜆2

,

then 𝑓𝑓𝑘𝑘𝑓𝑓𝑘𝑘−1

= 𝐴𝐴𝑘𝑘−1 10 = 𝑃𝑃 𝜆𝜆1 𝑘𝑘−1 0

0 𝜆𝜆2 𝑘𝑘−1 𝑃𝑃−1 10 .


› In many applications in computer science and engineering, we have vector sequences of the type

𝑣𝑣,𝐴𝐴𝑣𝑣,𝐴𝐴2𝑣𝑣, … ,𝐴𝐴𝑘𝑘𝑣𝑣, …

› Do the entries of the vectors• Tend to infinity?• Decay to zero?• Converge to a fixed vector?

› If 𝐴𝐴 =𝜆𝜆1 0

⋱0 𝜆𝜆𝑛𝑛

is a diagonal matrix, then these questions are

easy to answer (depends on whether 𝜆𝜆𝑖𝑖 < 1 or not)› Otherwise, try to find an invertible matrix P such that A = PDP-1 with D diagonal, and do the analysis on D instead!


› Assume that the n x n matrix A is of the form A = PDP-1, with

𝑃𝑃 = 𝒑𝒑1 ⋯ 𝒑𝒑𝑛𝑛 , 𝐷𝐷 =𝜆𝜆1 0

⋱0 𝜆𝜆𝑛𝑛

› Rewriting A = PDP-1 as AP = PD and comparing columns, we see that

𝐴𝐴𝒑𝒑1 ⋯ 𝐴𝐴𝒑𝒑𝑛𝑛 = 𝜆𝜆1𝒑𝒑1 ⋯ 𝜆𝜆𝑛𝑛𝒑𝒑𝑛𝑛so

𝐴𝐴𝒑𝒑1 = 𝜆𝜆1𝒑𝒑1, … , 𝐴𝐴𝒑𝒑𝑛𝑛 = 𝜆𝜆𝑛𝑛𝒑𝒑𝑛𝑛where p1, … pn are non-zero vectors and λ1, …, λn are scalars.

› The 𝒑𝒑𝑗𝑗 are called eigenvectors of A, and the λj are called eigenvalues of A.


› An example from physics (optional):

› URL: http://www.youtube.com/watch?v=tliBfYdddhUCHAPTER 5: EIGENVALUES AND EIGENVECTORS 6

http://www.youtube.com/watch?v=tliBfYdddhU

› Vibration model:𝑑𝑑2𝒚𝒚𝑑𝑑𝑡𝑡2

= 𝐴𝐴𝒚𝒚

› Let y = P-1z and simplify:𝑑𝑑2𝒛𝒛𝑑𝑑𝑡𝑡2

= 𝐷𝐷𝒛𝒛

› Get n decoupled scalar ODEs!• Eigenvalues = square of the frequency

• Eigenvectors = shape of the vibrating plate at those frequencies

› The Chladni figures can be calculated numerically!


› Definition: • An eigenvector of an nxn matrix A is a non-zero vector x such that Ax = λx for some scalar λ.

• A scalar λ is called an eigenvalue of A if there is a non-trivial solution x of Ax = λx; such an x is called an eigenvector corresponding to λ.


› Warning: Eigenvalues and eigenvectors are only defined for square matrices.

› When x is an eigenvector of A, the image of x under A is particularly simple: A only stretches x by a factor λ.

› In general, Av need not be along the same direction as v, when v is not an eigenvector.

› Example: Let 𝐴𝐴 = 1 65 2 ,𝑢𝑢 = 6

−5 , 𝑣𝑣 = 3−2 . Are u and v

eigenvectors of A?› Solution: Calculate Au and Av explicitly:

𝐴𝐴𝑢𝑢 = −2420 = −4 6

−5 , 𝐴𝐴𝑣𝑣 = −911 .

• Since Au = -4u, u is an eigenvector of A.• Since Av is not a scalar multiple of v, v is not an eigenvector of A.


› Example: Let 𝐴𝐴 = 1 65 2 . Show that λ=7 is an eigenvalue of A, and

find its corresponding eigenvector.› Solution: We seek a non-zero solution x to Ax = 7x. This is the same as finding a non-trivial solution to 𝐴𝐴 − 7𝐼𝐼 𝑥𝑥 = 0. This can be done by row reduction:

𝐴𝐴 − 7𝐼𝐼 = −6 65 −5 ∼ 1 −1

0 0The general solution of 𝐴𝐴 − 7𝐼𝐼 𝑥𝑥 = 0 is 𝑥𝑥 = 𝑡𝑡 1

1 , t∈R. Thus, any scalar multiple of [1,1]𝑇𝑇 is an eigenvector of A corresponding to λ=7.

› Remarks: • Eigenvectors can never be zero (but eigenvalues can!).• Eigenvectors are not unique. If v is an eigenvector of A, any scalar multiple of v must also be an eigenvector of A.


› Definition: Let λ be an eigenvalue of A. Then the set of all solutions of the equation

𝐴𝐴 − 𝜆𝜆𝐼𝐼 𝑥𝑥 = 0is called the eigenspace of A corresponding to λ.

› An eigenspace is a subspace of the form Nul(𝐴𝐴 − 𝜆𝜆𝐼𝐼) and has dimension at least 1 (why?). It is also possible to have larger eigenspaces.

› Example: Let 𝐴𝐴 =4 −1 62 1 62 −1 8

and λ=2. Then

𝐴𝐴 − 2𝐼𝐼 =2 −1 62 −1 62 −1 6

∼2 −1 60 0 00 0 0

Thus, λ=2 is an eigenvalue of A, and a basis for the corresponding eigenspace is

𝐵𝐵 =1210

,−301

.



§5.2 Characteristic Equation› Row reduction allows us to find eigenvectors once an eigenvalue is known.

› But: How to find eigenvalues in the first place?› Recall: if 𝐴𝐴 − 𝜆𝜆𝐼𝐼 𝑥𝑥 = 0 has a non-trivial solution, then

det 𝐴𝐴 − 𝜆𝜆𝐼𝐼 = 0.

› The equation (*) is a polynomial equation in λ whose roots are precisely the eigenvalues of A.

› Definition:• Equation (*) is called the characteristic equation of A.• The polynomial 𝑝𝑝𝐴𝐴 𝜆𝜆 = det(𝐴𝐴 − 𝜆𝜆𝐼𝐼) is called the characteristic polynomial of A.


(*)

› Example: Find the eigenvalues of 𝐴𝐴 = 2 33 −6 .

› Solution: det 𝐴𝐴 − 𝜆𝜆𝐼𝐼 = det 2 − 𝜆𝜆 3

3 −6 − 𝜆𝜆= 𝜆𝜆 − 2 𝜆𝜆 + 6 − 9= 𝜆𝜆2 + 4𝜆𝜆 − 21= 𝜆𝜆 + 7 𝜆𝜆 − 3 .

So the eigenvalues of A are -7 and 3. Their corresponding eigenvectors are:• Av1 = -7v1:

9 33 1 ∼ 3 1

0 0 ⟹ 𝑣𝑣1 = 𝑡𝑡 1−3

• Av2 = 3v2:−1 33 −9 ∼ 1 −3

0 0 ⟹ 𝑣𝑣2 = 𝑡𝑡 31

• Verify: for t=1, we indeed get

𝐴𝐴𝑣𝑣1 = −721 = −7 1

−3 , 𝐴𝐴𝑣𝑣2 = 93 = 3 3

1 .


› Remark: Some authors prefer to define the characteristic polynomial as

𝑝𝑝𝐴𝐴 𝜆𝜆 = det(𝜆𝜆𝐼𝐼 − 𝐴𝐴) .No matter which definition you use, you always get the same eigenvalues.

› Special cases:• A is singular (i.e. non-invertible) if and only if 0 is an eigenvalue of A. (why?)

• If A is triangular (upper or lower), then its eigenvalues are precisely its diagonal entries:

𝑝𝑝𝐴𝐴 𝜆𝜆 = det 𝐴𝐴 − 𝜆𝜆𝐼𝐼 = det

𝑎𝑎11 − 𝜆𝜆 ∗ ∗ ∗𝑎𝑎22 − 𝜆𝜆 ∗ ∗

⋱ ∗0 𝑎𝑎𝑛𝑛𝑛𝑛 − 𝜆𝜆

= 𝑎𝑎11 − 𝜆𝜆 𝑎𝑎22 − 𝜆𝜆 ⋯ 𝑎𝑎𝑛𝑛𝑛𝑛 − 𝜆𝜆 = 0.So the eigenvalues are 𝑎𝑎11,𝑎𝑎22, … ,𝑎𝑎𝑛𝑛𝑛𝑛.


› Example: Let c and s be two real numbers. Calculate the eigenvalues of

𝐴𝐴 = 𝑐𝑐 −𝑠𝑠𝑠𝑠 𝑐𝑐

› Solution:det 𝐴𝐴 − 𝜆𝜆𝐼𝐼 = det 𝑐𝑐 − 𝜆𝜆 −𝑠𝑠

𝑠𝑠 𝑐𝑐 − 𝜆𝜆 = 𝜆𝜆2 − 2𝑐𝑐𝜆𝜆 + (𝑐𝑐2 + 𝑠𝑠2) .• Since the discriminant is

Δ = 4𝑐𝑐2 − 4 𝑐𝑐2 + 𝑠𝑠2 = −4𝑠𝑠2 ≤ 0,The characteristic polynomial does not have real roots unless s=0. The two complex roots are

𝜆𝜆1,2 = 𝑐𝑐 ± 𝑖𝑖𝑠𝑠, where 𝑖𝑖 = −1.› Remarks:

• In general, matrices with real entries may have complex eigenvalues and eigenvectors. See Section 5.5 for details.

• However, if A=AT, i.e., when A is real and symmetric, we always have real eigenvalues. See Chapter 6.


§5.3 Diagonalization› Recall: our goal is to produce an invertible matrix P such that A = PDP-1.

› What we have shown so far: columns of P must be eigenvectors of A.› Question: Do we have enough eigenvectors to form a basis of Rn?

• If YES, then an invertible P exists A is diagonalizable.• If NO, then A is not diagonalizable.

› Definition: An nxn matrix A is said to be diagonalizable if there exists an invertible matrix P such that P-1AP is diagonal.

› Fact: Not every matrix is diagonalizable, even if we allow complex eigenvalues!


› Theorem 5 (Diagonalization Theorem)An nxn matrix A is diagonalizable if and only if A has n linearly independent eigenvectors.

› Proof: • We have seen previously that if A is diagonalizable, then the n columns of P must be eigenvectors. Since P is invertible, the columns must be linearly independent.

• Conversely, suppose A has n linearly independent eigenvectors v1,…, vn, with corresponding eigenvalues λ1,…,λn. Then letting

𝑃𝑃 = 𝑣𝑣1 ⋯ 𝑣𝑣𝑛𝑛leads to

𝐴𝐴𝑃𝑃 = 𝐴𝐴𝑣𝑣1 ⋯ 𝐴𝐴𝑣𝑣𝑛𝑛 = 𝜆𝜆1𝑣𝑣1 ⋯ 𝜆𝜆𝑛𝑛𝑣𝑣𝑛𝑛

= 𝑣𝑣1 ⋯ 𝑣𝑣𝑛𝑛𝜆𝜆1

⋱𝜆𝜆𝑛𝑛

= 𝑃𝑃𝐷𝐷.

Since P is invertible, we see that P-1AP = D is a diagonal matrix, so A is diagonalizable.


› Example: Diagonalize the matrix 𝐴𝐴 =1 3 3−3 −5 −33 3 1

, if possible.

› Step 1: Find the eigenvalues of A.

• det 𝐴𝐴 − 𝜆𝜆𝐼𝐼 =1 − 𝜆𝜆 3 3−3 −5 − 𝜆𝜆 −33 3 1 − 𝜆𝜆

= −𝜆𝜆3 − 3𝜆𝜆2 + 4 = 1 − 𝜆𝜆 𝜆𝜆 + 2 2

• So the eigenvalues are 1 and -2 (with algebraic multiplicity 2).

› Step 2: Try to find 3 linearly independent eigenvectors of A.

• λ=1: 0 3 3−3 −6 −33 3 0

∼1 0 −10 1 10 0 0

⟹ 𝑣𝑣1 = 𝑡𝑡1−11


• λ=-2:3 3 3−3 −3 −33 3 3

∼1 1 10 0 00 0 0

⟹ 𝑥𝑥 = 𝑡𝑡−110

+ 𝑠𝑠−101

• One eigenvector for λ=1, two lin. ind. eigenvectors for λ=-2.• If there are not enough linearly independent eigenvectors, then A is not diagonalizable.

› Step 3: Construct P by collecting the eigenvectors:

• 𝑃𝑃 =1 −1 −1−1 1 01 0 1

, det 𝑃𝑃 = 2 ≠ 0 ⟹ P is invertible.

› Step 4: Construct D from the corresponding eigenvalues:

• 𝐷𝐷 =1 0 00 −2 00 0 −2

(same order as the eigenvectors!)

› Optional: make sure that AP = PD and that P is invertible.


› Example: Diagonalize 𝐴𝐴 =1 0 00 −2 10 0 −2

, if possible.

• Eigenvalues: 𝜆𝜆1 = 1 (alg. mult.=1), 𝜆𝜆2 = −2 (alg. mult.=2)• Eigenvectors:

› 𝜆𝜆1 = 1:0 0 00 −3 10 0 −3

∼0 1 00 0 10 0 0

⟹ 𝑣𝑣1= 𝑡𝑡100

› 𝜆𝜆2 = −2:3 0 00 0 10 0 0

∼1 0 00 0 10 0 0

⟹ 𝑣𝑣2 = 𝑡𝑡010

• Since we do not have enough eigenvectors, the matrix is not diagonalizable.

• In general, A is diagonalizable if and only ifdim Nul 𝐴𝐴 − 𝜆𝜆𝐼𝐼 = alg. mult. of 𝜆𝜆

for every eigenvalue λ. (Theorem 7 in textbook, proof omitted)


› Theorem 2: If v1,…, vr are eigenvectors corresponding to different eigenvalues λ1, …, λr, then the set {v1,…, vr} is linearly independent.

› Corollary (Theorem 6):If an nxn matrix A has n distinct eigenvalues, then it is diagonalizable.

› Proof: Consider the eigenvectors v1,…,vn corresponding to the distinct eigenvalues λ1,…, λn. By theorem 2, these vectors are linearly independent, so Theorem 5 implies that A is diagonalizable.

› Warning: If A has repeated eigenvalues, then A may or may not be diagonalizable. One has to compute the eigenspaces to find out.


› Proof of Theorem 2: By induction on r. The case r=1 is clear (since v1≠0). Assume the result holds for r-1, and consider the equation

𝑐𝑐1𝑣𝑣1 + ⋯+ 𝑐𝑐𝑟𝑟𝑣𝑣𝑟𝑟 = 0.

Multiply by A to get𝑐𝑐1𝐴𝐴𝑣𝑣1 + ⋯+ 𝑐𝑐𝑟𝑟𝐴𝐴𝑣𝑣𝑟𝑟 = 0𝑐𝑐1𝜆𝜆1𝑣𝑣1 + ⋯+ 𝑐𝑐𝑟𝑟𝜆𝜆𝑟𝑟𝑣𝑣𝑟𝑟 = 0

But we can also multiply by λr to get𝑐𝑐1𝜆𝜆𝑟𝑟𝑣𝑣1 + ⋯+ 𝑐𝑐𝑟𝑟𝜆𝜆𝑟𝑟𝑣𝑣𝑟𝑟 = 0

Subtracting from (*) gives𝑐𝑐1 𝜆𝜆1 − 𝜆𝜆𝑟𝑟 𝑣𝑣1 + ⋯+ 𝑐𝑐𝑟𝑟−1 𝜆𝜆𝑟𝑟−1 − 𝜆𝜆𝑟𝑟 𝑣𝑣𝑟𝑟−1 = 0

By induction hypothesis, {v1,…,vr-1} is linearly independent, so all the weights arezero. Thus,

𝑐𝑐𝑖𝑖 𝜆𝜆𝑖𝑖 − 𝜆𝜆𝑟𝑟 = 0 ⟹ 𝑐𝑐𝑖𝑖 = 0 (since λi ≠ λr by assumption)

We are left with crvr=0, which implies cr=0. So {v1,…,vr-1,vr} is linearly independent.


(*)

Similar Matrices› Recall: two square matrices A and B are similar if and only if there is an invertible matrix P such that A = PBP-1.

› Theorem: Let A and B be nxn similar matrices. Then A and B havei. the same determinant,ii. the same characteristic polynomial,iii. the same eigenvalues, with the same algebraic multiplicities.Moreover, the eigenspaces corresponding to the same eigenvalues have the same dimensions.

› Proof (Sketch): Note that𝐴𝐴 − 𝜆𝜆𝐼𝐼 = 𝑃𝑃𝐵𝐵𝑃𝑃−1 − 𝜆𝜆𝐼𝐼 = 𝑃𝑃 𝐵𝐵 − 𝜆𝜆𝐼𝐼 𝑃𝑃−1

Then (i)-(iii) follows from the fact thatdet 𝐴𝐴 − 𝜆𝜆𝐼𝐼 = det 𝑃𝑃 det 𝐵𝐵 − 𝜆𝜆𝐼𝐼 det 𝑃𝑃−1 = det 𝐵𝐵 − 𝜆𝜆𝐼𝐼 ,

whereas the last assertion comes from the fact that 𝐴𝐴 − 𝜆𝜆𝐼𝐼 and 𝐵𝐵 − 𝜆𝜆𝐼𝐼 have the same rank.


Non-diagonalizable matrices› A typical non-diagonalizable matrix is the Jordan matrix of the form

𝐽𝐽 =𝜆𝜆 1 0

𝜆𝜆 ⋱⋱ 1

0 𝜆𝜆 𝑟𝑟×𝑟𝑟

› Theorem (Jordan Canonical Form, optional):For every (complex) square matrix A, there exists an invertible (complex) matrix P such that

𝑃𝑃−1𝐴𝐴𝑃𝑃 =𝐽𝐽1

⋱𝐽𝐽𝑝𝑝

where J1,…,Jp are Jordan matrices. The matrix A is diagonalizable if and only if all the Jordan matrices are 1x1 (i.e., scalars).


› Self-testing questions: True or False?1. An nxn matrix A is diagonalizable if and only if it is similar to a

diagonal matrix.2. A real matrix always has real eigenvalues.3. If v is an eigenvector of A, then it is also an eigenvector of A3.4. If v is an eigenvector of A3, then it is also an eigenvector of A.5. If λ is an eigenvalue of A, then λ3 is an eigenvalue of A3. 6. If A is non-diagonalizable, then A has repeated eigenvalues.7. If A has repeated eigenvalues, then A is not diagonalizable.8. If u and v are eigenvectors of A belonging to the same

eigenvalue λ, then u and v are linearly dependent.9. If A is nxn, then its characteristic polynomial is of degree n.10. If A and B have the same set of eigenvalues with the same

algebraic multiplicities, then A and B are similar.


Date post:	31-Mar-2018
Category:	Documents
Upload:	lamdiep
View:	239 times
Download:	3 times

Eigenvalues and Eigenvectors - Hong Kong Baptist …felix_kwok/math2207/Ch5.pdf · ·...

Documents