Linear algebra: matrix Eigen-value Problems

Eng. Shubham Kumbhar

Part 3

Eigenvalue Problems

1. Eigenvalues and eigenvectors

2. Vector spaces

3. Linear transformations

4. Matrix diagonalization

The Eigenvalue Problem Consider a nxn matrix A Vector equation: Ax = x

» Seek solutions for x and » satisfying the equation are the eigenvalues» Eigenvalues can be real and/or imaginary; distinct and/or

repeated» x satisfying the equation are the eigenvectors

Nomenclature» The set of all eigenvalues is called the spectrum» Absolute value of an eigenvalue:

» The largest of the absolute values of the eigenvalues is called the spectral radius

22 baiba jj

Determining Eigenvalues Vector equation

» Ax = x (A-x = 0» A- is called the characteristic matrix

Non-trivial solutions exist if and only if:

» This is called the characteristic equation Characteristic polynomial

» nth-order polynomial in » Roots are the eigenvalues {1, 2, …, n}

0)det(

21

22221

11211

nnnn

n

n

aaa

aaaaaa

IA

Eigenvalue Example

Characteristic matrix

Characteristic equation

Eigenvalues: 1 = -5, 2 = 2

4321

1001

4321

IA

0103)3)(2()4)(1( 2 IA

Eigenvalue Properties Eigenvalues of A and AT are equal Singular matrix has at least one zero eigenvalue Eigenvalues of A-1: 1/1, 1/2, …, 1/n

Eigenvalues of diagonal and triangular matrices are equal to the diagonal elements

Trace

Determinant

n

jjTr

1

)( A

n

jj

1

A

Determining Eigenvectors

First determine eigenvalues: {1, 2, …, n} Then determine eigenvector corresponding to

each eigenvalue:

Eigenvectors determined up to scalar multiple Distinct eigenvalues

» Produce linearly independent eigenvectors Repeated eigenvalues

» Produce linearly dependent eigenvectors» Procedure to determine eigenvectors more complex (see

text)» Will demonstrate in Matlab

0)(0)( kk xIAxIA

Eigenvector Example Eigenvalues

Determine eigenvectors: Ax = x

Eigenvector for 1 = -5

Eigenvector for 1 = 2

31

or 9487.03162.0

03026

1121

21 xxxxxx

12

or 4472.08944.0

06302

2221

21 xxxxxx

25

4321

2

1

A

0)4(302)1(

432

21

21

221

121

xxxx

xxxxxx

Matlab Examples

>> A=[ 1 2; 3 -4];>> e=eig(A)e = 2 -5>> [X,e] = eig(A)X = 0.8944 -0.3162 0.4472 0.9487e = 2 0 0 -5

>> A=[2 5; 0 2];>> e=eig(A)e = 2 2>> [X,e]=eig(A)X = 1.0000 -1.0000 0 0.0000e = 2 0 0 2

Vector Spaces

Real vector space V» Set of all n-dimensional vectors with real elements» Often denoted Rn

» Element of real vector space denoted

Properties of a real vector space » Vector addition

» Scalar multiplication

Vx

0aawvuwvua0aabba

)()()(

aaaaaaababa

1)()()()(

kckcckkcccc

Vector Spaces cont. Linearly independent vectors

» Elements:» Linear combination:» Equation satisfied only for cj = 0

Basis» n-dimensional vector space V contains exactly n linearly

independent vectors» Any n linearly independent vectors form a basis for V» Any element of V can be expressed as a linear

combination of the basis vectors Example: unit basis vectors in R3

021 (m)(2)(1) aaa mccc V(m)(2)(1) aaa ,,,

3

2

1

3213321

100

010

001

ccc

cccccc )((2)(1) aaax

Inner Product Spaces Inner product

Properties of an inner product space

Two vectors with zero inner product are called orthogonal Relationship to vector norm

» Euclidean norm

» General norm

» Unit vector: ||a|| = 1

n

knnkk

T babababa1

2211),( bababa

0 ifonly and if 0),(0),(),(),(

),(),(),( 2121

aaaaaacca

cbcacba qqqq

222

21),( n

T aaa aaaaa

babababa ),(

Linear Transformation Properties of a linear operator F

» Linear operator example: multiplication by a matrix» Nonlinear operator example: Euclidean norm

Linear transformation

Invertible transformation

» Often called a coordinate transformation

)()()()()( xxxvxv cFcFFFF

AxyA

yx

tionTransformaOperator

,Elementsxnm

mn

RRR

yAxAxy

Ayx

1

x

tionTransforma InversetionTransforma

,,Dimensions

nnnn RRR

Orthogonal Transformations Orthogonal matrix

» A square matrix satisfying: AT = A-1

» Determinant has value +1 or -1» Eigenvalues are real or complex conjugate pairs with

absolute value of unity» A square matrix is orthonormal if:

Orthogonal transformation» y = Ax where A is an orthogonal matrix» Preserves the inner product between any two vectors

» The norm is also invariant to orthogonal transformation

bavuAbvAau ,

kjkj

kTj if 1

if 0aa

vbua

Similarity Transformations

Eigenbasis» If a nxn matrix has n distinct eigenvalues, the

eigenvectors form a basis for Rn

» The eigenvectors of a symmetric matrix form an orthonormal basis for Rn

» If a nxn matrix has repeated eigenvalues, the eigenvectors may not form a basis for Rn (see text)

Similar matrices» Two nxn matrices are similar if there exists a

nonsingular nxn matrix P such that:» Similar matrices have the same eigenvalues» If x is an eigenvector of A, then y = P-1x is an

eigenvector of the similar matrix

APPA 1ˆ

Matrix Diagonalization Assume the nxn matrix A has an eigenbasis Form the nxn modal matrix X with the eigenvectors

of A as column vectors: X = [x1, x2, …, xn] Then the similar matrix D = X-1AX is diagonal with

the eigenvalues of A as the diagonal elements

Companion relation: XDX-1 = A

nnnnn

n

n

aaa

aaaaaa

00

0000

2

1

1

21

22221

11211

AXXDA

Matrix Diagonalization Example

2005

21545

1321

71

1321

4321

1321

71

1321

71

1321

12

,23

1,5

4321

1

1

121

2211

AXXD

AXXD

XxxX

xxA

Matlab Example>> A=[-1 2 3; 4 -5 6; 7 8 -9];>> [X,e]=eig(A)X = -0.5250 -0.6019 -0.1182 -0.5918 0.7045 -0.4929 -0.6116 0.3760 0.8620e = 4.7494 0 0 0 -5.2152 0 0 0 -14.5343>> D=inv(X)*A*XD = 4.7494 -0.0000 -0.0000 -0.0000 -5.2152 -0.0000 0.0000 -0.0000 -14.5343