USTH Master linear algebra lecture 2

8/13/2019 USTH Master linear algebra lecture 2

1/24

Eigenvalues and Singular values

Juliette Ryan

Paris 13

February 2014


2/24

Singular values

Numerical Computation

Unitary matrix factorization, normal matrices

Theorem :For any A(N,N),A= UTU

where U is an unitary matrix (UU =UU=Id)and T is an upper triangular matrix

Definition :A is normal if AA

=A

ATheorem : A is normalthere exists U unitary matrixsuch thatA= UDU and D is diagonal

Corollary : A hermitian is diagonalizable

Corollary : A real symmetric is diagonalizable

Corollary : A hermitian matrix is positive definite alleigenvaluesi >0

Ryan Eigenvalues and Singular values


3/24


4/24

Singular values


Eigenvalue localization : Proof theorem 1

, eigenvalue, u eigenvector such thatmax|ui|= 1,with k such thatuk=1

( akk)uk =

j=1,j=kakjujthus(| akk|

j=1,j=k|akj|

andDkk not known but you are sure that k=1,...,NDk



5/24

Singular values



possible eigenvalue but not


Si l l


6/24

Singular values



not in

k=1,...,NDk, thus for any k,| akk| kisuch that| aii| i | aii|= iI={i, |ui|= 1= max|uk|} =, and

j=1,N,j=i|aij

||uj

| |(

aii)ui

|= i=

j=1,N,j=i|aij

|thus j=1,N,j=i|aij|(1 |uj|)0By definition|uj| 1 thus|aij|(1 |uj|) =0letJsuch thatJ

I={1, 2,...N}andJ I=

J=otherwise the matrix would be reducible and|akk|=|(akk)uk|=|k=jakjuj| k=j|akj|= k,for any k

k=1,...,NDk andk=1,...,NDk Dkfor all k


Si l l


7/24

Singular values


Dominant matrices

A(N,N)

Definitions

A diagonally dominanti=1,...N, |aii|

j=i|aij|A strictly diagonally dominant i=1,...N, |aii|> j=i|aij|A strongly diagonally dominant A is diagonallydominant andk such that|akk|>

j=i|akj|

Propositions

If A strictly diagonally dominant, A is invertible

If A strongly diagonally dominant and irreductible, A isinvertible

If A is either strictly diagonally dominant or irreductible and

strongly diagonally dominant and if all diagonal elements

aiiare real andaii >0 thei,Re(i)>0


Singular values


8/24

Singular values


Singular Values

(See http ://en.wikipedia.org/wiki/Singular value decomposition

and

Definition : A (M,N) rectangular matrix M>N. Singular

values of A(i)are the square roots0 ofAAeigenvalues (AAis (N,N) square matrix)

Theorem : A (M,N) rectangular matrix M>N. There exits

U(M,M) and V(N,N) unitary square matrices such that

UAV = where =

12

... N

andiare A singular values


Singular values


9/24

Singular values


Singular Values

The right singular vectors of A are eigenvectors of AA

The left singular vectors of A are eigenvectors of AA

Corollary: rank(A) = number of non zero valuesi


Singular values


10/24

Singular values


Pseudo inverse

Definition :Let =

12

...

N

,

the pseudo inverse of is

+ =

1 |2

|... | N |


Singular values


11/24

Singular values


Pseudo inverse

Definition :Let A =UV, the pseudoinverse of a isA+ =V+U

Note :If A is a square regular matrix, AA+ =A+A=Id


Singular values Finite precision computation


12/24

Singular values


Finite precision computation

Numerical instabilities

Real number representation

References :

http://docs.sun.com/source/806-3568/ncg_goldberg.html

http://www5.in.tum.de/huckle/bugse.html

Decimal representation : infinite computer memory ;

Approximated representation : floating point ;x m.bp withb: base,mmantissa,pexponent ;mantissa : number written as

m=0,a1a2. . .aN=N

k=1

akbk,b1 m


13/24

g


p p


Finite accuracy Arithmetics

Let real numbers represented with 3 significant digits

Computex+ y+ zwithx=8, 22 y=0, 00317 z=0, 00432

x+ y=8, 223178, 22(x+ y) + z8, 224328, 22y+ z=0, 00749

x+ (y+ z)8, 227498, 23Sum is not associative !




14/24

g


p p


Round off errors on arithmetics operators

Round off error on a sum

(x+ y)(|x| + |y|)

Ifsn=

nk=1

uk, thensnsn1+ sn(sn)(|un| + 2|un1| + 3|un2| + + (n 1)|u2| + (n 1)|u1|)Better accuracy starting the sum with small absolutevalue terms

Round off error on mutiply/Divide(xy)|xy|(x11 x

22 xnn )(|x11 |+|x22 |+ +|xnn )|x11 x22 xnn |




15/24

Numerical Computation Numerical instabilities

Solve equationx2 1634x+ 2 = 0

Computation is made with 10 significative digits.

Classical formulas :

=667487,

816, 9987760x1= 817 +

1633, 998776

x2= 817 0, 0012240

5 significative digits are lost onx2 !x1x2=2, thus

x2= 2x1

1, 223991125.103




16/24


Approximate computation ofe10

Use the seriese10

n

k=0

(

1)k 10k

k!

general term :|uk|= 10kk! ;|u9|=|u10|= 101010! 2, 755.103

e10 4, 5.105Comparisonu10ete

10 :

u10: 2 7 5 5, . . . . . .e10 : 0, 0 0 0 0 4 5

Loss of significative digits !

Cure :e10 = 1e10

withe10 n

k=0

10k

k!


Singular values

N i l C i


N i l i bili i


17/24


Recurrence computation

ComputeIn= 1

0

xn

10 + x, n N

Simple recurrence :

I0=ln

1110

In= 1n 10In1

Error :In10In110n

I0ComputationI361022 !Bound : 1

11(n+1) In 110(n+1)Inverse iterationIn1 =

110

1n In

Cure :In1 110 In;Approximate I46 11147ComputeI36from approximation ofI46:I361010 !


Singular values

N i l C t ti


N i l i t biliti


18/24


Iterative computation

Compute the sequenceu0=2, un+1=| ln(un)|u0 2,000000000 2,000000001 1,999999999 5.10

10

u5 5,595485181 5,595484655 5,595485710 9.108

u10 0,703934587 0,703934920 0,703934252 5.107

u15 1,126698502 1,126689382 1,126707697 8.106

u20 1,266106839 1,266256924 1,265955552 104

u24 1,000976376 1,001923276 1,000022532 103

u25 0,000975900 0,001921429 0,000022532 100%u26 6,932150628 6,254686211 10,700574400 50%

u30 0,880833175 0,691841353 1,915129896 100%

Numerical error :f(x) =|f(x)|xf(x)|f(x)| =

|f(x)||x||f(x)|

x|x| =

1| ln x|

x|x|


Singular values





19/24


Errors due to the condition number

SolveAx=betAy=b+ bwith

A=

10 7 8 77 5 6 58 6 10 97 5 9 10

,b=

32233331

, b=

0, 010, 01

0, 01

0, 01

x=

1

1

11

, y=

1, 82

0, 36

1, 350, 79

, 1x=y x=

0, 82

1, 36

0, 350, 21

Relative perturbation of3.104 error of0.8(2500).


Singular values





20/24


Errors due to the condition number. . .

SolveAx=bet(A + A)z=bwith

A=

10 7 8 77 5 6 58 6 10 97 5 9 10

,A=

0 0 0, 1 0, 20, 08 0, 04 0 0

0 0, 02 0, 11 00, 01 0, 01 0 0, 02

, b=

32233331

Solving the 2 systems :

x=

1

1

11

, z=

81137

3422

, 2x=zx=

82136

3521

Relative error of102 relative perturbation of80.


Singular valuesNumerical Computation

Finite precision computationNumerical instabilities


21/24


Analysis

det(A) = 1 ;

A1 =

25 41 10 641 68 17 10

10 17 5 3

6 10 3 2

Eigenvalues(i)i=1, ,4 ofA ?

mini=1, ,4

|i| 0, 01

maxi=1, ,4

|i| 30

maxi=1, ,4

|i|min

i=1, ,4|i| 3000= C

Coincidence ?

1x

2

x2 =0, 82C.

b2b2

=1





22/24


Error Analysis by right hand side perturbation

Recall perhaps : matrix norm induced from a vector norm

A=supx=0

Axx = supx=1

Ax

xsolution ofA.x=bety=x+ 1xsolution ofA.y=b+ b.

1x = A

1bb = A.x

1x A1.b

b A.x

1xx

AA1

bb





23/24


Error Analysis by perturbation of the matrix

xsolution ofAx=betzsolution of(A + A).z=b.

2x=A1

A(x+ 2x) A1

.A.x+ 2x2x

x+ 2xA.A1

AA





24/24

p

Definition of the condition number and properties

Definition

Cond(A) =AA1Properties

1 cond(A)12 cond(A) = cond(A1)

3 cond(A) = cond(A) ;=04 cond2(A) =

A

2

A1

2=

max

min

whereminandmaxare

respectively highest and lowest singular values of A


Date post:	03-Jun-2018
Category:	Documents
Upload:	agony1091
View:	219 times
Download:	0 times

USTH Master linear algebra lecture 2

Documents