[KTH - DN2266] Matrices and Linear Systems

8/13/2019 [KTH - DN2266] Matrices and Linear Systems

1/24

Mathematical Models - DN2266

Homework 1 - Matrices and Linear Systems

Teachers

Anna-Karin TORNBERG

Rikard OJALA

Authors

Pierre-Alexandre BEAUFORT

Hadrien VAN LIERDE

September 2013


2/24

Contents

1 Rank and eigenvalues of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 Rank of A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Eigenvalues of A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Apositive definite ? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Solving a linear system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.1 Form of the residuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32.2 First inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.3 Second inequality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Least squares approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4.1 Normal equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84.2 QR factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94.3 SVD factorization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104.4 Function mldivide inMATLAB . . . . . . . . . . . . . . . . . . . . . . . . . 124.5 Comparison of the four methods. . . . . . . . . . . . . . . . . . . . . . . 13

5 Singular value decomposition for image compression . . . . . . . . . . . . . . . . 15A Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

A.1 uh.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19A.2 LeastSquare.m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

0


3/24

1 Rank and eigenvalues of a matrix

Let us consider the following n n matrix Awith the elements:

aii= , i= 1,...,n, aij = 1, i =j (1)

1.1 Rank of A

Let us consider the n vectors formed by the columns a1,...,an ofA and see in which casesthese are linearly independent. Suppose we have 1,...,n such that:

1a1+...+nan = 0 (2)

In other words,1 + 2 + ... + n = 01 + 2 + ... + n = 0

...1 + 2 + ... + n = 0

(3)

If these equations are only satisfied for 1 = 2 = ...= n = 0 then the columns are linearlyindependent and the rank of the matrix is n. If these equations can be satisfied whereas i= 0for some i then the columns are linearly dependent and the rank of the matrix is strictly lessthan n.Assuming = 2 n, we do the following linear combinations of the equations 3, we denoteeach line by an index and the line li becomes:

li= 0 li 1+n

2

n

j=1,j=i

lj = 0 (4)

We obtain thus for the line i:

i( n1+n2) +n

j=1,j=ij(1 1+n2(+n 2)) = 0 i( n1+n2) = 0

(5)

Based on this, we obtain two cases:

if = 1 and = (n 1) then ( n1+n2

) = 0, it follows that 1 =2 =... = n = 0.The columns ofA are linearly independent and the rank of the matrix is n.

if = 1or = (n 1),( n1+n2) = 0, it follows that we can take some i non equal

to zero and the rank of the matrix is strictly less than n.

What is the rank of the matrix if= 1 or = (n 1) ? if = 1, then all the entries ofAare equal and nonzero, the rank ofAis obviously one.

if= (n1), we use an indirect method to calculate the rank: we consider the followingmatrix

S=

0 1 . . . 11 0 . . . 1... ... ... ...1 1 . . . 0

(6)

1


4/24

We may compute the eigenvalues of this symmetric (and thus diagonalizable) matrix byfinding for which value, S Iis singular. Sis only a special form of Awith = 0. Aswe showed above,1 is an eigenvalue of order n 1 as rank(S + I) = 1. We also knowthat n 1is an eigenvalue and its order is necessarily 1 as the order of the eigenvalue 1is already n 1. We then know that rank(S (n 1)I) =n 1 and rank(A) =n 1for =

(n

1).

What about = (n 2)? We can show that with this value of, Ais necessarily nonsingu-lar: if it was singular, then, using the same matrix Sas above, S (n 2)Iwould be singular.(n 2) would be an eigenvalue Sand this is absurd as 1 is an eigenvalue of order n 1 of Sand(n 1) of order 1.

To summarize these results:

if= 1,rank(A) = 1,

if= (n 1),rank(A) =n 1. else rank(A) =n,

1.2 Eigenvalues ofA

A value R is an eigenvalue of A if and only if A I is singular. This matrix is of thefollowing form:

1 ... 1

1 ... 1...1 1 ...

(7)

As we have seen previously, this matrix is singular in two cases: either = 1, then thematrix A Iis of rank 1 and the order of the eigenvalue is (n 1), either = (n 1),then the matrix A Iis of rank (n 1) and the order of the eigenvalue is 1. In other words, = 1 is an eigenvalue of order n 1, = + (n 1) is an eigenvalue of order 1.

1.3 A positive definite ?

We conclude directly that Ais positive definite if all its eigenvalues are positive, which meansif >1.

2


5/24

2 Solving a linear system

Let us consider A, a real symmetric n nmatrix with eigenvalues 0 < 1 2 ... n. Weuse the following iterative scheme to solve Ax= b, given an initial guess x0

xk+1 = xk rk, rk= Axk b, k= 0, 1,... (8)

Let ek = A1

b xk and = n1 , the condition number of A.

2.1 Form of the residuals

We have:ek+1 = A

1b xk+1= A1b xk+rk= A1b xk+(Axk b)= A1b

xk

A(A1b

xk))

= ek Aek= (I A)ek

(9)

As the identity matrix commutes with any other matrix, it commutes with A. Then weknow that the eigenvalues of B= I Aare 1 1, 1 2, ..., 1 n.

2.2 First inequality

Let us show thateTk+1ek+1

eTk ek max

1ln(1 l)2 (10)

We can develop this expression:

eTk+1ek+1

eTkek

= e

TkB

TBek

eTkek

= Bek

22

ek22

(11)

As Ais symmetric and B= I A, Bis also symmetric and thus diagonalizable. Let us note(v1, ..., vn), an orthonormal basis ofR

n made of eigenvectors ofB , with associated eigenvalues1 1, 1 2, ..., 1 n. We can rewrite ek as

ek =1

v1+...+n

vn (12)

for some real parameters 1,...,n. Then we observe that

Bek22 = 1(1 1)v1+...+n(1 n)vn22 1v1+...+nvn22max1ln(1 l)2= ek22max1ln(1 l)2

(13)

This is the result we wanted to obtain.

3


6/24

2.3 Second inequality

Let us find such that

ek+12 1+ 1

ek2 (14)

Let us consider the following value of:

= 1

1

1 n 1

2n 21

=

1if= 1 n 1

2n 21

(15)

We note that 0 1.

Before this point, it is strictly decreasing and after, it is stricly growing. Thus the maximumvalue is necessarily reached on the bounds of the domain, either in l =1, either in l =n.Let us show it is reached in 1 by proving the following inequality:

1 1

1

2= (1 )2

1 n

1

2(17)

We first show that 1

n

1 n12n

21

1n

1 1n

n12n

21

(18)

As the two members of this inequality are positive, we can square them:

(1 1n

)2 (n1)22n

21

1

2n 1

2n21

(19)

And this proves the result shown by equation 18. We are now able to prove the assertion ofequation17. As 1

n, 1 n

1 0, we can take the square root of the two members of the

equation17and we obtain

1 1 n1

(20)

This inequality is verified if n1

1which is verified by definition. The maximum value is thengiven by

(1

)2 = (n1)

2

2n21

= n1n+1= 1

+1

(21)

4


7/24

We show then thatek+122 ek22 max

1ln(1 l)2

1+1

ek2 (22)

This result shows us that choosing an appropriate value of increases the speed of convergenceof the iterative scheme. In particular, if all eigenvalues of A are equal, then the algorithm

converges after one step. For a diagonalizable matrix in general, the lower the condition numberof the matrix, the faster the scheme converges1.

1The condition number of a diagonalizable matrixA is given by max(A)min(A)

.

5


8/24

3 Approximation

Here we approximate the function 1x

on the interval[1; 2]with a linear combination of the func-tionsexp(x),sin(x)and (x). We want this approximation to be as close as possible accordingto the norm L2.2

However, we are going to use a discrete approximation of the norm L2[1;2] to build the approxi-mation :

||f(x)||2L2[1;2]

1n 1

ni=2

f

xi+xi12

2(23)

where n >3 3 is the number of points we use to discretize the interval [1; 2]; these points areequally distant (but the function is evaluated inn1points, one in each interval). The discreteapproximation ofL2[1;2] is built thank to the approximation of the integral with a Riemanns

sum4. As||1||L2[1;2]

= 1 and xi = xi, we have multiplied the Riemanns sum by a factor1

(n1)x.

Now, we are able to build the approximation of 1x

by using our discrete approximation ofL2[1;2].

Obviously, we want to minimize the error || 1x (aexp(x) +bsin(x) +c(x))||L2

[1;2].

Thus, with our discrete norm (23) :

mina,b,cR

ni=2

2

xi xi1

aexp

xi xi12

+bsin

xi xi1

2

+c

xi xi1

2

2

We know this problem admit a single minimum (if there are enough points of discretization).Therefore it is sufficient to compute the partial derivatives and determine when they are equalto zero :

ATAx= ATb

(24)

with :

A :=exp

xj+1xj

2

sin

xj+1xj

2

xj+1xj

2

row j s.t. 1 j n 1

b :=

2xj+1xj

row j s.t. 1 j n 1

x :=

ab

c

(25)

We obtain a linear system (24) to solve in order to compute the coefficients of the linear com-bination of the functions exp(x), sin(x) and (x) such that the approximation minimizes thenorm (23) of the error.

In order to solve this linear system, we wrote a MATLABprogram (cf. appendixA.1 uh.m) thatcomputes xwith at least 3 correct significant digits :

x= [0.107 0.00918 1.28]T

2||.||L2

=

(.)2dx3If n=3, the system (24) is near to be singular.4ba

f(x)dx ni=2f(xi+xi12 )xi , such that{x1= a; x2= x1+ x2 ; ...; xn1= xn xn ; xn= b}6


9/24

1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 20.4

0.6

0.8

1

1.2

x

y

Approximation obtained with 6 points for the discretization of [1;2] and displayed here for 100 points

1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 23

2

1

0

1x 10

3

x

Errorvalue

Error for each point

1/x

approximation

Figure 1: The upper graph displays 1x

in green versus the approximation in red. The othergraph displays the error evaluated in each point.

Figure 1 displays the result of this implementation. We may calculate the resulting erroraccording to the approximate norm L2, evaluating the function in a thousand points5. The

error is2.5

103

. To see if we could get a beter approximation using the same three functions,we runned the program and computed the results with at least eighteen significant digits. Theerror is approximately 6 104. This shows that we cannot fit the three functions perfectlybut, with three significant digits, we already reached a good precision6.

5The points chosen to evaluate the error have nothing to do with the points we chose for the discretization.6Increasing a lot the number significant digits does not reduce the error significantly.

7


10/24

4 Least squares approximations

Let us define t as a vector ofm uniformly spaced points from 0 to 1. Given a function f, wewant to fit a polynomial of degree n 17 using the m data points (ti, f(ti), i= 1,...,m. To thatend, we introduce the following notation:

A=

1 t1 t

2

1

... tn1

11 t2 t22 ... tn12

...1 tm t

2m ... t

n1m

and b=

f(t1)f(t2)

...f(tm)

(26)

We find the least square coefficients vector by finding an approximate solution of the equationAx = b, in other words, we find x that minimizesAx b2. We let f(t) = cos(4t) andm= 50, n= 12. We then compute the coefficients using four different methods.

4.1 Normal equations

We can compute analytically the vector that minimizesAx b2:

argminj

mi=1

(f(ti) n

j=1

jtj1i )

2 (27)

We need the derivative of the left term of this equation with respect to j, j = 1,...,n to beequal to zero. The equations that have to be satisfied are :m

i=1tk1i (f(ti)

nj=1jt

j1i ) = 0

m

mi=1ti

m

i=1t

n1

i... ...

. . . ...m

i=1tn1i

mi=1t

ni

mi=1t

2n2i

1...

n

=m

i=1tif(ti)...mi=1t

n1i f(ti)

(28)

These equation can be summarized by the following form:

ATAx= ATb (29)

UsingMATLABfunctionA.2, we find the following coefficients and error (given by Axb).Thegraph of the solution obtained is given by the figure2.

=

999.999970929243e 0038.40732720203423e 006

8.00031246249825e+ 0004.55777147648062e 00310.6321201219237e+ 000154.480483584900e 003

6.12408716819671e+ 000798.025601096974e 003661.926103693841e 003765.336512704183e 003

690.872291676884e 003145.173350125393e 003

and error= 136.744782296283 109 (30)

7n 1< m

8


11/24

0 0.2 0.4 0.6 0.8 11

0.8

0.6

0.4

0.2

0

0.2

0.4

0.6

0.8

1

t

fandpolynomial

Figure 2: Graph obtained for the first method (normal equations) for m=50 and n=12.

4.2 QR factorization

We introduce the following factorization: A= QR whereQ is anm nmatrix with orthogonalcolumns and R is an n n matrix, non-singular and upper triangular. The normal equationsthen become:

ATAx= ATb (QR)TQRx= RTQTb RTRx= RTQTb Rx= Q

T

b

(31)

Using MATLAB functionA.2, we find the following coefficients and error (given byAx b).With MATLAB, we use the command qr(A, 0)to calculate the "economy size" decomposition (the

9


12/24

full QR factorization is unuseful). The graph of the solution obtained is given by the figure 3.

=

1.00000000099661e+ 000422.742971234461e 0097.99998123568819e+ 000

318.763196927325e

006

10.6694307956399e+ 00013.8202868522896e 0035.64707563050777e+ 00075.3160186619973e 0031.69360695695264e+ 0006.03211344200864e 003

374.241705346037e 00388.0405764032993e 003

and error= 7.99915456681444 109 (32)

0 0.2 0.4 0.6 0.8 11

0.5

0

0.5

1

1.5

t

fandpolynomial

Figure 3: Graph obtained for the second method (QR factorization) for m=50 and n=12

4.3 SVD factorization

We introduce the following factorization: A = UVT where U and V are matrices with or-thonormal columns and is a diagonal matrix with the "singular values" (i) of A in the

10


13/24

diagonal. The normal equations then become:

(UVT)T(UVT)x= (UVT)Tb VTUTUVTx= (UVT)Tb TVTx= TUTb

(33)

Let us assume that the columns ofA are linearly independent. Then all singular values ofAare strictly positive and is non-singular. We can thus solve the problem in the following way:

1. solvey=UTb

2. computex = V y

Using MATLAB functionA.2, we find the following coefficients and error (given byAx b).The graph of the solution obtained is given by the figure 4.

=

1.00000000099661e+ 000422.742984376510e 0097.99998123568772e+ 000318.763203314361e 00610.6694307956869e+ 000

13.8202870626617e 0035.64707562990608e+ 00075.3160197864053e 0031.69360695831683e+ 0006.03211240787543e 003

374.241704901470e 00388.0405763206151e 003

and error= 7.99915402401412 109 (34)

11


14/24

0 0.2 0.4 0.6 0.8 11

0.5

0

0.5

1

1.5

t

fandpolynomial

Figure 4: Graph obtained for the third method (SVD factorization) for m=50 and n=12

4.4 Function mldivide inMATLAB

We can also use the function \ofMATLAB (mldivide), as the following code lines:

1 a l ph a = A \ b ;

Using MATLAB functionA.2, we find the following coefficients and error (given by

Ax

b

).

The graph of the solution obtained is given by the figure 5.

=

1.00000000099661e+ 000422.743086756281e 0097.99998123568417e+ 000318.763247153980e 00610.6694307959661e+ 000

13.8202881049225e 0035.64707562748576e+ 00075.3160233538701e 0031.69360696161712e+ 0006.03211058750612e 003

374.241704373879e 00388.0405762637911e 003

and error= 7.99915374376384 109 (35)

12


15/24

0 0.2 0.4 0.6 0.8 11

0.5

0

0.5

1

1.5

t

fandpolynomial

Figure 5: Graph obtained for the fourth method (mldivide) for m=50 and n=12

How does this method (the function mldivide in MATLAB) is coded ? Actually, it was codedusing the QR-factorization8.

4.5 Comparison of the four methods

Obviously, the method that produces the worst result is the first one (the error is 106 timesbigger than in the other cases). To understand this, we observe that in each case, the problemof finding the least square coefficients is solved by solving another system ofn equations with

nunknowns. We can compute the condition number of this matrices to see how large they are.We find in the four cases:

1. normal equations: c(ATA) = 13.4831962182673 1015

2. qr factorization: c(R) = 117.177656148794 106

3. svd factorization: c() = 117.177656148794 106

4. mldivide: c(A) = 117.177656148794 106.

We observe that qr, svd and mldivide give exactly the same condition number. The definition

of the condition number of a square matrix is given by c(A) = AA1. But in general when8 See the documentation given by : ANONYMOUS, mldivide, mrdivide,

http://www.maths.lth.se/na/courses/NUM115/NUM115-11/backslash.html.

13


16/24

the matrix is not squared, we use the following definition: c(A) = max(A)/min(A), the ratio ofthe largest and smallest singular values of A (this definition is equivalent to the first one forsquare matrices). We can then observe the three following properties :

1. Aand (as defined in the SVD method) have the same condition number ; this result isan obvious consequence of the definition of.

2. A and R have the same condition number : let us prove this result, given Q and R, theQR factorization ofA, Let us assume that is a singular value ofR. Then there existsunit length vectors uand v Rn such that:

Rv = uRTu = v

(36)

We can multiply the first equation by Q, we then obtain

QRv= Qu (37)

In the second equation, we can write

RTQTQu= v (QR)TQu= v (38)As Q is orthogonal,Qu =u = 1. We now see that the equations37and38 showthat is also a singular value ofA9

3. The singular values ofAare the square roots of the eigenvalues ofATA. AsATAis squareand diagonalizable, c(ATA) = max(ATA)/min(ATA). As a consequence, c(ATA) = c2(A).We can actually calculate that this result is almost perfectly verified in this case.

We know that "the computer algorithms for solving the linear system looses about log(c) dec-imals to round off error"10. Thats why the first method used gives a less accurate result. Asexplained in the same document, it is better to use QR or SVD algorithms. However, the SVDalgorithm is theoretically more stable if A is rank defficient (which is not very likely in this caseifm is large compared to n).

Finally, we may calculate the time of execution of the four algorithms:

1. Normal equations: 0.000092 seconds.

2. QR factorization: 0.000075 seconds.

3. SVD factorization: 0.000281 seconds.

4. mldivide: 0.000064 seconds.

We see that the three methods are approximately as fast, except the SVD factorization. Astheoretically expected, this method is supposed to be more stable but it is less time efficientbecause the program that computes the SVD factorization takes more time to be executed. Wealso observe that the times of execution of the QR factorization and mldivide are very close,as expected.

9In a similar way, we can prove that any singular value ofAis a singular value ofR.10TORNBERG (A.-K.), DN2266, Mathematical Models, Analysis and Simulation, Royal Institute of Tech-

nology, Fall semester, 2013.

14


17/24

5 Singular value decomposition for image compression

The singular value decomposition11 of a m n matrixAconsists in A = UVT, with : an m n matrix U = [u1 u2 ... un] with{ui} forming a basis for the column space

of A

an n nmatrixV = [v1 v2 ... vn] with{ui} forming a basis for the row space of A an n nmatrix =diag(i) such that Avi =iui and ATui =ivi

Here, we use this decomposition for a 256 256 matrix basketpic.mat that represents animage with gray levels [0; 255]:

basketpic.mat= UVT

1 l o a d ( b a s k e t p i c . m a t );

2 i m a g e s c ( b a s k e t p i c ) ;

3 c o l o r m a p ( g r a y ) ;

Listing 1: With the following MATLAB lines we can display the image represented by

basketpic.mat. See figure6.

50 100 150 200 250

50

100

150

200

250

Figure 6: Image corresponding to the matrix basketpic.matand produced by listing1.

First, we observe that the matrix basketpic.mathas the following property about the Frobe-nius norm12: m

i=1

nj=1

(basketpic.mat)2ij =

ni=1

2i (39)

We check this property (39) with the listing2.


2 Amat=double(basketpic);

3 [ U ,S , V ] = sv d ( A m a t ) ;

4 n o rm F 1 = s q r t (sum( A m a t ( : ) . ^ 2 ) )

5 n o rm F 2 = s q r t (sum( d i a g (S).^2))

Listing 2: MATLABcode computes left and right members of (39) and returns for both3.399e+04.11SVD12||.||Fro =

mi=1

nj=1(.)

2ij

15


18/24

Actually, this property is true for every m nmatrix A. See proof Frobenius norm.Proof - Frobenius norm :

Let a m n matrix A.We use a SVD for A : A= UVT13 and compute ATA= VUTUVT.As the columns ofU forms a basis for the colum space of A, then the columns of Uare orthogonalbetween them. ThereforeUTU=Iand thus ATA= V2VT.

We observe that 2i are eigenvalues ofATA and thus, we know that trace(ATA) =

ni=12i .Besides,trace(ATA) =

nk=1diag(A

TA)k.Let(A)ij =aij and (A

T)ij =ij . We know ij =aji .Then, ATA=

nj=1ijajk =

nj=1ajiajk and thus diag(A

TA)i =n

j=1ajiaji =n

j=1a2ji .

We obtain trace(ATA) =m

i=1

nj=1a

2ij .

Finally, as trace(ATA) andn

i=12i are both positive, we can write : m

i=1

nj=1

a2ij =

ni=1

2i

On the figure 7, we can observe the singular values i ofbasketpic.mat according to theirposition on the diagonal of.

0 50 100 150 200 250 30010

2

101

100

101

102

103

104

105

Singularvalue

ith entry on the diagonal of Sigma

Singular values according to their position on the diagonal of Sigma

Figure 7: Singular values ofbasketpic.mataccording to their position on the diagonal of.

13We remind that is a diagonal matrix.

16


19/24

Now, we are going to underline the interest of SVD for image compression, by using the rankk approximation ofAwhich we define as:

Ak =k

i=1

iuiviT

We use this approximation for matrix basketpic.matin a MATLAB code (cf. listing3).



3 [ U ,S , V ] = sv d ( A m a t ) ;

4 Vt = V ;

5

6 f o r k = 1 : 2 56

7

8 A k = U ( : , 1: k ) * S ( 1: k , 1: k ) * V t ( 1: k , :) ;

9

10 f i g u r e()

11 i m a g e s c ( A k ) ;

12 c o l o r m a p ( g r a y )

13 t i t l e ( [ A p p ro x im a ti o n o f r an k num 2str ( k ) ] ,Fontsize ,15)14

15 en d

Listing 3: MATLAB code computes and displays approximation Ak. The result begins to be

nearly perfect when k is about 25 (see figure 8).

Relative error = 0.098581 , for k = 25

50 100 150 200 250

50

100

150

200

250

Figure 8: Approximation A25 of figure6.

Finally, we observe the error due to this approximation of rank k. We use the relative error inthe Frobenius norm :

Ek =||Ak A||Fro

||A||FroOn the figure9, we can observe the relative error for different values ofk.

17


20/24


50 100 150 200 250

50

100

150

200

250


50 100 150 200 250

50

100

150

200

250


50 100 150 200 250

50

100

150

200

250

Relative error = 2.4495e15 , for k = 256

50 100 150 200 250

50

100

150

200

250

Figure 9: Approximations and relative errors for different value ofk . This subplot is producedby listing4



3 n o rm F 1 = s q r t (sum( A m a t ( : ) . ^ 2 ) )

4 [ U ,S , V ] = sv d ( A m a t ) ;

5 Vt = V ;

6

7 A _k = z e r o s (s i z e(Amat));

8

9 i=0;

10 f o r k = [1 1 00 1 75 2 56 ]

11 i=i+1;

12

13 A k = U ( : ,1 :k ) * S ( 1: k ,: ) * V t (: , 1: k );

14 A _k ( : , 1: k ) = A k ;

15 E = s q r t (sum((A_k(:)-Amat(:)).^2))/normF1;

16

17

s u b p l o t (2,2,i)18 i m a g e s c ( A k ) ;

19 c o l o r m a p ( g r a y ) ;

20 t i t l e ( [ R e la t iv e e r ro r = num 2str (E ) , for k = num 2str ( k ) ] , F o n t s i z e ,15)

21

22 en d

Listing 4: MATLABcode computes approximations of rank k - displays the resulting images and

returns the relative error according to the Frobenius norm (see figure 9).

The preceding results show us that we can use SVD factorization for image compression. The

quality of the compression is quite good : with only 10% of the information, we get an imagenearly perfect ! Actually, the error14 decreases fastly.

14according to Frobenius norm

18


21/24

A Appendix

A.1 uh.m

1 % D N 22 6 6 - H o me w or k 1

2 % Q u es t io n 3 , P ie rr e - A l e xa n dr e B E AU F OR T & H a dr i en V AN L I ER D E

3 %%4 % T hi s f u nc t io n c o mp u te s t he c o ef f ic i en t s o f t he l i ne a r c o mb i na t io n o f

5 % e xp ( x ) , s i n ( x ) a n d G a mm a ( x ) t h a t a p pr o xi m at e s 1 / x t h e c l os e st a s p o ss i bi l e

6 % a cc or di ng t o t h e n o rm L ^2

7 % T he f u nc t io n p lo t 1 / x v s t he a p pr o xi m at i on , t he e rr o r i n e ac h p o in t s a nd

8 % r et ur ns t he v al ue o f t h e n or m o f t h e e rr or a nd t he c oe ff ic ie nt s [ a b c ]

9 % w it h a t l e as t t h re e c o rr e ct s i gn i fi a nt d i gi t s .

10 f u n c t i o n [X , e r r o r ] = uh ()

11

12 % i n i t i l i z a t i on

13 n = 4;

14 x = l i n s p a c e(1,2,n);

15

16 % n o rm a l s y s t em

17 z = ( x( 2: en d ) + x (1: end -1))/2;

18 A = [ ex p ( z ) s i n ( z ) gamma(z)];

19 b = 1 ./ z ;

20

21 C = A * A;

22 d = A * b;

23 X ne w = C \ d;

24 y = -f l o o r( l o g 1 0 ( ab s ( X n e w ) ) ) ;

25

26 % i t e r a t i v e c o m p u t a t i on

27 X ol d = X ne w + 1;

28 w h i l e m ax( a bs ( X ne w - X ol d ) . * [ 10 ^( y ( 1) ) 1 0^ ( y (2 ) ) 1 0 ^( y (3 ) )] ) >= 1 e -3 0 & & n < 1 00 0

29

30 n = n + 1;

31 X ol d = X n ew ;

32

33 x = l i n s p a c e(1,2,n);

34 z = ( x( 2: en d ) + x (1: end -1))/2;

35 A = [ ex p ( z ) s i n ( z ) gamma(z)];

36 b = 1 ./ z ;

37 C = A * A;

38 d = A * b;

39 X ne w = C \ d;

40

41 y = -f l o o r( l o g 1 0 ( ab s ( X n e w ) ) ) ;

42

43 en d

44 % c o ef f ic i en t s f or t he l i ne a r c o mb i na t io n

45 X = X ne w;

46

47 % e v al u at i on o f t h e e r ro r i n 1 0 0 0 p o i n ts a c co r di n g t o L ^ 248 z p ri m = l i n s p a c e(1,2,1000);

49 B = [ ex p ( z p r i m ) s i n ( z p r i m ) gamma(zprim)];

50 b p r im = 1 . / z pr i m ;

51 e r r o r = 1 /( n -1 ) * sum( ( b pr i m - B * X ) .^ 2 ) ;

52 e r r o r = s q r t ( e r r o r );

53

54 %display

55 y = l i n s p a c e(1,2,100);

56 s u b p l o t (2,1,1)

57 p l o t (y,1./y, -g , l i n e w i d t h ,5); h o l d on

58 p l o t (y,[ ex p ( y ) s i n ( y ) gamma( y ) ] * X , . r , Markersize ,10)

59 x l a b e l ( x , F o n t s i z e ,10)

60 y l a b e l ( y )

61 l e g e n d ( 1 / x , a p p r o x i m a t i o n )

62 t i t l e ( [ A p p r o x i ma t i o n o b t a i n e d w i t h num 2str (n-1) p o in t s f or t he d i sc r e ti z at i on o f [ 1 ;2 ]

a nd d i sp l ay e d h er e f or 1 00 p o in t s ]) ;

63 g r i d on

64 s u b p l o t (2,1,2)

19


22/24

65 p l o t ( y ,1 ./ y - [ ex p ( y ) s i n ( y ) gamma( y ) ] * X ) ;

66 x l a b e l ( x )

67 y l a b e l ( E r r o r v a l u e )

68 t i t l e( E rr o r f or e ac h p o in t )

69 g r i d on

70

71 en d

20


23/24

A.2 LeastSquare.m

1 f u n c t i o n [ c , c n , e r r o r ]=LeastSquare(m,n,method)

2 % L EA ST - S Q U A RE : f i nd s t he l e as t s q ua r es c o ef f ic i en t s f or a g i ve n f u nc t io n

3 % f i s d e fi ne d a s a n i n te rn al f un ct io n ( h er e , f ( t) = co s (t ) )

4 %

5 % @INPUT :

6 % 1) m i s t he n um be r o f d a ta p oi nt s ( t ,f (t )) u se d t o c o mp ut e t he7 % c oe ff ic ie nt s ,

8 % 2 ) n -1 i s t h e d e gr ee o f t h e p o ly no mi al u se d t o a p pr ox im at e f ,

9 % 3 ) m et ho d a l lo ws t o c ho os e b e tw ee n f o ur m et ho ds t o c om pu te t he

10 % c oe ff ic ie nt s : m et ho d 1 u se s t he t he l ea st n or ma l e qu at io ns , m et ho d 2

11 % u se s t he Q R f a c to ri za ti on , m et ho d 3 t he S VD f ac to ri z at io n a nd m et ho d 4

12 % t he f un ct io n ml di vi de .

13 %

14 % @OUTPUT :

15 % 1 ) c c on ta in s t h e c o e ff ic ie nt s c om pu te d ,

16 % 2) c n i s t he c on di ti on n um be r o f t he m at ri x ( d en ot ed by R )

17 % o f t he l i n ea r s y st em u se d t o c o mp ut e t he s ol ut io n ( i t d ep en ds o n t h e

18 % m et ho d use d) ,

19 % 3) e rr or i s t he n or m o f t he e rr or n or m( A* c- b) if th e s ys te m t o b e

20 % s ol ve d is of t he f or m A* x= b.

21

22

23 t= l i n s p a c e(0,1,m);

24 A= z e r o s ( m , n ) ;

25 f o r i = 1 : m

26 A ( i , 1 : n ) = t ( i ) . ^ ( 0 : n - 1 ) ;

27 en d

28 b=f(t);

29 f o r m a t longeng

30

31 t i c

32 % % M E T H OD 1 : N o rm a l e q u a ti o ns

33 i f method==1

34 R = A * A ;

35 c = R \ ( A * b ) ;36 en d

37

38 % % M E T H OD 2 : Q R f a c t o ri z at i on

39 i f method==2

40 [Q,R]= q r (A,0);

41 B = Q * b ;

42 c=R\B;

43 en d

44

45 % % M E T H OD 3 : S VD f a c to r iz a ti o n

46 i f m e t ho d = = 3

47 [U,R,V]= sv d ( A ) ;

48 y = R \ ( U * b ) ;

49 c=V*y;

50 en d51

52 % % M ET HO D 4 : L ea st s qu ar e i n M at la b

53 i f method==4

54 c=A\b;

55 R=A;

56 en d

57 t o c

58

59 % % G r a ph

60 M=50;

61 mes=l i n s p a c e(0,1,M);

62 y= z e r o s (1,M);

63 f o r i = 1 : M

64 y ( i ) =sum(c.*(mes(i).^(0:n-1)));

65 en d

66

67 p l o t ( m e s , f ( m e s ) ,r )

68 h o l d on

21


24/24

69 p l o t ( m e s , y ,o )

70

71 % % E r ro r a nd c o nd i ti o n n u mb e r

72 e=b-A*c;

73 e r r o r =norm (e);

74 cn = cond ( R ) ;

75

76 % % F u n c ti o n a p p r o xi m a t e d

77 f u n c t i o n y = f ( t )78 y= c o s (4*t);

79 en d

80

81 en d

Date post:	04-Jun-2018
Category:	Documents
Upload:	lambdastudentepl
View:	224 times
Download:	0 times

[KTH - DN2266] Matrices and Linear Systems

Documents