NumericalLinearAlgebra - Lunds universitetctr.maths.lu.se/na/courses/FMNN01/course_media/... ·...

Numerical Linear AlgebraUnit 5: Orthogonalization: QR

Numerical Analysis, Lund UniversityClaus Führer and Philipp Birken

Numerical Analysis, Lund University, 2018 48

Range of A

Let A ∈ Rm×n and full rank, with m ≥ n

(see overdetermined linear systems).

Goal: Find an orthogonal basis of Rn = range(A)

Idea: Produce a chain of subspaces and a basis

span(a1)︸︷︷︸=:R1

⊂ span(a1, a2)︸︷︷︸=:R2

⊂ · · · ⊂ span(a1, a2, . . . , an)︸︷︷︸=:Rn


Range of A: orthogonal basis

span(a1)︸︷︷︸=:R1

⊂ span(a1, a2)︸︷︷︸=:R2

⊂ · · · ⊂ span(a1, a2, . . . , an)︸︷︷︸=:Rn

We aim for an orthogonal basis Rj with:

Rj = span(q1, q2, . . . , qj) = span(a1, a2, . . . , aj)

(a1 a2 . . . an

)︸︷︷︸

=A

=(q1 q2 . . . qn

)︸︷︷︸

=:Q

r11 r12 . . . r1n

r22 . . . r2n. . . ...

rnn

︸︷︷︸

=:R


Range of A: reduced QR factorization

a1 = r11q1a2 = r12q1 + r22q2

...an = r1nq1 + r2nq2 + · · ·+ rnnqn

A = QR


Range of A: full QR factorization

Complete Q by m − n additional orthogonal columns and R by a(m − n)× n zero matrix:

A Q R=


Gram-Schmidt orthogonalization

Algorithm to compute Q and R:

Step j: Given: q1, ..., qj−1 orthonormal basis of span(a1, ..., aj−1).Find: qj ∈ span(a1, . . . , aj) with qj ⊥ qi

vj = aj − qT1 aj q1 − qT

2 aj q2 − · · · − qTj−1aj qj−1

andqj = vj

‖vj‖2


Gram-Schmidt orthogonalization (Cont.)

Set rĳ = qTi aj for i 6= j and rii = ‖vi‖2

Compute q1, . . . , qj from

v1 = a1v2 = a2 − r12q1

...

vn = an −n−1∑i=1

rinqi

and

qi = virii


Gram-Schmidt orthogonalization - Projections

Note, rĳqi = qiqTi aj is a projection of aj onto span(qi).

Thus, Gram-Schmidt orthogonalization can be written as asequence of projections:

vj = aj − Pq1aj − Pq2aj − . . .Pqj−1aj = (I− Qj−1QTj−1)︸︷︷︸

=:Pj

aj

Problem: Not a very good method (see assignment). Look foralternative idea!


Householder reflection in 2D

Reflection across span(v)⊥


Householder triangularization (basic idea)

The idea:Select a vector v so that H reflects a onto the coordinate axis!

Ha = H

a1a2...am

︸︷︷︸

a

=

‖a‖20...0

︸︷︷︸

a

.

This leads to v = a − a.

(see also the figure in the next slide) Check that Ha = ‖a‖e1.


Householder triangularization (Cont.)Householder transformation for annihilating (nollställa) entries in avector


A Matlab code for a single Householder step

function [Q]=householder(A)% [Q]=function house(a)% computes a Householder matrix to% introduce zeros in the first column of A% by an orthogonal transformation%m = sizedim(A,1)a=A[:,1]la=norm(a)ahat=[la;zeros(n-1,1)]v=a-ahat;v=v/norm(v)Q=eye(n)-2*(v*v’)/(v’*v);


A Python code for a single Householder step

extendedcharsextendedcharsextendedcharsextendedcharsextendedchars extendedchars extendedcharsfrom numpy import array , eye , outerextendedcharsextendedcharsextendedchars extendedchars extendedcharsfrom scipy. linalg import normextendedcharsextendedcharsextendedchars extendedchars extendedcharsextendedcharsextendedcharsextendedchars extendedchars extendedcharsdef Householder_step (A):extendedcharsextendedcharsextendedchars extendedchars extendedchars" " "extendedcharsextendedcharsextendedchars extendedchars extendedcharsOne Householder stepextendedcharsextendedcharsextendedchars extendedchars extendedcharsExample code forextendedcharsextendedcharsextendedchars extendedchars extendedcharsFMNN01 / NUMA11 Numerical Linear Algebraextendedcharsextendedcharsextendedchars extendedchars extendedchars" " "extendedcharsextendedcharsextendedchars extendedchars extendedcharsa=A[:,0]extendedcharsextendedcharsextendedchars extendedchars extendedcharsm=A.shape[0]extendedcharsextendedcharsextendedchars extendedchars extendedcharsahat=array ([ norm(a)]+(m-1)*[0.])extendedcharsextendedcharsextendedchars extendedchars extendedcharsv=a-ahatextendedcharsextendedcharsextendedchars extendedchars extendedcharsv=v/norm(v)extendedcharsextendedcharsextendedchars extendedchars extendedcharsQ=eye(m)-2*outer(v,v)extendedcharsextendedcharsextendedchars extendedchars extendedcharsreturn Q

extendedcharsextendedchars


Multiplication with a Householder matrix

Multiplying with a Householder matrix is cheap:

Ha = (I− 2 vvT

vTv )a = a − 2vTv (vTa)v

Thus we never need to form and store this matrix!


First step

>> A=[1,2;4,5;7,8]>> H1=housholder(A);A1=H1*AA1 =

-8.1240 -9.60110.0000 -0.08600.0000 -0.9004


Second step

We look now for a matrix H2, which transformsA1(2 :, 2) into the vector −9.6011a2

0


Second step (Cont.)

We formH2 =

(1

H22

)

where H22 is a 2× 2 matrix whichreflects

(−0.08600.9004

)onto

(a20

)The same procedure as before but in one dimension less!


Second step (Cont.)

>> A1=H1*A % first stepA1 =

-8.1240 -9.60110.0000 -0.08600.0000 -0.9004

>> H22=householder(A1(2:end,2:end));H22 =

-0.0950 -0.9955-0.9955 0.0950

>> H2=[1 0 0 ;[0 0]’,H22]H2 =

1.0000 0 00 -0.0950 -0.99550 -0.9955 0.0950

>> A2=H2*A1 % second stepA2 =

-8.1240 -9.60110 0.9045

-0.0000 0.0000


Rank of a Matrix

The rank of a matrix is the number of non-zero diagonal elementsin R:

>> AS=[1,2,3;1,2,3;2,5,7]AS =

1 2 31 2 32 5 7

>> [Q,R]=qr(AS)Q = R =

-0.4082 -0.5774 -0.7071 -2.4495 -5.7155 -8.1650-0.4082 -0.5774 0.7071 0 0.5774 0.5774-0.8165 0.5774 0.0000 0 0 -0.0000

>> nnz(diag(R)) %Wrong result >> nnz(find(abs(diag(R))>1.e-15))ans = %due to round-off ans =

3 2


Null space of a matrix

DefinitionLet A be a n ×m matrix. The set of all x ∈ Rm with Ax = 0 iscalled the null space (or kernel) of A:

N (A) := {x ∈ Rm|Ax = 0}

In MATLAB

>> null(A)


Null space of a matrix (Cont.)

>> A=[1,2,3,4;1,2,3,4;1,2,3,4;5,6,7,8]A =

1 2 3 41 2 3 41 2 3 45 6 7 8

>> null(A)>> A*ans % Test

ans = ans =-0.5026 0.2178 1.0e-014 *0.8324 0.0842 0.0333 -0.0444

-0.1571 -0.8218 0.0333 -0.0444-0.1727 0.5198 0.0333 -0.0444

0.3997 -0.0888


Null space of a matrix with QR

Ax = QRx = 0.

If A has rank k < n then R takes the following form

R =(R1 S0 0

),

(possibly after a previous permutation of the columns of A)

R1 is a k × k upper triangular matrix with no zeros on its diagonaland S a k × (n − k)-matrix.


Null space of a matrix with QR (Cont.)

This gives

Ax = QRx = (Q1,Q2)(R1 S0 0

)(x1x2

)

first k columns of Q denoted by Q1first k rows of x denoted by x1.



Multiplication gives

Q1R1x1 + Q1Sx2 = 0

or due to the orthogonality of Q1

x1 = −R−11 Sx2.

Thus, the general solution of Ax = 0 is given by(x1x2

)=(−R−11 Sx2

x2

)=(−R−11 S

I

)x2.



Components of x2 are the free parameters in the solution of thehomogenous problem and the columns of

V =(−R−11 S

I

)

form a basis of the null space.(Note again, we did not consider any permutations in thederivation)The column space (or range space) is given by the columns of Q1.


Null space of a matrix in Python

extendedcharsextendedcharsextendedcharsextendedcharsextendedchars extendedchars extendedcharsn=3extendedcharsextendedcharsextendedchars extendedchars extendedcharsA=array ([[1,2,3],[4,5,6],[6,7,8]])extendedcharsextendedcharsextendedchars extendedchars extendedchars# QR factorization with permutationsextendedcharsextendedcharsextendedchars extendedchars extendedchars[Q,R,P]=qr(A, pivoting =True)extendedcharsextendedcharsextendedchars extendedchars extendedcharsrankA=sum(diag(abs(R)>1.e-15))extendedcharsextendedcharsextendedchars extendedchars extendedcharsR1=R[: rankA ,: rankA]extendedcharsextendedcharsextendedchars extendedchars extendedcharsS=R[: rankA ,rankA :]extendedcharsextendedcharsextendedchars extendedchars extendedcharsV= vstack ((- solve(R1 ,S),eye(n-rankA )))extendedcharsextendedcharsextendedchars extendedchars extendedcharsV=V[P[P],:] # backpermuteextendedcharsextendedcharsextendedchars extendedchars extendedchars# Testextendedcharsextendedcharsextendedchars extendedchars extendedcharsnorm(A@V )) # gives 4.61 e-16

extendedcharsextendedchars


Date post:	24-Jun-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

NumericalLinearAlgebra - Lunds universitetctr.maths.lu.se/na/courses/FMNN01/course_media/... ·...

Documents