Thomas algorithm to solve tridiagonal matrices

rTx

nnnn

nnn

nnn

r

r

r

x

x

x

ba

cba

cba

cba

cba

cb

2

1

2

1

111

222

333

222

11

*

Basically sets up an LU decomposition

three parts

1) decomposition

2) forward substitution

3) backward substitution

1) decompositionloop from rows 2 to n

ai = ai/bi-1

bi = bi-ai*ci-1

end loop

2) forward substitutionloop from 2 to n

ri = ri - ai*ri-1

end loop

3) back substitutionxn = rn/bn

loop from n-1 to 1xi = (ri-ci*xi+1)/bi

end loop

Example

16

55

13

49

9

21

543

311

1214

25

5

4

3

2

1

x

x

x

x

x

First decompose T

loop from rows 2 to 5ai = ai/bi-1

bi = bi-ai*ci-1

end loop

21

543

311

1214

25

0515.11*0515.01

0515.0)4.19/(1

5/2 192*)5/4(21

5/4

3

3

2

2

b

a

b

a

0968.32194.0

55588.48529.2

30515.10515.0

14.198.0

25

forward substitutionloop from 2 to n

ri = ri - ai*ri-1

end loop

16

55

13

49

9

r

0968.32194.0

55588.48529.2

30515.10515.0

14.198.0

25

T

1546.158.41*0515.013

8.419*8.049

3

2

r

r

5806.18

7647.11

1546.15

8.41

9

r

back substitutionxn = rn/bn

loop from n-1 to 1xi = (ri-ci*xi+1)/bi

end loop

0968.32194.0

55588.48529.2

30515.10515.0

14.198.0

25

T

5806.18

7647.11

1546.15

8.41

9

r

15/)2*29(

24.19/)3*18.41(

30515.1/)4*)3(1546.15(

45588.4/)6*)5(7647.11(

60968.3/5806.18

1

2

3

4

5

x

x

x

x

x

6

4

3

2

1

x

Crout algorithm - alternate LU decomposition

Instead of 1’s on diagonal of L, get 1’s on diagonal of U

Operate on rows and columns sequentially, narrowing down to single element

Algorithm

n to1ifor 1,1, ii al

5554535251

4544434241

3534333231

2524232221

1514131211

aaaaa

aaaaa

aaaaa

aaaaa

aaaaa

5554535251

4544434241

3534333231

2524232221

1514131211

aaaal

aaaal

aaaal

aaaal

aaaal

5554535251

4544434241

3534333231

2524232221

1514131211

aaaal

aaaal

aaaal

aaaal

uuuul

for j=2 to n-1

n to1jkfor

1

1,

jj

j

iikjijk

kj l

ulau

n tojifor 1

1,,

ul-alj

kkjikjiji

5554535251

4544434241

3534333231

2524232221

1514131211

aaall

aaall

aaall

aaall

uuuul

5554535251

4544434241

3534333231

2524232221

1514131211

aaall

aaall

aaall

uuull

uuuul

5554535251

4544434241

3534333231

2524232221

1514131211

aalll

aalll

aalll

uuull

uuuul

n to2jfor 11

11

l

au j

j

5554535251

4544434241

3534333231

2524232221

1514131211

aalll

aalll

uulll

uuull

uuuul

5554535251

4544434241

3534333231

2524232221

1514131211

allll

allll

uulll

uuull

uuuul

For j=n

1

1

n

kknnknnnn ulal

5554535251

4544434241

3534333231

2524232221

1514131211

allll

ullll

uulll

uuull

uuuul

5554535251

4544434241

3534333231

2524232221

1514131211

lllll

ullll

uulll

uuull

uuuul

Cholesky decomposition

A decomposition for symmetric matrices

Symmetric matrix e.g. a covariance matrix

332313

322212

312111

covcovcov

covcovcov

covcovcov

xxxxxx

xxxxxx

xxxxxx

For symmetric matrices, can write

T

T

LU

LLA

i.e.

Can develop relationships for l

1

1

2

1

1 1-k to1ifor

k

j

kjkkkk

ii

i

jkjijki

ki

lal

l

lla

l

Example:

135.025.015.0

35.013.04.0

25.04.012.0

15.04.02.01

A

1st row

11

1

12

1111

1

111

1 1-1 to1ifor

j

j

ii

i

jjiji

i

lal

l

lla

lSkip this equation

1111 al

135.025.015.0

35.013.04.0

25.04.012.0

15.04.02.01

A

1

L

Row 2

1

1

22

2222

0

122

2 1ifor

j

j

ii

jjiji

i

lal

l

lla

l

9798.02.01

1

2.0

221

22222

11

2121

lal

l

al

135.025.015.0

35.013.04.0

25.04.012.0

15.04.02.01

A

9798.02.0

1

L

Row 3

8886.02245.04.01

2245.09798.0

4.0*2.03.0

1)(i 1

4.0

2232

231

23333

22

31213232

11

3131

llal

l

llal

l

al

135.025.015.0

35.013.04.0

25.04.012.0

15.04.02.01

A

8886.02245.04.0

9798.02.0

1

L

9243.02696.02245.015.01

2696.08886.0

2245.0*2245.015.0*4.035.0

2245.09798.0

15.0*2.025.0

1

15.0

22243

242

241

24444

33

423241314343

22

41214242

11

4141

lllal

l

llllal

l

llal

l

al

Row 4

135.025.015.0

35.013.04.0

25.04.012.0

15.04.02.01

A

9243.02696.02245.015.0

8886.02245.04.0

9798.02.0

1

L

9243.0

2696.08886.0

2245.02245.09798.0

15.04.02.01

TL

Iterative methods for solving matrix equations

1. Jacobi

2. Gauss-Seidel

3. Successive overrelaxation (SOR)

Idea behind iterative methods: Convert Ax=b into Cx=d, which has sequence of approximations x(1), x(2), … with

dCxx kk 1

What are C and d?

Recall from fixed-point iteration

xxxx

xxxx

sin0sin2

3032

22

Rewrite matrix equation in same way

4444343242141

3434333232131

2424323222121

1414313212111

bxaxaxaxa

bxaxaxaxa

bxaxaxaxa

bxaxaxaxa

becomes

44

43

44

432

44

421

44

414

33

34

33

342

33

321

33

313

22

24

22

243

22

231

22

212

11

14

11

143

11

132

11

121

a

bx

a

ax

a

ax

a

ax

a

bx

a

ax

a

ax

a

ax

a

bx

a

ax

a

ax

a

ax

a

bx

a

ax

a

ax

a

ax

Then

0

0

0

0

44

43

44

42

44

41

33

34

33

32

33

31

22

24

22

23

22

21

11

14

11

13

11

12

a

a

a

a

a

aa

a

a

a

a

aa

a

a

a

a

aa

a

a

a

a

a

C

44

4

33

3

22

2

11

1

a

ba

ba

ba

b

d

Jacobi method is like fixed point iteration

Example: Shape of a stretched membrane

Shape can be described by potential function

0 yyxx uu

Let us give some boundary conditions

11,

0,

1,1

,0

10 ,10 0

2

2

xu

xxu

yu

yyu

yxuu yyxx

1111111

164.08.0

136.06.0

116.04.0

104.02.0

164.036.016.004.00

18.06.04.02.0

16151413

1211109

8765

4321

uuuu

uuuu

uuuu

uuuu

y

x

Problem look likes this

Solve for u’s

2

1

1

64.1

1

0

0

36.0

1

0

0

16.0

64.1

36.0

16.0

08.0

411

1411

1411

141

1411

11411

11411

1141

1411

11411

11411

1141

141

1141

1141

114

16

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

u

Leads to this system of equations

Choose an initial u=[1 1 1 1 …1]’

Iterate using x=Cx+d

Matlab solution, 49 iterations

Gauss-Seidel method differs from Jacobi by sequential updating

- use new xi’s as they become available

Example:

154

62

13223

321

321

321

xxx

xxx

xxx

4

15

4

1

4

1

32

1

2

13

13

3

2

3

2

213

312

321

oldoldnew

oldoldnew

oldoldnew

xxx

xxx

xxx

4

15

4

1

4

1

32

1

2

13

13

3

2

3

2

213

312

321

newnewnew

oldnewnew

oldoldnew

xxx

xxx

xxx

Jacobi Gauss-Seidel