solution of linear equations system

8/10/2019 solution of linear equations system

1/33

SOLUTION OF LINEAR EQUATION SYSTEMS

DR. AJMAL SHAH, PIEAS

Where Ais the square sparse coefficient matrix

is a vector(or column matrix) containing the

variable values at the grid nodes

Qis the vector containing the terms on the right-

hand side

QA

In CFD we always get the sparse matrices

(matrices with many zero elements) afterdiscretization of partial differential equations


2/33



All the non-zero coefficients lie on the main

diagonal and the two neighboring diagonals for

1Dcase

The next two neighboring diagonals are added for2D case and even the next two diagonals for 3D

case

All other coefficients are zero

This structure allows the use of efficient iterative

solvers

QA


3/33

For 1Dand 2Dthe coefficient matrix A, vector and

Qvector will have the following shape

QA



5

4

3

2

1

5

4

3

2

1

*0

0

0

0

0

0

0

0

0

0

00

Q

QQ

Q

Q

A

A

A

AA

AA

A

A

A

A

A

A

p

E

W

p

E

W

p

E

W

p

E

W

p

5

4

3

2

1

5

4

3

2

1

*

0

00

00

0

Q

Q

Q

Q

Q

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

A

p

N

E

S

p

N

E

W

S

p

N

E

W

S

p

N

W

S

p

QAAA EEWWPP QAAAAA EENNSSWWPP

QAA k kkPP


4/33

Direct Methods

Gauss Elimination

LU Decomposition

Tri-diagonal SystemsCyclic Reduction



Iterative Methods

Jacobi Iteration

Gauss-Seidel Iteration

Successive OverRelaxation (S.O.R)


5/33

An analytical method to solve simultaneous linear

system of equations of the form [A][X]=[C]

The solution obtained is exact

Its basis is the systematic reduction of large

systems of equations to smaller onesWe use full-matrix notation, because it is the basic

method and most of the methods are based on it

It has two major steps

1. Forward Elimination2. Back Substitution


GAUSS ELIMINATION METHOD


6/33

2.2792.177

8.106

1121441864

1525

3

2

1

xx

x

The goal of forward elimination is to transform the

coefficient matrix into an upper triangular matrix

735.0

21.96

8.106

7.000

56.18.40

1525

3

2

1

x

x

x


FORWARD ELIMINATION


7/33DR. AJMAL SHAH, PIEAS

FORWARD ELIMINATION

112144

1864

1525

A

11214456.18.40

1525

56.212;56.225

64

RowRow

76.48.160

56.18.40

1525

76.513;76.525

144

RowRow

ExampleConsider a matrix Aof 3*3

Perform the forward elimination

step

7.000

56.18.40

1525

5.323;5.38.4

8.16

RowRow


8/33

Solve each equation starting from the last equation

Example of a system of 3equations

735.0

21.96

8.106

7.00056.18.40

1525

3

2

1

xx

x


BACK SUBSTITUTION

735.07.0 3x

21.9656.18.4 32 xx


9/33

We have to make a21

in equation 2equal to zero

Divide Equation 1by a11and multiply by a21

).. .( 1131321211111

21 bxaxaxaxaa

ann

1

11

211

11

21212

11

21121 ... b

a

axa

a

axa

a

axa nn


FORWARD ELIMINATION

nnnnnnn

n

n

n

b

b

b

b

x

x

x

x

aaaa

aaaa

aaaa

aaaa

3

2

1

3

2

1

321

3333231

2232221

1131211


10/33

)...( 111

21

1

11

21

212

11

21

121 ba

a

xaa

a

xaa

a

xa nn

1

11

2121

11

212212

11

2122 .. . b

a

abxa

a

aaxa

a

aa nnn

'

2

'

22

'

22 ... bxaxa nn

22323222121 ... bxaxaxaxa nn

Subtract the result from Equation 2, and replace

equation2 with this new equation


FORWARD ELIMINATION

nnnnnnn

n

n

n

b

b

b

b

x

x

x

x

aaaa

aaaa

aaaa

aaaa

3

2

1

3

2

1

321

3333231

2232221

1131211


11/33

Repeat this procedure for the remaining equations toreduce the set of equations as

11313212111 ... bxaxaxaxa nn '

2

'

23

'

232

'

22 ... bxaxaxa nn '

3

'

33

'

332

'

32 ... bxaxaxa nn

''

3

'

32

'

2 ... nnnnnn bxaxaxa

. . .

. . .

. . .


FORWARD ELIMINATION


12/33

Repeat the same procedure for the2

nd term of Equation

3 and so on

11313212111 ... bxaxaxaxa nn

'

2

'

23

'

232

'

22 ... bxaxaxa nn "

3

"

33

"

33 ... bxaxa nn

""

3

"

3 ... nnnnn bxaxa

. .

. .

. .


FORWARD ELIMINATION


13/33

At the end of (n-1)Forward Elimination steps, thesystem of equations will look like

'

2

'

23

'

232

'

22 ... bxaxaxa nn

33333 ... bxaxa nn

nnnn

bxa

. .

. .

. .

11313212111 ... bxaxaxaxa nn


FORWARD ELIMINATION


14/33

n

n

'

n

n

nn

nnnn

n

'

n

''

n

b

b

b

b

b

x

x

x

x

x

a

aa

aa

aaa

aaaa

1

3

2

1

1

3

2

1

)1()1)(1(

333

22322

1131211

00

0

0

0

0

0

00

0


MATRIX FORM AT THE END OF FORWARD ELIMINATION


15/33

'

2

'

23

'

232

'

22 ... bxaxaxa nn

"

3

"

33

"

33 ... bxaxa nn

nnnn bxa

. .. .

. .

11313212111 ... bxaxaxaxa nn


BACK SUBSTITUTION

"'

1

'

1,

' ... iiiniiiiii bxaxaxa

. .

. .

. .


16/33

1,...,1for1

ni

a

xab

xii

n

ijjiji

i

nn

nn

abx

1,...,1for... ,22,11,

ni

a

xaxaxabx

ii

nniiiiiiii

i


BACK SUBSTITUTION

''

1

'

1,

' ... iiiniiiiii bxaxaxa

nnnn bxa Calculating x

nfrom

last row (equation)

The ithrow (equation) is


17/33

The operation count for gaussian elimination The number of operations required in the forward

elimination is proportional to n3/3 arithmetic

operations

The number of operations required in the backsubstitution is proportional to n2/2 arithmetic

operations

The high cost of gauss elimination providesincentive to search for more efficient special

solvers for matrices such as the sparseones arising

from the discretization of differential equations


OPERATION COUNT


18/33

A number of variations on gauss elimination have been

proposedOne variant of value to CFDis LUdecomposition

Any matrix A can be factored into the product of lower

(L)and upper (U)triangular matrices

The diagonal elements of L, lii, or U, uii, are kept

equal to 1

QA

LUA

QLUA

YU


LU DECOMPOSITION

QLY


19/33

LU factorization can be performed without knowing the

vector Q

If many systems involving the same matrix are to be

solved, considerable savings can be obtained

U is the same as the coefficient matrix at the end of the

forward elimination step

L is obtained using the multipliers that were used in the

forward elimination process


LU DECOMPOSITION

33

2322

131211

3231

21

00

0

1

01001

.

u

uuuuu

ULA



LU DECOMPOSITION

112144

1864

1525

A

11214456.18.40

1525

56.212;56.225

64

RowRow

76.48.160

56.18.40

1525

76.513;76.525

144

RowRow

ExampleConsider a matrix Aof 3*3

Perform the forward elimination

step

7.000

56.18.40

1525

5.323;5.38.4

8.16

RowRow



LU DECOMPOSITION

Using the multipliers used during the Forward

Elimination Procedure to find L

7.000

56.18.40

1525

U

1

01

001

3231

21

L 112144

1864

1525

A

56.225

64

11

2121

a

a 76.5

25

144

11

3131

a

a 5.3

8.4

8.16

22

32

32

a

a

15.376.5

0156.2001

L

7.000

56.18.401525

15.376.5

0156.2001

.UL ?

QA LUA

QLUA YUQLY ;


22/33

ii

i

Ei

i

Pi

i

W QAAA 11

For 1Dproblems the resulting algebraic equations

have an especially simple structureThey constitute a tri-diagonalsystem


TRI-DIAGONAL SYSTEMS

n

n

i

n

n

i

nn

nn

nn

nnnn

iiiiii

Q

Q

Q

Q

Q

Q

A

A

A

AA

AAA

AAA

AAA

AA

1

3

2

1

1

3

2

1

,

,1

1,

1,1,1

1,,1,

343332

232221

1211

*

0.........0.........

00.

0.

.....

.0

.0

...

.0

00..

00

00

00

..

....

....

....

....

0

0

00


23/33

1

1

i

Ei

P

i

Wi

P

i

P AA

AAA



Only one element needs to be eliminated from each

row during the forward elimination process

When the algorithm has reached the ith row, onlyAPneeds to be modified; the new value is:

n

n

i

n

n

i

nn

nn

nn

nnnn

iiiiii

Q

Q

Q

Q

Q

Q

AA

AAA

AAA

AAAAAA

AA

1

3

2

1

1

3

2

1

,

,1

1,

1,1,1

1,,1,

343332

232221

1211

*

0.........

0.........

00.0.

....

.

.0

.0

...

.0

00.0

0000

00

..

....

....

....

....

00

00

ii

ii

ii

iiii AA

AAA ,1

1,1

1,

,,


24/33

The forcing term is also modified:

This tri-diagonal solution method is sometimes called

the Thomas Algorithm or the Tri-diagonal Matrix

Algorithm (TDMA)

1

*

1*

i

P

i

i

Wii

A

QAQQ

*

1 ii

i

Ei

i

P QAA



n

n

i

n

n

i

nn

nnnn

iiii

Q

Q

Q

Q

Q

Q

A

AA

AA

AA

AA

AA

1

3

2

1

1

3

2

1

,

,11,1

1,,

3433

2322

1211

*

0.........0.........

000.

00.

....0.

.0

.0

...

.0

0

0

0

0

00.0

00

00

00

....

00

00

00

n

P

nn

A

Q *

i

P

i

i

Eii

A

AQ 1*


25/33

A Fortran code requires 8 executable lines forTDMA method

The number of operations is proportional to n,

the number of unknowns, rather than the n3of full

matrix Gauss elimination

The cost per unknown is independent of the n

Thus, Very economical method

But applicable to tridiagonal matrices only




26/33

Jacobi Iteration

Gauss-Seidel Iteration

Successive Over Relaxation (S.O.R) SORis a method used to accelerate the

convergence

Gauss-Seidel Iteration is a special case of SOR

method


DIFFERENT ITERATIVE METHODS


27/33

The cost of direct methods is very high

The discretization error is usually much largerthan the solver accuracy

Therefore no need of solving equations so

accurately This leaves an opening for iterative methods

They must be used for non linear systems butare more economical for linear systems as

well


DIFFERENT ITERATIVE METHODS


28/33

The truncation error in the discretization procedure is

proportional to grid spacing, x So as the number of grid points is increased and xis reduced,

the error in the numerical solution would decrease and the

agreement between the numerical and exact solutions would

get better


GRID CONVERGENCE


29/33

nnnnnn

nn

nn

bxaxaxa

bxaxaxabxaxaxa

2211

22222121

11212111

0

0

2

01

0

n

x

x

x

x

)(1 0

1

0

2121

11

1

1 nnxaxaba

x

)(1 0

11

0

22

0

11

1

nnnnnnnn

n xaxaxaba

x

)(1 0203230121222

12 nnxaxaxab

ax


JACOBI ITERATION

1

1 1

1 1 i

j

n

ij

k

jij

k

jiji

ii

k

i xaxaba

x


30/33

nnnnnn

nn

nn

bxaxaxa

bxaxaxa

bxaxaxa

2211

22222121

11212111

0

0

2

01

0

nx

x

x

x

)(1 0

2

0

323

1

1212

22

1

2 nnxaxaxaba

x

Use the latest

update


GAUSS-SEIDEL (GS) ITERATION

)(1 0

1

0

2121

11

1

1 nnxaxaba

x

)(1 1

11

1

22

1

11

1

nnnnnnnn

n xaxaxaba

x

1

1 1

11 1 i

j

n

ij

k

jij

k

jiji

ii

k

i xaxaba

x


31/33

Gauss-seidel iteration converges more rapidlythan the jacobi iteration since it uses the latest

updates

But there are some cases thatjacobiiteration does

converge but gauss-seideldoes not

To accelerate the gauss-seidel method even

further, successive over relaxation method can be

used


COMPARISON


32/33

1

1 1

11

1)1(

i

j

n

ij

kjijkjiji

ii

kiki xaxabaxx

1


33/33

The operation count for Gaussian Elimination orLU Decomposition was O (n3) (order of n3)

For iterative methods, the number of scalar

multiplications is O (n2

)at each iteration If the total number of iterations required for

convergence is much less than n, then iterative

methods are more efficient than direct methods

Also iterative methods are well suited for sparsematrices

OPERATION COUNT

Date post:	02-Jun-2018
Category:	Documents
Upload:	attique-javaid
View:	228 times
Download:	0 times

solution of linear equations system

Documents