ECIV 301

Programming & Graphics

Numerical Methods for Engineers

Lecture 20

Solution of Linear System of Equations - Iterative Methods

Iterative Methods

Recall Techniques for Root finding of Single Equations

Initial Guess

New Estimate

Error Calculation

Repeat until Convergence

Gauss Seidel

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

11

31321211 a

xaxabx

22

32312122 a

xaxabx

33

23213133 a

xaxabx

Gauss Seidel

11

1

11

1312111

00

a

b

a

aabx

22

23112121

2

0

a

axabx

33

1232

113131

3 a

xaxabx

First Iteration: 0,0,0 321 xxx

Better Estimate

Better Estimate

Better Estimate

Gauss Seidel

11

1313

121212

1 a

xaxabx

22

1323

212122

2 a

xaxabx

33

2232

213132

3 a

xaxabx

Second Iteration: 13

12

11 ,, xxx

Better Estimate

Better Estimate

Better Estimate

Gauss SeidelIteration Error:

%1001

, ji

ji

ji

ia x

xx

s

Convergence Criterion:

n

jij

ijii aa1

nnn2n1

2n2221

1n1211

aaa

aaa

aaa

Jacobi Iteration

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

11

31321211 a

xaxabx

22

32312122 a

xaxabx

33

23213133 a

xaxabx

Jacobi Iteration

11

1

11

1312111

00

a

b

a

aabx

22

2321212

00

a

aabx

33

3231313

00

a

aabx

First Iteration: 0,0,0 321 xxx

Better Estimate

Better Estimate

Better Estimate

Jacobi Iteration

11

1313

121212

1 a

xaxabx

22

1323

112122

2 a

xaxabx

33

1232

113132

3 a

xaxabx

Second Iteration: 13

12

11 ,, xxx

Better Estimate

Better Estimate

Better Estimate

Jacobi Iteration

Iteration Error:

%1001

, ji

ji

ji

ia x

xx

s

Example

4.71

3.19

85.7

102.03.0

3.071.0

2.01.03

3

2

1

x

x

x

3

2.01.085.7 321

xxx

7

3.01.03.19 312

xxx

10

2.03.04.71 213

xxx

Determinants

nnn2n1

2n2221

1n1211

aaa

aaa

aaa

A

nnn2n1

2n2221

1n1211

aaa

aaa

aaa

det

AA

Are composed of same elements

Completely Different Mathematical Concept

Determinants

2221

1211

aa

aaA

Defined in a recursive form

2x2 matrix

122122112221

1211det aaaaaa

aaA

DeterminantsDefined in a recursive form

3x3 matrix

3231

222113

3331

232112

3332

232211

det

aa

aaa

aa

aaa

aa

aaa

A

333231

232221

131211

aaa

aaa

aaa

333231

232221

131211

aaa

aaa

aaa

Determinants

3332

232211 aa

aaa

3231

222113

3331

232112 aa

aaa

aa

aaa

3332

2322

aa

aaMinor a11

333231

232221

131211

aaa

aaa

aaa

Determinants

3331

2321

aa

aaMinor a12

3332

232211 aa

aaa

3331

232112 aa

aaa

3231

222113 aa

aaa

333231

232221

131211

aaa

aaa

aaa

Determinants

3231

2221

aa

aaMinor a13

3332

232211 aa

aaa

3331

232112 aa

aaa

3231

222113 aa

aaa

Solution of Small Systems of Equations – Cramer’s Rule

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

333231

232221

131211

det

aaa

aaa

aaa

D A

1. Compute

Solution of Small Systems of Equations – Cramer’s Rule

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

33323

23222

131211det1

aab

aab

aab

D A

2. Compute

Solution of Small Systems of Equations – Cramer’s Rule

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

33331

23221

131112det2

aba

aba

aba

D A

3. Compute

Solution of Small Systems of Equations – Cramer’s Rule

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

33231

22221

112113det3

baa

baa

baa

D A

4. Compute

Solution of Small Systems of Equations – Cramer’s Rule

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

D

Dx

11

D

Dx

22

D

Dx

33

If D=0 solution does NOT exist

Singular Matrices

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

If D=0 solution does NOT exist

Regardless of Method

Singular Matrices

0det if AD

3

2

1

3

2

1

333231

232221

131211

b

b

b

x

x

x

aaa

aaa

aaa

bxA

bAx 1

For Example

{x} does not exist

[A]-1 does not exist

Determinants and LU Decomposition

064.62

30

10

645.700

2.162.60

835

z

y

x

24

6

10

23610

3112

835

z

y

x

{x} is not affected

Determinants and LU Decomposition

{x} is not affected

)operations pivoting no (if detdet UA D

Determinants and LU Decomposition

nnaaaaD 332211det U

33

2322

131211

00

0

a

aa

aaa

Example

23610

3112

835

610

1128

2310

3123

236

315

237)1072(8)30276(3)1823(5

Example

645.700

2.162.60

835

After Elimination [A] becomes

995.236 )645.7)(2.6)(5(