SSCE2393NUMERICALMETHODS
CHAPTER2LINEAREQUATIONSYSTEMS
FarhanaJohar,DepartmentofMathematicalSciences,FacultyofScience,[email protected]
OverviewChapter2
LinearEquationSystems
GAUSSELIMINATION JACOBI GAUSSELIMINATIONWITH GAUSS-SEIDEL
PARTIALPIVOTING
• DOOLITTLE• CROUT• THOMASMETHOD• THOMASALGORITHM• CHOLESKY
ELIMINATIONMETHOD
LUFACTORIZATION
ITERATION METHOD
mnmnmm
nn
nn
bxaxaxa
bxaxaxabxaxaxa
=+++
=+++
=+++
…!!!!!
…
…
2211
22222121
11212111
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
nnmnmm
n
n
b
bb
x
xx
aaa
aaaaaa
!!…
!!!!……
2
1
2
1
21
22221
11211
⎥⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢⎢
⎣
⎡
nmnmm
n
n
b
bb
aaa
aaaaaa
!…
!!!!……
2
1
21
22221
11211
2.1INTRODUCTIONOFLINEAREQUATIONSYSTEMSLinearequations: bax =
Inmatrixform( bxA =×nm )Inaugmentedmatrixform
Wewillonlyconsider nn×A matrix(squarematrix).
EliminationmethodSolutions LU Iteration
Backward subst.
⎥ ⎥ ⎥
⎦
⎤
⎢ ⎢ ⎢
⎣
⎡
3
2
1
33 32 31
23 22 21
13 12 11
b
b
b
a a a
a a a
a a a
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
3
2
1
33
2322
131211
000
fff
uuuuuu
3331321211
2233322
3333
/)(/)(
uxuxudxuxufx
ufx
−−=
−=
=
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛=
3
2
1
3
2
1
33
2322
131211
000
fff
x
x
x
uuuuuu
2.1EliminationMethod
• GaussElimination• Gausseliminationwithpartialpivoting
2.1.1GaussEliminationGiven: Writteninaugmentedmatrixformas:
fUx =
bAx =
3333232131
2323222121
1313212111
bxaxaxabxaxaxabxaxaxa
=++
=++
=++
Elementary row operations
Elementary row operations
212 59.0 BBB +−→
Example1:SolvethefollowinglinearsystemusingGausseliminationmethod.
84.148.104.368.275.030.193.048.156.553.448.151.2
321
321
321
−=−+
−=−+
=++
xxxxxxxxx
Solution
⎥⎥⎥
⎦
⎤
−
−
⎢⎢⎢
⎣
⎡
−
−
84.175.056.5
48.104.368.230.193.048.153.448.151.2
2.51 1.48 4.530 0.06 −3.972.68 3.04 −1.48
⎡
⎣
⎢⎢⎢
5.56−4.03−1.84
⎤
⎦
⎥⎥⎥
multiplier, 59.0112121 == aam multiplier, 07.1113131 == aam
313 07.1 BBB +−→
2.51 1.48 4.530 0.06 −3.970 1.46 −6.33
⎡
⎣
⎢⎢⎢
5.56−4.03−7.79
⎤
⎦
⎥⎥⎥B3→−24.33B2 +B3
multiplier,m32 = a32 a22 = 24.33
2.51 1.48 4.530 0.06 −3.970 0 90.26
⎡
⎣
⎢⎢⎢
5.56−4.0390.26
⎤
⎦
⎥⎥⎥
∴Usingbackwardsubstitution,weget:
x3 =1x2 = −1x1 =1
Exercise:SolvethefollowinglinearsystemsusingGausseliminationmethod:1.Use4decimalplaces.
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
−
=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ −
11210
511042115
3
2
1
xxx
2.Use3decimalplaces.
8x1 + 2x2 − x3 + 2x4 = −2x1 −8x2 + x3 − 2x4 = 42x1 − x2 + 7x3 − x4 = −1
−2x1 + x2 −3x3 −8x4 = 3
Ans: 5555.21,7222.12,0555.13 ==−= xxx 083.01,458.02,231.03,325.04 −=−=−=−= xxxx
2.1.2GaussEliminationwithPartialPivotingwhy????• tosolveproblemthatinvolvedivisionbyzero(incase 11a =0or 22a =0)
• reducetheround-offerrors.ThealgorithmfollowstheGausseliminationmethodexcept:• Interchangerowswhenneededatthek-thstepsothatthe
absolutevalueofpivotelement kka isthelargestelementcomparetotheotherelementsunderneaththepivot.
Then,dotheeliminationprocess.Example2:UseGausseliminationmethodwithpartialpivotingtosolve:
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
9868
0015491001924300
4
3
2
1
xxxx
Solution
0 0 3 42 9 1 00 1 9 45 1 0 0
8689
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
B1↔ B4
5 1 0 02 9 1 00 1 9 40 0 3 4
9688
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
m21=25
B2 :−m21B1 +B2⎯ →⎯⎯⎯⎯⎯⎯
5 1 0 00 8.6 1 00 1 9 40 0 3 4
92.488
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
m32 =18.6
B3 :−m32B2 +B3⎯ →⎯⎯⎯⎯⎯⎯
5 1 0 00 8.6 1 00 0 8.8837 40 0 3 4
92.4
7.72098
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
m43=3
8.8837B4 :−m43B3 +B4⎯ →⎯⎯⎯⎯⎯⎯
5 1 0 00 8.6 1 00 0 8.8837 40 0 0 2.6492
92.4
7.72095.3927
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
Thenwehave:5x1 + x2 = 9
8.6 x2 + x3 = 2.48.8837 x3+ 4 x4 = 7.7209
2.6492 x4 = 5.3927
Usebackwardsubst.togetx4 = 2.0356x3 = −0.0474 or x = x̂ = (1.743, 0.285, − 0.048, 2.036)x2 = 0.2846x1 =1.7431
Exercise:SolvethefollowingsystemsusingGausseliminationmethodwithpartialpivoting.Use3decimalplaces.1.
2.
4 1 −15 1 26 1 1
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟
x1x2x3
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟=
−246
⎛
⎝
⎜⎜⎜
⎞
⎠
⎟⎟⎟
Ans:1. 2.
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
jihgfed
cba
000000
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
tsrqpunmlk
00000
0
2.2LUFACTORIZATIONMETHODForalinearsystem
Ax=bUsesubstitutionofA=LU,whereLisalowertriangularmatrix,andUisuppertriangularmatrix.
LUx=bLet
Ux=YYields
LY=bL=U=
Procedure:Step1:FromA=LU,solveforLandU.
Step2:FromLY=b,solveforYbyforwardsubstitution.
Step3:FromUx=Y,solveforxbybackwardsubstitution.
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
1010010001
ihged
b
2.2.1DoolittleMethod
A=LU
Diagonalelementformatrix 1=L
Ax=b
objectiveàtogetthevalueof x Step:1. LUA = ,findthematrixfor L andU 2. bLy = ,solvefor y usebackwardandforward substitution3. yUx = ,solvefor x
13425146228923
321
321
321
−=−+
=+−
=++
xxxxxxxxx
08.0226.65.03
5.45.132
2134
32
21
1
=−−++
−=+−
=+
=
xxxxxx
xxx
Example3:SolvethisequationsystemusingDoolittlemethod.Dothecalculationin4decimalplaces.Example4:SolvethisequationsystemusingDoolittlemethod.
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
1000100
101
sqpnml
bAx =
2.2.2CroutMethod
LUA =
Diagonalelementformatrix 1=U objectiveàtofindthevalueof x Step:1. LUA = ,determine L andU
2. bLy = ,solvefor y useforwardandbackward
3. yUx = ,solvefor x substitutions
13425146228923
321
321
321
−=−+
=+−
=++
xxxxxxxxx
Example5:SolvethislinearsystemusingCroutmethod.
bAx =
bAx =
2.2.3ThomasMethod Checkthisfirst!MakesurethatmatrixAmustbeTridiagonalMatrix
d1 e1 0 ! 0c2 d2 e2 ! "
0 ! ! ! 0" ! cn−1 dn−1 en−10 0 0 cn dn
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
Ifnotàrearrangethematrix
L =
α1 0 0 0 0c2 α2 0 0 00 c3 α3 0 00 0 ! ! 00 0 0 cn αn
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
U =
1 β1 0 0 00 1 β2 0 00 0 1 β3 00 0 ! ! βn−10 0 0 0 1
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
objectiveàtofindthevalueof x
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
−
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−−
112011
3200514001210031
4
3
2
1
xxxx
Step:1. LUA = ,determinethematrixfor L andU 2. bLw= ,solvefor y useforwardandbackward substitutions3. wUx = ,solvefor x Example6:SolvethislinearsystemequationusingThomasmethod.
2.2.4ThomasAlgorithmGeneralizationofThomasmethod
Ø suitabletosolvelargesystemØ Easyforprogrammingcoding
Remember!AlwayscheckwhetherMatrixAistridiagonalmatrixornot…Checkthisfirst!
d1 e1 0 0 0c2 d2 e2 0 00 c3 d3 e3 00 0 ! ! en−10 0 0 cn dn
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
Ifnotàrearrangethematrix
d1 e1 0 0 0c2 d2 e2 0 00 c3 d3 e3 00 0 ! ! en−10 0 0 cn dn
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
x1x2x3"xn
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
=
b1b2b3"bn
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥
ei
id ic
Formula:Thomas’sAlgorithm:
11 d=α
1−−= iiii cd βα , ni ...,,3,2=
i
iieα
β = , 1...,,3,2,1 −= ni
111 /αby = yi = (bi − ciyi−1) /αi , ni ...,,3,2=
xn = yn xi = yi −βi xi+1 , 1...,,2,1 −−= nni
Table:
i 1 2 … n
id
ie
ic
ib
iα
iβ
iy
ix
92849692
843
21
432
321
43
=+
=++
=++
=+
xxxxxxxx
xx
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
−
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−−
112011
3200514001210031
4
3
2
1
xxxx
Example7:SolvethislinearsystemusingThomasmethod.Example8:Given(i) Whatconditionthatneedstobefulfilledbeforeusing
Thomas’sAlgorithm?(ii) Ifthesystemabovesatisfiedthecondition,solvethe
equationsystem.
bAx =
bAx =
/
2.2.5CholeskyMethodMatrixAmustbesymmetricpositive-definite
Definition Rules(theorem)
0,0 ≠∀> xAxxT 1. 0≠A
2. niaii ,,2,1,0 …=∀>
3. iikj aani
njnk
maxmax1
11 ≤≤
≤≤≤≤
≤
4. njiji
jjaiiaija ,,2,1,,)( 2 …=∀≠
<
LUA =
where TLU =
Targetàtofindthevalueof x Step:1. LUA = ,determine L andU 2. bLy = ,solvefor y useforwardandbackward substitutions3. yUx = ,solvefor x
153637
74
321
321
321
=++
=++−
=+−
xxxxxx
xxx
Example9:ShowthatmatrixAforthefollowinglinearsystemissymmetricpositive-definitebyusingdefinition.Then,solvethesystemoflinearequationsbyCholeskyMethod.
2.3ITERATIVEMETHOD
IsAStrictlyDiagonallyDominantMatrix?
∑≠=
>n
ijj
ijii aa1
ifnotàrearrangerows.e.g.
Showthat⎥⎥⎦
⎤
⎢⎢⎣
⎡−
721152023
isaSDDmatrix.
Solution:
127:3
125:2
023:1
+>
+>−
+>
B
B
B
Mustsatisfyconvergencecriterioni.e.Amustbestrictlydiagonallydominantmatrix
2.3.1JacobiMethodFormula:
33
)(232
)(1313)1(
3
22
)(323
)(1212)1(
2
11
)(313
)(2121)1(
1
axaxabx
axaxabx
axaxabx
kkk
kkk
kkk
−−=
−−=
−−=
+
+
+
Initialguess:
Tx )000()0( …= Stoptheiterationwhen:
{ } ε<−−≤≤
)1()(1
kix
kix
nimax
foragivenvalueofε andtake)(kxx ≈
Example10:SolvethefollowinglinearsystemusingJacobimethod.Set 0x =)0( .Take 05.0=ε .
192343112
3271
321
3231
=+
=−+
=+
+
−
xxxxxxxxx
SolutionIsmatrixASDD?Rearrangerows:
311219234
3231
3271
321
=+
=+
=−+
−
+
xxxxxxxxx
Theiterationformula:
1231
72194
3
)(2
)(1)1(
3
)(3
)(1)1(
2
)(3
)(2)1(
1
kkk
kkk
kkk
xxx
xxx
xxx
+−=
−−=
+−=
+
+
+
k )(
1kx )(
2kx )(
3kx )1(
1)(
1−− kk xx )1(
2)(
2−− kk xx )1(
3)(
3−− kk xx
0 0 0 0 1 2 3 4 5
{ }ε<==
−=−≤≤
01.0}01.0,01.0,01.0{max
)4()5(
31
)4()5( max iii
xxxx
)00.3,00.2,01.1()5( =≈∴ xx
2.3.2GaussSeidelMethodFormula:
33
)1(232
)1(1313)1(
3
22
)(323
)1(1212)1(
2
11
)(313
)(2121)1(
1
axaxabx
axaxabx
axaxabx
kkk
kkk
kkk
+++
++
+
−−=
−−=
−−=
Initialguess:Tx )000()0( …=
Stoptheiterationwhen:
{ } ε<−+
≤≤
)()1(1
kix
kix
nimax
foragivenvalueofε andtake)(kxx ≈
Example11:SolvethefollowinglinearsystemusingGauss-Seidelmethod.Set 0x =)0( .Take 05.0=ε .
192343112
3271
321
3231
=+
=−+
=+
+
−
xxxxxxxxx
SolutionIsmatrixASDD?Rearrangerows:
311219234
3231
3271
321
=+
=+
=−+
−
+
xxxxxxxxx
Theiterationformula:
1231
72194
3
)1(2
)1(1)1(
3
)(3
)1(1)1(
2
)(3
)(2)1(
1
+++
++
+
+−=
−−=
+−=
kkk
kkk
kkk
xxx
xxx
xxx
k )(
1kx )(
2kx )(
3kx )1(
1)(
1−− kk xx )1(
2)(
2−− kk xx )1(
3)(
3−− kk xx
0 0 0 0 1 2 3 4
{ }ε<==
−=−≤≤
00.0}00.0,00.0,00.0{max
)3()4(
31
)3()4( max iii
xxxx
)00.3,00.2,00.1()5( =≈∴ xx
Exercise:1.WritetheGauss-Seidelformulafor
2171782115
321
321
321
=++−
=−+
=+−
xxxxxxxxx
Thenfindthevalueofx1,x2,andx3.Dothecalculationin3decimalplaces.2.SolvethefollowingsystemusingGauss–Seidelmethodandstoptheiterationwhen 0005.0)1()( <−
∞
−kk xx .
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
−
−−
−−
−
1511256
8130110123111102110
4
3
2
1
xxxx