Major: All Engineering Majors Author(s): Autar KawForward Elimination = 279.2 177.2 106.8 144 12 1...

Gaussian Elimination

Major: All Engineering Majors

Author(s): Autar Kaw

http://numericalmethods.eng.usf.eduTransforming Numerical Methods Education for STEM

Undergraduates

http://numericalmethods.eng.usf.edu/

Naïve Gauss Elimination

http://numericalmethods.eng.usf.edu


Naïve Gaussian EliminationA method to solve simultaneous linear equations of the form [A][X]=[C]

Two steps1. Forward Elimination2. Back Substitution

Forward Elimination

=

2.2792.1778.106

11214418641525

3

2

1

xxx

The goal of forward elimination is to transform the coefficient matrix into an upper triangular matrix

−=

−−

735.021.968.106

7.00056.18.40

1525

3

2

1

xxx

Forward EliminationA set of n equations and n unknowns

11313212111 ... bxaxaxaxa nn =++++

22323222121 ... bxaxaxaxa nn =++++

nnnnnnn bxaxaxaxa =++++ ...332211

. .

. .

. .

(n-1) steps of forward elimination

Forward EliminationStep 1 For Equation 2, divide Equation 1 by and multiply by .

)...( 1131321211111

21 bxaxaxaxaaa

nn =++++

111

211

11

21212

11

21121 ... ba

axaaaxa

aaxa nn =+++

11a21a

Forward Elimination

111

211

11

21212

11

21121 ... ba

axaaaxa

aaxa nn =+++

111

2121

11

212212

11

2122 ... ba

abxaaaaxa

aaa nnn −=

−++

−

'2

'22

'22 ... bxaxa nn =++

22323222121 ... bxaxaxaxa nn =++++Subtract the result from Equation 2.

−_________________________________________________

or

Forward EliminationRepeat this procedure for the remaining equations to reduce the set of equations as

11313212111 ... bxaxaxaxa nn =++++'2

'23

'232

'22 ... bxaxaxa nn =+++

'3

'33

'332

'32 ... bxaxaxa nn =+++

''3

'32

'2 ... nnnnnn bxaxaxa =+++

. . .

. . .

. . .

End of Step 1

Step 2Repeat the same procedure for the 3rd term of Equation 3.

Forward Elimination

11313212111 ... bxaxaxaxa nn =++++'2

'23

'232

'22 ... bxaxaxa nn =+++

"3

"33

"33 ... bxaxa nn =++

""3

"3 ... nnnnn bxaxa =++

. .

. .

. .

End of Step 2

Forward EliminationAt the end of (n-1) Forward Elimination steps, the system of equations will look like

'2

'23

'232

'22 ... bxaxaxa nn =+++

"3

"33

"33 ... bxaxa nn =++

( ) ( )11 −− = nnnn

nn bxa

. .. .. .

11313212111 ... bxaxaxaxa nn =++++

End of Step (n-1)

Matrix Form at End of Forward Elimination

=

− )(n-n

"

'

n)(n

nn

"n

"

'n

''n

b

bbb

x

xxx

a

aaaaaaaaa

1

3

2

1

3

2

1

1

333

22322

1131211

0000

000

Back SubstitutionSolve each equation starting from the last equation

Example of a system of 3 equations

−=

−−

735.021.968.106

7.00056.18.40

1525

3

2

1

xxx

Back Substitution Starting Eqns

'2

'23

'232

'22 ... bxaxaxa nn =+++

"3

"3

"33 ... bxaxa nn =++

( ) ( )11 −− = nnnn

nn bxa

. .. .. .

11313212111 ... bxaxaxaxa nn =++++

Back SubstitutionStart with the last equation because it has only one unknown

)1(

)1(

−

−

= nnn

nn

n ab

x

Back Substitution

( ) ( )

( ) 1,...,1for11

11

−=∑−

= −+=

−−

nia

xabx i

ii

n

ijj

iij

ii

i

)1(

)1(

−

−

= nnn

nn

n ab

x

( ) ( ) ( ) ( )

( ) 1,...,1for...

1

1,2

12,1

11,

1

−=−−−−

= −−

+−++

−+

−

nia

xaxaxabx i

ii

ninii

iiii

iii

ii

i

THE END



Additional Resources

For all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit

http://numericalmethods.eng.usf.edu/topics/gaussian_elimination.html


Naïve Gauss EliminationExample



Example 1The upward velocity of a rocket is given at three different times

Time, Velocity, 5 106.88 177.212 279.2

The velocity data is approximated by a polynomial as:

( ) 12.t5 , 3221 ≤≤++= atatatvFind the velocity at t=6 seconds .

( )s t ( )m/s vTable 1 Velocity vs. time data.

Example 1 Cont. Assume

( ) 12.t5 ,atatatv ≤≤++= 3221

=

3

2

1

323

222

121

111

vvv

aaa

tttttt

3

2

1

Results in a matrix template of the form:

Using data from Table 1, the matrix becomes:

=

2.2792.1778.106

11214418641525

3

2

1

aaa

Example 1 Cont.

⇒

=

22791121442177186481061525

227921778106

11214418641525

3

2

1

.

.

.

.

.

.

aaa

1. Forward Elimination2. Back Substitution

Forward Elimination

Number of Steps of Forward Elimination

Number of steps of forward elimination is (n−1)=(3−1)=2

Divide Equation 1 by 25 and

multiply it by 64, .

Forward Elimination: Step 1

.

[ ] [ ]408.27356.28.126456.28.1061525 =×

[ ][ ][ ]208.96 56.18.4 0

408.273 56.2 8.1264177.2 1 8 64

−−−

−

2.2791121442.17718648.1061525

−−−

2.279112144208.9656.18.408.1061525

56.22564

=

Subtract the result from Equation 2

Substitute new equation for Equation 2

Forward Elimination: Step 1 (cont.)

.

[ ] [ ]168.61576.58.2814476.58.1061525 =×

−−−

2.279112144208.9656.18.408.1061525

[ ][ ][ ]968.335 76.48.16 0

168.615 76.5 8.28 144279.2 1 12 144

−−−

−

−−−−−−

968.33576.48.160208.9656.18.408.1061525


multiply it by 144, .76.525

144=




[ ] [ ]728.33646.58.1605.3208.9656.18.40 −−−=×−−−

−−−−−−

968.33576.48.160208.9656.18.408.1061525

[ ][ ][ ].760 7.0 0 0

728.33646.516.80335.968 76.416.80

−−−−−−

−−−

76.07.000208.9656.18.408.1061525

Divide Equation 2 by −4.8

and multiply it by −16.8,

.5.38.48.16=

−−



Back Substitution

Back Substitution

−=

−−⇒

−−−

76.0208.968.106

7.00056.18.40

1525

7.07.0002.9656.18.408.1061525

3

2

1

aaa

08571.17.076.076.07.0

3

3

3

=

=

=

a

a

aSolving for a3

Back Substitution (cont.)

Solving for a2

690519. 4.8

1.085711.5696.208

8.456.1208.96

208.9656.18.4

2

2

32

32

=−

×+−=

−+−

=

−=−−

a

a

aa

aa

−=

−−

76.0208.968.106

7.00056.18.40

1525

3

2

1

aaa


Solving for a1

290472.025

08571.16905.1958.1062558.106

8.106525

321

321

=

−×−=

−−=

=++aaa

aaa

−=

−−

76.02.968.106

7.00056.18.40

1525

3

2

1

aaa

Naïve Gaussian Elimination Solution

=

227921778106

11214418641525

3

2

1

.

.

.

aaa

=

08571.16905.19

290472.0

3

2

1

aaa

Example 1 Cont.

SolutionThe solution vector is

=

08571.16905.19

290472.0

3

2

1

aaa

The polynomial that passes through the three data points is then:

( )125 ,08571.16905.19290472.0 2

322

1

≤≤++=

++=

tttatatatv

( ) ( ) ( ).m/s 686.129

08571.166905.196290472.06 2

=++=v

THE END



Naïve Gauss EliminationPitfalls



95511326

3710

321

321

32

=+−=++

=−

xxxxxx

xx

Pitfall#1. Division by zero

=

−

−

9113

5153267100

3

2

1

xxx

Is division by zero an issue here?

95514356

1571012

321

321

321

=+−=++=−+

xxxxxx

xxx

=

−

−

91415

51535671012

3

2

1

xxx

Is division by zero an issue here? YES

2852414356

1571012

321

321

321

=+−=++=−+

xxxxxx

xxx

=

−

−

281415

512435671012

3

2

1

xxx

−=

−

−

25.6

15

1921125.60071012

3

2

1

xxx

Division by zero is a possibility at any step of forward elimination

Pitfall#2. Large Round-off Errors

=

−−

9751.145

3157249.23

101520

3

2

1

xxx

Exact Solution

=

111

3

2

1

xxx


=

−−

9751.145

3157249.23

101520

3

2

1

xxx

Solve it on a computer using 6 significant digits with chopping

=

999995.005.1

9625.0

3

2

1

xxx


=

−−

9751.145

3157249.23

101520

3

2

1

xxx

Solve it on a computer using 5 significant digits with chopping

=

99995.05.1

625.0

3

2

1

xxx

Is there a way to reduce the round off error?

Avoiding Pitfalls

Increase the number of significant digits• Decreases round-off error

• Does not avoid division by zero

Avoiding Pitfalls

Gaussian Elimination with Partial Pivoting• Avoids division by zero

• Reduces round off error

THE END



Gauss Elimination with Partial Pivoting



Pitfalls of Naïve Gauss Elimination

• Possible division by zero• Large round-off errors

Avoiding Pitfalls

Increase the number of significant digits• Decreases round-off error

• Does not avoid division by zero

Avoiding Pitfalls

Gaussian Elimination with Partial Pivoting• Avoids division by zero

• Reduces round off error

What is Different About Partial Pivoting?

pka

At the beginning of the kth step of forward elimination, find the maximum of

nkkkkk aaa .......,,........., ,1+

If the maximum of the values is

in the p th row, ,npk ≤≤ then switch rows p and k.

Matrix Form at Beginning of 2ndStep of Forward Elimination

=

'

'3

2

1

3

2

1

''4

'32

'3

'3332

22322

1131211

0

00

n

'

nnnnn'n

n'

'n

''n

b

bbb

x

xxx

aaaa

aaaaaaaaaa

Example (2nd step of FE)

−

=

−

−

3986

5

43111217086239011112402167067.31.5146

5

4

3

2

1

xxxxx

Which two rows would you switch?

Example (2nd step of FE)

−

=

−

−

69835

21670862390

111124043111217067.31.5146

5

4

3

2

1

xxxxx

Switched Rows

Gaussian Elimination with Partial Pivoting

A method to solve simultaneous linear equations of the form [A][X]=[C]

Two steps1. Forward Elimination2. Back Substitution

Forward Elimination

Same as naïve Gauss elimination method except that we switch rows before eachof the (n-1) steps of forward elimination.

Example: Matrix Form at Beginning of 2nd Step of Forward Elimination

=

'

'3

2

1

3

2

1

''4

'32

'3

'3332

22322

1131211

0

00

n

'

nnnnn'n

n'

'n

''n

b

bbb

x

xxx

aaaa

aaaaaaaaaa

Matrix Form at End of Forward Elimination

=

− )(n-n

"

'

n)(n

nn

"n

"

'n

''n

b

bbb

x

xxx

a

aaaaaaaaa

1

3

2

1

3

2

1

1

333

22322

1131211

0000

000

Back Substitution Starting Eqns

'2

'23

'232

'22 ... bxaxaxa nn =+++

"3

"3

"33 ... bxaxa nn =++

( ) ( )11 −− = nnnn

nn bxa

. .. .. .

11313212111 ... bxaxaxaxa nn =++++

Back Substitution

( ) ( )

( ) 1,...,1for11

11

−=∑−

= −+=

−−

nia

xabx i

ii

n

ijj

iij

ii

i

)1(

)1(

−

−

= nnn

nn

n ab

x

THE END




Example



Example 2

=

227921778106

11214418641525

3

2

1

.

.

.

aaa

Solve the following set of equations by Gaussian elimination with partial pivoting

Example 2 Cont.

⇒

=

227921778106

11214418641525

3

2

1

.

.

.

aaa

1. Forward Elimination2. Back Substitution

2.2791121442.17718648.1061525

Forward Elimination

Number of Steps of Forward Elimination

Number of steps of forward elimination is (n−1)=(3−1)=2

Forward Elimination: Step 1• Examine absolute values of first column, first row

and below.144,64,25

• Largest absolute value is 144 and exists in row 3.• Switch row 1 and row 3.

⇒

8.10615252.17718642.279112144

2.2791121442.17718648.1061525


.

[ ] [ ]1.1244444.0333.599.634444.02.279112144 =×

8.10615252.17718642.279112144

[ ][ ][ ]10.53.55560667.2 0

124.1 0.44445.33363.99177.21 8 64

−

8.106152510.535556.0667.20

2.279112144


multiply it by 64, .4444.014464

=




.

[ ] [ ]47.481736.0083.200.251736.0279.2112144 =×

[ ][ ][ ]33.588264.0 917.20

48.470.17362.08325106.81 5 25

−

8.106152510.535556.0667.20

2.279112144

33.588264.0917.2010.535556.0667.20

2.279112144


multiply it by 25, .1736.014425

=



Forward Elimination: Step 2• Examine absolute values of second column, second row

and below.2.917,667.2

• Largest absolute value is 2.917 and exists in row 3.• Switch row 2 and row 3.

⇒

10.535556.0667.2033.588264.0917.20

2.279112144

33.588264.0917.2010.535556.0667.20

2.279112144


.

[ ] [ ]33.537556.0667.209143.058.330.82642.9170 =×

10.535556.0667.2033.588264.0917.20

2.279112144

[ ][ ][ ]23.02.0 0 0

53.33 0.75562.667053.10 0.55562.6670

−−

−

−− 23.02.00033.588264.0917.202.279112144

Divide Equation 2 by 2.917 andmultiply it by 2.667,

.9143.0917.2667.2

=



Back Substitution

Back Substitution

1.152.023.023.02.0

3

3

=−−

=

−=−

a

aSolving for a3

−=

−⇒

−− 2303358

2279

20008264091720112144

23.02.00033.588264.0917.202.279112144

3

2

1

...

aaa

.

..


Solving for a2

6719.917.2

15.18264.033.58917.2

8264.033.5833.588264.0917.2

32

32

=

×−=

−=

=+aa

aa

−=

− 2303358

2279

20008264091720112144

3

2

1

...

aaa

.

..


Solving for a1

2917.0144

15.167.19122.279144122.279

2.27912144

321

321

=

−×−=

−−=

=++aaa

aaa

−=

− 2303358

2279

20008264091720112144

3

2

1

...

aaa

.

..

Gaussian Elimination with Partial Pivoting Solution

=

227921778106

11214418641525

3

2

1

.

.

.

aaa

=

15.167.19

2917.0

3

2

1

aaa


Another Example



Partial Pivoting: ExampleConsider the system of equations

655901.36099.23

7710

321

321

21

=+−=++−

=−

xxxxxx

xx

In matrix form

−−

−

5156099.230710

3

2

1

xxx

6901.37

=

Solve using Gaussian Elimination with Partial Pivoting using five significant digits with chopping

Partial Pivoting: ExampleForward Elimination: Step 1

Examining the values of the first column

|10|, |-3|, and |5| or 10, 3, and 5

The largest absolute value is 10, which means, to follow the rules of Partial Pivoting, we switch row1 with row1.

=

−−

−

6901.37

5156099.230710

3

2

1

xxx

=

−−

5.2001.67

55.206001.000710

3

2

1

xxx

⇒Performing Forward Elimination

Partial Pivoting: ExampleForward Elimination: Step 2

Examining the values of the first column

|-0.001| and |2.5| or 0.0001 and 2.5

The largest absolute value is 2.5, so row 2 is switched with row 3

=

−−

5.2001.67

55.206001.000710

3

2

1

xxx

⇒

=

−

−

001.65.2

7

6001.0055.200710

3

2

1

xxx

Performing the row swap

Partial Pivoting: Example


Performing the Forward Elimination results in:

=

−

002.65.2

7

002.60055.200710

3

2

1

xxx

Partial Pivoting: ExampleBack Substitution

Solving the equations through back substitution

1002.6002.6

3 ==x

15.255.2 3

2 −=−

=xx

010

077 321 =

−+=

xxx

=

−

002.65.2

7

002.60055.200710

3

2

1

xxx

Partial Pivoting: Example

[ ]

−=

=

11

0

3

2

1

xxx

X exact[ ]

−=

=

11

0

3

2

1

xxx

X calculated

Compare the calculated and exact solution

The fact that they are equal is coincidence, but it does illustrate the advantage of Partial Pivoting

THE END



Determinant of a Square MatrixUsing Naïve Gauss Elimination

Example



Theorem of Determinants

If a multiple of one row of [A]nxn is added or

subtracted to another row of [A]nxn to result in

[B]nxn then det(A)=det(B)

Theorem of Determinants

The determinant of an upper triangular matrix

[A]nxn is given by

( ) nnii aaaa ×××××= ......Adet 2211

∏=

=n

iiia

1

Forward Elimination of a Square Matrix

Using forward elimination to transform [A]nxn to an

upper triangular matrix, [U]nxn.

[ ] [ ] nnnn UA ×× →

( ) ( )UA detdet =

Example

11214418641525

Using naïve Gaussian elimination find the determinant of the following square matrix.

Forward Elimination


.

[ ] [ ]56.28.126456.21525 =×[ ][ ][ ]56.18.40

56.2 8.12641 8 64

−−

−

11214418641525

11214456.18.40

1525

−−



=




.

[ ] [ ]76.58.2814476.51525 =×[ ][ ][ ]76.48.16 0

76.5 8.281441 12 144

−−

−

76.48.16056.18.40

1525

−−−−

11214456.18.40

1525

−−



144=




.

76.48.16056.18.40

1525

−−−−

[ ]( ) [ ]46.58.1605.356.18.40 −−=×−−[ ][ ][ ]7.0 0 0

46.58.16076.48.160

−−−−−

7.00056.18.40

1525

−−

Divide Equation 2 by −4.8

and multiply it by −16.8,

.5.38.48.16=

−−



Finding the Determinant

.

7.00056.18.40

1525

11214418641525

−−→

After forward elimination

( )( )00.84

7.08.425Adet 332211

−=×−×=

××= uuu

Summary

-Forward Elimination

-Back Substitution

-Pitfalls

-Improvements

-Partial Pivoting

-Determinant of a Matrix

Additional Resources

For all resources on this topic such as digital audiovisual lectures, primers, textbook chapters, multiple-choice tests, worksheets in MATLAB, MATHEMATICA, MathCad and MAPLE, blogs, related physical problems, please visit



THE END



Gaussian EliminationNaïve Gauss Elimination�� http://numericalmethods.eng.usf.edu��Naïve Gaussian EliminationForward EliminationForward EliminationForward EliminationForward EliminationForward EliminationForward EliminationForward EliminationMatrix Form at End of Forward EliminationBack SubstitutionBack Substitution Starting EqnsBack SubstitutionBack SubstitutionSlide Number 16Additional ResourcesNaïve Gauss Elimination�Example�� http://numericalmethods.eng.usf.eduExample 1Example 1 Cont. �Example 1 Cont.�Forward EliminationNumber of Steps of Forward EliminationForward Elimination: Step 1Forward Elimination: Step 1 (cont.)Forward Elimination: Step 2Back SubstitutionBack SubstitutionBack Substitution (cont.)Back Substitution (cont.)Naïve Gaussian Elimination SolutionExample 1 Cont.Slide Number 33Naïve Gauss Elimination�Pitfalls�� http://numericalmethods.eng.usf.eduSlide Number 35Is division by zero an issue here?Is division by zero an issue here? YESSlide Number 38Slide Number 39Slide Number 40Avoiding PitfallsAvoiding PitfallsSlide Number 43Gauss Elimination with Partial Pivoting�� http://numericalmethods.eng.usf.eduPitfalls of Naïve Gauss EliminationAvoiding PitfallsAvoiding PitfallsWhat is Different About Partial Pivoting?Matrix Form at Beginning of 2nd Step of Forward EliminationExample (2nd step of FE)Example (2nd step of FE)Gaussian Elimination �with Partial PivotingForward EliminationExample: Matrix Form at Beginning of 2nd Step of Forward EliminationMatrix Form at End of Forward EliminationBack Substitution Starting EqnsBack SubstitutionSlide Number 58Gauss Elimination with Partial Pivoting�Example�� http://numericalmethods.eng.usf.edu�Example 2��Example 2 Cont.�Forward EliminationNumber of Steps of Forward EliminationForward Elimination: Step 1Forward Elimination: Step 1 (cont.)Forward Elimination: Step 1 (cont.)Forward Elimination: Step 2Forward Elimination: Step 2 (cont.)Back SubstitutionBack SubstitutionBack Substitution (cont.)Back Substitution (cont.)Gaussian Elimination with Partial Pivoting SolutionGauss Elimination with Partial Pivoting�Another Example�� http://numericalmethods.eng.usf.eduPartial Pivoting: ExamplePartial Pivoting: ExamplePartial Pivoting: ExamplePartial Pivoting: ExamplePartial Pivoting: ExamplePartial Pivoting: ExampleSlide Number 81Determinant of a Square Matrix�Using Naïve Gauss Elimination�Example�� http://numericalmethods.eng.usf.eduTheorem of Determinants Theorem of Determinants Forward Elimination of a �Square Matrix�Example�Forward EliminationForward Elimination: Step 1Forward Elimination: Step 1 (cont.)Forward Elimination: Step 2Finding the DeterminantSummaryAdditional ResourcesSlide Number 94

Date post:	07-Feb-2021
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Major: All Engineering Majors Author(s): Autar KawForward Elimination = 279.2 177.2 106.8 144 12 1...

Documents