DIRECT METHODS TO THE SOLUTION OF LINEAR EQUATIONS SYSTEMS

Lizeth Paola Barrero Riaño

Numerical Methods – Industrial Univesity of Santander

•Symmetric Matrix•Transposes Matrix•Determinant•Upper Triangular Matrix•Lower Triangular Matrix•Banded Matrix•Augmented Matrix•Matrix Multiplication

BASIC FUNDAMENTALS

Matrix

A matrix consists of a rectangular

array of elements represented by a single symbol. As depicted in Figure

[A] is the shorthand

notation for the matrix and

designates an individual

element of the matrix.

A horizontal set of elements is called a row (i) and a vertical

set is called a column (j).

11 12 13 1

21 22 23 2

1 2 3

...

...

...

m

m

n n n nm

a a a a

a a a aA

a a a a

Column 3

Row 2

Symmetric Matrix

It is a square

matrix in which the

elements are

symmetric about the main diagonal

is a Symmetric Matrix if , nm ij ij jiA a a a ij

Example:

If A is a symmetric matrix, then:

a. The product is defined and is a symmetric matrix.

b. The sum of symmetric matrices is a symmetric matrix.

c. The product of two symmetric matrices is a symmetric matrix if the matrices commute

tA A

Scalar, diagonal and

identity matrices, are

symmetric matrices.

Transposes Matrix

Let any matrix

A=(aij) of mxn order, the matrix B=(bij) de

order nxm is the A

transpose if the A rows are the B columns .

This operation is

usually denoted by At = A' = B

t A such that , i,jm n ij n m ij ji ijA a b b a

Example:

2 3 3 2

1 2 2 1

1 01 2 2

2 40 4 3

2 3

22 9 B

9

t

t

A A

B

a. ( )t tA Ab. ( )t t tA B A B c. ( )t t tA B B A d. ( ) , t tk A k A k

Properties:

Determinant

Given a square matrix A of n size,

its determinant is defined as the sum of

the any matrix line elements

product (row or column) chosen, by

their correspondin

g attachments.

Example:

2 4 5

To the matrix 6 7 3

3 0 2

A

4 5 2 5 2 4det( ) 3 +0 +2

7 3 6 3 6 7

=3 (-12-35)+0 (-(6-30))+2 (-14-24)=-141+0-76

=-217

A

applying the definition, if we choose the third row is:

Determinant Properties

a. If a matrix has a line (row or column) of zeros, the determinant is zero.

b. If a matrix has two equal rows or proportional, the determinant is null

c. If we permute two parallels lines of a square matrix, its determinant changes sign.

d. If we multiply all elements of a determinant line by a number, the determinant is multiplied by that number.

e. If a matrix line is added another line multiplied by a number, the determinant does not change.

f. The determinant of a matrix is equal to its transpose,g. If A has inverse matrix, A-1, it is verified that:

1 1det( )

det( )A

A

Upper and Lower Triangular Matrix

Upper Triangular Matrix Lower Triangular Matrix

is Upper Triangular if 0 , ij ijA a a i j

2 0 2

0 1 0

0 0 3

A

Example:

It is a square matrix in which all the items under the main

diagonal are zero.

is Lower Triangular if 0 , n n ij ijA a a i j

Example:

22 0 0

2 1 0

1 1 0

D

It is a square matrix in which all the elements above the

main diagonal are zero.

Banded Matrix

A band matrix is a sparse matrix whose non-zero entries are confined to a diagonal band,

comprising the main diagonal and zero or more diagonals on either side.

4 0 0 0 0 8 7 6 0 02 0 0 0

7 8 1 0 0 9 3 0 2 00 1 0 0

D= M=0 0 5 2 0 3 1 8 9 100 0 5 0

0 0 1 3 5 0 0 3 5 80 0 0 7

0 0 0 3 4 0 0 7 4 0

C

DiagonalTriadiagonal Pentadiagonal

Example:

Augmented Matrix

It is called extended or augmented

matrix that is formed by

the coefficient matrix and

the vector of independent terms, which are usually separated

with a dotted line

1 3 2 4

2 0 1 , B= 3

5 2 2 8

The augmented matrix

is represented as follows:

1 3 2 4

2 0 1 3

5 2 2 1

A

A B

A B

Example:

Matrix Multiplication

To define A ⋅ B is necessary that the number of columns in the first matrix coincides with the number of rows in the second matrix. The product order

is given by the number of rows in the first matrix per

the number of columns in the second matrix.

That is, if A is mxn order and B is nxp order, then C = A ⋅

B is mxp order.

11 1 11 1

1 1

11 11 1 1 11 1 1 1

1 11 1 1 1

y B=

then

n p

m mn n np

n n p n p

m mn n m p mn np

a a b b

A

a a b b

a b a b a b a b

AB

a b a b a b a b

.

y ,m n n p ijA aij B b

m pA

1

.n

ij ik kjk

c a b

Given the product A.B is

another matrix in which each Cij is the nth

row product of A per the nth column of B, namely the

element

Matrix Multiplication

1 3 5 1 3 4 0

0 0 1 B= 0 8 9 1

4 1 2 7 5 5 1

36 4 56 8

7 5 5 1

18 14 35 3

A

C A B

Graphically:

Example:

Solution of Linear Algebraic Equation

Linear algebra is one of the corner stones of

modern computational mathematics. Almost all numerical schemes

such as the finite element method and

finite difference method are in act techniques

that transform, assemble, reduce, rearrange, and/or approximate the

differential, integral, or other types of

equations to systems of linear algebraic

equations.

A system of linear algebraic equations can be expressed as

11 12 1 1 1

21 22 2 2 2

1 2

where a and b are constants,

i=1,2,..., , 1,2,..., .

n

n

m m nm m m

ij i

a a a x b

a a a x b

a a a x b

m j n

Solution of Linear Algebraic Equation

Or:AX=B

Solving a system with a coefficient

matrix is equivalent to finding

the intersection point(s) of all m

surfaces (lines) in an n dimensional space.

If all m surfaces happen to pass through a single point then the

solution is unique

• If the intersected part is a line or a surface, there are an infinite number of solutions, usually expressed by a particular solution added to a linear combination of typically n-m vectors. Otherwise, the solution does not exist.

• In this part, we deal with the case of determining the values x1, x2,…,xn that simultaneously satisfy a set of equations.

mxnA

Small Systems of Linear Equations

1. Graphical Method2. Cramer’s Rule3. The Elimination of Unknows

1. Graphical Method

When solving a system with two

linear equations in two variables, we are looking for the

point where the two lines cross.

This can be determined by

graphing each line on the same

coordinate system and estimating the

point of intersection.

When two straight lines are

graphed, one of three possibilities may result:

When two lines cross in exactly one point, the

system is consistent and independent

and the solution is the one

ordered pair where the two lines cross. The coordinates of

this ordered pair can be

estimated from the graph of the

two lines:

Case 1

Independent system:

one solution point

Graphical Method

This graph shows two distinct lines that are parallel.

Since parallel lines never cross, then there can be no

intersection; that is, for a system of

equations that graphs as parallel lines, there can be no solution. This is

called an "inconsistent"

system of equations, and it has no solution.

Case 2

Inconsistent system:

no solution andno

intersection point

Graphical Method

This graph appears to show

only one line. Actually, it is the same line drawn

twice. These "two" lines,

really being the same line,

"intersect" at every point along their

length. This is called a

"dependent" system, and the "solution" is the

whole line.

Case 3

Dependent system:

the solution is the

whole line

Graphical Method

ADVANTAGES:The graphical method is good because it clearly illustrates the principle involved.

DISADVANTAGES:•It does not always give us an exact solution. •It cannot be used when we have more than two variables in the equations.

Graphical Method

For instance, if the lines cross at a shallow angle it can be just about impossible to tell where the lines cross:

Example

Solve the following system by graphing.

2x – 3y = –24x + y = 24

First, we must solve each equation for "y=", so we can graph easily:

2x – 3y = –2 4x + y = 242x + 2 = 3y y = –4x + 24(2/3)x + (2/3) = y

Graphical Method

x y = (2/3)x + (2/3) y = –4x + 24

–4–8/3 + 2/3 = –6/3 = –

216 + 24 = 40

–1 –2/3 + 2/3 = 0 4 + 24 = 28

2 4/3 + 2/3 = 6/3 = 2 –8 + 24 = 16

510/3 + 2/3 = 12/3 =

4–20 + 24 = 4

816/3 + 2/3 = 18/3 =

6–32 + 24 = –

8

Graphical Method

Solution: (x, y) = (5, 4)

The second line will be easy to graph using just the slope and intercept, but it is necessary a T-chart for the first line.

Cramer’s Rule

Cramer’s rule is another technique that is best

suited to small numbers of equations. This rule

states that each unknown in a system of

linear algebraic equations may be

expressed as a fraction of two determinants

with denominator D and with the numerator obtained from D by

replacing the column of coefficients of the

unknown in question by the constants b1, b2,

… ,bn.

1 12 13

2 22 23

3 32 331

b a a

b a a

b a ax

D

For example, x1 would be computed as

Example

Solution:

We begin by setting

up and evaluating the three determina

nts:

Use Cramer’s Rule to solve the system:

5x – 4y = 26x – 5y = 1

1 1

2 2

5 4 5 5 6 4 25 4 1

6 5

a bD

a b

1 1

2 2

2 4 2 5 1 4 10 4 6

1 5

c bDx

c b

1 1

2 2

5 4 5 1 6 2 5 12 7

6 1

a cDy

a c

From Cramer’s Rule, we have:

6 6

1

Dxx

D

and

7 7

1

Dyx

D

The solution is (6,7)

Cramer’s Rule does not apply if D=0. When D=0 , the system is either inconsistent or dependent.

Another method must be used to solve it.

Example

The Elimination of Unknows

The basic strategy is to multiply the equations by

constants so that of the unknowns will

be eliminated when the two equations are combined. The result is a single

equation that can be solved for the

remaining unknown. This value can then be substituted into

either of the original equations to

compute the other variable.

The elimination of unknowns by combing equations is an algebraic approach that can be illustrated for a set of two equations:

11 1 12 2 1

21 1 22 2 2

1

2

a x a x b

a x a x b

For example, these equations might be multiplied by a21 and a11 to give

11 21 1 12 21 2 1 21

21 11 1 22 11 2 2 11

3

4

a a x a a x ba

a a x a a x b a

Subtracting Eq. 3 from 4 will, therefore, eliminate the xt term from the equations to yield

The Elimination of Unknows

22 11 2 12 21 2 2 11 1 21a a x a a x b a ba

Which can be solve for

11 2 21 1

211 22 12 21

a b a bx

a b a a

This equation can then be substituted into Eq. 1, which can be solved for

22 1 12 2

111 22 12 21

a b a bx

a b a a

The Elimination of UnknowsNotice that these equations follow directly from Cramer’s rule, which states

EXAMPLEUse the elimination of unknown to solve,

1 12

2 22 1 22 12 21

11 12 11 22 12 21

21 22

11 1

21 2 11 2 1 212

11 12 11 22 12 21

21 22

b a

b a ba a bx

a a a a a aa a

a b

a b a b bax

a a a a a aa a

1 2

1 2

1

2

3 2 18

2 2

Solution

2(18) 2(2)4

3(2) 2( 1)

3(2) ( 1)183

3(2) 2( 1)

x x

x x

x

x

Gaussian Elimination

Gaussian Elimination is

considered the workhorse of

computational science for the

solution of a system of linear equations.

Karl Friedrich Gauss, a great 19th

century mathematician, suggested this

elimination method as a part of his

proof of a particular theorem.

Computational scientists use this “proof” as a direct

computational method.

Gaussian Elimination is a systematic application of elementary row operations to a system of linear equations in order to convert the system to upper triangular form.

Once the coefficient matrix is in upper triangular form, we use back substitution to find a solution.

Gaussian Elimination

The general procedure for Gaussian Elimination can be summarized in the

following steps: 1. Write the augmented matrix for the system of

linear equations.

2. Use elementary row operations on the augmented matrix [A|b] to transform A into upper triangular form. If a zero is located on the diagonal, switch the rows until a nonzero is in that place. If you are unable to do so, stop; the system has either infinite or no solutions.

3. Use back substitution to find the solution of the problem.

Gaussian Elimination

1. Write the augmented matrix for the system of linear equations.

2 4

2 6

2 7

y z

x y z

x y z

0 2 14

1 1 2 6

2 1 17

2.Use elementary row operations on the augmented matrix [A|b] to transform A into upper triangular form.

2

1

0 2 14 ( )

1 1 2 6 ( )

2 1 17

r

r

3 1

1 1 2 6

0 2 14

2 1 17 ( ) ( 2 )r r

Change row 1 to row 2

and vice versa

Example

Gaussian Elimination

Notice that the original coefficient matrix had a “0”

on the diagonal in row 1. Since we needed to use

multiples of that diagonal element to eliminate the

elements below it, we switched two rows in order to move a nonzero element into that position. We can use the same technique

when a “0” appears on the diagonal as a result of calculation. If it is not

possible to move a nonzero onto the diagonal by

interchanging rows, then the system has either

infinitely many solutions or no solution, and the

coefficient matrix is said to be singular.

3 2

1 1 2 6 1 1 2 6

0 2 1 4 0 2 1 4

0 1 3 5 1 5 30 0( ) ( ) 22

r r

Since all of the nonzero elements are now located in the “upper triangle” of the matrix, we have completed the first phase of solving a system of linear equations using Gaussian Elimination.

Gaussian EliminationThe second and final phase of Gaussian Elimination is back substitution. During this phase, we solve for the values of the unknowns, working our way up from the bottom row.

3. Use back substitution to find the solution of the problem

1 1 2 6

0 2 1 4

5 30 0 2

The last row in the augmented matrix represents the equation:

5 63 z=

2 5z

Gaussian Elimination

4 4 6 5 72 4

2 2 5z

y z y y

117 62 6 x=6-y-2z=6- 2 5 5 5x y z x

The second row of the augmented matrix represents the equation:

Finally, the first row of the augmented matrix represents the equation

Gaussian-Jordan EliminationAs in Gaussian

Elimination, again we are transforming

the coefficient matrix into

another matrix that is much

easier to solve, and the system represented by

the new augmented

matrix has the same solution

set as the original system

of linear equations.

In Gauss-Jordan Elimination, the goal is to transform the coefficient matrix into a diagonal matrix, and the zeros are introduced into the matrix one column at a time. We work to eliminate the elements both above and below the diagonal element of a given column in one pass through the matrix.

Gaussian-Jordan Elimination

The general procedure for Gauss-Jordan Elimination can be summarized in the following steps:

1. Write the augmented matrix for the system of linear equations.

2. Use elementary row operations on the augmented matrix [A|b] to transform A into diagonal form. If a zero is located on the diagonal, switch the rows until a nonzero is in that place. If you are unable to do so, stop; the system has either infinite or no solutions.

3. By dividing the diagonal element and the right-hand-side element in each row by the diagonal element in that row, make each diagonal element equal to one.

Gaussian-Jordan Elimination

We will apply Gauss-Jordan Elimination to the same example that was used to demonstrate Gaussian Elimination

1-Write the augmented matrix for the system of linear equations.

Example

2 4

2 6

2 7

y z

x y z

x y z

0 2 14

1 1 2 6

2 1 17

2

1

0 2 14 ( )

1 1 2 6 ( )

2 1 17

r

r

3 1

1 1 2 6

0 2 14

2 1 17 ( ) ( 2 )r r

2. Use elementary row operations on the augmented matrix [A|b] to transform A into diagonal form.

Gaussian-Jordan Elimination

Notice that the coefficient matrix is now a diagonal matrix with ones on the diagonal. This is a special matrix called the identity matrix.

3-By dividing the diagonal element and the right-hand-side element in each row by the diagonal element in that row, make each diagonal element equal to one.

1 31 2

2 3

3 2

3 111 3 ( ) ( )( ) ( ) 1 0 5 51 1 2 6 4 1 0 02 22 14 0 2 1 4 0 2 1 4 ( ) ( ) 0 2 05 5

0 1 3 5 1 5 3 5( ) ( ) 0 0 0 0 32 2 2

r rr r

r r

r r

2

3

1111551 0 0 1 0 0

714 1 0 2 0 ( ) ( ) 0 1 05 2 55 0 0 10 0 23 6( ) ( )2 5 5

Entonces,

11 7 6x= , y= , and z=

5 5 5

r

r

LU DecompositionJust as was the case with Gauss elimination,

Lu decomposition requires pivoting to

avoid division by zero. However, to simplify

the following description, we will defer the issue of

pivoting until after the fundamental approach

is elaborated. In addition, the following explanation is limited

to a set of three simultaneous

equations. The results can be directly extended to n-

dimensional systems.method.

Linear algebraic notation can be rearranged to give

0 1A X B

Suppose that this equation could be expressed as an upper triangular system:

11 12 13 1 1

22 23 2 2

33 3 3

0 2

0 0

u u u x d

u u x d

u x d

Elimination is used to reduce the system to upper triangular form. The above equation can also be expressed in matrix notation and rearranged to give 0 3U X D

LU Decomposition

Now, assume that there is a lower diagonal matrix with 1’s on the diagonal,

21

31 32

1 0 0

1 0 4

1

L l

l l

That has the property that when Eq. 3 is premultiplied by it, Eq. 1 is the result. That is,

5L U X D A X B

If this equation holds, it follows from the rules for matrix multiplication that

6

and

7

L U A

L D B

LU DecompositionA two-step strategy for obtaining solutions can be based on Eqs. 3, 6 y 7.

• LU decomposition step. [A] is factored or “decomposed” into lower [L] and upper [U] triangular matrices.

• Substitution step. [L] and [U] are used to determine a solution {X} for a right-hand side {B}. This step itself consists of two steps. First, Eq. 7 is used to generate an intermediate vector {D} by forward substitution. Then, the result is substituted into Eq. 3, which can solved by back substitution for [X].

In the other hand, Gauss Elimination can be implemented in this way.

Bibliography

CHAPRA, Steven. Numerical Methods for engineers. Editorial McGraw-Hill. 2000.http://www.efunda.com/mathhttp://www.purplemath.comhttp://ceee.rice.edu/Books