MÉTODOS DE RESOLUCIÓN

Systems of Linear Algebraic Equations

General Representation

as are constant coefficients, bs are constants, x are the unknown variables, and n is the number of equations

Analysis of System LAE

Consistent, independent equationsIf equations are not contradictory and not simply multiplies of the other, then solution to LAE problems will be unique.Example:

Inconsistent equationsIf equations are contradictory, then no solution exists.Example:

Analysis of System LAE (2)

Dependent equationsIf equations are multiplies of others, many solutions can be foundExample:

Ill-conditioned equationsIf a small change in any of the coefficients will change the solution set from unique to either infinite or empty, the system of LAE is called to be ill-conditioned.Numerical solution will be difficultExample:

Curso de modelizacin y simulacin de procesos. ETSII. Manuel Rodrguez

MTODOS DE RESOLUCINEcuaciones algebraicas lineales

Gauss Elimination - Concept

If two equations have a point in common, then that point also satisfies any linear combination of the two equations. Suppose that are the solutions to linear equations

then also satisfies the linear combination of these linear equations:

Gauss Elimination Algorithm

Add a multiple of row onto row to form a new row

Repeat (1) until we get an upper (or lower) triangular matrix of LAE coefficients;

Apply back substitution to solve for each variable

Step 1: Forward elimination until we get the upper (or lower) triangular matrix of LAE coefficientsStep 2: Backward substitution

Pitfalls of the Gauss Elimination

When does the Gauss elimination get solution? When inverse of exists

When equations are consistent and independent Satisfy the following rank condition:

Limitation of algorithmWhen pivot element equals zeroWhen pivot element is significantly smaller than the coefficient being used to eliminate (or Ill-conditioned)Round-off errors



LU Decomposition - Concept

A matrix A is decomposed (or factorised) into lower and upper matrix factors, for example if A is a 3x3 matrix:

Methods to find LU factorsGauss elimination (Doolittle form)Direct computation Crout form ( )Cholesky form ( )

Motivation for LU Decomposition

Solve the LAE Problems in a more efficient way

Algorithm for solving an LAE Problem using the LU Decomposition

Find the LU factor of the A matrix: A = LUAlgebraic problem is transformed from Ax = b to LUx = b Define y = UxSolve for y using the forward substitutionSolve for x using the backward substitution


Ecuaciones algebraicas linealesMTODOS DE RESOLUCIN

Step 2: Rows 1 and 2 are unchanged. Rows 3 and 4 are transformed by

to yield:

Step 3: The fourth row is modified to complete the forward eliminating stage by:

to yield:

Hence, the LU factors are:

LU Decomposition using Gauss Elimination

Suppose we try to find LU factors of the following matrix:

Initialization: Let U = A and

Step 1: Row 1 is unchanged, and rows 2-4 are modified by:to give:



LU Decomposition using Crout Method

Based on equating the elements of the product with the corresponding elements of , where the diagonal elements of are 1s . For a 3 x 3 matrix

AlgorithmStep 1: Set , then solve for the remaining elements in the first row of and the first column of . Step 2: Find , then solve for the remainder of the second row of and the second column of Continue for the rest

General Formula for Crouts Method

General formula for the first column of L, the first row of U and the diagonal element of U

General formula for the j-th row of L and the j-th column of U:For j= 2, 3, , n-1; i = j, j+1, , n; and k = j+1, j+2, , n

General formula for the last row of L



LU Decomposition using Choleskys Method

The matrix is symmetric and positive definite.

For a 3 x 3 matrix:

Algorithm:Find , then solve for the remaining elements in the first row of and the first column of .Find , then solve for the remainder of the second row of and the second column of Continue for the rest.

Generalization of Choleskys Method

For the k-th row



Variations of Algorithm

Jacobi MethodBased on the transformation of to , in which the matrix has zeros on the diagonal. The vector is updated using the previous estimate for all components of to evaluate the right hand side of the equation (simultaneous updating).Gauss-Seidel MethodBased on the transformation of to , in which the matrix has zeros on the diagonal as in Jacobi method, but each component of the vector is updated immediately as each iteration progresses (sequential updating). Successive Over Relaxation (SOR) MethodLike Gauss-Seidel method, but we introduce an additional parameter , that may accelerate the convergence of iteration. For , the method is known as successive under relaxation, while for is known as successive over relaxation.

Iterative Methods: Concept

Basic IdeaTo solve the i-th equation in the system for the i-th variable by converting the given system to be suitable for finding the solution in the iterative manner.

AlgorithmTransform to so that we can solve iteratively for as .

Stopping ConditionsTo stop the iterations when the norm of the change in the solution vector from one iteration to the next is sufficiently small.To stop the iterations when the norm of the residual vector is below a specified tolerance.


Ecuaciones algebraicas linealesMTODOS DE RESOLUCIN

SOR Method

Example

SOR Iterative Scheme

It can be seen that if , it becomes the iterative scheme for Gauss-Seidel method. The parameter is selected to improve the convergence of algorithm.

Implementation for Different Methods

Jacobi MethodIterative Scheme:

Initial condition/guess:Stopping criteriaGauss-Seidel MethodIterative Scheme

Initial condition/guessStopping criteria


MTODOS DE RESOLUCINEcuaciones algebraicas no lineales

Problem Statement

Given f(x) as nonlinear (algebraic or not) function in x, find the value of x, say x*, that makes f(x*) ~ 0.x* is sometimes called a zero or root of f(x).x* can be sought analytically (exact solution) or numerically (approximate solution)

General consideration on the selection of numerical methodsIs this a special function that will be evaluated often?How much precision needed?How fast and robust must the method be?Is the function a polynomial?Does the function have singularities?There is no single root-finding method that is best for all situation

Basic strategy of root-finding

Select a range of xPlot the function f(x) versus xFind the intersection of the function f(x) with the x-axisIf no intersection is found, select another range of x and repeat Step 2-3.

f(x)

x

x1

x2

x*

Feasible range

Infeasible range

x3

f(x1)

f(x2)

Graphical Method



Bracketing Methods

Principle:A function typically changes sign in the vicinity of a root.A root is bracketed on the interval [a,b], if f(a) and f(b) have the opposite sign.However a sign change occurs for singularities as well as rootsCommon Usage:Bracketing methods are used to make initial guesses at the roots, not to accurately estimate the values of the roots.

A simple test for sign changeAlternative 1: f(a) f(b) < 0 ?Alternative 2: Use built in sign function: sign(fa) ~= sign(fb) ?



Bisection Algorithm

Select an interval [a,b] in which the location of zero is detected Apply the sign test procedure (i.e. {f(a) * f(b) < 0} or {sign(f(a)) ~= sign(f(b))} )Calculate a mid-point as the next estimate of the zero

Apply the sign test to determine a feasible interval for the new estimate of the zeroTest the function whether it changes sign for the candidates of new interval, [a,m] or [m,b] .If f(a)*f(m) = 0, the root equals m; terminate the computation Repeat step 2-3 for the feasible interval

Bisection Method

a

b

f(x)

x

Mid-point

Next estimate of Bisection

f(a)

f(b)



False-position Algorithm

Select an interval [a,b] in which the location of zero is detected Apply the sign test procedure (i.e. {f(a) * f(b) < 0} or {sign(f(a)) ~= sign(f(b))} )Calculate an intersection point as the next estimate of the zero

Apply the sign test to determine a feasible interval for the new estimate of the zeroTest the function whether it changes sign for the candidates of new interval, [a,m] or [m,b] .If f(a)*f(m) = 0, the root equals m; terminate the computation Repeat step 2-3 for the feasible interval

False-Position Method

a

b

f(x)

x

Intersection point

Next estimate of False-position

f(a)

f(b)



Convergence Rate

Number of iterations

Relative Errors

False-position method

Bisection method

10

1

Comparisons of Bisection and False-position

Similarity:Both methods make use of two initial guesses of the root (lower and upper bound)Require a sign test procedure to determine the feasible range for searching the rootAlways converge to the true value with certain tolerance.Difference:Updating the new estimate of the root using different formula/strategyBisection: use the mid-point strategyFalse-position: use the triangular principle (or a linear interpolation)In general, false position method converges faster than the bisection method. However, for some special functions, the false-position method may be very slow to converge.



Open Methods: Principles

Use a functional approximation (i.e. straight-line, quadratic or polynomial) to generate new estimate of the root we wish to find

MethodsFixed-point IterationNewton-Raphson method (a straight-line approximation using gradient information)Secant method (a straight-line approximation of two points)Mullers Method (a quadratic approximation of three points)


Ecuaciones algebraicas no linealesMTODOS DE RESOLUCIN

Bracketing vs Open Methods

Bracketing methodsThe root is located within a lower and upper bound (or prescribed interval), and then the estimate is updated to eventually always converge to the true root with a certain tolerance.

Open methodsUse a single starting point (or two starting points that do not necessarily bracket the root) and a formula to find the root, and (hopefully) end up to the true value.These methods sometimes diverge or move away from the true root, but when these methods converge, they usually do so much more quickly than the bracketing methods



Fixed-point Iteration

Transform f(x) to the form of: x = g(x)Select initial estimate of the root, xo , and set i = 0Apply a fixed point iteration to calculate the next estimate of the root as follows:

Repeat step 3 for i = 1, 2, . until satisfying the convergent criteria ( is a specified tolerance)

Problem of finding the root: f(x) = 0

Converge

Diverge

Convergent Analysis

The fixed point iteration may or may not be convergent, depending upon the nature of g(x)If |g(x)| < 1, the fixed point algorithm will linearly converge to the true root If |g(x)| 1, the fixed point algorithm will diverge

To guarantee the convergence, add a checking step into the algorithmFor example:

dg = derg(x);% derivative of g(x)if abs(dg(x)) < 1,Apply the fixed point iterationelse Fixed point iteration divergesend



Newton-Raphson

f(x)

x

Next estimate of Newton-Raphson

Tangent-line

xi

xi+1

Newton-Raphson Algorithm

Select an initial estimate, xo of the root Calculate f(xo) and f /(xo)Apply a tangent-line approximation with a gradient information to calculate the next estimate of the zero

Repeat step 3 for k = 1,2, until satisfying the convergent criteria

OR

Characteristics

Only require a single initial estimate. But need to know the derivative of the function.Usually converge faster (quadratically convergent!). But there may hit a singular point when f(xk) = 0.As the fixed point iteration, the convergence of Newton-Raphson method depends on the nature of the function and initial guess.Initial guess is expected to sufficiently close to the true valueGood initial guess is helped byPhysical understanding on the problem, orGraphical method



Secant method

f(x)

x

Straight-line

Intersectionpoint

Next estimate of Secant

xi

xi+2

xi+1

Secant Algorithm

Select two initial estimates xo and x1 Calculate the function values at those points f(xo) and f(x1) Apply a straight-line approximation passing through two points to calculate the next estimate of the zero

4.Repeat step 3 for k = 2, 3, . until satisfying the convergent criteria



Comparison with False-position method

Require two initial guessesBut the points does not need to bracket the rootThe estimate of the root at the second iteration is different due t the nature of the derivative approximation

Secant method

False-position method

Comparison with Newton-Raphson method

Does not require the derivative of the functionThe secant method is suitable if we do not know the function (just have the data points)Require two initial guessesApproximate the derivative of the function using the two data points:

At the first iteration, both methods produce the same estimate. Afterthat, each method has a different estimate.



Difficulties with multiple root problems

The fact that the function does not change sign at even multiple roots. So, the bracketing strategy cannot be applied!The fact that not only f(x), but also f(x), goes to zero at the root. Possible problems for Newton-Raphson and Secant methods!SOLUTION: Use the modified Newton-Raphson methodsAlternative one: If we have m multiple roots, the updated estimate follows:

Alternative two: Use the ratio of the function to its derivative, and then apply the Newton-Raphson formula for this ratio function

Systems of nonlinear algebraic equation

Find x that solves for the following set of equations

Or in the compact form



Newton-Raphson Method

Use the Jacobian (matrix of partial derivatives) of the system; At each stage of the iteration process, an updated approximate solution vector is found from the current approximate solution according to the equationwhere is the Jacobian matrix at points defined as



f(x)

x

x2

Next estimate of Muller

Parabolic

Mullers Method

x0

x1

Mullers Algorithm

Select three points [xo,f(xo)]; [x1,f(x1)] and [x2,f(x2)] Compute the coefficients a, b and cApply a quadratic formula to calculate the next estimate of the zero

Repeat step 2-3 until satisfying the convergent criteria

Real and complex rootCan be located

Which point is discarded in Muller Algorithm

Two general strategies are typically used:

If only real roots are being located, we choose the two original points that are nearest the new root estimate, x3

If both real and complex roots are being evaluated, a sequential approach is employed. That is just like the secant method, x1, x2 and x3 take place of x0, x1 and x2.



MATLAB Built in Function (1)

The built in fzero function (Optimization Toolbox) is a hybrid method that combines bisection, secant and reverse quadratic interpolation to a find the root of f(x) = 0

Syntax:

x = fzero(fun,x0)

X0 can be scalar or a two element vectorIf x0 is a scalar, fzero tries to create its own bracketIf x0 is a two element vector, fzero uses the vector as a bracket

MATLAB Built in Function (2)

The built in fsolve function (Optimization Toolbox) is an M-file function to solve a system of nonlinear equations:

Syntax:

x = fsolve(fun,x0)

x0 is a vector of initial estimate of the roots

Date post:	31-Dec-2015
Category:	Documents
Upload:	sawyer-stephens
View:	37 times
Download:	0 times

MÉTODOS DE RESOLUCIÓN

Documents