The Islamic University of GazaFaculty of Engineering

Civil Engineering Department

Numerical Analysis ECIV 3306

Chapter 6

Open Methods

Open Methods

• Bracketing methods are based on assuming an interval of the function which brackets the root.

• The bracketing methods always converge to the root.

• Open methods are based on formulas that require only a single starting value of x or two starting values that do not necessarily bracket the root.

• These method sometimes diverge from the true root.

1 .Simple Fixed-Point Iteration

• Rearrange the function so that x is on the left side of the equation:

)()(0)(

1 ii xgxxxgxf

• Bracketing methods are “convergent”.• Fixed-point methods may sometime “diverge”,

depending on the stating point (initial guess) and how the function behaves.

Simple Fixed-Point Iteration

Examples:1.

2. f(x) = x 2-2x+3 x = g(x)=(x2+3)/23. f(x) = sin x x = g(x)= sin x + x3. f(x) = e-x- x x = g(x)= e-x

xxg

orxxg

orxxg

xxxxf

21)(

2)(

2)(

02)(2

2

Simple Fixed-Point Iteration Convergence

• x = g(x) can be expressed as a pair of equations:y1= xy2= g(x)…. (component equations)

• Plot them separately.

Simple Fixed-Point Iteration Convergence

• Fixed-point iteration converges if :

( ) 1 (slope of the line ( ) )g x f x x

• When the method converges, the error is roughly proportional to or less than the error of the previous step, therefore it is called “linearly convergent.”

Simple Fixed-Point Iteration-Convergence

Steps of Simple Fixed Pint Iteration• 1. Rearrange the equation f(x) = 0 so that x is on

the left hand side and g(x) is on the right hand side. – e.g f(x) = x2-2x-1 = 0 x= (x2-1)/2

g(x) = (x2-1)/2

• 2. Set xi at an initial guess xo.• 3. Evaluate g(xi)• 4. Let xi+1 = g(xi)• 5. Find a=(Xi+1 – xi)/Xi+1, and set xi at xi+1

• 6. Repeat steps 3 through 5 until |a|<= a

Example: Simple Fixed-Point Iteration

1. f(x) is manipulated so that we get x=g(x) g(x) = e-x

2. Thus, the formula predicting the new value of x is: xi+1 = e-xi

3. Guess xo = 0

4. The iterations continues till the approx. error reaches a certain limiting value

f(x)

Root x

f(x)

x

f(x)=e-x - x

g(x) = e-x

f1(x) = x

f(x) = e-x - x

Example: Simple Fixed-Point Iteration

i xi g(xi) a% t%

0 0 1.01 1.0 0.367879 100 76.32 0.367879 0.692201 171.8 35.13 0.692201 0.500473 46.9 22.14 0.500473 0.606244 38.3 11.85 0.606244 0.545396 17.4 6.896 0.545396 0.579612 11.2 3.837 0.579612 0.560115 5.90 2.28 0.560115 0.571143 3.48 1.249 0.571143 0.564879 1.93 0.70510 0.564879 1.11 0.399

Example: Simple Fixed-Point Iteration

ixig(xi) a% t%

001.011.00.367879 100 76.320.3678790.692201 171.835.130.6922010.50047346.922.140.5004730.60624438.311.850.6062440.54539617.46.8960.5453960.57961211.23.8370.5796120.5601155.902.280.5601150.5711433.481.2490.5711430.5648791.930.705

100.5648791.110.399

Ex 5.1

Flow Chart – Fixed Point

Start

Input: xo , s, maxi

i=0a=1.1s

1

Stop

1

whilea< s &

i >maxi

xn=0

x0=xn

100%n oa

n

x xx

Print: xo, f(xo) ,a , i 0

1nx g x

i i

False

True

2. The Newton-Raphson Method

• Most widely used method.• Based on Taylor series expansion:

)()(

)(0g,Rearrangin

0)f(x when xof value theisroot The

...!2

)()()()(

1

1

1i1i

2

1

i

iii

iiii

iiii

xfxfxx

xx)(xf)f(x

xxfxxfxfxf

Solve for

Newton-Raphson formula

The Newton-Raphson Method

• A tangent to f(x) at the initial point xi is extended till it meets the x-axis at the improved estimate of the root xi+1.

• The iterations continues till the approx. error reaches a certain limiting value.

f(x)

Root x

xixi+1

f(x) Slope f /(xi)

f(xi)

)()(

)()(

/

/

i

ii1i

1ii

ii

xfxfxx

xx0xfxf

Example: The Newton Raphson Method

11)()(

/1

x

x

ix

x

ii

iii e

xexe

xexxfxfxx

• Use the Newton-Raphson method to find the root of e-x-x= 0 f(x) = e-x-x and f`(x)= -e-x-1; thus

Iter. xi t%0 0 1001 0.5 11.82 0.566311003 0.1473 0.567143165 0.000024 0.567143290 <10-8

Flow Chart – Newton Raphson

Start

Input: xo , s, maxi

i=0a=1.1s

1

Stop

1

whilea >s & i <maxi

xn=0

x0=xn

100%n oa

n

x xx

Print: xo, f(xo) ,a , i

00 '

0

1

n

f xx x

f x

i i

False

True

Pitfalls of The Newton Raphson Method

Cases where Newton Raphson method diverges or exhibit poor convergence.

a) Reflection point b) oscillating around a local optimumc) Near zero slop , and d) zero slop

3. The Secant Method

The derivative is sometimes difficult to evaluate by the computer program. It may be replaced by a backward finite divided difference

)()())((

i1i

i1iii1i xfxf

xxxfxx

Thus, the formula predicting the xi+1 is:

/ 1

1

( ) ( )( ) i i

ii i

f x f xf x

x x

/ ( )if x

The Secant Method

• Requires two initial estimates of x , e.g, xo, x1. However, because f(x) is not required to change signs between estimates, it is not classified as a “bracketing” method.

• The scant method has the same properties as Newton’s method. Convergence is not guaranteed for all xo, x1, f(x).

Secant Method: Example

• Use the Secant method to find the root of e-x-x=0; f(x) = e-x-x and xi-1=0, x0=1 to get x1 of the first iteration using:

Iter xi-1 f(xi-1) xi f(xi) xi+1 t%1 0 1.0 1.0 -0.632 0.613 8.02 1.0 -0.632 0.613 -0.0708 0.5638 0.583 0.613 -0.0708 0.5638 0.00518 0.5672 0.0048

)()())((

i1i

i1iii1i xfxf

xxxfxx

Comparison of convergence of False Position and Secant Methods

False Position Secant Method

Use two estimate xl and xu Use two estimate xi and xi-1

f(x) must changes signs between xl and xu

f(x) is not required to change signs between xi and xi-1

Xr replaces whichever of the original values yielded a function value with the same sign as f(xr)

Xi+1 replace xiXi replace xi-1

Always converge May be diverge

Slower convergence than Secant in case the secant converges.

If converges, It does faster then False Position

11

1

( )( )( ) ( )

i i ii i

i i

f x x xx x

f x f x

( )( )( ) ( )

u l ur u

l u

f x x xx x

f x f x

Comparison of convergence of False Position and Secant Methods

• Use the false-position and secant method to find the root of f(x)=lnx. Start computation with xl= xi-1=0.5, xu=xi = 5.

1. False position method

2. Secant method Iter xi-1 xi xi+1

1 0.5 5.0 1.8546

2 5 1.8546 -0.10438

Iter xl xu xr 1 0.5 5.0 1.8546 2 0.5 1.8546 1.2163

3 0.5 1.2163 1.0585

False Position and Secant Methods

xi-1

xixu

xl

Although the secant method may be divergent, when it converges it usually does so at a quicker rate than the false position method

See the next figure

• Comparison of the true percent relative Errors Et for the methods to the determine the root of

f(x)=e-x-x

Flow Chart – Secant Method

Start

Input: x-1 , x0,s, maxi

i=0a=1.1s

1

Stop

1

whilea >s & i < maxi

Xi+1=0

Xi-1=xi

Xi=xi+1

1

1

100%i ia

i

x xx

Print: xi , f(xi) ,a , i11

1

( )( )( ) ( )

1

i i ii i

i i

f x x xx x

f x f xi i

False

True

Modified Secant Method

Rather than using two initial values, an alternative approach is using a fractional perturbation of the independent variable to estimate

1( )

( ) ( )i i

i ii i i

x f xx x

f x x f x

is a small perturbation fraction

/ ( ) ( )( ) i i i

ii

f x x f xf x

x

/ ( )if x

Modified Secant Method: Example

• Use the modified secant method to find the root of f(x) = e-x-x and, x0=1 and =0.01

0 0

0 0 0 0

1 1

1 1

1 1 1 1

2 1

1 0.63212

1.01 0.64578

( ) 0.537263

First Iteration

Second Iteration

5.3%( ) ( )

0.537263 0.047083

0.542635 0.038579

( )(

i ii i t

i i i

i ii i

x f x

x x f x x

x f xx x xf x x f x

x f x

x x f x x

x f xx x xf x

0.56701 0.0236%

) ( ) ti i ix f x

Multiple Roots

x

f(x)= (x-3)(x-1)(x-1) = x3- 5x2+7x -3

f(x)

1x

3Double roots

f(x)= (x-3)(x-1)(x-1)(x-1) = x4- 6x3+ 125 x2- 10x+3

f(x)

1 3

triple roots

Multiple Roots

•“Multiple root” corresponds to a point where a function is tangent to the x axis.

•Difficulties- Function does not change sign with double

(or even number of multiple root), therefore, cannot use bracketing methods.

- Both f(x) and f′(x)=0, division by zero with Newton’s and Secant methods which may diverge around this root.

4. The Modified Newton Raphson Method

• Another u(x) is introduced such that u(x)=f(x)/f /(x); • Getting the roots of u(x) using Newton Raphson

technique:

)()()()()(

)]([)()()()()(

)()(

//2/

/

1

2/

/////

/1

iii

iiii

i

iiiii

i

iii

xfxfxfxfxfxx

xfxfxfxfxfxu

xuxuxx

This function has roots at all the same locations as the original function

Modified Newton Raphson Method: Example

Using the Newton Raphson and Modified Newton Raphson evaluate the multiple roots of f(x)= x3-5x2+7x-3 with an initial guess of x0=0

)106)(375()7103()7103)(375(

)()()()()(

2322

223

//2/

/

1

iiiiii

iiiiii

iii

iiii

xxxxxxxxxxxx

xfxfxfxfxfxx

7x10x33x7x5xx

xfxfxx 2

i

i2i

3i

ii

ii1i

)(

)(/

•Newton Raphson formula:

•Modified Newton Raphson formula:

Newton Raphson Modified Newton-RaphsonIter xi t% iter xi t%0 0 100 0 0 1001 0.4286 57 1 1.10526 112 0.6857 31 2 1.00308 0.313 0.83286 17 3 1.000002 000244 0.91332 8.75 0.95578 4.46 0.97766 2.2

•Newton Raphson technique is linearly converging towards the true value of 1.0 while the Modified Newton Raphson is quadratically converging.•For simple roots, modified Newton Raphson is less efficient and requires more computational effort than the standard Newton Raphson method

Modified Newton Raphson Method: Example

Systems of Nonlinear Equations

• Roots of a set of simultaneous equations:f1(x1,x2,…….,xn)=0

f2 (x1,x2,…….,xn)=0

fn (x1,x2,…….,xn)=0

• The solution is a set of x values that simultaneously get the equations to zero.

Systems of Nonlinear Equations

Example: x2 + xy = 10 & y + 3xy2 = 57 u(x,y) = x2+ xy -10 = 0 v(x,y) = y+ 3xy2 -57 = 0• The solution will be the value of x and y which makes

u(x,y)=0 and v(x,y)=0 • These are x=2 and y=3• Numerical methods used are extension of the open

methods for solving single equation; Fixed point iteration and Newton-Raphson. (we will only discuss the Newton Raphson)

Systems of Nonlinear Equations: 2. Newton Raphson Method

• Recall the standard Newton Raphson formula:

1( )'( )

ii i

i

f xx xf x

• which can be written as the following formula

1

( )'( )

'( ) ( )

i i i

ii

i

i i i

x x xf xwhere xf x

f x x f x

• By multi-equation version (in this section we deal only with two equation) the formula can be derived in an identical fashion:

• u(x,y)=0 and v(x,y)=0

1

i i

i i

i ii i

i i

i i

i ii i

u ux ux yy vv v

x y

u ux ux yy vv v

x y

Systems of Nonlinear Equations: 2. Newton Raphson Method

• And thus

1

1i i i i

i i i ii i i i

u u v ux y y y

u v v uv v v ux y x yx y x x

1

i ii i

i ii i i i

v uu vy yx x u v v u

x y x y

1

i ii i

i ii i i i

v uu vx xy y u v v u

x y x y

Systems of Nonlinear Equations: 2. Newton Raphson Method

• x 2+ xy =10 and y + 3xy 2 = 57 are two nonlinear simultaneous equations with two unknown x and y they

can be expressed in the form: use the point (1.5,3.5) as initial guess.

2

2 ,

3 , 1 6

u ux y xx yv vy xyx y

i xi yi Ui Vi ui,x ui,y vi,x vi,y a,x a,y

0 1.5 3.5 -2.5 1.625 6.5 1.5 36.75 32.5

1 2.03603 2.84388 -.06435 -4.7560 6.91594 2.03603 24.26296 35.74135 26.3 23.1

2 1.9987 3.00229 1.87 5.27

Systems of Nonlinear Equations: 2. Newton Raphson Method