E. T. S. I. Caminos, Canales y Puertos 1

EngineeringComputation

Lecture 4

E. T. S. I. Caminos, Canales y Puertos 2

Error Analysis for N-R :

Recall that

Taylor Series gives:

ii 1 i

i

f (x )x x

f '(x )

2r i i r i r i

f "( )f (x ) f (x ) f '(x )(x x ) (x x )

2!

where xr x xi and f(xr) = 0

Open Methods (Newton-Raphson Method)

E. T. S. I. Caminos, Canales y Puertos 3

Dividing through by f '(xi) yields

Ei+1 is proportional to Ei2 ==> quadratic rate of convergence.

2ir i r i

i i

f (x ) f "( )0 (x x ) (x x )

f '(x ) 2f '(x )

2i 1 i r i r i

i

f '' ( )(x x ) (x x ) (x x )

2f '(x )

2r i 1 r i

i

f '' ( )(x x ) (x x )

2f '(x )

OR

2i 1 i

i

f "( )E E

2!f '(x )

Open Methods (Newton-Raphson Method)

E. T. S. I. Caminos, Canales y Puertos 4

Summary of Newton-Raphson Method:

Advantages:1. Can be fast

Disadvantages:1. May not converge

2. Requires a derivative

Open Methods (Newton-Raphson Method)

E. T. S. I. Caminos, Canales y Puertos 5

Secant Method

Approx. f '(x) with backward FDD:

Substitute this into the N-R equation:

to obtain the iterative expression:

i 1 i

i 1 i

f (x ) f (x )f '(x)

x x

ii 1 i

i

f (x )x x

f '(x )

i i 1 ii 1 i

i 1 i

f (x )(x x )x x

f (x ) f (x )

Open Methods (Secant Method)

E. T. S. I. Caminos, Canales y Puertos 6

Secant Method

xi = xi+1

i i 1 ii 1 i

i 1 i

f (x )(x x )x x

f (x ) f (x )

x

f(x)

f(xi)

xi

f(xi-1)

i-1 ii

i-1 i

f(x ) - f(x )f '(x )

x - x

f(x)

xi-1xi+1x

f(xi)

xi

f(xi-1)

i-1 ii

i-1 i

f(x ) - f(x )f '(x )

x - x

xi-1xi+1

Open Methods (Secant Method)

E. T. S. I. Caminos, Canales y Puertos 7

1) Requires two initial estimates: xi-1 and xi

These do NOT have to bracket root !

2) Maintains a strict sequence:

Repeated until:

a. | f(xi+1) | < k with k = small number

b.

c. Max. number of iterations is reached.

3. If xi and xi+1 were to bracket the root, this would be the same as the False-Position Method. BUT WE DON'T!

i i 1 ii 1 i

i 1 i

f (x )(x x )x x

f (x ) f (x )

i 1 ia s

i 1

x x100%

x

Open Methods (Secant Method)

E. T. S. I. Caminos, Canales y Puertos 8

Fixed point Method

predict a value of xi+1 as a function of xi.

Convert f(x) = 0 to x = g(x)

iteration steps: xi+1 = g(xi )

x(new) = g(x(old) )

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 9

Example II:

x = sin(x) –> xi+1 = sin(xi) OR

x = arcsin(x) –> xi+1 = arcsin(xi)

sin(x)f (x) 1.0 0.0

x

201 1 i

7,500 1,000i

20

i 11 1 i

i 1.07.5

Example I:

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 10

Convergence:

Does x move closer to real root (?)

Depends on:1. nature of the function2. accuracy of the initial estimate

Interested in:1. Will it converge or will it diverge?2. How fast will it converge ?

(rate of convergence)

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 11

Convergence of the Fixed point Method:

Root satisfies: xr = g(xr)

The Taylor series for function g is:

xi+1 = g(xr) + g'(x)(xi - xr) xr < x < xi

Subtracting the second equation from the first yields

(xr – xi+1) = g'(x) (xr – xi) or

1. True error for next iteration is smaller than the true error in the previous iteration if |g'(x)| < 1.0 (it will converge).

2. Because g'(x) is almost constant, the new error is directly proportional to the old error (linear rate of convergence).

E i1 g' () E i

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 12

Further Considerations:

Convergence depends on how f(x) = 0 is

converted into x = g(x)

So . . .

Convergence may be improved

by recasting the problem.

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 13

can be small, even though xnew is not close to root.

Remedy: Do not completely rely on a to ensure that the problem is solved.

Check to make sure |f(xnew) | < .

new olda

new

x xx 100%

x

Convergence Problem:

For slowly converging functions

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 14

Open Methods (Fixed point method)

E. T. S. I. Caminos, Canales y Puertos 15

Open Methods

E. T. S. I. Caminos, Canales y Puertos 16

Why do open methods fail?Function may not look linear.

Remedy: recast into a linear form. For example,

201 (1 i)f (i) 7500 0

i

Is a poorly constrained problem in that there is a large, nearly flat zone for which the derivative is near zero. Recast as:

i f(i) = 0 = 7,500 i - 1000 [ 1 - (1+i)-20 ]

Open Methods

E. T. S. I. Caminos, Canales y Puertos 17

Recast as: i f(i) = 0 = 7,500 i - 1000 [ 1 - (1+i)-20 ]

– The recast function, "i f(i) will have the same roots as f(i) plus an additional root at i = 0.

– It will not have a large, flat zone, thus:

h(i) = i f(i) = 7,500 i – 1000 [ 1 – (1+ i)–20]

– To apply N-R we also need the first derivative:

h'(i) = 7,500 - 20,000 (1+ i)-21

201 (1 i)f (i) 7500 0

i

Open Methods

E. T. S. I. Caminos, Canales y Puertos 18

Cases of Multiple Roots

Multiple Roots: f(x) = (x – 2)2 (x – 4)

x = 2 represents two of the three roots.

-3

-2

-1

0

1

2

3

4.03.02.01.0

Open Methods

E. T. S. I. Caminos, Canales y Puertos 19

Problems and Approaches:Cases of Multiple Roots

1.Bracketing Methods fail locating x = 2. Note that f(x) f(xr) > 0.

2. At x = 2, f(x) = f '(x) = 0. • Newton-Raphson and Secant methods may experience

problems.

• Rate of convergence drops to linear.

• Luckily, f(x) 0 faster than f '(x) 0

3. Other remedies, recasting problem: Find x such that u(x) = 0 where:

Note that u(x) and f(x) have same roots.

f (x)u(x) 0

f '(x)

Open Methods

E. T. S. I. Caminos, Canales y Puertos 20

m = 1: linear convergence

m = 2: quadratic convergence

Method m

Bisection 1

False Position 1

Secant, mult. root 1

NR, mult. root 1

Secant, single root 1.618 "super linear"

NR, single root 2

Accel. NR, mult. root (f(x)/f'(x)=0) 2

i 1mi

i

Elim A 0

E

Summary -- Rates of Convergence

E. T. S. I. Caminos, Canales y Puertos 21

A real rootfinding problem can be viewed as having three phases:

1) Opening moves: One needs to find the region of the parameter space in which desired root can be found. Understanding of problem, physical insight, and

common sense are valuable.

2) Middle Game: Use robust algorithm to reduce initial region of uncertainty.

3) End game: Generate a highly accurate solution in a few iterations.

Three Phase Rootfinding Strategy