Root Finding without Derivatives
1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence
2 Method of False Positiona bad case for the secant methodcombine secant with bisection
MCS 471 Lecture 3(b)Numerical Analysis
Jan Verschelde, 22 June 2018
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 1 / 27
Root Finding without Derivatives
1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence
2 Method of False Positiona bad case for the secant methodcombine secant with bisection
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 2 / 27
the secant line
Consider the secant line through (a, f (a)) and (b, f (b)):
y − f (b) =
(f (b)− f (a)
b − a
)(x − b).
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 3 / 27
derivation of the methodConsider the secant line through (a, f (a)) and (b, f (b)):
y − f (b) =
(f (b)− f (a)
b − a
)(x − b)
Set x = c and y = 0 and solve for c:
0− f (b) =
(f (b)− f (a)
b − a
)(c − b)
−f (b)
(b − a
f (b)− f (a)
)= c − b
Then we obtain the formula
c = b − f (b)
(b − a
f (b)− f (a)
)
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 4 / 27
the secant method
Rename a = x1, b = x2, c = x3:
x3 = x2−f (x2)
(x2 − x1
f (x2)− f (x1)
)︸ ︷︷ ︸
∆x
Input: f , x1, x2, N, δ, ε.
for k = 1,2, . . . ,N do
∆x := f (x2)
(x2 − x1
f (x2)− f (x1)
)x1 := x2x2 := x2 −∆xif |∆x | ≤ δ or |f (x2)| ≤ ε
report success, return x2
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 5 / 27
Root Finding without Derivatives
1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence
2 Method of False Positiona bad case for the secant methodcombine secant with bisection
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 6 / 27
a Julia function
function secant(f::Function,x1::Float64,x2::Float64,dxtol::Float64=1.0e-8,fxtol::Float64=1.0e-8,N::Int64=10)
## Applies the secant method to the function f,# starting at the points x1 and x2.# Stops when the sequence reaches the point x# where |f(x)| < fxtol or when the distance between# two consecutive points is less than dxtol.# Failure is reported when the accuracy requirement# is not satisfied in N steps,# otherwise fail is false on return.#
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 7 / 27
the definition of the function
println("running the secant method...")fx1 = f(x1)fx2 = f(x2)for i = 1:N
dx = fx2*(x2 - x1)/(fx2 - fx1)x1 = x2 # store x2 for next stepx2 = x2 - dxfx1 = fx2fx2 = f(x2)
Observe that there is only one new function evaluationin each iteration of the loop: fx2 = f(x2).
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 8 / 27
printing the results and the stop criterion
stri = @sprintf("%3d", i)strx = @sprintf("%.16e", x2)strdx = @sprintf("%.4e", abs(dx))strfx = @sprintf("%.4e", abs(fx2))println("$stri : $strx $strdx $strfx")if((abs(fx2) < dxtol) | (abs(dx) < fxtol))
stri = string(i)println("succeeded after $stri steps")return [x2, false]
endendstrN = string(N)println("failed requirements after $strN steps")return [x2, true]
end
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 9 / 27
running a test
result = secant(cos,pi/4,2*pi/3)
is executed in the script:
$ julia secantmethod.jlrunning the secant method...step : root |dx| |f(x)|1 : 1.5521908171562901e+00 5.4220e-01 1.8604e-022 : 1.5716418753086792e+00 1.9451e-02 8.4555e-043 : 1.5707962802269275e+00 8.4560e-04 4.6568e-084 : 1.5707963267949021e+00 4.6568e-08 5.4899e-15
succeeded after 4 steps$
Observe the quadratic convergence.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 10 / 27
two exercises
Exercise 1: Consider running the secant method on x3 + x − 1and choose 0.5 as the first value for x1.Experiment with different values for the second point x2,choosing 0.500001, 0.6, and 0.7 for x2.For which choice of x2 does the secant method work best?
Exercise 2: The polynomial x3 + x − 1 has two complex conjugatedroots but the secant method cannot compute the complex roots,unless the arithmetic is complex.Adjust the Julia function so it works with initial values with nonzeroimaginary parts.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 11 / 27
Root Finding without Derivatives
1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence
2 Method of False Positiona bad case for the secant methodcombine secant with bisection
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 12 / 27
superlinear convergence
TheoremAssume the secant method applied to f (x) = 0 converges:
1 xk → x∞, f (x∞) = 02 f ′(x∞) 6= 0.
Denote ek = xk − x∞, the error in the k-th step. Then,
ek+1 ≈ Ce(1+√
5)/2k ≈ Ce1.618
k ,
for some constant C.
Note that1 +√
52
is the golden ratio.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 13 / 27
a first lemma
LemmaUnder the assumptions and notations of the theorem:
ek+1 ≈12
f ′′(x∞)
f ′(x∞)ek−1ek .
Proof. Using the definition of xk+1, we find
ek+1 = xk+1 − x∞ = xk − f (xk )xk − xk−1
f (xk )− f (xk−1)− x∞.
We can replace xx+1 by xk + ek and xk by xk−1 + ek−1, so that
ek+1 = x∞ + ek − f (x∞ + ek )x∞ + ek − x∞ − ek−1
f (x∞ + ek )− f (x∞ + ek−1)− x∞. (1)
To simplify this expression, we apply the Taylor expansion off (x∞ + ek ) and f (x∞ + ek−1) about x∞.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 14 / 27
proof of the first lemma continued 1
f (x∞ + ek ) = f (x∞) + f ′(x∞)ek +12
f ′′(x∞)e2k + O(e3
k ),
f (x∞ + ek−1) = f (x∞) + f ′(x∞)ek−1 +12
f ′′(x∞)e2k−1 + O(e3
k−1).
Subtracting f (x∞ + ek−1) from f (x∞ + ek ):
f (x∞ + ek )− f (x∞ + ek−1)
= f ′(x∞)(ek − ek−1) + 12 f ′′(x∞)(e2
k − e2k−1) + O(e3
k )−O(e3k−1).
Since O(e3k )−O(e3
k−1) is of a smaller order than ek and ek−1 we omitthis term. Using e2
k − e2k−1 = (ek − ek−1)(ek + ek−1), we organize the
above expression as
f (x∞+ek )−f (x∞+ek−1) ≈ (ek−ek−1)(f ′(x∞)+f ′′(x∞)(ek +ek−1)). (2)
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 15 / 27
proof of the first lemma continued 2
The left of (2):
f (x∞ + ek )− f (x∞ + ek−1) ≈ (ek − ek−1)(f ′(x∞) + f ′′(x∞)(ek + ek−1)).
appears at the right of (1):
ek+1 = x∞ + ek − f (x∞ + ek )x∞ + ek − x∞ − ek−1
f (x∞ + ek )− f (x∞ + ek−1)− x∞.
so we derive the following expression
ek+1 ≈ ek − f (x∞ + ek )ek − ek−1
(ek − ek−1)(f ′(x∞) + f ′′(x∞)(ek + ek−1)).
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 16 / 27
proof of the first lemma continued 3
Using a Taylor expansion for f (x∞ + ek ) about x∞ (recall f (x∞) = 0)we have
ek+1 ≈ ek − ekf ′(x∞) + 1
2 f ′′(x∞)ek
f ′(x∞) + 12 f ′′(x∞)(ek + ek−1)
.
Now we put everything on the same denominator:
ek+1 ≈ ekf ′(x∞) + 1
2 f ′′(x∞)(ek + ek−1)− f ′(x∞)− 12 f ′′(x∞)ek
f ′(x∞) + 12 f ′′(x∞)(ek + ek−1)
,
which can be simplified as
ek+1 ≈ ek
12 f ′′(x∞)ek−1
f ′(x∞) + 12 f ′′(x∞)(ek + ek−1)
.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 17 / 27
end of the proof of the first lemma
Because ek → 0 as k →∞, 12 f ′′(x∞)(ek + ek−1) is negligible
compared to f ′(x∞), so we omit the second term in the denominator,to find the estimate
ek+1 ≈12
f ′′(x∞)
f ′(x∞)ekek−1.
Q.E.D.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 18 / 27
a second lemma
LemmaThere exists a positive real number r such that:
ek+1 ≈ Cek−1ek ⇒ e1+1/rk ≈ Ker
k , for some constants C and K .
Proof. Assuming the convergence rate is r , there exists someconstant A, so we can write
ek+1 ≈ Aerk and ek ≈ Aer
k−1 or(
1A
ek
)1/r
≈ ek−1.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 19 / 27
proof of the second lemma continued
Now we can replace the expressions for ek and ek−1 in the left handside of the equation of the second lemma:
ek+1 ≈ C(
1A
)1/r
e1/rk ek ≈ Be1+1/r
k .
Together with the assumption that ek+1 ≈ Aerk , we obtain
e1+1/rk ≈ A
B erk . So, we set K = A
B and the lemma is proven. Q.E.D.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 20 / 27
a third lemma
LemmaFor the r of the second lemma, we have
e1+1/rk ≈ Cer
k ⇒ r =1 +√
52
.
Proof. r satisfies the following equation
1 +1r
= r ⇒ r + 1 = r2 ⇒ r2 − r − 1 = 0.
The roots of r2 − r − 1 = 0 are r = 1±√
52 .
We take the positive value for r . Q.E.D.The constant r = 1+
√5
2 ≈ 1.618 is the golden ratio.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 21 / 27
Root Finding without Derivatives
1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence
2 Method of False Positiona bad case for the secant methodcombine secant with bisection
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 22 / 27
a bad case for the secant method
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 23 / 27
Root Finding without Derivatives
1 Secant Methodderivation of the methoda Julia functionsuperlinear convergence
2 Method of False Positiona bad case for the secant methodcombine secant with bisection
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 24 / 27
combine secant with bisection
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 25 / 27
the method of false position or regula falsi
Input: [a,b] such that f (a)f (b) < 0;N is the maximum number of iterations;ε > 0, tolerance on |f (x)|.
for i = 1,2, . . . ,N do
c :=b f (a)− a f (b)
f (a)− f (b)
if |f (c)| ≤ ε then return c
if f (a)f (c) < 0 thenb := c
elsea := c.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 26 / 27
write a Julia function
Exercise 3: Write a Julia script to run the method of false position.Test your script by running the experiment of Exercise 1.
Numerical Analysis (MCS 471) Bisection and Fixed-Point Iterations L-3(b) 22 June 2018 27 / 27