2.3 The Fixed-Point Algorithm

1. Mean Value Theorem:Theorem Rolles Theorem : Suppose that f is continuous on a, b and is differentiable on a, b .

If fa fb, then there exists a number c in a, b such that f c 0.

Theorem Mean Value Theorem: Suppose that f is continuous in a, b and is differentiable ona, b . Then there exists a number c in a, b such that

f c fb fab a .Note that:a. Rolles Theorem is a special case of theMean Value Theorem.b. By Rolles Theorem, we know if f x 0 for all x in a, b , then fa fb. On the other hand,

the condition fa fb alone is not enough for us to determine if there exists a number c ina, b such that f c 0 .

Graphically, Rolles Theorem and the Mean Value Theorem can be described as follows.

-0.5 0.5 1.0 1.5 2.0 2.5

-2

-1

1

2y

f0 f2, f 1 0

-3 -2 -1 1 2 3

-10

-5

5

10

x

y

c 1.4, c 2.2c. When b is close to a (|b a| is small), f c fb fab a and f

x fb fab a for x a,b.

Example First show that the equation x ex 0 has a solution in 0,1. Then Determine if thesolution is unique.

Let fx x ex. Since f0f1 0 1 1 1e 0.632 0, by the Intermediate ValueTheorem there exists a number c in 0, 1 such that fc 0. Therefore, the equation x ex 0 has asolution in 0, 1 . Now let us check to see if f x 0 for x in 0, 1 ,

f x 1 ex 0 for all x.So, f x 0 for all x in 0, 1 and therefore, fx 0 only once and the solution is unique.

1

Example The graph of fx xcosx 0.2 for x in 2, 2 is given below. Find graphically allpossible c in 2, 2 satisfying the conclusion given in the Mean Value Theorem.Approximate f c for each c.

-2 -1 1 2

-1.4

-1.2

-1.0

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

x

y

y fx

Approximately, c1 1.19 and c2 0.89.

f c1 f1 f1.51 1.5 0.697 0.401

0.5 0.592

f c2 f1 f0.51 0.5 0.362 0.382

0.5 0.04

Comparison:

f c1 0.446 , f c2 0.327

2. Fixed-Point of a Function:Definition A number p is said to be a fixed point of a function gx if gp p.Graphically, a function has a fixed point at x p if its graph y gx intersects with the liney x intersect at x p. For example, the following graphs show that y ex and y x intersect at apower where x is near 0.58.

0 1 20

1

2

x

y

ep p, when p 0.58Hence, fx ex has a fixed point near 0.58. Some functions may have more than one fixed points andsome functions may not have a fixed point. For example, the function in (i) has no fixed point becausey x2 1 and y x do not intersect; and the function in (ii) has infinitely many fixed points.

2

-5 -4 -3 -2 -1 1 2 3 4 5

-5

5

10

x

y

(i) y x2 1

-10 -8 -6 -4 -2 2 4 6 8 10

-10

-5

5

10

x

y

(ii) y x cosxAlgebraically, we solve the equation gx x (or gx x 0 ) to determine if a function has any fixedpoint over a given interval.

Example Find all fixed points of g1x x2 1 and g2x x cosx if they exist.a. Set g1x x : x2 1 x, x2 x 1 0. Using the quadratic formula:

x 1 1 412 1 3

2 no real solution.

So, g1x has no fixed point for x .b. Set g2x x : x cosx x, cosx 0, x 2n 12 , n 1,2,3, . . .

So, g2x has infinitely many fixed points for x .

3. Existence and Uniqueness of a Fixed Point:Theorem Existence and Uniqueness: Let g be continuous on a, b .

i. If a gx b for all x in a, b , then gx has a fixed point p in a, b .ii. If, in addition, gx exists on a, b and there exists a constant 0 K 1 such that

gx K for all x in a, b ,then p is unique.

Note that:a. Both conditions: a gx b for all x in a, b and gx K for all x in a, b are

sufficient conditions. So, in the case where the condition in i. does not hold, it is possible that gxhas a fixed point; and in the case where the condition in ii. is not satisfied, it is also possible thefixed point of gx is unique.

b. Because gx is the slope of the tangent line to the curve y gx at x, |gx| K 1 means thatthe graph of gx does not grow as faster than y x and not slower than y x.

Proof of the existence and uniqueness:i. If ga a or gb b, then p a or p b and g has a fixed point. Now let ga a and

gb b, and let hx gx x. Since gx is continuous on a, b , hx is continuous ona, b . Observe that ha ga a 0 and hb gb b 0. So, by the Intermediate Value

Theorem, we know there exists a number c in a, b such that hc 0, that is3

gc c 0 or gc c.So, c is a fixed point of g in a, b .

ii. Now let also gx K for all x in a, b where 0 K 1. Suppose that gx has two fixedpoints, say p q in a, b . Then by the Mean Value Theorem, we know these exists a point c inq, p such that

gc gp gqp q .

Since gp p and gq q, gp gqp q p qp q 1. So, gc 1, this contradicts the given

condition |gx| 1 for all x in a, b . So, g cannot have two fixed points in a,b.

Example Let gx 13 x2 1 for x in 1,1. Determine if g has a fixed point in 1, 1 . If so,determine if the fixed point is unique.

Check conditions given in the theorem for the existence and uniqueness:i. Observe that gmin g0 13 1 and gmax g1 g1 0 1. Since 1 gx 1 for

all x in 1, 1 , g has a fixed point in 1, 1 .ii. Compute gx 23 x. Since g

x 23 |x| 23 1, g has a unique fixed point in 1, 1 .

For gx, we can solve its fixed point p algebraically.

gx x 13 x2 1 x x2 3x 1 0 x 3 9 412

3 132

Since 3 132 1, p 3 13

2 0.302776 is a unique fixed point in 1,1.

Check the graph of gx : -1.0 -0.5 0.5 1.0

-1.0

-0.5

0.5

1.0

x

y

y 13 x2 1, -.-. y x, y

Example Let gx 3x for x in 0,1. Determine if g has a fixed point in 0, 1 . If so, determine ifthe fixed point is unique.

Check conditions given in the theorem for the existence and uniqueness:i. Observe that gmin g1 31 0, and gmax g0 1. Since 0 gx 1, gx has a fixed

point in 0, 1 .ii. Compute gx 3x ln3. gx 3x ln3. Since |g0| ln3 1, there is no conclusion

4

about the uniqueness.From the graph of g below, we can see that g has a unique fixed point p 0.55 in 1,1. But wecannot solve p algebraically. How can solve a fixed point numerically?

0.0 0.2 0.4 0.6 0.8 1.00.0

0.2

0.4

0.6

0.8

1.0

x

y

y 3x, x in 0, 1

4. The Fixed-Point Algorithm:The Fixed-Point Algorithm is an algorithm that finds the fixed-point of a function over an intervalassuming the fixed point is unique in this interval.

Algorithm Fixed-Point Algorithm: Given gx, and a, b , choose p0 in a, b and computep1,p2, . . . , as follows:

pn gpn1 for n 1,2, . . . .Implement the algorithm in a programming language which does the following:Input gx, intervala, b , p0 in a, b , and Kmax, and compute pn gpn1 for n 1,2, . . . . The program terminates

ifi. |pn pn1 | and then p pn; orii. pn b or pn a, and the program fails; oriii. n Kmax.

The MatLab program fixpt.m implements the Fixed-Point Algorithm to find pn with input(1) the function gfun for gx;(2) an initial approximation p0 to the fixed-point;(3) an accuracy requirement ; and(4) the maximum number for iterations Kmax.The following two examples show graphically how the Fixed-Point Algorithm works.

5

0 0.1 0.2 0.3 0.4 0.50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

0.5

Fixed Point Iterations: {pn}, g(x)=(2-ex+x2-2)/3, p0=0

p0

(p0,g(p0))(p1,p1)=(g(p0),g(p0))

p1

(p1,g(p1))

(p2,p2)

p2

(1)

1 1.5 2 2.5 30

1

2

3

4

5

6

Fixed Point Iterations: {pn}, g(x)=(x-0.5)2, p0=2

p0

(p0,g(p0))

(p1,p1)

p1

(p1,g(p1))

y=x

(2)Clearly, the Fixed-Point Algorithm finds the fixed point p in (1) and diverges in (2).

5. Convergence and the Rate of Convergence of the Fixed-Point Algorithm:Questions: Assume that g has a unique fixed point p in a, b and p0 is in a, b . Let

pn gpn1, n 1,2, . . . .a. Under what condition(s), does pn converge to p?b. If limn pn p, what is the rate of converge?

Fixed-Point Theorem:Theorem Fixed-Point Theorem: Let g be continuous on a, b and a gx b. Suppose that gx

exists for all x in a, b , andgx K for all x in a, b where 0 K 1.

Then limn pn p for any p0 in a, b , andpn p Kn max p0 a, b p0 and

pn p Kn1 K p1 p0 , for all n 1,2, . . . .Proof: Let p0 be in a, b and pn be generated by the Fixed-Point Algorithm. Observe that

pn p gpn1 gp .By the Mean Value Theorem, we know there exists a number c in a, b such that

gpn1 gppn1 p gc or gpn1 gp gcpn1 p.

Since gx 1 for all x in a, b ,

6

pn p gpn1 gp gcpn1 p gc pn1 p K pn1 p K K pn2 p K2 pn2 p . . . . Kn| p0 p|

0 limn pn p limnKn| p0 p | 0.Therefore, limn pn p.Since p0 p p0 a or p0 p b p0 ,

pn p Kn| p0 p| Kn max p0 a, b p0 .Observe that

pn1 pn gpn gpn1 K pn pn1 K gpn1 gpn2 K2 gpn2 gpn3 . . . Kn p1 p0

So for any m n 1,pm pn |pm pm1 pm1 pm2 . . .pn1 pn|

pm pm1 pm1 pm2 . . . pn1 pn Km1 |p1 p0 | Km2 p1 p0 . . .Kn| p1 p0| KnKmn1 Kmn2 . . .K 1|p1 p0 | Kn 1 Kmn1 K p1 p0

Since limm pm p,p pn limm pm pn limmKn 1 K

mn1 K |p1 p0 |

Kn1 K p1 p0

Note that :a. Rate of convergence: pn p Kn1 K p1 p0 , for all n 1,2, . . . implies that

pn pKn

11 K p1 p0

11 K gp0 p0 .

So pn converges to pwith the rate of convergence of OKn, i.e.,pn p OKn.

b. Order of convergence: From pn p |gcn1||pn1 p| K pn1 p , we havepn ppn1 p

K.

Hence, pn converges to p linearly 1 with an asymptote error constant K.c. The smallest possible number of iterations: For a given , we can estimate the number N of

iterations needed to approximate p by pN. That is, find N such thatKN1 K p1 p0

KN1 K b a K

N 1 Kb a N lnK ln 1 Kb a

Since 0 K 1, lnK 0. So, N ln ln 1 Kb a

lnK .7

Example Determine whether or not the function has a fixed point in the given interval. If so, determineif the Fixed-point Algorithm will converge to the fixed point. In the case when it converges,estimate the number of iterations possibly needed to approximate the fixed point within 105.

(1) gx 13 2 ex x2, 0,1 (2) gx 12 10 x

31/2, 0,2(1) Check the range of g : From the graph of gx on 0,1, we have 0 gx 1.

0.0 0.2 0.4 0.6 0.8 1.00.10

0.15

0.20

0.25

0.30

x

y

(1) y gx 13 2 ex x2

0.0 0.2 0.4 0.6 0.8 1.0

0.22

0.24

0.26

0.28

0.30

0.32

x

y

(2) y |gx| 13 ex 2xSo, gx 13 2 ex x2 has a fixed point in 0,1.Check the maximum value of |gx| : gx 13 ex 2x. From the graph of |gx|,

| gx| g0 13 1 13 K 1 for all x in 0,1.

So, gx has a unique fixed-point in 0,1 and the sequence pn generated by the Fixed-PointAlgorithm converges to p.Estimate the number N of iterations:

N ln105 ln 1 13ln 13

10. 8486, let N 11.

Use the Fixed-Point Algorithm to solve the fixed point in 0,1 and p pn where|pn pn1 | 105.

Using MatLab program fixpt.m, we have the following.gfun@(x) 1/3*(2-exp(x)x.^2); fixptinput initial point p0 0input the tolerance for stopping criterion, ex:.0001,10^(-8) 10^(-5)input the maximum number of iterations 100Algorithm converges with number of iterations ans 9fixed point p 0.257531806267540Now we check the values of gx at pi: [p gfun(p)]0 0.3333333333333330.333333333333333 0.238499562008341

8

0.238499562008341 0.2625129636678660.262512963667866 0.2562399109200140.256239910920014 0.2578654070817870.257865407081787 0.2574433155536200.257443315553620 0.2575528599562190.257552859956219 0.2575244261304630.257524426130463 0.2575318062675400.257531806267540 0.257529890698957As you see, gp9 p9.

(2) Check the range of gx : Observe thatgmin g2 12 10 8

22 0, gmax g0

12 10 2.

So, 0 gx 2 for all x in 0,2. Hence, gx has a fixed point in 0,2.Check the maximum value of |gx| :

gx 123x210 x3

. |gx| 3x22 10 x3

From the graph of |gx|,

0.0 0.5 1.0 1.5 2.00

1

2

3

4

x

y

y |gx| 12 3x2

10x3|gx| 1 for some x in 0,2. So we cannot conclude the sequence pn generated by theFixed-Point Algorithm converges to p.Using MatLab program fixpt.m, we have the following. gfun@(x) 1/2*sqrt(10-x.^3); fixptinput initial point p0 0input the tolerance for stopping criterion, ex:.0001,10^(-8) 10^(-5)input the maximum number of iterations 100Algorithm converges with number of iterations 18fixed point p 1.365227242758129Check some values of pi and gpi: [p(13:18,1) gfun(p(13:18,1))]1.365076127110441 1.3653087860866491.365308786086649 1.3651896819350911.365189681935091 1.365250660802594

9

1.365250660802594 1.3652194425503751.365219442550375 1.3652354252341251.365235425234125 1.365227242758129gp18 p18.

6. Fixed-Point Algorithm for Solving The Equation: fx 0Let x be a solution of the equation fx 0. To solve x using the Fixed-Point Algorithm, a function gneeds to be defined first such that x is a fixed point of g, that is, x gx.

Example Consider solving x3 x 1 0. Find an interval a, b on which the equation has asolution. Find a function g such that the fixed point of g is the solution of the equation:fx 0. Determine if the sequence pn generated by the Fixed-Point Algorithm with thefunction g.

Consider a, b 1, 0 . Since f1f0 11 0, the equation has a solution in 1,0.(1) A naive choice of g : since x 1 x3, we can let

gx 1 x3.Check the range of g : gmin g0 1 and gmax g 1 0, so, 1 gx 0 and g has afixed point in 1,0.Check the maximum value of |gx| : gx 3x2, |gx| 3x2 1 for some x in 1,0.So, it is not certain by the Fixed-Point Theorem if pn converges to p.Observe that pnn0 0, 1, 0, 1, . . . .

(2) Rewrite the equation x3 x 1 0 as x3 1 x, x 3 x 1 . Letgx 3 x 1 .

Check the range of g : gmin g0 1, gmax g1 0, so, 1 gx 0 and g has a fixedpoint in 1,0.Check the maximum value of |gx| : gx 13

13 x 12

. Since gx is not defined at

x 1, gx is unbounded. So, it is not certain if pn converges to p.Observe that pnn0 0, 1, 0, 1, . . . .

(3) Rewrite the equation x3 x 1 0 as x3 x 1 and then xx2 1 1 or x 1x2 1 . Let

gx 1x2 1 .

Check the range of g : gmin g0 1, and gmax g1 12 , so, 1 gx 0 and g has afixed point in 1,0.

10

Check the maximum value of |gx| :gx 2xx2 12 ,

|gx| 2|x|x2 12x0 2xx2 12 .

-1.0 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0.00.0

0.1

0.2

0.3

0.4

0.5

0.6

x

y

y |gx| 2|x|x2 12From the graph of |gx| above, we see |gx| 0.7 K 1. So, p is unique in 1,0 and pnconverges to p. Algebraically, let hx |gx|. hx 21 3x

2x2 13 0, x

13

for x 0.

hx reaches its maximum at x 13

and hence, K 2 1

3

13

2 12 0.64951905 0.65

Estimate the number N of iterations needed:

N ln105 ln1 0.7ln0.7 35. 654, let N 36.

N ln105 ln1 0.65ln0.65 29. 162595, let N 30.

Use the Fixed-Point Algorithm to solve the fixed point in 0,1 and p pn wherepn pn1 105.

n 27 and p27 0.68232442571947.

Example Show that each of the following functions has a fixed point at p precisely when fp 0,where fx x4 2x2 x 3.

(1) gx 3 x 2x21/4 (2) gx 3x4 2x2 34x3 4x 1

(1) Set x4 2x2 x 3 0. Thenx4 3 x 2x2 x 3 x 2x21/4.

(2) Check if x 3x4 2x2 34x3 4x 1 , then

x 3x4 2x2 34x3 4x 1

4x4 4x2 x 3x4 2x2 34x3 4x 1

x4 2x2 x 34x3 4x 1 0.

So, x4 2x2 x 3 0.

Example The following four methods are proposed to compute 71/5. Rank them in order, based on theirapparent speed of convergence, assuming p0 1.

11

(1) pn 1 7 pn13

pn121/2

(2) pn pn1 pn15 7pn12

(3) pn pn1 pn15 75pn14

(4) pn pn1 pn15 712

The function g for each iteration is:

(1) g1x 1 7 x3x21/2

1 7x2

x1/2

(2) g2x x x5 7x2 x x3 7

x2

(3) g3x x x5 75x4 x 15 x

75x4

45 x 75x4

(4) g4x x x5 712 x x512

712

1 1.2 1.4 1.6 1.8 2-6

-4

-2

0

2

4

6

8graphs of g1, g2, g3 and g4

x

g(x)

y=g2(x)

y=x

y=g4(x)

y=g1(x)

y=g3(x)

All four functions have a fixed point on 1, 2 . Compute gx :(i) g1 x 12 1

7x2

x1/2

14x3

1(ii) g2 x 1 3x2 14x3(iii) g3 x 45

285x5

(iv) g4 x 1 512 x4

Check the value of |gx| at p 71/5 :|g1 p| 12 1

7p2

p1/2

14p3

1 1.61828 1

|g2 p| 1 3p2 14p3 9. 88953 1

|g3 p| 45 285p5

0 1

|g4 p| 1 512 p4 0.976365 1

12

So, the sequences pn generated by the Fixed-Point Algorithm using g1 and g2 do not converge. Thesequences pn generated by the Fixed-Point Algorithm using g3 and g4 converge and the third sequenceconverges faster than the the 4th one. The testing results show that

using g3x, 71/5 p7 1.47577316159456using g4x, 71/5 p355 1.47577807080213

Example Show that if A is any positive number, then the sequence defined byxn 12 xn1

A2xn1

, for n 1,converges to A whenever x0 0.

Let limn xn x. Thenlimn xn limn 12 xn1

A2xn1

x 12 x A2x

12 x

A2x x

2 A x A .

Example Let pn be generated by the Fixed-point Algorithm with the function gx and let p be thefixed point of gx such that limn pn p. Determine the orders of convergence and theasymptotic error constants of the sequence pn in the cases where(1) gp 0; and(2) gp 0 but gp 0.

(1) Assume that gp 0. By the Mean Value Theorem, we havepn1 p gpn gp gcnpn p gcn pn p

where cn is between pn and p.

limnpn1 ppn p

limn gcn gp .

So, the order of convergence is 1 and the asymptotic error constant |gp|.(2) Assume that gp 0 but gp 0. By the Taylor Theorem, we have for x p,

gx gp gp1! x p

gx2! x p

2, where x is between x and p.Then

gpn gp gp1! pn p

gxn2! pn p

2 gp gpn

2! pn p2

pn1 p gpn gp gp gpn

2! pn p2 gp 12 g

pn pn p 2

pn1 ppn p 2

12 gpn , limn

pn1 ppn p 2

12 gp

So, the order of convergence is 2 and the asymptotic error constant 12 |gp|.

13

Exercises:

1.

The graph of fx for 3 x 5 is given at the left.Find graphically all possible c in 3, 5 thatsatisfies the conclusion given by the Mean ValueTheorem.

-3 -2 -1 1 2 3 4 5

-5

x

y

y fx

2.

The graph of g for x in 0, 1 is given

at the left. Let p0 0. Computegraphically p1, p2, p3 generated bythe Fixed-point Algorithm.

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.00.0

0.2

0.4

0.6

0.8

1.0y

y gx, -.-. y x

3. Use the Fixed-Point Algorithm to approximate the solution of the equation x3 x 1 0 on 1,2 within105 with p0 1. Give gx first and explain how you choose gx.

4. Consider solving the equation cosx x 0 for x in 0, 2 .a. Show that the equation cosx x 0 has a unique solution in 0, 2 by two steps:

(i) show the equation has a solution in 0, 2 by the Intermediate Value Theorem;(ii) show the solution is unique by Rolles Theorem.

b. Approximate the solution of the equation within 108 by the following methods:(i) the Bisection Method (bisect.m);(ii) the Newton Method (newton.m) with p0 0;(iii) the Fixed Point Method (fixpt.m) with gx cosx and p0 0.Report the number of iterations for each method. Rank the methods based on numbers of iterations.

c. Estimate (the best you can) the asymptotic error constant for the Newton Method. Does matchwith the performance of the Newton Method?

14

5. Consider the function gx 1 x 18 x3.

a. Show that gx has a unique fixed point on the real line.b. Can we show gx has a unique fixed point using the theorem for the existence and uniqueness?

Explain.c. What is the order of convergence if we use the Fixed-Point Algorithm to find this fixed-point? Show

your work in detail.

6. Consider the function gx ex2 .a. Show that g has a unique fixed point on the interval 0,1.b. Approximate the fixed point of gx within 108 using the Fixed-Point Algorithm.c. Estimate (algebraically) the number of iterations to approximate the fixed point within 108 (using

the formula given in Notes c. after the Fixed-Point Theorem.

7. The following four methods are proposed to compute 3 21 . Rank them in order, based on their apparentspeed of convergence, assuming p0 1.

(1) pn 20pn1 21pn12

21 (2) pn pn1 pn13 213pn12

(3) pn pn1 pn14 21pn1pn12 21

(4) pn 21pn1

15