Post on 19-Apr-2018
transcript
Solving Nonlinear Equation
0x1x2x
)( 1xf
)( 0xf
Root r
Numerical Methods © Wen-Chieh Lin 2
Nonlinear Equations
Given function f, we find value x for which
Solution x is root of equation, or zero offunction f
So problem is known as root finding or zerofinding
0)( xf
Numerical Methods © Wen-Chieh Lin 3
Nonlinear Equations: two cases Single nonlinear equation in one unknown, where
Solution is scalar x for which f(x) = 0
System of n coupled nonlinear equations in nunknowns, where
Solution is vector x for which all components of fare zero simultaneously, f(x) = 0
Numerical Methods © Wen-Chieh Lin 4
Example of 1-D nonlinear equation
for which x = 0.3604 is one approximate solution
Example of system of nonlinear equations in twodimensions
for which x = [-1.8, 0.8] is one approximatesolution vector
1
4
2
22
21
1
xe
xxx
Examples: Nonlinear Equations
0)sin(3 xexx
Numerical Methods © Wen-Chieh Lin 5
Multiplicity If f(R) = f’(R) = f”(R) = … = f(m-1)(R) = 0 but
f(m)(R) ≠ 0, then root R has multiplicity m
If m = 1 (f(R) = 0, f’(R) ≠ 0 ), then R is simpleroot
Numerical Methods © Wen-Chieh Lin 6
Bisection method begins with initial bracket andrepeatedly halves its length until solution hasbeen isolated as accurately as desired
http://www.cse.uiuc.edu/iem/nonlinear_eqns/Bisection/
Interval Halving (Bisection)
while |b-a| > tol,m = a + (b-1)/2;If f(a)*f(m) < 0,
b = m;else
a = m;end;
end;
a bm
Numerical Methods © Wen-Chieh Lin 7
Bisection (cont.) Simple and guaranteed to work if
f is continuous in [a, b] [a, b] brackets a root
Needed iterations to achieve a specified accuracy isknown in advance Error after n iterations < |b - a| / 2n
Slow to converge Good for initial guess for other root finding
algorithms Finding the initial bracket may be a problem if f is not
given explicitly
Numerical Methods © Wen-Chieh Lin 8
Use graphing to assist root finding
Set the initial bracketDetect multiple roots
Numerical Methods © Wen-Chieh Lin 9
Can we find a root in a better way?
Bisection only utilizes function values f(x)We can find a root with fewer iterations if
other information is used linear approximation of f(x)
Secant line secant methodTangent line Newton’s method
Polynomial approximation of f(x)Muller’s method
Numerical Methods © Wen-Chieh Lin 10
Secant Method Approximate a function by a straight line
Compute the intersection of the line and x-axis
)()()(
)()(
10
10
1
21
xfxfxx
xfxx
)()()(
)(10
10112 xfxf
xxxfxx
0x1x2x
)( 1xf
)( 0xf
Root r
Numerical Methods © Wen-Chieh Lin 11
Secant Method (cont.) Update endpoints
Repeat
)()()(
)(1
11
nn
nnnnn xfxf
xxxfxx
0x
1x
)( 1xf
)( 0xf Root r
withswap,)()(if 1010 xxxfxf
Numerical Methods © Wen-Chieh Lin 12
Example f(x) = 3x + sin(x) –exp(x) Find the root in [0, 1] http://www.cse.uiuc.edu/iem/nonlinear_eqns/Secant/
0 0.5 1 1.5 2-1
-0.5
0
0.5
1
1.5
Numerical Methods © Wen-Chieh Lin 13
Method of False Position
Problem of secant method
RemedyAlways bracket a root in the interval [x0, x1]How to do this?
Numerical Methods © Wen-Chieh Lin 14
Newton’s methodBetter approximation using the first derivative
)( 0xf
10 xx 0x1x
10
00
)()(')tan(
xxxf
xf
)(')(
0
001 xf
xfxx
)(')(
1n
nnn xf
xfxx
Numerical Methods © Wen-Chieh Lin 15
Interpretation of Newton’s method Truncated Taylor series
is a linear function of h approximating f near xReplace nonlinear function f by this function,
whose zero is h = - f(x)/f’(x) Zeros of original function and linear
approximation are not identical, so repeatprocess
)(')()( xhfxfhxf
)(')(
1n
nnn xf
xfxx
Numerical Methods © Wen-Chieh Lin 16
Example: Newton’s method
http://www.cse.uiuc.edu/iem/nonlinear_eqns/Newton/
Numerical Methods © Wen-Chieh Lin 17
Comparison of Secant andNewton’s methods
)()()(
)(1
11
nn
nnnnn xfxf
xxxfxx
)('
)(1
n
nnn xf
xfxx
Secant method Newton’s method
Numerical Methods © Wen-Chieh Lin 18
Pros and Cons of Newton’s method
Pros efficient
ConsNeed to know the derivative function
Numerical Methods © Wen-Chieh Lin 19
)(')(
1n
nnn xf
xfxx
When will Newton’s method not converge?
x1=x6, loop
passingmaximum/minimum
Numerical Methods © Wen-Chieh Lin 20
Muller’s method Instead of linear approximation, Muller’s
method uses quadratic approximate Evaluation of derivatives are not required See the textbook for details
Numerical Methods © Wen-Chieh Lin 21
Fixed-point Iteration Method Rearrange f(x)=0 into an equivalent form x=g(x)
f(x) = x –g(x) = 0 If r is a root of f, then f(r) = r-g(r) = 0
r=g(r) In iterative form,
Also called function iteration For given equation f(x)=0, there may be many
equivalent fixed-point problems x=g(x) with differentchoice of g
Will the method always converge?
)(1 nn xgx
Numerical Methods © Wen-Chieh Lin 22
Example: Fixed-point Iteration
032)( 2 xxxf
32)(1 xxg2
3)(2
xxg
Numerical Methods © Wen-Chieh Lin 23
Example: Fixed-point Iteration (cont.)
032)( 2 xxxf
23
)(2
3
x
xg
Diverge!
Numerical Methods © Wen-Chieh Lin 24
Convergence Rate
For general iterative methods, define error atiteration n by
en = xn –Rwhere xn is approximate solution and R is truesolution
For methods that maintain interval known tocontain solution, rather than specificapproximate value for solution, take error to belength of interval containing solution
Numerical Methods © Wen-Chieh Lin 25
Convergence Rate (cont.) Sequence converges with rate r if
for some finite nonzero constant C Some cases of interest
r=1: linear (C<1) r>1: superlinear r=2: quadratic
Ce
er
n
n
n
1lim
Numerical Methods © Wen-Chieh Lin 26
Convergence Rate of Bisection
Length of interval containing solution reducedby half at each iteration
Linearly convergent r = 1C = 0.5
Numerical Methods © Wen-Chieh Lin 27
Convergence of Fixed-point Iteration
If R = g(R) and |g’(R)| < 1, then there is aninterval containing R such that iteration
xn+1 = g(xn)converges to R if started within that interval
If |g’(R)| > 1, then iterative scheme diverges
Numerical Methods © Wen-Chieh Lin 28
Proof of Convergent Condition
nnn ege )('1
)()()(1 nnn xgRgxgRxR
nn
nn xR
xRxgRg
xR
)()(
1
))(('1 nnn xRgxR Mean Value Theorem
),( nn xR
Ce
er
n
n
n
1limFixed-point iteration is linearly convergent
)(' RgC
Numerical Methods © Wen-Chieh Lin 29
Convergence of Newton’s MethodRepresent Newton’s method in fixed-point
iteration form
Condition for convergence
)()(')(
1 nn
nnn xg
xfxf
xx
1)]('[
)(")()(' 2
xfxfxf
xg
Numerical Methods © Wen-Chieh Lin 30
Convergence rate of Newton’s method)()(1 nn xgRgxR
2)()(")()(')()( 2nnn xRgxRRgRgxg
0)(')]('[
)(")()(' 0)(
2 Rgxf
xfxfxg Rf
2)()(")()( 2nn xRgRgxg
2)(")()( 211 nnnn egxgRgxRe
Ce
er
n
n
n
1limNewton’s method isquadratically convergent!
Recall that
Numerical Methods © Wen-Chieh Lin 31
Question from the last class
Is the initial solution important for theconvergence of fixed-point iteration?
Numerical Methods © Wen-Chieh Lin 32
Answer to the convergence problem offixed-point Iteration
|g’(R)|<1 is a necessary condition if |g’(R)|>1, the fixed-point iteration will diverge
even if the initial condition is very close to a rootsince the iteration will eventually reach the regioncausing divergence
Initial solution is important but less criticalThe algorithm may not converge if the initial
solution is far from the true solutionRecall the conditions that Newton’s method does
not converge?
Numerical Methods © Wen-Chieh Lin 33
Example: Newton’s Method for FindingComplex Roots
f(x) = x3 + 2x2 –x + 5
Numerical Methods © Wen-Chieh Lin 34
Start Newton’s method with a complex value
ix 10 ixf 52)( 0 ixf 103)(' 0
iii
ixfxf
xx 04587.1486238.010352
1)(')(
0
001
52)( 23 xxxxf 143)(' 2 xxxf
ixfxf
xx 23665.1448139.0)(')(
1
112
Numerical Methods © Wen-Chieh Lin 35
Newton’s Methods for Multiple Roots
Quadratically convergent for simple root,
Linearly convergent for multiple roots as
0)(')]('[
)(")()(' 0)(
2 Rgxf
xfxfxg Rf
00
)]('[)(")(
)(' 2 Rf
RfRfRg
Numerical Methods © Wen-Chieh Lin 36
Remedies for Multiple Roots withNewton’s method
If f(x) has a root of multiplicity k at x=R, wecan factor out (x-R)k from f(x) to get
With a slightly modified Newton’s method
It can be proved that andNewton’s method still converges quadratically
)()()( xQRxxf k
0)(' Rg k
)()(')(
1 nkn
nnn xg
xfxf
kxx
Numerical Methods © Wen-Chieh Lin 37
Remedies for Multiple Roots withNewton’s method (cont.)
In practice, we don’t know k in advanceRemedies
Try and errorDeflate f(x)
f(x)/(x-s) where s is an approximate RBe warned that an indeterminate form at x=R is
created
Numerical Methods © Wen-Chieh Lin 38
Systems of Nonlinear Equations Solving systems of nonlinear equations is
much more difficult than scalar case becauseWide variety of behavior is possible, so
determining existence and number of solutions orgood starting guess is much more complex
In general, there is no simple way to guaranteeconvergence to desired solution or to bracketsolution to produce absolutely safe method
Computational overhead increases rapidly withdimension of problem
Numerical Methods © Wen-Chieh Lin 39
Example: Systems in Two Dimensions
From Michael T. Heath
Numerical Methods © Wen-Chieh Lin 40
Newton’s Method In n dimensions, Newton’s method has form
where J(x) is Jacobian matrix of f
In practice, we do not explicitly invert J(xn),but instead solve linear system
for Newton step sn, then take as next iterate
)()( 1nnn1n xfxJxx
)()( nnn xfsxJ
nn1n sxx
Numerical Methods © Wen-Chieh Lin 41
Example: Newton’s Method
014
),(),(
)(2
22
21
212
211
1
xexx
xxfxxf
xxf
1
22)(
1
21
2
2
1
2
2
1
1
1
xexx
xf
xf
xf
xf
xJ
T7.110 x
0183.011.0
)( 0xf
17183.24.32
)( 0xJ
)()( 000 xfsxJ
0183.0
11.017183.2
4.320s
T]7298.10043.1[00 sxx1
Numerical Methods © Wen-Chieh Lin 42
Example: Newton’s Method
014
),(),(
)(2
22
21
212
211
1
xexx
xxfxxf
xxf
1
22)(
1
21
2
2
1
2
2
1
1
1
xexx
xf
xf
xf
xf
xJ
T7298.10043.11 x
49651.142653.8
)( 1 ee
xf
17300.24596.30086.2
)( 1xJ
)()( 111 xfsxJ
49651.142653.8
17300.24596.30086.2
1 ee
s
T]729637.1004169.1[112 sxx
Numerical Methods © Wen-Chieh Lin 43
Fixed-Point Iteration Fixed-point problem for is to find
vector x such thatCorresponding fixed-point iteration is
Converges if starts close enough to solution
)(xgx
)(1 nn xgx
nn RR :g
1))(( RJ
AA
RJ
matrixaofseigenvaluetheofnormcomplexmaximum:)(
solutiontrue:matrixJacobian:
Numerical Methods © Wen-Chieh Lin 44
Fixed-Point Iteration (cont.)Convergence rate is normally linear, with
constant C given by If then convergence rate is at least
quadratic, e.g., Newton’s method
nn RR :g
0)( RJ
))(( RJ
Numerical Methods © Wen-Chieh Lin 45
Example: Fixed-point Iteration
014
),(),(
)(2
22
21
212
211
1
xexx
xxfxxf
xxf
21
2
2
1
4)1ln(
xx
xx T7.110 x
1.00000000000000 -1.73205080756888
1 .00505253874238 -1.72912388057290
1 .00398063482304 -1.72974647995027
1 .00420874039761 -1.72961406264780
1 .00416023020985 -1.72964222660783
Numerical Methods © Wen-Chieh Lin 47
Next Monday
Introduction to Matlab/OctaveMatlab offers a student version with discounted
priceOctave is a shareware that has similar environment
as Matlab!
You are encouraged to bring your laptop withOctave or Matlab installed next Monday