1. Numerical Treatment of ODE€¦ · Numerical Treatment... Euler’s Method Discretization Error...

transcript

Numerical Treatment . . .

Euler’s Method

Discretization Error

Order of Convergence

Method of Runge and . . .

Alternative: Multistep . . .

Implicit Methods

Types of Problems for . . .

Ill-Conditioned Problems

Stability

Stiffness

Boundary Value . . .

Introduction to Scientific Computing

5. Numerical Methods forODE

Miriam Mehl

1. Numerical Treatment of ODE

• remember population dynamics: one or some ODE

– given as an initial value problem : starting point given

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– sometimes also as a boundary value problem: starting andfinal point given (think of a space shuttle’s trajectory, e.g.)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• prototypes of an initial value problem (IVP):

y(t) = f(t, y(t)), y(a) = ya, t ≥ a

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

y(t) = f(t, y(t)), y(a) = ya, t ≥ a

or for several unknowns yi, i = 1, . . . , n:

yi(t) = fi(t, y1(t), . . . , yn(t)), yi(a) = yi,a, t ≥ a, i = 1, . . . , n

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

y(t) = f(t, y(t)), y(a) = ya, t ≥ a

or for several unknowns yi, i = 1, . . . , n:

yi(t) = fi(t, y1(t), . . . , yn(t)), yi(a) = yi,a, t ≥ a, i = 1, . . . , n

• if f depends on t only: simple integration or quadrature

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

2. Euler’s Method

• standard approach for IVP: finite difference approximation (dif-ference quotient instead of derivative)

yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

2. Euler’s Method

yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,

which gives an explicit formula for yk+1 once yk is given:

yk+1 = yk + δt · f(tk, yk).

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

2. Euler’s Method

yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,

• this is the simplest strategy and called Euler’s method

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

2. Euler’s Method

yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,

• this is the simplest strategy and called Euler’s method

• other derivation: truncated Taylor expansion of y(t)

y(tk+1) = y(tk) + δt · y(tk) + O(δt2).= y(tk) + δt · f(tk, yk)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

3. Discretization Error

• local discretization error: local influence of using differences in-stead of derivatives; here:

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

l(δt) = max[a,b]{| y(t+δt)−y(t)δt

− f(t, y(t)) |}

with y(t) exact solution of the ODE.

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

− f(t, y(t)) |}

• global discretization error: maximum error of all computed dis-crete approximations:

e(δt) = max[a,b]{| yk − y(tk) |}

with y(t) exact solution of the ODE, yk, k = 1, 2, . . . solution ofthe discretized equation.

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

− f(t, y(t)) |}

• consistency:

l(δt) → 0 for δt → 0

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

− f(t, y(t)) |}

• consistency:

l(δt) → 0 for δt → 0

• convergence (stronger):

e(δt) → 0 for δt → 0

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

4. Order of Convergence

• Euler (some restrictions with respect to y and f ):

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– consistent of first order: l(δt) = O(δt)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– convergent of first order: e(δt) = O(δt)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– There are methods which are consistent but do not con-verge!

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• look for higher-order methods (faster convergence)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– start from Taylor expansion: leads to complicated formulas(higher derivatives of f )

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– use additional evaluations of f : Runge-Kutta-type methods

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– simplest representative: method of Heun

yk+1 = yk + δt2

(f(tk, yk) + f(tk+1, yk + δt · f(tk, yk))

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– simplest representative: method of Heun

yk+1 = yk + δt2

(f(tk, yk) + f(tk+1, yk + δt · f(tk, yk))

)both consistent and convergent of second order

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

5. Method of Runge and Kutta

• most famous representative: Runge-Kutta method

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk +δt

6(T1 + 2T2 + 2T3 + T4),

T1 = f(tk, yk),

T2 = f

2, yk +

T3 = f

2, yk +

T4 = f(tk+1, yk + δtT3)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk +δt

6(T1 + 2T2 + 2T3 + T4),

T1 = f(tk, yk),

T2 = f

2, yk +

T3 = f

2, yk +

T4 = f(tk+1, yk + δtT3)

• consistent and convergent of fourth order

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk +δt

6(T1 + 2T2 + 2T3 + T4),

T1 = f(tk, yk),

T2 = f

2, yk +

T3 = f

2, yk +

T4 = f(tk+1, yk + δtT3)

• consistent and convergent of fourth order

• Euler/Heun/Runge-Kutta correspond to rectangle/trapezoidal/Simp-son quadrature!

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

6. Alternative: Multistep Methods

• Runge-Kutta-type methods are expensive : many evaluationsof f (sometimes not given in closed form)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• different way to get higher order by profiting from history: Adams-Bashforth-type or multistep methods

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

– prominent representative: second-order method

yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

)– idea: use evaluations of f for previous approximations

yk−p+1, . . . , yk to find an approximation of the f -integral inthe integral form of the ODE:

yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.

– general form: take polynomial P (t) interpolating f in thediscrete points of time (tk−p+1, . . . , tk)

yk+1.= yk +

∫ tk+1

tkP (t)dt

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.

yk+1.= yk +

∫ tk+1

tkP (t)dt

– p = 1: Euler; p = 2: above method; generally: order p

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.

yk+1.= yk +

∫ tk+1

tkP (t)dt

– p = 1: Euler; p = 2: above method; generally: order p

– start: no/not enough predecessors available; hence modify!

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

• All schemes mentioned so far are explicit ones: The rule showsa direct way to do another time step.

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

• now: use the new value yk+1 on the right-hand side, too

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

• This leads us to Adams-Moulton multistep schemes:

– use interpolation and previous values as with Adams-Bashforth(fj = f(yj, tj))

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

– second order variant:

yk+1 = yk + δt2(fk + fk+1)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

yk+1 = yk + δt2(fk + fk+1)

– fourth order variant:

yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

yk+1 = yk + δt2(fk + fk+1)

yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)

• How to get fk+1 = f(yk+1, tk+1) in the implicit case?

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

yk+1 = yk + δt2(fk + fk+1)

yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)

– straightforward way: solve the (generally nonlinear) equa-tion

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

7. Implicit Methods

yk+1 = yk + δt2(fk + fk+1)

yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)

– straightforward way: solve the (generally nonlinear) equa-tion

– easier (and widespread): predictor-correctorapproach

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

8. Types of Problems for Numerical Methods for ODEs

• Ill-Conditioned Problems:

small changes in the input ⇒ big changes in the exact solutionof the ODE

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• Instabilities:

big errors in the approximate (i.e. numerical) solution comparedto the exact solution (for arbitrarily small time steps although themethod is consistent)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• Instabilities:

big errors in the approximate (i.e. numerical) solution comparedto the exact solution (for arbitrarily small time steps although themethod is consistent)

• Stiffness:

small time steps required for acceptable errors in the approxi-mate solution (although the exact solution is smooth)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

9. Ill-Conditioned Problems

• small changes in input entail completely different results

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• Numerical treatment of such problems is difficult!

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• an example:

– consider the ODE: y(t)−N · y(t)− (N + 1) · y(t) = 0

– initial conditions: y(0) = 1, y(0) = −1

– exact solution: y(t) = e−t

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• an example:

• slight change in initial condition:

– new value of y in t = 0: yε(0) = 1 + ε

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• an example:

– resulting new solution: yε(t) =

(1 + N+1

)e−t + ε

N+2e(N+1)t

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• an example:

(1 + N+1

)e−t + ε

N+2e(N+1)t

– arbitrarily small change leads to completely different result:t →∞

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• an example:

(1 + N+1

)e−t + ε

N+2e(N+1)t

– arbitrarily small change leads to completely different result:t →∞

• risk: non-precise input, round-off errors,. . .

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

• consider another IVP: y(t) = −2y(t) + 1, y(0) = 1

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

• exact solution: y(t) =

(e−2t + 1

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

(e−2t + 1

• well-conditioned: yε(0) = 1 + ε ⇒ yε(t)− y(t) = εe−2t

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

(e−2t + 1

• use the midpoint rule: yk+1 = yk−1 + 2δt · fk

yk+1 = yk−1 + 2δt(−2yk + 1) = yk−1 − 4δt · yk + 2δt, y0 = 1

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

(e−2t + 1

• 2-step rule: start with initial value and the exact y(δt)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

(e−2t + 1

– time step δt = 1.0 ⇒ y9 = −4945.5, y10 = 20953.9

– time step δt = 0.1 ⇒ y79 = −1725.3, y80 = 2105.7

– time step δt = 0.01 ⇒ y999 = −154.6, y1000 = 158.7

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

(e−2t + 1

– time step δt = 1.0 ⇒ y9 = −4945.5, y10 = 20953.9

– time step δt = 0.1 ⇒ y79 = −1725.3, y80 = 2105.7

– time step δt = 0.01 ⇒ y999 = −154.6, y1000 = 158.7

• midpoint rule is second-order consistent, but does not convergehere: oscillations or instable behaviour

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

10. Stability

(e−2t + 1

– time step δt = 1.0 ⇒ y9 = −4945.5, y10 = 20953.9

– time step δt = 0.1 ⇒ y79 = −1725.3, y80 = 2105.7

– time step δt = 0.01 ⇒ y999 = −154.6, y1000 = 158.7

• midpoint rule is second-order consistent, but does not convergehere: oscillations or instable behaviour

• there are stability conditions; generally:

consistency + stability = convergence

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

• for another phenomen, consider the IVP

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

• exact solution: y(t) = e−1000t + 1

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

• problem well-conditioned

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

• explicit Euler: stable (as all explicit 1-step methods or all Adams-Bashforth or all s-step Adams-Moulton methods for s > 1 are)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

yk+1 = yk + δt(−1000yk + 1000) = (1− 1000δt)yk + 1000δt

= (1− 1000δt)k+1 + 1

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

yk+1 = yk + δt(−1000yk + 1000) = (1− 1000δt)yk + 1000δt

= (1− 1000δt)k+1 + 1

• oscillations and divergence for δt > 0.002

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

11. Stiffness

y(t) = −1000y(t) + 1000, y(0) = y0 = 2

yk+1 = yk + δt(−1000yk + 1000) = (1− 1000δt)yk + 1000δt

= (1− 1000δt)k+1 + 1

• oscillations and divergence for δt > 0.002

• Why that? Consistency and stability are asymptoticterms. Rem-edy: Implicit methods (try implicit Euler)!

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

12. Boundary Value Problems: Outlook

• example:

y = f(t, y, y), ta ≤ t ≤ tb,

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• example:

y = f(t, y, y), ta ≤ t ≤ tb,

y(ta) = ya, y(tb) = yb

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• example:

y = f(t, y, y), ta ≤ t ≤ tb,

• special case:

y(t) = a(t)y(t) + b(t)y(t) + c(t), same b.c.

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• example:

y = f(t, y, y), ta ≤ t ≤ tb,

• special case:

• a vanishing and b positive: BVP has unique solution

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• example:

y = f(t, y, y), ta ≤ t ≤ tb,

• special case:

• a vanishing and b positive: BVP has unique solution

• discrete grid:

δt = (tb − ta)/n, t0 = ta, tn = tb, ti = ta + iδt

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

13. Boundary Value Problem 2

• y(t) = b(t)y(t) + c(t), y(ta) = ya, y(tb) = yb

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• finite difference approximation for second derivative:

y(t).= y(t+δt)−2y(t)+y(t−δt)

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• discrete analogon to ODE in each grid point:

δt−2 · (yi+1 − 2yi + yi−1)− biyi = ci, i = 1, . . . , n− 1

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• tridiagonal system of linear equations

Euler’s Method

Implicit Methods

Stability

Stiffness

Miriam Mehl

• tridiagonal system of linear equations

• alternative: shooting methods(reduction to IVPs)

1. Numerical Treatment of ODE€¦ · Numerical Treatment... Euler’s Method Discretization Error...

Documents