Numerical Methods for Partial Differential Equations

CAAM 452Spring 2005

Lecture 3AB2,AB3, Stability, Accuracy

Instructor: Tim Warburton

CAAM 452 Spring 2005

Recall: Euler-Forward Time Stepping

• By several different approximations we arrived at the following approximate ODE time-stepping scheme:

• Assuming:• We established the following stability region

• i.e. if mu*dt is in the yellow region the approximate solution will be bounded.

0

1

0

n n n

u u

u u dtf u

1 1dt

x Re

Im

duf u

dt

for some f u u

-1

CAAM 452 Spring 2005

Recall: Adams Bashford 2

• We went on to derive a similar method which uses

• Namely AB2:

• This was based on an interpolation of f(u(t))

• We next ask which values of mu*dt is this stable for?

1 and n nf u t f u t

1 1

3 12 2n n n nu u dt f u f u

CAAM 452 Spring 2005

Derivation of AB3

• Suppose we wish to use three previous values of f

• As previously we look for a fit through this data, although this time we choose a different path.

(i) Consider dt = 1 (will rescale later)

(ii) Will form the coefficients of the interpolant using a Vandermonde matrix and a monomial basis in t

(iii) Finally integrate the integrand (easy with this choice of basis).

1 2, and n n nf u t f u t f u t

Simpler Approach

22

2

1

0

1

2

1

0

1

22

2 1

0

1

212

2 1

0 00

2 4

1 2 4

1 1 1

1 0 0

1 2 4

1 1 1 1

1 0 0

1 2 4

1 1 1 1

1 0 0

If a bt ct

f a b c

f a b c

f a

a f

b f

c f

f

If t t f

f

f

If dt t t f dt

f

1

1

2

1 0 1 2

0

1 2 41 1 1

1 1 1 1 23 16 52 3 12

1 0 0

f

f f f f

f

(i) form quadratic interpolant:

(ii) Compute coefficients:

(iii) Integrate interpolant:

CAAM 452 Spring 2005

AB3

• We now have an interpolant for dt=1 based on three time levels:

• We rescale and shift:

• And finally have AB3:

1

2 0 1 2

0

123 16 5

12If dt f f f

1

2 1 223 16 512

n

n

t

n n n

t

dtIf dt f f f

1 1 223 16 512n n n n n

dtu u f f f

CAAM 452 Spring 2005

Absolute Stability of AB2

• We again assume:

• AB2 becomes:

• Tidying up:

• To start this method we require:

for some f u u

1 1

3 12 2n n n nu u dt u u

1 1

3 11 0

2 2n n nu u u

1

0 0

u u dt

u u

dt

CAAM 452 Spring 2005

Absolute Stability of AB2 cont

• We define the absolute stability region of this method as the set of nu in the complex plane for which the sequence {un} is bounded as

• We will make continuous reference to the stability polynomial of this recurrence relationship (obtained by setting : and canceling excess powers of z )

1 1

3 11 0

2 2n n nu u u

2 3 11

2 2z z z

nnu z

n

CAAM 452 Spring 2005

Note on Solutions of Recurrence Relations

• Given an s+1 term recurrence relationship (concisely written as):

• With stability polynomial:

• which (we suppose) has s distinct roots

• Then any solution to the recurrence relation can be written as:

• where the cm depend on the s initiating values for the recurrence.

• This follows because is a solution, and there is a limited rank of possible solutions because of the linearity of the recurrence in the initial values.

1

m s

m mr

1

m sn

n m mm

u r c

0

0j s

j n jj

u

0

j sj

jj

z z

0

1

n

n n su

nu rn m

Board demo

CAAM 452 Spring 2005

cont

• Since all solutions under these conditions can be written as:

• We immediately observe that this is bounded if and only if

• See Trefethen p43-46 for details of the case where the stability polynomial has 1 or more roots with multiplicity greater than 1.

1

m sn

n m mm

u r c

1mr

CAAM 452 Spring 2005

Root Condition for Absolute Stability

• A linear multistep formula is absolutely stable for a particular value of nu if and only if all the zeros of the stability polynomial satisfy and any zero with is a simple root.

z 1z 1z

See: Trefethen p58-60

CAAM 452 Spring 2005

Drawing the Stability Region

• We need to find the roots of the stability polynomial:

• i.e.

• Which lie on the margin of stability

• The method we adopt is to setand find the value of nu such that r is a root of the stability polynomial.

2 3 11

2 2z z z

such that 0r r

1r

for [0,2 )ir e

CAAM 452 Spring 2005

cont

• We can rearrange the root equation to find nu in terms of z:

• We can now ask matlab to plot this curve in the complex plane…

2

2 2

3 11 0

2 2

3 1 3 12 2 2 2

i i

i

r r r

r r e e

r e

CAAM 452 Spring 2005

Matlab Plot of Margin of Stability for AB2

CAAM 452 Spring 2005

AB2 v. AB3 v. AB4

• I repeated this for AB2, AB3, AB4

• Notice how the overall region of absolute stability gets smaller as we use more and more previous values of f

• This translates into stricter restrictions on dt to force nu=mu*dt into the stability regions.

CAAM 452 Spring 2005

Zooming Into The All Important Imaginary Axis

• Recall, we discounted Euler-Forward as useless for time-stepping the advection equation because it had zero coverage of the imaginary axis.

• This picture does not quite tell us ifthe imaginary axis is inside the absolutestability region for AB2, AB3 and AB4.

• So we had better check the root condition.

CAAM 452 Spring 2005

Root Condition for AB2

• We need to check that the roots of:

• Are bounded above by 1, and if any roots are on the unit circle we must make sure they are simple.

• We set nu=i*alpha

2 3 11 0

2 2z z

2 3 11 0

2 2z i z i

CAAM 452 Spring 2005

Always Experiment First

• So rather than waste time figuring out where the roots are, why not let matlab do the hardwork:

CAAM 452 Spring 2005

If at first..

• Since the plot is not quite clear, we plot:

sign(maximum|root|-1)*log10(|maximum|root|-1|)

In summary: the only stable point on the imaginary axis for AB2 is the origin.AB3 is stable for nu in i*[-.723…,.723…]

CAAM 452 Spring 2005

Accuracy & Consistency

• We already established experimentally that stability is not enough for a good solution – we also need to examine how accurate a solution will be.

Section 1.3 p21-29 Trefethen

CAAM 452 Spring 2005

Consistency

• Linear Multistep schemes (like Euler-Forward, AB2,AB3..) can be written as:

• It is natural to ask what is the local discretization error (or local truncation error) for the above formula when we substitute

• We define the residual operator

0 0

j s j s

j n j j n jj j

u dt f u

n for unu t

0 0

:j s j s

n j n j j n jj j

Lu t u t dt f u t

CAAM 452 Spring 2005

Consistency cont

• If the linear multistep method is accurate the truncation error should be small.

• Definition: if then we say the scheme is p’th order accurate.

0 0

:j s j s

n j n j j n jj j

Lu t u t dt f u t

1 as 0pnLu t O dt dt

CAAM 452 Spring 2005

Example

• Euler-Forward has:

• So:

• The next step is to estimate this in terms of dt and time derivatives of the solution.

0 0

1

1

:j s j s

n j n j j n jj j

n n n

n n n

Lu t u t dt f u t

u t u t dtf u t

duu t u t dt t

dt

0 1 0 01, 1, 1, 1, 1s

CAAM 452 Spring 2005

Recall: Taylor’s Theorem

• Assuming that f is sufficiently smooth (has n bounded derivatives):

• Then we can evaluate f at x+delta by a linear sum of f at x and its derivatives:

• The residual takes various forms:

2 2 3 3 1 1

2 3 1...2! 3! 1 !

n n

nn

df d f d f d ff t f t t t t t R

dt dt dt n dt

•http://mathworld.wolfram.com/TaylorsTheorem.html•http://mathworld.wolfram.com/LagrangeRemainder.html

1

1 !

nu t n

n nu t

u d fR u du

n dt

* * for some ,!

n n

n n

d fR t t t t

n dt

(Lagrange remainder)

CAAM 452 Spring 2005

Back to Forward-Euler

• We can now express the local truncation error for Forward-Euler in terms of:

• expanding about tn we obtain:

2

2, , ,...n n n

du d uu t t t

dt dt

1

2 2*

2

2 2*

2

2!

2!

n n n n

n n n n n

n

duLu t u t u t dt t

dt

du dt d u duu t dt t t u t dt t

dt dt dt

dt d ut

dt

CAAM 452 Spring 2005

cont

2 2

* 222!n n

dt d uLu t t O dt

dt

This indicates that the Euler-Forward methodis “first order in time”.

So what does this mean for solution accuracy?

CAAM 452 Spring 2005

Existance and Uniqueness for the Initial Value Problem

00

, for all 0,

u u

duf u t t t T

dt

Theorem (p13 Trefethen):

Let f(u,t) be continuous with respect to t and uniformly Lipschitz continuous with respect to u for .Then there exists a unique differentiable function u(t) that satisfies the initial-value problem:

0,t T

The important condition here is that the right hand side f is not too non-linear.With these conditions – we know we have a single solution to aim for.

CAAM 452 Spring 2005

Definition: Convergence

• A linear multistep method is convergent if, for all IVP’s satisfying the existance/uniqueness theorem conditions on an interval [0,T] and all starting values for the method tend to u(0) as

• the solution satisfies

• uniformly for all

0dt 0 1 1, ,..., sv v v

nv

1 as 0v t u t o dt

0,t T

The notation small ‘o’ of 1 as dt tends to zero, requires that the solution drops to zero as dt tends to 0.

CAAM 452 Spring 2005

Dahlquist Equivalence Theorem

Theorem:

A linear multistep formula is convergent if and only if it is consistent and stable.

Proving convergence of a method directly is difficult, however proving

consistency and stability is relatively straight forward.

p54 Trefethen

CAAM 452 Spring 2005

Finally: an estimate for solution accuracy

• Theorem (global p’th order accuracy):

• Consider an IVP that satisfies the existance/uniqueness theorem conditions on an interval [0,T] and in addition that the right hand side function f(u,t) is p times continuously differentiable with respect to u and t. Let an approximate solution be computed by a convergent linear multistep formula of accuracy >= p with starting values as in the convergence statement. Then this solution satisfies:

• uniformly for all

as 0pv t u t O dt dt

0,t T

CAAM 452 Spring 2005

Summary

• Stability + consistency convergence

• convergence + method accuracy global solution accuracy

CAAM 452 Spring 2005

Sketch of Solution Accuracy Proof (not entirely rigorous)

• Given the way we constructed the AB schemes it is instructive to write down the following three equations:

0

10

0

0

0

0

T

T n N

p nn

T

p

i u T u f u t dt

ii u T u I f u t dt L u t

iii u T u I f u t dt

N=T/dt

CAAM 452 Spring 2005

Sketch cont

0

10

0

0

0

0

T

T n N

p nn

T

p

i u T u f u t dt

ii u T u I f u t dt L u t

iii u T u I f u t dt

(i) Represents the actual solution (which we know exists and is unique by the Lipschitz assumptions on f)

(ii) Represents acknowledges the impact of using an interpolant for f on the ODE (i.e. the local truncation errors represent the residuals)

(iii)Represents the numerical solution u tilde.

CAAM 452 Spring 2005

Sketch cont

• Part 1: we used (i)-(ii) (with T=dt) to estimate L for each time step.

• Part 2: we look at (ii)-(iii)

1- pnii i L u t O dt

10

0

10

0

0

-(iii) 0 0

T n N

p nn

T

p

T n N

p nn

ii u T u I f u t dt L u t

iii u T u I f u t dt

ii u T u T u u I f u t f u t dt L u t

CAAM 452 Spring 2005

Sketch cont

• We can now try to bound each term:

10

10

0 0

0 0

T n N

p nn

T n N

p nn

u T u T u u I f u t f u t dt L u t

u u I f u t f u t dt L u t

Bounded by assumption on Initial conditions

Bounded by assumption on Stability of the Ip interpolationoperator

(i) Bounded by assumption on consistency.(ii) Estimated by accuracy study.

CAAM 452 Spring 2005

Euler-Forward Solution Accuracy

Consider:

1) the right hand side function is certainly Lipschitz!

2) It is certainly p=1 times differentiable

3) If we keep mu*dt in the absolute stability region (unit circle centered at -1 in the complex plane) then it is stable.

4) By Taylor’s theorem it is a p=1 order accurate method.

5) All of the above indicate global solution accuracy which is first order in dt, i.e. the error will be O(dt) as dt0

00 ,du

u u udt

CAAM 452 Spring 2005

AB2 Global Accuracy

• We already established the stability region for AB2 – so as long as we keep mu*dt in that region we have stability.

• Now we have to consider accuracy:

1 1

32 2 3*

2 3

32 2**

2 3

3 3 3 3* **

3 3

3

3 1

2 2

2! 3!

3 1

2 2 2!

3! 2!2

n n n n n

n n

du duLu t u t u t dt t t

dt dt

d udu dt d u dtu dt u

dt dt dt

d udu du d u dtdt dt

dt dt dt dt

dt d u dt d ut t

dt dt

O dt

AB2 is a 2nd orderacurate method

CAAM 452 Spring 2005

cont

• If the AB2 stability condition is met then the global accuracy result indicates that for a solution at fixed time the solution will be

• Thus we characterize the scheme as 2nd order method accurate, and 2nd order solution accurate.

2O dt

CAAM 452 Spring 2005

AB3 Convergence

1 1 2

2 2 3 3 4 4*

2 3 4

2 2 3 3 4**

2 3 4

2 2 3

2 3

123 16 5

12

2! 3! 4!

23 162! 3!1

12 4 85 2

2!

n n n n n n

du du duLu t u t u t dt t t t

dt dt dt

du dt d u dt d u dt d uu dt u

dt dt dt dt

du du d u dt d u dt d udt

dt dt dt dt dtdt

du d u dt d udt

dt dt dt

3 4***4

4 4 4 4 4 4* ** ***

4 4 4

4

3!

2 5

4! 9 9n n n

dt d u

dt

dt d u dt d u dt d ut t t

dt dt dt

O dt

Expand all terms in the local truncation error about tn

The AB3 scheme is 3rd order accurate as long as thesolution is sufficiently smooth.

CAAM 452 Spring 2005

Homework 1

Read chapter 1 of Trefethen’s book.

Q1) derive AB4 (use Matlab or Mathematica)

Q2) plot the stability region for AB4

Q3) indicate which part of the complex plane is inside the AB4 region of absolute stability (you may need to test at least one point in each closed loop of the marginal stability curve)

continued on next slide

CAAM 452 Spring 2005

Homework 1 cont

Q4a) estimate upper bounds for dt to solve the following with AB2,AB3, and AB4

Q4b) write a code which uses Euler-Forward for the first time step, AB2 for the second time step and AB3 for the third time step (use a suitable dt for AB3 for all time steps)

Q4c) use this code and plot the error as a function of time for dt, dt/2, dt/4, dt/8 (i.e. run the simulation 4 times).

Q4d) estimate the order of accuracy of the code based on the error at t=5

Q4e) (extra credit) compare the code with an AB3 method started with exact initial data.

0 1

.1 .5

u

duu

dti

0,5t