18.336|Numerical Methods for Partial Di erential Equations ...plamen/18.336/math336.pdf1.3. VON...

18.336—Numerical Methods for Partial Differential Equations,

Spring 2005

Plamen Koev

September 6, 2012

2

Contents

1 Hyperbolic PDEs 51.1 Consistency, Stability, Well-posedness, and Convergence . . . . . . . . . . . . . . . . . . . . . 51.2 Fourier Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Von Neumann Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.4 The Leap-Frog Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91.5 Dissipation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.6 Dispersion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131.7 Group Velocity and Propagation of Wave Packets . . . . . . . . . . . . . . . . . . . . . . . . . 151.8 Summary of Schemes for the Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2 Parabolic equations 192.1 The Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2 The Du Fort–Frankel Scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.3 The Convection-Diffusion Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.4 Summary of Schemes for the Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Systems of PDEs in Higher Dimensions 273.1 The Equation ut +Aux = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 The Equation ut +Aux = Bu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.3 The Equation ut +Aux +Buy = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.4 The equation ut = b1uxx + b2uyy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303.5 ADI methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.6 Boundary conditions for ADI methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4 Elliptic Equations 354.1 Steady-State Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354.2 Numerical methods for uxx + uyy = f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384.3 Jacobi, Gauss–Seidel, and SOR(ω) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3

4 CONTENTS

Chapter 1

Hyperbolic PDEs

We consider the hyperbolic equationut + aux = 0

for t ≥ 0 and initial condition u(0, x) = u0(x). The unique solution to this problem is given by

u(t, x) = u0(x− at),i.e., the solution is wave traveling right if a > 0 and left if a < 0.

We will show (a) how to generate schemes for its numerical solution, (b) verify that these schemes area good approximation to the differential equation (i.e., are consistent) and (c) that the numerical solutionconverges to the solution to the differential equation.

The idea in using finite differences to solve a PDE is to select a grid in time and space (with meshlengthsk and h, respectively) and to approximate the values u(mk, nh) for integer m,n. All all that follows u willdenote the exact solution to the PDE and

v(time)(space) = vnm ≈ u(mh, kn)

will denote the approximate finite difference solution.We will approximate derivatives of a function f as follows:

δ+f(x) =f(x+ h)− f(x)

hforward difference

δ−f(x) =f(x)− f(x− h)

hbackward difference

δ0f(x) =f(x+ h)− f(x− h)

2hcentered difference

For a grid function v = (. . . , v−2, v−1, v0, v1, v2, . . .) we have:

δ+vm =vm+1 − vm

hforward difference

δ−vm =vm − vm−1

hbackward difference

δ0vm =vm+1 − vm−1

2hcentered difference

1.1 Consistency, Stability, Well-posedness, and Convergence

Definition 1 (Consistency). We say that a finite difference scheme Pk,hv = f is consistent with the PDEPu = f of order (r, s) if for any smooth function φ

Pφ− Pk,hφ = O(kr, hs) (1.1)

5

6 CHAPTER 1. HYPERBOLIC PDES

To verify consistency expand φ in Taylor series and make sure (1.1) holds.

Definition 2 (L2 norm). For a function w = (. . . , w−2, w−1, w0, w1, w2, . . .) on a grid with step size h:

‖w‖ =

(h

∞∑m=−∞

|wm|2)1/2

For a function f on the real line:

‖f‖ =(∫ ∞−∞|f(x)|2dx

)1/2

Definition 3 (Stability). A finite one-step difference scheme Pk,hvnm = 0 for a first-order PDE is stable ifthere exist numbers k0 > 0 and h0 > 0 such that for any T > 0 there exists a constant CT such that

‖vn‖ ≤ CT ‖v0‖

for 0 ≤ nk ≤ T, 0 < h ≤ h0, 0 < k ≤ k0.

Definition 4 (Well-posedness). The initial value problem for the first-order PDE Pu = 0 is well-posed iffor any time T ≥ 0, there is a constant CT , such that any solution u(t, x) satisfies

‖u(t, x)‖ ≤ CT ‖u(0, x)‖, for 0 ≤ t ≤ T.

Definition 5 (Convergence). A one-step finite difference scheme approximating a PDE is convergent if forany solution to the PDE, u(t, x), and solution to the finite difference scheme vnm, such that v0

m → u(0, x) asmh→ x, we have vnm → u(t, x) as (nk,mh)→ (t, x) (as h, k → 0).

Theorem 1 (Lax). A consistent finite difference scheme for a PDE for which the initial value problem iswell-posed is convergent if and only if it is stable.

Theorem 2 (The Courant–Friedrichs–Lewy Condition). A necessary condition for stability of the explicitscheme for the hyperbolic equation ut + aux = 0:

vn+1m = αvnm−1 + βvnm + γvnm+1

with k/h = λ held constant is|aλ| ≤ 1.

Proof. The solution is u(t, x) = u0(x − at) and u(1, 0) = u0(−a). The finite difference scheme vn0 dependsonly on v0

m for |m| ≤ n. Therefore |hn| ≥ | − a|. Since kn = 1, we have n = 1/k and |n/k| ≥ |a| or|aλ| ≤ 1.

1.2 Fourier Analysis

Fourier Transform and Inversion formula

• For u defined on R

u(ω) =1√2π

∫ ∞−∞

e−iωxu(x)dx, u(x) =1√2π

∫ ∞−∞

eiωxu(ω)dω

• For a grid function v = (. . . , v−2, v−1, v0, v1, v2, . . .) with grid spacing h (here ξ ∈ [−π/h, π/h])

v(ξ) =1√2π

∞∑m=−∞

e−imhξvmh, vm =1√2π

∫ π/h

−π/heimhξ v(ξ)dξ

Theorem 3 (Parseval).‖u(x)‖ = ‖u(ω)‖, ‖v‖ = ‖v‖.

(where ‖v‖2 =∫ π/h−π/h |v(ξ)|2dξ.)

1.3. VON NEUMANN ANALYSIS 7

1.3 Von Neumann Analysis

Provides an uniform way of verifying if a finite difference scheme is stable.

Example 1. Let’s study the forward-time backward-space scheme

vn+1m − vnm

k+ a

vnm − vnm−1

h= 0.

Rewrite asvn+1m = (1− aλ)vnm + aλvnm−1,

where λ = k/h. Use the Fourier inversion formula

vnm =1√2π

∫ π/h

−π/heimhξ vn(ξ)dξ

and substitute to obtain

1√2π

∫ π/h

−π/heimhξ vn+1(ξ)︸︷︷︸

∗

dξ =1√2π

∫ π/h

−π/heimhξ [(1− aλ) + aλe−ihξ]vn(ξ)︸︷︷︸

∗∗

dξ.

The Fourier transform is unique, so (*) must equal (**):

vn+1(ξ) = [(1− aλ) + aλe−ihξ]vn(ξ).

Denote g(hξ) ≡ (1− aλ) + aλe−ihξ, called amplification factor. We have

vn(ξ) = g(hξ)vn−1(ξ) = . . . =(g(hξ)

)nv0(ξ).

Now

‖vn‖2 =∫ π/h

−π/h|vn(ξ)|2dξ =

∫ π/h

−π/h|g(hξ)|2n|v0(ξ)|2dξ.

Therefore ‖vn‖ ≤ ‖v0‖ (i.e., the scheme is stable), if |g(hξ)| ≤ 1. Write θ = hξ and evaluate |g(θ)|2

|g(θ)|2 = |(1− aλ) + aλe−ihξ|2

= (1− aλ+ aλ cos θ)2 + a2λ2 sin2 θ

= (1− 2aλ sin2 θ2 )2 + 4a2λ2 sin2 θ

2 cos2 θ2

= 1− 4aλ(1− aλ) sin2 θ2 .

Thus |g(θ)| ≤ 1 if 0 ≤ aλ ≤ 1. Then ‖vn‖ ≤ ‖v0‖ and the scheme is stable if 0 ≤ aλ ≤ 1.

Theorem 4 (Stability condition). A one-step finite difference scheme is stable if and only if there existpositive constants K,h0, k0 such that

|g(θ, k, h)| ≤ 1 +Kk

for all θ, 0 < k ≤ k0, 0 < h ≤ h0. If g is independent of k, then the condition is

|g(θ, k, h)| ≤ 1.

Proof. Assume g(θ, k, h) ≤ 1 +Kk for some K.

‖vn‖2 =∫ π/h

−π/h|g|2n|v0(ξ)|2dξ ≤ (1 +Kk)2n

∫ π/h

−π/h|v0(ξ)|2dξ ≤ (1 +Kk)2n‖v0‖2


Now nk ≤ T and (1 +Kk)n ≤ (1 +Kk)T/k ≤ eKT , meaning ‖vn‖ ≤ eKT ‖v0‖ and the scheme is stable.Conversely, assume that for any C there exists an interval [θ1, θ2] such that |g| ≥ 1+Ck for θ ∈ [θ1, θ2], h ∈

(0, h0], and k ∈ (0, k0]. Let

v0(ξ) =

0 if hξ 6∈ [θ1, θ2],√h(θ2 − θ1)−1 if hξ ∈ [θ1, θ2].

Now ‖v0‖ = 1 and

‖vn‖2 =∫ π/h

−π/h|g|2n|v0(ξ)|2dξ =

∫ θ2/h

θ1/h

|g|2n h

θ2 − θ1dξ ≥ (1 + Ck)2n ≥ 1

2e2TC‖v0‖2

for n near T/k. Therefore the scheme is unstable if C can be arbitrarily large. If g is independent of h andk, then |g| ≤ 1 +Kk must hold for any 0 < k ≤ k0, therefore |g| ≤ 1.

In practice to analyze a finite difference scheme we do not write integrals. Instead we replace vnm bygneimθ and solve for g.

Example 2. Forward-time centered space

vn+1m − vnm

k+ a

vnm+1 − vnm−1

2h= 0.

0 =gn+1eimθ − gneimθ

k+ a

gnei(m+1)θ − gneim−1θ

2h= gneimθ

(g − 1k

+ aeiθ − e−iθ

2h

).

So g(θ) = 1− iaλ sin θ, with λ = k/h. If λ is constant, then |g(θ)|2 = 1 + a2λ2 sin2 θ > 1 and the scheme isunstable.

Example 3. The Lax–Wendroff scheme:

vn+1m = vnm −

aλ

2(vnm+1 − vnm−1) +

a2λ2

2(vnm+1 − 2vnm + vnm−1),

so the amplification factor is:

g(θ) = 1− aλ

2(eiθ − e−iθ) +

a2λ2

2(eiθ − 2 + e−iθ)

= 1− iaλ sin θ − a2λ2(1− cos θ)= 1− 2a2λ2 sin2 θ

2 − iaλ sin θ

Thus

|g(θ)|2 = (1− 2a2λ2 sin2 θ2 )2 +

(2aλ sin θ

2 cos θ2)2

= 1− 4a2λ2(1− a2λ2) sin4 θ2

The scheme is stable only when |g(θ)| ≤ 1, i.e., when |aλ| ≤ 1.

Example 4. For the Crank–Nicolson scheme

vn+1m = vnm −

aλ

4(vn+1m+1 − v

n+1m−1 + vnm+1 − vnm−1)

we obtain

g(θ) =1− 1

2 iaλ sin θ1 + 1

2 iaλ sin θthus |g(θ)|2 =

1 + ( 12aλ sin θ)2

1 + ( 12aλ sin θ)2

= 1

so this scheme is unconditionally stable.

1.4. THE LEAP-FROG SCHEME 9

1.4 The Leap-Frog Scheme

In this section we prove that Leap-frog is stable if and only if |aλ| < 1. The scheme is

vn+1m − vn−1

m

2k+ a

vnm+1 − vnm−1

2h= 0, i.e., vn+1

m = vn−1m + aλ(vnm+1 − vnm−1).

Write δ0vm = vm+1−vm−12h . Then

[vn+1m

vnm

]=[−2kaδ0 1

1 0

]·[

vnmvn−1m

].

Fourier transform for vectors = Fourier transform in each component:

[vn+1m

vnm

]=

1√2π

∫ π/h

−π/heimhξ

[vn+1(ξ)vn(ξ)

]dξ,[

vn+1(ξ)vn(ξ)

]=

1√2π

∞∑m=−∞

e−imhξ[vn+1m

vnm

]h

and Parseval for vectors (| · | means the 2-norm for vectors or matrices so we can tell it apart from theL2-norm)

∥∥∥∥[ vn+1

vn

]∥∥∥∥2

= h

∞∑m=−∞

∣∣∣∣[ vn+1m

vnm

]∣∣∣∣2 =∫ π/h

−π/h

∣∣∣∣[ vn+1(ξ)vn(ξ)

]∣∣∣∣2 dξ =∥∥∥∥[ vn+1(ξ)

vn(ξ)

]∥∥∥∥2

.

Now Fourier of Leap-Frog:

1√2π

∫ π/h

−π/heimhξ

[vn+1(ξ)vn(ξ)

]dξ =

[−2kaδ0 1

1 0

]1√2π

∫ π/h

−π/heimhξ

[vn(ξ)vn−1(ξ)

]dξ

=1√2π

∫ π/h

−π/h

[−2kaδ0eimhξ vn(ξ) + eimhξ vn−1(ξ)

eimhξ vn(ξ)

]dξ

=1√2π

∫ π/h

−π/heimhξ

[−2ka e

ihξ−e−ihξ2h 1

1 0

] [vn(ξ)vn−1(ξ)

]dξ

=1√2π

∫ π/h

−π/heimhξ

[−2iaλ sin(hξ) 1

1 0

] [vn(ξ)vn−1(ξ)

]dξ

Therefore

[vn+1(ξ)vn(ξ)

]=[−2iaλ sin(hξ) 1

1 0

]︸︷︷︸

G(hξ)

[vn(ξ)vn−1(ξ)

]= G ·

[vn(ξ)vn−1(ξ)

]= . . . = Gn ·

[v1(ξ)v0(ξ)

].


Parseval now gives ∥∥∥∥[ vn+1

vn

]∥∥∥∥2

=∥∥∥∥[ vn+1(ξ)

vn(ξ)

]∥∥∥∥2

=∫ π/h

−π/h

∣∣∣∣(G(hξ))n ·[v1(ξ)v0(ξ)

]∣∣∣∣2 dξ≤

∫ π/h

−π/h|G(hξ)|n ·

∣∣∣∣[ v1(ξ)v0(ξ)

]∣∣∣∣2 dξ≤ max

|hξ|≤π|G(hξ)|2n

∥∥∥∥[ v1(ξ)v0(ξ)

]∥∥∥∥2

= max|hξ|≤π

|G(hξ)|2n∥∥∥∥[ v1

v0

]∥∥∥∥2

.

Remains to see when the 2-norm of G is bounded. Jordan form: G = TΛT−1 and Gn = TΛnT−1.Characteristic polynomial:

g2 + 2iaλ sin(hξ)g − 1 = 0

Λn bounded only if the roots (not to be confused with λ): |λ1,2| ≤ 1, but λ1λ2 = −1, so we must have|λ1| = |λ2| = 1. Eigenvalues (denote s ≡ sin(hξ) for short):

λ1,2 = −iaλs±√

1− (aλs)2.

If |aλ| > 1, then there exists ξ, s.t. |aλs| > 1 and both λ1 and λ2 are purely imaginary and distinct, so oneof them will be > 1 and the other < 1 in magnitude. So we must have |aλ| ≤ 1.

When |aλ| ≤ 1 we have |λ1,2|2 = (aλs)2 +1− (aλs)2 = 1. Therefore both λ1 and λ2 are on the unit circle.If λ1 6= λ2, then Jordan form of G is (exercise):

G =[

1 1−λ2 −λ1

] [λ1

λ2

] [ −λ1 −1λ2 1

]−λ1 + λ2

Exercise: |A| ≤ ‖A‖F = (∑i,j |aij |2)1/2 (Frobenius norm).

Now

|Gn| ≤ |TΛnT−1| ≤ |T | · 1 · |T−1| ≤ ‖T‖F ‖T−1‖F ≤√

4 ·√

4| − 2

√1− |aλ|2|

≤ 2√1− |aλ|2

is nicely bounded. Going back to Parseval∥∥∥∥[ vn+1

vn

]∥∥∥∥ ≤ √2

(1− |aλ|2)1/4

∥∥∥∥[ v1

v0

]∥∥∥∥and Leap-frog is stable.

Next case: λ1 = λ2. It occurs when sin(hξ) = ±1 and |aλ| = 1. Assume aλs = 1, (the −1 case isanalogous).

G =[−2i 1

1 0

]=[

1 1i 0

] [−i −i

−i

] [0 −i1 i

]= TΛT−1

A little unusual to write a Jordan block with −i in position (1, 2) but legal and, in this case, convenient.

Gn = T (−i)n[

1 10 1

]nT−1 = (−i)nT

[1 n0 1

]T−1,

1.5. DISSIPATION 11

i.e., G(±π/2) will blow up. You’d think that there may be cancellation, but no:

n =∣∣∣∣[ 1 n

0 1

]∣∣∣∣ = |inT−1GnT | ≤ |T | · |Gn| · |T |

and both |T | and |T−1| are nicely bounded by (say) 2, so |Gn| ≥ n/4 = T/(4k)→∞ as k → 0.The stability condition is therefore |aλ| < 1.

1.5 Dissipation

We would expect the wave equation to propagate the initial condition with a constant speed a, including allfrequencies that make up that initial condition.

Unfortunately the discrete nature of our data means that instead of the initial condition u0(x) we havea discrete version of it—vn0 .

The initial condition u0(x) is a superposition (in theory) of an infinite number of frequencies (thinkFourier expansion), whereas vn0 only inherits the frequencies ξ ∈ [−π/h, π/h]. All higher frequencies areignored by our discrete initial condition. Recall that the Fourier transform vn(ξ) of vn is only defined forξ ∈ [−π/h, π/h].

Obviously different frequencies are treated differently and we would like to get a better understanding ofthat treatment. Example is the best way to go here. Consider Lax–Friedrichs:

vn+1m − 1

2 (vnm+1 + vnm−1)k

+ avnm+1 − vnm−1

2h= 0,

equivalently,vn+1m = 1−aλ

2 vnm+1 + 1+aλ2 vnm−1.

Von Neumann analysis implies

g(hξ) = cos(hξ)− iaλ sin(hξ),|g(hξ)|2 = cos2(hξ) + (aλ)2 sin2(hξ).

Let θ = hξ as usual.We see that θ = 0 and θ = π are not dampened, but all other θ are. Let’s observe closely. Pick aλ to be

(say) 1/2. Then

vn+1m =

14vnm+1 +

34vnm−1

• θ = π/2. Then eimhξ = eimθ = eimπ/2 = . . . , 1, 0,−1, 0, 1, 0,−1, 0, 1, . . .

n = 4 1/16n = 3 1/8 0 −1/8n = 2 1/4 0 −1/4 0 1/4n = 1 1/2 0 −1/2 0 1/2 0 −1/2n = 0 1 0 −1 0 1 0 −1 0 1

• θ = π, we have eimhξ = eimθ = eimπ = . . . , 1,−1, 1,−1, 1, . . . = (−1)m.We can verify that vnm = (−1)m+n is a solution to Lax–Friedrichs, so θ = π is not dampened at all.

We don’t really expect good results for wildly oscillating solutions, so we can expect that the higherfrequencies will not be well-represented in our calculation. However it is unacceptable for higher frequenciesto be less dampened than the middle-range ones.

Another example. Look at Lax–Wendroff: |g|2 = 1− 4a2λ2(1− a2λ2) sin4 θ2 ≤ 1− const · sin4 θ

2 .This is very important—says that all frequencies, except ξ = 0 (then θ = 0) are decreasing and the

highest frequencies are suppressed the most. This is exactly what we want and will call schemes that havethis property dissipative.


Definition 6 (Dissipative Scheme). A scheme is dissipative of order 2r if

|g(θ)| ≤ 1− c · sin2r θ2 .

The reason we like dissipative schemes is that if we are not doing a good job with the high frequenciesanyway, why not kill them.

Remark 1. A dissipative scheme is always stable.

How can we make a non-dissipative scheme dissipative? This calls for another example. Crank–Nicolson,which is second order accurate.

vn+1m − vnm

k+ a

vn+1m+1 − v

n+1m−1 + vnm+1 − vnm−1

4h= 0

So adding a fourth derivative in there will not affect the order of accuracy of the approximation, since fourthderivatives get ignored anyway. When we do the Fourier analysis the fourth derivative will bring a sin4 θ

2 .

vn+1m − vnm

k+ a

vn+1m+1 − v

n+1m−1 + vnm+1 − vnm−1

4h+ Cε

vnm−2 − 4vnm−1 + 6vnm − 4vnm+1 + vnm+2

h4= 0

Now select C appropriately so that the fourth derivative only brings sin4 θ2 into the picture, without any

weird powers of k and h: C = h4

16k . Then after some simplification

g(θ) =1− ε sin4 θ

2 − iaλ2 sin θ

1 + iaλ2 sin θ

implying

|g(θ)|2 =1 +

(aλ2 sin θ

)2 − 2ε sin4 θ2 + ε2 sin8 θ

2

1 +(aλ2 sin θ

)2

= 1−ε sin4 θ

2 −

>0 for ε<1︷︸︸︷ε sin4 θ

2 (1− ε sin4 θ2 )

1 +(aλ2 sin θ

)2≤ 1−

ε sin4 θ2

1 +(aλ2 sin θ

)2 .If we now restrict |aλ| (say) ≤ 10, then 1 +

(aλ2 sin θ

)2 ≤ 26, and

|g|2 ≤ 1− ε

26sin4 θ

2 .

We want a bound on |g|, not |g|2:

|g|2 ≤ 1− ε

26sin4 θ

2 ≤ |g|2 ≤ 1− ε

26sin4 θ

2 +( ε

52

)2

sin8 θ2 =

(1− ε

52sin4 θ

2

)2

,

so|g| ≤ 1− ε

52sin4 θ

2 .

The scheme is all of a sudden dissipative of order 4 (since 2r = 4). Although Crank–Nicolson is stable forall aλ we cannot make it dissipative without restricting aλ.

The exact same trick works for Leap-frog.

1.6. DISPERSION 13

1.6 Dispersion

In this section we investigate whether in the numerical solution of ut + aux = 0 different frequencies travelwith the same speed a as they should. They, of course, do not and we will see that in fact travel with speedα(hξ) ≈ a.

Look for a solution tout + aux = 0, u(0, x) = f(x)

(which has a unique solution u(t, x) = f(x− at)) using separation of variables

u(t, x) = g(t)eixξ,

assuming u(0, x) = g(0)eixξ = eixξ =periodic wave (here we assume g(0) = 1). Then

ut + aux = g′(t)eixξ + ag(t)iξeixξ = (g′(t) + aiξg(t))eixξ = 0.

Since |eixξ| = 1 we have g′(t) + aiξg(t) = 0, which implies g(t) = e−iatξg(0). Insert back and get

u(t, x) = g(0)e−iatξeixξ = ei(x−at)ξ,

since g(0) = 1.Therefore the initial condition is translated with speed a for all ξ.

Example 5. Same Fourier analysis can be used for other equation also to study the speed of differentfrequency waves, e.g.,

ut + aux + uxxx = 0.

For u(t, x) = g(t)eixξ we get

ut + aux + uxxx = (g′ + iξ(a− ξ2)g)eixξ = 0,

thusg′ + iξ(a− ξ2)g = 0,⇒ g(t) = e−iξ(a−ξ

2)tg(0),

so the solution isu(t, x) = ei(x−(a−ξ2)t)ξg(0).

now the speed of the waves depends on ξ.

Definition 7 (Dispersion). The phenomenon of waves with different frequencies moving with different speedsis called dispersion.

Return now to the solution of the difference equation. Take Lax–Friedrichs:

vn+1m =

12

(vnm+1 + vnm−1)− aλ

2(vnm+1 − vnm−1).

Separation of variables: vnm = gneimhξ and substitute above to get

g = cos(hξ)− iaλ sin(hξ),

so the solution isvnm = (cos(hξ)− iaλ sin(hξ))neimhξ,

which looks nothing like ei(x−at)ξ. Let

g(hξ) ≡ ρe−iω = ρ cosω − ρi sinω = cos(hξ)− iaλ sin(hξ).


Therefore tanω = aλ tan(hξ), ρ2 = cos2(hξ) + a2λ2 sin2(hξ). For |hξ| ≤ π/2 we have

vnm = (ρe−iω)neimhξ = ρneimhξ−iωn = ρnei(mh−ωn/ξ)ξ = ρnei(x−ωnξ

tnk )ξ = ρnei(x−

ωkξ t)ξ = ρnei(x−α(hξ)t)ξ,

where x = mh, t = nk and

α(hξ) ≡ ω

kξ=

arctan(aλ tan(hξ))λhξ

(Recall tan ε ≈ ε and arctan ε ≈ ε so α ≈ arctan(aλ tan(hξ))λhξ ≈ a.)

We havevnm = |g(hξ)|n · ei(x−α(hξ)t)ξ.

Definition 8 (Phase speed). The quantity α(hξ) is called phase speed, and is the speed at which waves offrequency ξ are propagated by the difference scheme.

Once again, waves with different frequencies travel with different speeds. Thus we say that the scheme isdispersive. We want the scheme to be dispersive as little as possible (i.e., α(hξ) ≈ a), so that the numericalsolution looks like the exact solution.

Time to study the Taylor series for α(hξ) to obtain a better estimate of the closeness to a.

tan z = z +13z3 +O(z5)

arctan z = z − 13z3 +O(z5)

Let z = hξ

α(z) =arctan(aλ tan z)

λz

=aλ tan zλz

− (aλ tan z)3

3λz+ . . .

= a · z + z3/3 + . . .

z− a3λ3

λ

z3

3z+ . . .

= a

(1 + (1− a2λ2)

(hξ)2

3+ . . .

)So, if ξ is given and h is small, then the wave speed is slightly higher than a, and the high frequencies travelfastest. Let’s look at some special cases.

Take hξ = π/2. Then ω = π/2 and ρ = |aλ|, so

vnm = |aλ|neimhξ · e−inπ/2 = |aλ|nei(xξ−πn/2) = |aλ|nei(x−t/λ)ξ

(since nπ/2 = (t/k)(hξ) = tξ/λ). So the speeds can be quite different. Exact = a; Computed = 1/λ = aaλ ,

so it is not a good idea to take aλ small. The closer to the stability limit (i.e., the closer to 1) the better.

1.7. GROUP VELOCITY AND PROPAGATION OF WAVE PACKETS 15

1.7 Group Velocity and Propagation of Wave Packets

Consider the numerical solution to ut + aux = 0 with initial data u(0, x) = eiξ0xp(x), where p(x) decaysrapidly at ∞. The function u(0, x) is called a wave packet—see Figure 1.1.

−1 −0.5 0 0.5 1 1.5 2−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

1

Figure 1.1: Wave packet cos 5π cos2 πx2 on [−1, 1]

The exact solution is u(t, x) = eiξ0(x−at)p(x− at).

Proposition 1. A finite difference scheme will have a solution that is approximately

v∗(t, x) = eiξ0(x−α(hξ0)t)p(x− γ(hξ0)t),

where α(hξ0) is the phase speed, and γ(hξ0) is the group velocity, given by

γ(θ) = α(θ) + θα′(θ).

The rest of this section is devoted to proving Proposition 1 and may be skipped on a first reading.Consider a class of one-step schemes with the property

vn(ξ) = g(hξ)vn−1(ξ) = . . . = (g(hξ))nv0(ξ).

In addition, for simplicity, assume |g(hξ)| = 1. The numerical method will give

vnm =1√2π

∫ π/h

−π/heiξxm(g(hξ))nv0(ξ)dξ.

On the other sideu(0,mh) =

1√2π

∫ ∞−∞

eimhξu(0, ξ)dξ.

Split up the interval (−∞,∞) =⋃∞l=−∞[l 2πh −

πh , l

2πh + π

h ) to obtain

u(0,mh) =1√2π

∑l

∫ l 2πh +πh

l 2πh −πh

eimhξu(0, ξ)dξ.

Set ξ = l 2πh + ξ′, meaning dξ = dξ′ and

u(0,mh) =1√2π

∑l

∫ πh

−πheimh(l 2πh +ξ′)u(0, l

2πh

+ ξ′)dξ′

=1√2π

∫ πh

−πheimhξ

′∞∑

l=−∞

u(0, ξ′ + l2πh

)dξ′.


This gives a formula for

v0(ξ) =∞∑

l=−∞

u(0, ξ + l2πh

).

If u is smooth, then its Fourier transform decays rapidly, and only the middle l = 0 term really matters

v0(ξ) ∼ u(0, ξ),

with the error bounded by h to some high power depending on the smoothness of u(0, x).Consider

u(0, ξ) =1√2π

∫ ∞−∞

e−ixξu(0, x)dx

=1√2π

∫ ∞−∞

e−ixξeiξ0xp(x)dx

=1√2π

∫ ∞−∞

e−ix(ξ−ξ0)p(x)dx

= p(ξ − ξ0)

Let’s recall how we handle the phase speed

g(hξ) = |g(hξ)|e−iω = e−iω ⇒ (g(hξ))n = e−iωn = e−i ωnkk

nhξξ

= e−iωλhξ tξ = e−iα(hξ)tξ.

We can return to

vnm =1√2π

∫ π/h

−π/heiξxm · e−iα(hξ)tnξ · v0(ξ)dξ.

Exercise 1. Verify that eiξxm , v0(ξ), g(hξ), and ω are periodic functions of ξ with period 2πh .

Since v0(ξ) ∼ u(0, ξ) = p(ξ − ξ0), we change the variables φ = ξ − ξ0. Then ξ = φ+ ξ0, and

vnm =1√2π

∫ π/h+ξ0

−π/h+ξ0

ei(φ+ξ0)xm · e−iα(h(φ+ξ0))tn(φ+ξ0) · v0(φ+ ξ0)dφ.

Since all functions are periodic with period 2πh we can shift the interval of integration back and get

vnm =1√2π

∫ π/h

−π/hei(φ+ξ0)xm · e−iα(h(φ+ξ0))tn(φ+ξ0) · v0(φ+ ξ0)dφ.

This begins to look right. Factor out the phase speed

vnm = eiξ0(xm−α(hξ0)tn) · 1√2π

∫ π/h

−π/heiφxm · e−i[(φ+ξ0)α(hφ+hξ0)−ξ0α(hξ0)]tn · v0(φ+ ξ0)dφ.

This begins to look like a Fourier transform

∼ eiξ0(x−α(hξ0)t) · 1√2π

∫ π/h

−π/heiφ“x− (φ+ξ0)α(hφ+hξ0)−ξ0α(hξ0)

φ t”· v0(φ+ ξ0)dφ.

Since v0(ξ) ∼ p(ξ− ξ0) we have v0(φ+ ξ0) ∼ p(φ+ ξ0 − ξ0) = p(φ). The next step is to replace v0(φ+ ξ)by p(φ). Also since p(φ) goes to zero rapidly as φ→∞ we may as well let the integral go to infinity. Hence

vnm ∼ eiξ0(x−α(hξ0)t) · 1√2π

∫ ∞−∞

eiφ(x−γt) · p(φ)dφ = eiξ0(x−α(hξ0)t) · p(x− γt).

1.7. GROUP VELOCITY AND PROPAGATION OF WAVE PACKETS 17

The last step is wrong because γ depends on φ, but it does tell us where we are going. We have

γ ≡ (φ+ ξ0)α(hφ+ hξ0)− ξ0α(hξ0)φ

=(hφ+ ξ0)α(hφ+ hξ0)− hξ0α(hξ0)

hφ

=(θ + θ0)α(θ + θ0)− θ0α(θ0)

θ

=G(θ + θ0)−G(θ0)

θ

= G′(θ0) + θ2G′′(θ∗),

where θ ≡ hφ, θ0 ≡ hξ0 and G(θ) ≡ θα(θ) and θ∗ is between θ0 and θ. The beauty of the above expressionis that G′(θ0) does not depend on φ, but only on hξ0 = θ0.

Let’s go back to

vnm ∼ eiξ0(xm−α(hξ0)tn) · 1√2π

∫ π/h

−π/heiφ(xm−G′(hξ0))tn · e−iφ

hφ2 G′′(θ∗)tp(φ)dφ.

The idea is to replace e−iφhφ2 G′′(θ∗)t by one. By doing so we are making an error bounded by∫ ∞

−∞

∣∣∣e−iφhφ2 G′′(θ∗)t − 1∣∣∣ · |p(φ)|dφ.

We will now show that this error is at most O(h).Let’s first bound |p(φ)|

p(φ) =1√2π

∫ ∞−∞

e−iφxp(x)dx

φ4p(φ) =1√2π

∫ ∞−∞

(−iφ)4e−iφxp(x)dx;

=1√2π

∫ ∞−∞

∂4

∂x4(e−iφx) · p(x)dx

=1√2π

∫ ∞−∞

e−iφx · p′′′′(x)dx

Thus

|p(φ)| ≤ 1√2π·∫∞−∞ |p

′′′′(x)|dx|φ|4

=C

|φ|4.

Also |eiz − 1|2 = 4 sin2 z2 ≤ |z|

2, so |eiz − 1| ≤ |z|. By using this estimate we have∫ ∞−∞

∣∣∣e−iφhφ2 G′′(θ∗)t − 1∣∣∣ · |p(φ)|dφ ≤

∫ ∞−∞

∣∣∣∣hφ2 12G′′(θ∗)t

∣∣∣∣ · |p(φ)|dφ

≤ h · const ·∫ ∞−∞|φ2p(φ)|dφ

≤ h · const ·∫ ∞−∞

∣∣∣∣ 1φ2

∣∣∣∣ dφ≤ h · const.

If we work in L2 we can bound the error by h2 in norm, but not pointwise.Either way we have shown that

vnm = eiξ0(x−α(hξ0)t)p(x−G′(hξ)t) + small terms.


Definition 9. The quantity γ(θ) = G′(θ) = α(θ) + θ · α′(θ) is called group velocity.

We have α(θ) → a as h → 0. So the phase speed is different from the group velocity, but both tend tothe correct speed a as h→ 0. Otherwise the numerical method will not converge.

1.8 Summary of Schemes for the Wave Equation ut + aux = 0

Notation: λ = kh , θ = hξ.

Name Scheme g(θ) Stable dissipative α(θ)/aForward timeforward space

vn+1m −vnm

k + avnm+1−v

nm

h = 0

Forward timebackward space

vn+1m −vnm

k + avnm−v

nm−1h = 0 1− aλ+ aλe−iθ 0 ≤ aλ ≤ 1

order 2, if0 < aλ < 1

Forward-timecentered space

vn+1m −vnm

k + avnm+1−v

nm−1

2h = 0 1− iaλ sin θ No No

Backward-timecentered space

vnm−vn−1mk + a

vnm+1−vnm−1

2h = 0 1− θ26 (1 + 2a2λ2)

Lax–Wendroffvn+1m =vnm − aλ

2 (vnm+1 − vnm−1)

+ a2λ22 (vnm+1 − 2vnm + vnm−1)

1− 2a2λ2 sin2 θ2

−iaλ sin θ|aλ| ≤ 1 order 4 1− 1

6 θ2(1− a2λ2)

Lax–Friedrichs

vn+1m − 1

2 (vnm+1+vnm−1)

k

+amnm+1−v

nm−1

2h = 0cos θ − iaλ sin θ |aλ| ≤ 1 No 1 + (1− a2λ2) θ

23

Leap-Frogvn+1m −vn−1

m2k + a

vnm+1−vnm−1

2h =0−iaλ sin θ

±p

1− (aλ sin θ)2|aλ| < 1 No

atan aλ sin θ√1−(aλ sin θ)2

a

Crank–Nicolson

vn+1m −vnm

k +

avn+1m+1−v

n+1m−1+vnm+1−v

nm−1

4h = 0

1− 12 iaλ sin θ

1+ 12 iaλ sin θ

Always No 1− θ26 (1 + 1

2a2λ2)

Chapter 2

Parabolic equations

2.1 The Heat Equation

ut = buxx

Schemes:

• Lax–Friedrichsvn+1m − 1

2 (vnm+1 + vnm−1)k

= bvnm+1 − 2vnm + vnm−1

h2

• Lax–Wendroff

u(t+ k) = u+ kut +k2

2utt

= u+ kbuxx +k2b2

2uxxxx

vn+1m = vnm + kb

vnm+1 − 2vnm + vnm−1

h2+k2b2

2·vnm+2 − 4vnm+1 + 6vnm − 4vnm−1 + vnm−2

h4

• Forward in time, centered in space

vn+1m − vnm

k= b


h2

• Backward in time, centered in space

vn+1m − vnm

k= b

vn+1m+1 − 2vn+1

m + vn+1m−1

h2

• Leap-Frogvn+1m − vn−1

m

2k= b


h2

• Du Fort–Frankelvn+1m − vn−1

m

2k= b

vnm+1 − (vn+1m + vn−1

m ) + vnm−1

h2

• Crank–Nicolson

vn+1m − vnm

k=b

2

(vn+1m+1 − 2vn+1

m + vn+1m−1

h2+vnm+1 − 2vnm + vnm−1

h2

).

19

20 CHAPTER 2. PARABOLIC EQUATIONS

For parabolic equations it appears natural to have a “new” λ, which we will call µ.

Definition 10 (µ).

µ ≡ k

h2.

Von Neumann analysis works the same way: vnm = gneimhξ.

Example 6. Forward in time, centered in space.

g(hξ) = 1− 4bµ sin2 hξ2 .

Stability requires |g| ≤ 1, i.e.,

0 ≤ 4bµ sin2 hξ2 ≤ 2 for all |hξ| ≤ π, meaning bµ ≤ 1

2 .

The scheme is dissipative of order 2 as long as bµ is strictly less than 12 (check!). For bµ = 1

2 we haveg = 1 − 2 sin2 hξ

2 and the frequency ξ = πh is not damped at all: v0

m = eimhξ = eimπ = (−1)m remainsunchanged by the scheme.

Definition 11. Let

‖u(t, x)‖x ≡(∫ ∞−∞|u(t, x)|2dx

) 12

mean the L2 norm of u(t, x) with respect to x for a fixed t.

Remark 2. Let u be a solution to ut = buxx. Then the overall energy E(t) ≡ ‖u(t, x)‖2x decreases with time

‖u(t, x)‖x ≤ ‖u(s, x)‖x, when t ≥ s

and the solution becomes smoother with time

‖ux(t, x)‖2x ≤1

2bt‖u(0, x)‖2.

The dissipative schemes possess the same qualities

‖vn+1‖ = ‖vn+1‖ = ‖g(hξ)vn‖ ≤ ‖vn‖ = ‖vn‖

and‖δ+vn‖2 ≤

4µCt‖v0‖2,

where δ+vm ≡ vm+1−vmh , and C is a constant.

Proof. We will show that E′(t) ≤ 0 meaning E(t) is decreasing:

E′(t) =∫ ∞−∞

2uutdx =∫ ∞−∞

2buuxxdx = 2buux∣∣∣∞−∞− 2b

∫ ∞−∞

u2xdx = −2b‖ux‖2x ≤ 0,

because u(t, x)→ 0 as x→ ±∞ for ‖u(t, x)‖x to exist.The above implies (after integrating from 0 to t):

E(t)− E(0) = −2b∫ t

0

‖ux(τ, x)‖2xdτ ⇒ E(0) ≥ 2b∫ t

0

‖ux(τ, x)‖2xdτ

The derivative ux = ∂∂xu(t, x) is also a solution to ut = buxx because (ux)t = (ut)x = buxxx = b(ux)xx,

therefore ‖ux(t, x)‖x ≤ ‖ux(s, x)‖x for t ≥ s. Now we get

E(0) ≥ 2b∫ t

0

‖ux(τ, x)‖2xdτ ≥ 2bt‖ux(t, x)‖2x,

2.1. THE HEAT EQUATION 21

meaning

‖ux(t, x)‖2x ≤1

2bt‖u(0, x)‖2x,

i.e., the solution get smoother and smoother as t→∞.Now repeat the same analysis for a difference scheme that is dissipative of order (say) 2:

vn+1(ξ) = g(hξ)vn(ξ), where |g(hξ)| ≤ 1− C sin2 hξ2 ,

which implies

‖vn+1(ξ)‖2 ≤∥∥∥(1− C sin2 hξ

2

)vn(ξ)

∥∥∥2

and (after some major reworking):

‖vn+1‖2 + C∥∥∥sin hξ

2 · vn∥∥∥2

≤ ‖vn‖2

(these are, of course, the discrete L2 norms). Now comes the big moment,

sin(hξ/2) · vn(ξ) =eihξ/2 − e−ihξ/2

2i· vn(ξ) =

e−ihξ/2

2i(eihξ − 1)vn(ξ).

Next, observe that

δ+vnm ≡

1h

(vnm+1 − vnm) =1√2π

∫ π/h

−π/heimhξ

eihξ − 1h

vn(ξ)dξ.

On the other side,

δ+vnm =

1√2π

∫ π/h

−π/heimhξ δ+vn(ξ)dξ,

soeihξ − 1

hvn(ξ) = δ+vn(ξ),

and inserting we get‖ sin(hξ/2) · vn‖ = 1

2‖(eihξ − 1)vn‖ = 1

2 · h · ‖δ+vn‖.

We can therefore simplify our inequality

‖vn+1‖2 + Ch2

4‖δ+vn‖2 ≤ ‖vn‖2.

Parseval says “hats = no hats”, so

‖vn+1‖2 +Ck

4µ‖δ+vn‖2 ≤ ‖vn‖2, (2.1)

where, again, µ = k/h2. In particular,‖vn+1‖ ≤ ‖vn‖.

Next, we prove that ‖δ+vn+1‖ ≤ ‖δ+vn‖. The only property we used was that vnm was solution to

vn+1m − vnm

k= b


h2.

Now tryvn+1m+1 − vnm+1

k= b

vnm+2 − 2vnm+1 + vnmh2

.


Subtract first from second, divide by h and get

vn+1m+1−v

n+1m

h − vnm+1−vnm

h

k= b

vnm+2−vnm+1

h − 2 vnm+1−v

nm

h + vnm−vnm−1h

h2.

Or in a simpler language

(δ+v)n+1m − (δ+vn)m

k= b

(δ+v)nm+1 − 2(δ+v)nm + (δ+v)nm−1

h2,

meaning ‖δ+vn+1‖ ≤ ‖δ+vn‖.The inequality (2.1) works for all time steps

‖vn+1‖2 +Ck

4µ‖δ+vn‖2 ≤ ‖vn‖2

‖vn‖2 +Ck

4µ‖δ+vn−1‖2 ≤ ‖vn−1‖2

...

‖v1‖2 +Ck

4µ‖δ+v0‖2 ≤ ‖v0‖2,

which we sum up and cancel common terms to obtain

‖vn+1‖2 +Ck

4µ

n∑k=0

‖δ+vk‖2 ≤ ‖v0‖2 ⇒

‖vn+1‖2 +Ck(n+ 1)

4µ‖δ+vn+1‖2 ≤ ‖v0‖2 ⇒

‖δ+vn+1‖2 ≤ 4µCt‖v0‖2.

This means that the numerical solution will smooth out—as long as the scheme is dissipative.

2.2. THE DU FORT–FRANKEL SCHEME 23

2.2 The Du Fort–Frankel Scheme

This is an example of an explicit and unconditionally stable scheme for ut = buxx.The problem with schemes like forward time, centered space is that they are stable for bµ = bk/h2 ≤ 1

2 ,which puts a terrible restriction k ≤ h2

2b on the timestep. The Du Fort–Frankel scheme,

vn+1m − vn−1

m = 2bµ(vnm+1 − (vn+1m + vn−1

m ) + vnm−1),

is a slight modification of the unstable Leap–Frog scheme. We rewrite the Du Fort–Frankel scheme as

(1 + 2bµ)vn+1m − (1− 2bµ)vn−1

m = 2bµ(vnm+1 + vnm−1).

To study the stability, we substitute vnm = gneimhξ to get

(1 + 2bµ)g2 − (1− 2bµ) = 2bµ(eihξ + e−ihξ)g,

which implies

g± =2bµ cos(hξ)±

√1− 4b2µ2 sin2(hξ)

1 + 2bµ.

The scheme is not dissipative since g−(π) = −1. To determine stability we consider two cases:

• 1− 4b2µ2 ≥ 0⇒ |g±| ≤ 2bµ| cos(hξ)|+√

11+2bµ ≤ 2bµ+1

1+2bµ = 1.

• 1− 4b2µ2 < 0⇒ |g±|2 = (2bµ cos(hξ))2+4b2µ2 cos2(hξ)−11+2bµ = 4b2µ2−1

(1+2bµ)2 = 2bµ−11+2bµ ≤ 1.

In addition, we do not want double roots on the unit circle. Double root occurs when 1 − 4b2µ2 = 0, butthen |g±| ≤ 2bµ| cos(hξ)|

1+2bµ < 1.So we have stability for any value of µ. But how is that possible? The catch is in the consistency. In

order for the scheme to be consistent we must have k/h→ 0, as we will now demonstrate.Rewrite Du Fort–Frankel as

vn+1m − vn−1

m

2k= b


h2− bv

n+1m + vn−1

m − 2vnmh2

then expand in Taylor to see that it approximates

ut +k2

6uttt = b

(uxx +

h2

12uxxxx

)− b(k2

h2utt +

k4

12h2utttt

).

Now think numerically. For hyperbolic systems we could (at best) hope for kh ≈ 1. However, if we use Du

Fort–Frankel with such a timestep, k = h, the solution will not converge to the solution of ut = buxx, butinstead to the solution of butt+ut = buxx (i.e., the solution to a wave equation). This was not the purpose ofthe exercise. So the scheme will only converge to the solution of ut = buxx if k

h → 0. Even so the truncationerror will be dominated by b k

2

h2utt, which is not small unless kh2 is constant, but then we are back where we

started—with the same restrictions as the ones for forward in time centered in space.We, of course have two explicit schemes—backward in time, centered in space (which is O(k + h2) and

dissipative) and Crank–Nicolson (which is O(k2 + h2) and not dissipative if kh is constant.).


2.3 The Convection-Diffusion Equation: ut + aux = buxx

If a = 0, we have the heat equation, and if b = 0, we have the wave equation. Define a new function w suchthat w(t, x− at) = u(t, x), i.e., w(t, x) = u(t, x+ at). Then

bwxx = buxx = ut + aux = wt − awx + awx = wt.

So u is simply the solution to the heat equation translated with speed a. The problem occurs when theviscosity coefficient b is very small compared to a. Then the obvious numerical methods trick you. We havea choice of

vn+1m − vnm

k+ a

vnm+1 − vnm−1

2h= b


h2, (2.2)

vn+1m − vnm

k+ a

vnm+1 − vnmh

= bvnm+1 − 2vnm + vnm−1

h2,

vn+1m − vnm

k+ a

vnm − vnm−1

h= b


h2. (2.3)

We expand in Taylor series to see that the orders of accuracy are

O(k + h2), O(k + h), and O(k + h),

respectively, which speaks strongly in favor of (2.2).The heat equation has one nice property. The maximum at a later time is less than the maximum at an

earlier time. We will copy that. Rewrite (2.2) as

vn+1m =

(bµ− aλ

2

)vnm+1 + (1− 2bµ)vnm +

(bµ+

aλ

2

)vnm−1

= bµ

(1− aλ

2bµ

)vnm+1 + (1− 2bµ)vnm + bµ

(1 +

aλ

2bµ

)vnm−1.

Let α ≡ aλ2bµ = ah

2b . Now if all the coefficients are positive (i.e., |α| = | aλ2bµ | < 1), then

|vn+1m | ≤ bµ(1− α) max

m|vnm|+ (1− 2bµ) max

m|vnm|+ bµ(1 + α) max

m|vnm|

≤ [bµ(1− α) + (1− 2bµ) + bµ(1 + α)] maxm|vnm|

≤ maxm|vnm|.

The maximum is a decreasing function of time if |α| ≤ 1, i.e., if

|a|b· h

2≤ 1, which is the same as h ≤ 2b

|a|. (2.4)

This will, of course, be satisfied eventually as h→ 0, but who can wait that long? Say a = 10, b = 10−2 ⇒h ≈ 10−3 is needed. And remember, for stability we must have 1/2 ≥ bµ = 10−2 · k/(10−3)2. This impliesk ≈ 10−4/2, which is terribly small.

Now look at (2.3) instead:

vn+1m − vnm + aλ(vnm − vnm−1) = bµ(vnm+1 − 2vnm + vnm−1),

which we rewrite as

vn+1m = (bµ+ aλ)vnm−1 + (1− 2bµ− aλ)vnm + bµvnm+1

= bµ(1 +aλ

bµ)vnm−1 + (1− 2bµ(1 +

aλ

2bµ))vnm + bµvnm+1

= bµ(1 + 2α)vnm−1 + (1− 2bµ(1 + α))vnm + bµvnm+1.

2.4. SUMMARY OF SCHEMES FOR THE HEAT EQUATION 25

Say a > 0, so α > 0. Now the requirement for max-norm stability becomes

2bµ(1 + α) < 1 or 2bµ+ aλ < 1,

which is a lot less restrictive than (2.4). We can pick h to be 10−2 instead of 10−3, i.e., 10 times larger. Wetry k = 10−3

5 , h = 10−2, a = 10, b = 10−2. Then

2bk

h2+ a

k

h= 2 · 10−2 ·

10−3

5

(10−2)2=

250

+15< 1.

So we increased the timestep by a factor of 10. But at what price?We can rewrite (2.3) as

vn+1m − vnm

k+ a

vnm+1 − vnm−1

2h=(b+

ah

2

)vnm+1 − 2vnm + vnm−1

h2.

We introduced an artificial viscosity ah2 = bα. This artificial viscosity comes from our numerical method.

Therefore solving ut + aux = b(1 +α)uxx using (2.2) is equivalent to solving ut + aux = buxx using (2.3).

2.4 Summary of Schemes for the Heat Equation ut = buxx

Notation: µ ≡ kh2 , θ = hξ.

Name Scheme g(hξ) Stable Dissipative

Lax–Friedrichsvn+1m − 1

2 (vnm+1+vnm−1)k

= bvnm+1−2vnm+vnm−1

h2

Lax–Wendroffvn+1m = vnm + kb

vnm+1−2vnm+vnm−1h2

+ k2b22 ·

vnm+2−4vnm+1+6vnm−4vnm−1+vnm−2h4

Forward timecentered space

vn+1m −vnm

k= b

vnm+1−2vnm+vnm−1h2 1 − 4bµ sin2 hξ

2 bµ ≤ 12 2, if bµ < 1

2

Backward timecentered space

vn+1m −vnm

k= b

vn+1m+1−2vn+1

m +vn+1m−1

h2 Always if µ ≥ c, c > 0

Leap-Frogvn+1m −vn−1

m2k = b

vnm+1−2vnm+vnm−1h2 No

Du Fort–Frankelvn+1m −vn−1

m2k = b

vnm+1−(vn+1m +vn−1

m )+vnm−1h2

2bµ cos θ±q

1−4b2µ2 sin2 θ1+2bµ Always No

Crank–Nicolson

vn+1m −vnm

k= b

2

0@ vn+1m+1−2vn+1

m +vn+1m−1

h2

+vnm+1−2vnm+vnm−1

h2

! Always order 2


Chapter 3

Systems of PDEs in HigherDimensions

3.1 The Equation ut + Aux = 0

We consider ut + Aux = 0, where u is a d-vector of functions and A and B are d-by-d matrices. Initialconditions u(0, x) = f(x) (all d-vectors). If A is symmetric A = QΛQT , where QTQ = I, then

ut +QΛQTux = 0⇒ QTut + ΛQTux = 0⇒ wt + Λwx = 0,

where w = QTu. The matrix Λ is diagonal, so the problem falls apart to d independent equations.

(wi)t + λi(wi)x = 0.

The solution is wi(t, x) = gi(x− λit), where g(x) = QT f(x). In general

u(t, x) = Qw(t, x) = Q · (g1(x− λ1t), . . . , gd(x− λdt))T .

How do we generate and analyze a numerical scheme?Take Lax–Wendroff:

u(t+ k) = u+ kut +k2

2utt = u− kAux +

k2

2A2uxx.

Therefore

vn+1m = vnm − kA

vnm+1 − vnm−1

2h+k2

2A2 v

nm+1 − 2vnm + vnm−1

h2

= vnm −λ

2A(vnm+1 − vnm−1) +

λ2

2A2(vnm+1 − 2vnm + vnm−1).

Here vnm is a d-vector. We can use the same Fourier analysis as before. Fourier transform on a vector meansFourier transform in each component, therefore, as before

vn+1(ξ) =[I − iλA sin(hξ)− 2λ2A2 sin2 hξ

2

]vn(ξ) = G(hξ)vn(ξ) = . . . = (G(hξ))nv0(ξ)

We call G the amplification matrix. Stability will mean

‖vn+1‖ ≤ (1 + Ck)‖vn‖, or ‖vn+1‖ ≤ Const · ‖v0‖,

27

28 CHAPTER 3. SYSTEMS OF PDES IN HIGHER DIMENSIONS

when C does not depend on h, k. Let’s analyze (G(hξ))n. Assume A = TΛT−1. Then (after some verysimple math)

Gn(hξ) = T

[I − iλΛ sin(hξ)− 2λ2Λ2 sin2 hξ

2

]nT−1.

Therefore |Gn(hξ)| is bounded if [I−iλΛ sin(hξ)−2λ2Λ2 sin2 hξ2 ]n is. The latter is a diagonal matrix, meaning

each element on the diagonal needs to be bounded. We have reduced the problem to the one-dimensionalcase (which implies λ|λj(A)| < 1) , therefore the stability condition is

λ · max1≤j≤d

|λj(A)| ≤ 1.

3.2 The Equation ut + Aux = Bu

Take Lax–Wendroff for ut +Aux = 0 and just add the lower order term

vn+1m = vnm −

λ

2A(vnm+1 − vnm−1) +

λ2

2A2(vnm+1 − 2vnm + vnm−1) + kBvnm

The amplification factor for the scheme becomes

vn+1(ξ) = (G+ kB)vn(ξ),

so we end up with the question when |(G + kB)n| is bounded by a constant, where |Gn(hξ)| ≤ C. We useStrang’s idea:

(G+ kB)n = Gn + k(Gn−1B +Gn−2BG+ . . . BGn−1

)+ k2

(Gn−2B2 +Gn−3BGB +Bn−4BG2B + . . .+B2Gn−2

)+ . . .

+ kn(G0BG0B . . .G0BG0).

Now estimate upward using |Gn| ≤ C to obtain

|(G+ kB)n| ≤ C +(n

1

)kC2|B|+

(n

2

)k2C3|B|2 + . . .

≤ C

(1 +

(n

1

)kC · |B|+

(n

2

)k2C2 · |B|2 + . . .

)≤ C(1 + k|B|C)n ≤ Cenk|B|C ≤ CeT |B|C

for all nk ≤ T .Therefore the condition for stability is once again |Gn| ≤ C.

3.3 The Equation ut + Aux + Buy = 0

The equation ut + Aux + Buy = 0, where A and B are d-by-d matrices, is a baby problem for an under-lying three dimensional problem, which linearized and reduced leads to the above equation, namely Euler’sequation in fluid mechanics. For a numerical scheme pick

vn+1ml − v

n−1ml

2k+A

vnm+1,l − vnm−1,l

2h+B

vnm,l+1 − vnm,l−1

2h= 0.

Using Fourier transforms in two dimensions we get

vn+1(ξ1, ξ2)− vn(ξ1, ξ2) = 2iλ[A sin(hξ1) +B sin(hξ2)]vn(ξ1, ξ2),

3.3. THE EQUATION UT +AUX +BUY = 0 29

which is the same as[vn+1

vn

]=[

2iλ(A sin θ1 +B sin θ2) II 0

] [vn

vn−1

]= G(θ1, θ2)

[vn

vn−1

].

So we have to study |Gn(θ1, θ2)| and see when it is bounded. Depending on the assumptions one makes thiscan be easy or hard. If we make the simplifying assumptions that A and B are simultaneously diagonalizable,A = PΛ′P−1, B = PΛ′′P−1. Then

A sin(hξ1) +B sin(hξ2) = P diag(λ′i sin(hξ1) + λ′′i sin(hξ2))P−1

and we can write

G =[P

P

] [2iλ(Λ′ sin θ1 + Λ′′ sin θ2) I

I 0

] [P−1

P−1

],

so the whole analysis breaks down to the analysis of the scalar case. Without repeating the previous analysiswe want

|λ(λ′i sin θ1 + λ′′i sin θ2)| < 1− ε

for all θ1 = hξ1, θ2 = hξ2 and i = 1, 2, . . . , d. Our condition for stability is then

λmaxi

(|λ′i|+ |λ′′i |) < 1− ε.

In general if the assumption is that A and B are only symmetric, then the stability criterion is the same,but the proof is a bit more complicated.


3.4 The equation ut = b1uxx + b2uyy

Introduce a notation

A1u = b1∂2

∂x2u, A2u = b2

∂2

∂y2u.

The discrete versions of these operators are

A1hvnml = b1

vnm+1,l − 2vnml + vnm−1,l

h2and A2hv

nml = b2

vnm,l+1 − 2vnml + vnm,l−1

h2,

where u(t, x, y) = u(nk,mh, lh) ∼ vnml (assume ∆x = ∆y = h and ∆t = k). Next we attack ut =b1uxx + b2uyy = A1u+A2u using the Crank–Nicolson idea

vn+1ml − vnml

k= (A1h +A2h)

vn+1ml + vnml

2,

which is equivalent to (I − k

2(A1h +A2h)

)vn+1 =

(I +

k

2(A1h +A2h)

)vn,

and also equivalent to(I − k

2A1h

)(I − k

2A2h

)vn+1 =

(I +

k

2A1h

)(I +

k

2A2h

)vn +A1hA2h

k2

4(vn+1 − vn)︸︷︷︸

(∗)

The expression (*) is O(k3) (since vn+1−vn = O(k)), so dropping it does not affect the order of the accuracyof the scheme.

Our scheme thus becomes(I − k

2A1h

)(I − k

2A2h

)vn+1 =

(I +

k

2A1h

)(I +

k

2A2h

)vn.

3.5. ADI METHODS 31

3.5 ADI methods

Solve the above as (I − k

2A1h

)vn+1/2 =

(I +

k

2A2h

)vn(

I − k

2A2h

)vn+1 =

(I +

k

2A1h

)vn+1/2

This is the Peaceman–Rachford Algorithm, which is an ADI method—alternating direction implicitmethod. Meaning that the two-dimensional problem has been reduced to two one-dimensional implicitproblems by factoring the scheme.

Let’s now perform stability analysis of this scheme.(I − k

2A2h

)vn+1 =

(I +

k

2A1h

)vn+1/2(

1− b2k

2h2(eihξ2 − 2 + e−ihξ2)

)vn+1 =

(1 + b1

k

2h2(eihξ1 − 2 + e−ihξ1)

)vn+1/2

(1 + 2µb2 sin2(hξ2/2)

)vn+1 =

(1− 2µb1 sin2(hξ1/2)

)vn+1/2

vn+1 =1− 2µb1 sin2(hξ1/2)1 + 2µb2 sin2(hξ2/2)

vn+1/2.

Similarly vn+1/2 =1− 2µb2 sin2(hξ2/2)1 + 2µb1 sin2(hξ1/2)

vn

Meaning vn+1 =1− 2µb1 sin2(hξ1/2)1 + 2µb1 sin2(hξ1/2)

· 1− 2µb2 sin2(hξ2/2)1 + 2µb2 sin2(hξ2/2)

· vn.

Since∣∣∣ 1−x1+x

∣∣∣ ≤ 1 for any x ≥ 0, we conclude |vn+1| ≤ |vn|.Now using Parseval ∑

m,l

|vn+1ml |

2 · h2 =∫ π/h

−π/h

∫ π/h

−π/h|vn+1(ξ1, ξ2)|2dξ1dξ2

≤∫ π/h

−π/h

∫ π/h

−π/h|vn(ξ1, ξ2)|2dξ1dξ2

=∑m,l

|vnml|2 · h2.

Therefore we have stability for all values of m and l, and the order of accuracy of the scheme is O(k2+h2).Thus we can take k = h and the scheme is efficient and accurate.

The Douglas–Rachford method starts with the backward-time, central-space scheme for ut = A1u+A2u

(I − kA1h − kA2h)vn+1ml = vnml,

to obtain (after dropping an O(k3) term)

(I − kA1h)(I − kA2h)vn+1ml = (I + k2A1hA2h)vnml.

The method is

(I − kA1h) vn+1/2 = (I + kA2h) vn

(I − kA2h) vn+1 = vn+1/2 − kA2hvn.


The Mitchell–Fairweather method is second-order accurate in time and fourth-order accurate in space.Recall the operator δ2:

δ2f(x) ≡ f(x+ h)− 2f(x) + f(x− h)h2

and

δ2xvnml =

vnm+1,l − 2vnml + vnm−1,l

h2and δ2

yvnml =

vnm,l+1 − 2vnml + vnm,l−1

h2.

Then

δ2f = f ′′ +h2

12f ′′′′ +O(h4) = f ′′ +O(h2),

so f ′′ = δ2f +O(h2). Then

δ2f = f ′′ +h2

12f ′′′′ +O(h4)

= f ′′ +h2

12(f ′′)′′ +O(h4)

= f ′′ +h2

12(δ2f ′′ +O(h2)) +O(h4)

= f ′′ +h2

12δ2f ′′ +O(h4)

= δ2f =(

1 +h2

12δ2

)d2

dx2f +O(h4). (3.1)

Now start with Peaceman–Rachford idea for ut = A1u+A2u:(I − k

2A1

)(I − k

2A2

)un+1 =

(I +

k

2A1

)(I +

k

2A2

)un +O(k3)

Multiply both sides by (1 +

h2

12δ2x

)(1 +

h2

12δ2y

)and replace (

1 +h2

12δ2x

)∂2

∂x2

by δ2x +O(h4) (see (3.1)). Similar changes are made for the derivatives with respect to y. The result is(1 +

h2

12δ2x −

k

2b1δ

2x

)(1 +

h2

12δ2y −

k

2b2δ

2y

)un+1

=(

1 +h2

12δ2x +

k

2b1δ

2x

)(1 +

h2

12δ2y +

k

2b2δ

2y

)un +O(k3) +O(kh4).

We obtain the Mitchell–Fairweather scheme:[1− 1

2

(b1µ1 −

16

)h2

1δ2x

]vn+1/2 =

[1 +

12

(b2µ2 +

16

)h2

2δ2y

]vn[

1− 12

(b2µ2 −

16

)h2

2δ2y

]vn+1 =

[1 +

12

(b1µ1 +

16

)h2

1δ2x

]vn+1/2.

3.6. BOUNDARY CONDITIONS FOR ADI METHODS 33

3.6 Boundary conditions for ADI methods

One can obtain the boundary conditions for the intermediate step vn+1/2 by solving for it using the boundaryconditions at steps n and n+ 1.

For example by subtracting the equations of the Peaceman–Rachford method(I − k

2A1

)un+1/2 =

(I +

k

2A2

)un(

I − k

2A2

)un+1 =

(I +

k

2A1

)un+1/2

(note that we use u’s and not v’s and keep everything is operator form for the moment) we get

un+1/2 =12

(I +

k

2A2

)un +

12

(I − k

2A2

)un+1.

Now if the indices i and j in vnij range from 1 to m, the desired boundary conditions at vn+1/21i and v

n+1/2mi

for i = 2, 3, . . . ,m− 1 are computed as

vn+1/21i =

12

(1 +

k

2A2

)vn1i +

12

(1− k

2A2

)vn+1

1i

=b2µv

n1,i−1 + 2(1− b2µ)vn1i + b2µv

n1,i+1

4+−b2µvn+1

1,i−1 + 2(1 + b2µ)vn+11i − b2µvn+1

1,i+1

4.

And similarly for vn+1/2mi for i = 2, 3, . . . ,m− 1.

The boundary conditionvn+1/2ij = u(tn+1/2, xi, yj)

is only first order accurate and if used with the Peaceman–Rachford method (or other similar second orderaccuracy) will result in the overall accuracy being only first order.

Boundary conditions for the Mitchell–Fairweather scheme are obtained as follows. First we eliminateδ2xvn+1/2 terms by multiplying the first equation by 1+ 1

2 (b1µ1+ 16 )h2

1δ2x and the second by 1− 1

2 (b1µ1− 16 )h2

1δ2x

to obtain:

vn+1/21i =

(b1µ1 + 1

6

) (1 + 1

2 (b2µ2 + 16 )h2

2δ2y

)vn1i +

(b1µ1 − 1

6

) (1− 1

2 (b2µ2 − 16 )h2

2δ2y

)vn+1

1i

2b1µ1

for i = 2, 3, . . . ,m− 1, and similarly for the other boundary.


Chapter 4

Elliptic Equations

4.1 Steady-State Heat Equation

The steady-state heat equation isuxx + uyy = f(x, y). (4.1)

We solve it numerically by introducing a rectangular grid on a finite domain Ω (which need not be rectan-gular).

The numerical scheme is then

vm+1,l − 2vml + vm−1,l

h2+vm,l+1 − 2vml + vm,l−1

h2= fml, (4.2)

with boundary conditions vml specified on the boundary of Ω. We write Ωh for the set of grid points in Ω,∂Ω for the boundary of Ω and ∂Ωh for the set of grid points on the boundary of Ω.

Existence and uniqueness of the solution of (4.2)

Theorem 5. The equation (4.2) has a unique solution.

Proof. Assume there is another solution to (4.2); call it wml, and let eml = vml − wml. Subtract (4.2) for vand w to obtain

em+1,l − 2eml + em−1,l

h2+em,l+1 − 2eml + em,l−1

h2= 0,

with eml = 0 on the boundary. Assume now that eml attains its maximum at an inner point (m, l) on Ω.Then

eml =em+1,l + em−1,l + em,l+1 + em,l−1

4.

But eml ≥ em±1,l±1, so equality is only possible if eml = em±1,l±1. Continuing that way we observe that themaximum must also occur on the boundary, where eml = 0. Thus the maximum of eml is zero and occurs onthe boundary. Similarly for the minimum. Thus eml ≡ 0 for all m, l, which implies vml = wml for all m, l.

The equations (4.2) represent a linear system for vml, which we can write as Av = f . Since Ae = 0 hasthe unique solution e = 0, we conclude that detA 6= 0 and thus Av = f has an unique solution.

Convergence

Next we study convergence, namely does vml → u(xm, yl) as h→ 0? Let

∆hv ≡vm+1,l − 2vml + vm−1,l

h2+vm,l+1 − 2vml + vm,l−1

h2.

35

36 CHAPTER 4. ELLIPTIC EQUATIONS

To prove convergence we need two results. The first is analogous to the argument above:

∆hv ≥ 0⇒ maxΩh

vml = max∂Ωh

vml

and is called The Discrete Maximum Principle. Second, we establish the inequality

‖v‖∞,Ωh ≤18‖∆hv‖∞,Ωh ,

where Ω is the unit square, vml = 0 on ∂Ωh and ‖v‖∞,Ωh ≡ max(m,l)∈Ωh |vml|. The constant 18 is connected

with the shape of the domain. Start with the obvious

−‖f‖∞,Ωh ≤ fml ≤ ‖f‖∞,Ωh . (4.3)

Definewml =

14

[(xm − 1

2

)2 +(yl − 1

2

)2].

Check that (a) ∆hwml = 1, and (b) wml ≤ 18 on ∂Ωh (done in class).

Rewrite (4.3) as−‖f‖∞,Ωh∆hwml ≤ fml ≤ ‖f‖∞,Ωh∆hwml.

Therefore

∆h(v + ‖f‖∞,Ωhw) ≥ 0 (4.4)∆h(‖f‖∞,Ωhw − v) ≥ 0. (4.5)

Now from the discrete maximum principle we have

maxΩh

(vml + ‖f‖∞,Ωhwml) ≤ max∂Ωh

(vml + ‖f‖∞,Ωhwml).

Then

vml ≤ vml + ‖f‖∞,Ωhwml since w ≥ 0≤ max

Ωh(vml + ‖f‖∞,Ωhwml)

≤ max∂Ωh

(vml + ‖f‖∞,Ωhwml)

≤ 18‖f‖∞,Ωh since vml = 0 on ∂Ωh and wml ≤

18

on ∂Ωh.

From (4.5) (in class) we get − 18‖f‖∞,Ωh ≤ vml, therefore |vml| ≤ 1

8‖f‖∞,Ωh and we have

maxΩh|vml| ≤

18‖f‖∞,Ωh =

18‖∆hv‖∞,Ωh .

Talyor series implies

∆huml = (∆u)xm,yl +h2

12(∂4xu(xm + θmlh, yl) + ∂4

yu(xm, yl + θ′mlh) = fml +h2

12(∂4xu+ ∂4

yu)︸︷︷︸truncation error

.

As usual define eml =computed − exact = vml − uml, then

∆heml = ∆hvml −∆huml

= fml −[fml +

h2

12(∂4xu+ ∂4

yu)]

= −h2

12(∂4xu+ ∂4

yu).

4.1. STEADY-STATE HEAT EQUATION 37

Note that eml = 0 on ∂Ωh, thus

maxΩh|eml| ≤

18

maxΩh|∆heml|

≤ h2

8 · 12·max

Ωh|∂4xu+ ∂4

yu|

≤ h2

48·max

Ω(|∂4

xu(x, y)|, |∂4yu(x, y)|).

So the error goes to 0 as h→ 0. Remember that the constant 18 was married to the unit square.

The continuous case

We will now prove that the equation uxx + uyy = 0 with u(x, y) = 0 on ∂Ω has a unique solution u ≡ 0. Westart with the theorem of Gauss ∫

Ω

uxidx =∫∂Ω

uνids,

where νi is the normal in the ith direction. Since uuxx + uuyy = 0 we have (uux)x + (uuy)y − u2x − u2

y = 0,which we integrate over Ω using Gauss’ theorem to obtain∫

∂Ω

(uuxν1 + uuyν2)−∫

Ω

(u2x + u2

y) = 0.

The first integral is 0, since u = 0 on ∂Ω, therefore ux = uy = 0 on Ω, which along with u = 0 on ∂Ω impliesu ≡ 0 on Ω.


4.2 Numerical methods for uxx + uyy = f

We have

vml −14

(vm−1,l + vm+1,l + vm,l−1 + vm,l+1) = −h2

4fml.

Where 0 ≤ m, l ≤ N . The above represents a linear system with (N − 1)2 unknowns vml, 1 ≤ m, l ≤ N − 1.Write it as Av = b. Solving Av = b directly using Gaussian elimination would cost O(((N −1)2)3) = O(N6),which is prohibitive. We’d better use the banded structure of the system. For example when N = 4 withzero boundary conditions we have

1 −1/4 −1/4−1/4 1 −1/4 −1/4

−1/4 1 −1/4−1/4 1 −1/4 −1/4

−1/4 −1/4 1 −1/4 −1/4−1/4 −1/4 1 −1/4

−1/4 1 −1/4−1/4 −1/4 1 −1/4

−1/4 −1/4 1

·

v11

v12

v13

v21

v22

v23

v31

v32

v33

= −h

2

4·

f11

f12

f13

f21

f22

f23

f31

f32

f33

(4.6)

or

Av = b.

The idea in Jacobi, Gauss–Seidel, and SOR is to split A = B + C and solve (B + C)v = b iteratively asvn+1 = B−1(b− Cvn), where n is the iteration, not the time. The error e = vn − v (= computed − exact)satisfies

en+1 = B−1Cen

or en+1 = Fen for short.

Lemma 1. Say en+1 = Fen for n = 1, 2 . . .. Then en → 0 if and only if ρ(F ) < 1, where

ρ(F ) = maxj|λj(F )|

is the spectral radius of F .

Proof. If ρ(F ) ≥ 1 and (say) |λ1| = ρ(F ), then by picking e1 as the eigenvector of F corresponding to λ1 wehave en+1 = λn1 e

1 which will never converge to 0.Now assume ρ(F ) < 1 and let F = T−1JT be the eigenvalue decomposition of F , where

J =

λ1 c1

λ2 c2. . . . . .

λk

and ci = 0 or 1 depending on whether there is a Jordan block. The choice of ones as superdiagonal elements ofJordan blocks is is a matter of convention, we can put any positive number there, since if S = diag(ε, . . . , εk),

4.3. JACOBI, GAUSS–SEIDEL, AND SOR(ω) 39

then

J = S−1JS

=

ε−1

ε−2

. . .ε−k

λ1 c1

λ2 c2. . . . . .

λk

ε1

ε2

. . .εk

=

λ1 εc1

λ2 εc2. . . . . .

λk

.Selecting ε = |1− ρ(F )|/2 guarantees that

maxj

(|λj |+ ε|cj |) < 1− ε < 1,

i.e., ‖J‖∞ < 1− ε. Now F = TSJ(TS)−1 and Fn = TSJn(TS)−1, which implies

‖en+1‖∞ = ‖TSJn(TS)−1e1‖∞≤ ‖TS‖∞ · ‖J‖n∞ · ‖(TS)−1‖∞ · ‖e1‖∞≤ ‖TS‖∞ · (1− ε)n · ‖(TS)−1‖∞ · ‖e1‖∞ → 0 as n→∞.

4.3 Jacobi, Gauss–Seidel, and SOR(ω)

All start with an initial guess and iterate

Jacobi vn+1ml =

14

(vnm+1,l + vnm−1,l + vnm,l+1 + vnm,l−1)− h2

4fml;

Gauss–Seidel vn+1ml =

14

(vnm+1,l + vn+1m−1,l + vnm,l+1 + vn+1

m,l−1)− h2

4fml;

SOR(ω) vn+1ml = (1− ω)vnml +

ω

4(vnm+1,l + vn+1

m−1,l + vnm,l+1 + vn+1m,l−1)− ωh2

4fml.

In matrix form we write the system (4.6) Av = b as

(I − L− U)v = b,

where L (U) is strictly lower (upper) triangular. Then

Jacobi vn+1 = (L+ U)vn + b

Gauss–Seidel vn+1 = (I − L)−1(Uvn + b)

SOR(ω) vn+1 =(

1ω I − L

)−1 (( 1ω I − I + U)vn + b

).

The convergence of each method will be determined by looking at

maxj|λj(L+ U)| < 1

maxj|λj((I − L)−1U)| < 1

maxj

∣∣λj (( 1ω I − L)−1( 1

ω − I + U))∣∣ < 1


Convergence of Jacobi’s method

We must find the eigenvalues of L+ U , i.e., solve

vm+1,l + vm−1,l + vm,l+1 + vm,l−1 = 4λvml.

A direct verification (exercise in manipulating trigonometric functions) confirms that

vml = sinamπ

N· sin blπ

N

satisfy the above equation for any a, b = 1, 2, . . . , N − 1. Thus we have found the (N − 1)2 orthogonaleigenvectors. The eigenvalues are

λab =12

(cos

aπ

N+ cos

bπ

N

).

The spectral radius is now

ρ(L+ U) = cosπ

N≈ 1− 1

2

( πN

)2

.

Convergence of Gauss–Seidel method

We must find the eigenvalues of (I − L)−1U , i.e., solve

vm+1,l + λvm−1,l + vm,l+1 + λvm,l−1 = 4λvml.

The trick is to now set vml = λ(m+l)/2wml (we may lose a zero eigenvalue this way, but a zero eigenvaluewill not hinder convergence; it will turn out later there were no zero eigenvalues). Then

wm+1,l + wm−1,l + wm,l+1 + wm,l−1 = 4√λvml.

We already solved this problem for Jacobi. Therefore

ρ((I − L)−1U) = cos2 π

N≈ 1− π2

N2.

Gauss–Seidel converges about twice as fast as Jacobi. To compare convergence look at the number ofsteps it will take to decrease the error by a factor of e (think, e.g., e = 102 or e = 104, etc.). For Jacobi,(1− π2

2N2 )j = e−1 implies j log(1− π2

2N2 ) = log e−1. Since log(1−ρ) ≈ −ρ for small ρ, we have j ≈ 2N2

π2 · log e,whereas for Gauss–Seidel (1 − π2

N2 )g = e−1 implies g ≈ N2

π2 · log e, i.e., Gauss–Seidel converges twice as fastas Jacobi.

Convergence of SOR(ω)

Using the same substitution as in Gauss–Seidel we obtain

wm+1,l + wm−1,l + wm,l+1 + wm,l−1 = 4λ+ ω − 1ω√λ

wml,

i.e., µ = λ+ω−1√λω

is an eigenvalue of L + U . (Do not confuse ω with wml). Selecting ω to minimizeρ(( 1ω I − L)−1( 1

ω I − I + U))

we obtain

ωopt =2

1 +√

1− µ2=

21 + sin π

N

ρ(

( 1ωopt

I − L)−1( 1ωopt

I − I + U))

=cos2 π

N

(1 + sin πN )2 ≈ 1− 2π

N,

i.e., SOR(ω) is N times faster than Gauss–Seidel.

Index

L2 norm, 6

ADI methods, 31boundary conditions, 33

amplification factor, 7

consistency, 5convergence, 6

dispersion, 13dissipation, 11Douglas–Rachford, 31

Fourier Analysis, 6

Gauss–Seidel method, 39convergence, 40

group velocity, 15

Jacobi’s method, 39convergence, 40

Mitchell–Fairweather, 32

Parseval identity, 6phase speed, 14

schemeheat equation

Crank–Nicolson, 25Du Fort–Frankel, 23, 25Lax–Friedrichs, 25Lax–Wendroff, 25Leap-frog, 25

wave equationforward time centered space, 8Lax-Wendroff, 8Leap-frog, 9

SOR(ω), 39convergence, 40

stability, 6

Von Neumann analysis, 7

wave packet, 15well-posedness, 6

41

Date post:	17-Mar-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	1 times

18.336|Numerical Methods for Partial Di erential Equations ...plamen/18.336/math336.pdf1.3. VON...

Documents