236861 Numerical Geometry of Images
Tutorial 2
Introduction to numerical schemes
Guy Rosman
c©2012
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Classifying PDEs
Looking at the PDEAuxx + 2Buxy + Cuyy + Dux + Euy + Fu + .. = 0,and its discriminant, B2 − AC , the equation is said to be:
I Parabolic if B2 − AC = 0 - for example, the heat equation,ut = uxx
I Hyperbolic if B2 − AC > 0 - for example, the second orderwave equation, utt = uxx
I Elliptic if B2 − AC < 0 - for example, Laplace equation,uxx + uyy = 0
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Classifying PDEs (Cont.)
This can be generalized to second order equations in N dimensionsvia the positive definiteness of the second order operator involved:
(∇)T(
A BB C
)(∇) u + {lower order terms} = 0
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Heat equation
The simple parabolic PDE
∂u
∂t= K
∂2u
∂2x
with the initial values
u(0, x) = u0(x)
and some boundary conditions is called the (one-dimensional) heatequation.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
I This equation describes the thermal energy transport in a 1Drod, where u(t, x) describes the temperature at a point x attime t and K denotes the thermal conductivity constant.
I In 2D it describes the effect of defocusing on an image inoptics
I The solution is given by convolution of the initial data with aGaussian kernel:
u(x , t) = G (x , t) ∗ u0(x) =∫RG (x̄ , t)u0(x − x̄)dx̄
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Boundary conditions
Dirichlet:
u(t, 0) = a u(t, 1) = b
Neumann:
ux(t, 0) = a ux(t, 1) = b
Reflective - Neumann with a = b = 0.Mixed boundary conditions:
u(t, 0) = a, ux(t, 1) = b.
Periodic:
u(t, 0−) = u(t, 1+)
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Discretization of the heat equation
Replace the continuous system of coordinates (t, x) by a discretegrid (n,m) = (n∆t,m∆x), and the continuous function u(t, x) bya discrete version unm = u(n∆t,m∆x).
Figure: The grid
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Finite differences
1. Replace the first-order time derivative ut(t, x) by a forwardfinite difference in time
D+t unm =
un+1m − unm
∆t≈ ut(t).
2. Replace the second-order space derivative uxx(t, x) by acentral difference in space
D0xxu
nm =
unm+1 − 2unm + unm−1
(∆x)2≈ uxx(x).
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Finite differences’ motivation
From the Taylor expansion,
u(t +∆t) = u(t) + ut(t)∆t +1
2utt(t)(∆t)2 + ...
we obtain
ut(t) =u(t +∆t)− u(t)
∆t− 1
2utt(t)∆t + ...
=u(t +∆t)− u(t)
∆t+ O(∆t).
This first-order approximation of the first derivative is called theforward finite-difference approximation and is denoted by D+
t unm.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Finite differencesForward difference
D+t unm =
un+1m − unm
∆t≈ ut(t) + O(∆t).
In the same manner, backward difference can be defined
D−t unm =
unm − un−1m
∆t≈ ut(t) + O(∆t).
To approximate a second-order derivative, use the centraldifference approximation
D0xxu
nm =
unm+1 − 2unm + unm−1
(∆x)2≈ uxx(x) + O((∆x)2).
Note: We can also write
D0xxu
nm = D−
x D+x unm.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Finite differences
Proving the order of convergence – by writing a fourth order Taylorseries and canceling out terms,
unm+1 − 2unm + unm−1
(∆x)2=
unm + (ux)nm ∆x + 1
2 (uxx)nm (∆x)2
+ 13! (uxxx)
nm (∆x)3 + 1
4! (uxxxx)nm (∆x)4
−2unm+unm − (ux)
nm ∆x + 1
2 (uxx)nm (∆x)2
− 13! (uxxx)
nm (∆x)3 + 1
4! (uxxxx)nm (∆x)4
(∆x)2=
12 (uxx)
nm (∆x)2 + 1
4! (uxxxx)nm (∆x)4+
12 (uxx)
nm (∆x)2 + 1
4! (uxxxx)nm (∆x)4
(∆x)2
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Finite differences
= uxx +1
12(uxxxx)
nm (∆x)2
= uxx +O((∆x)2
)
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
The discrete heat equation
The continuous equation ut(t, x) = Kuxx(t, x) is replaced by
un+1m − unm
∆t= K
unm+1 − 2unm + unm−1
(∆x)2
or
un+1m = unm +
∆t
(∆x)2K(unm+1 − 2unm + unm−1
),
This scheme is explicit (called also Euler scheme) and very simpleto implement.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
I We denote the continuous solution by U = u(t, x) and thediscrete solution by u = unm.
I We denote the continuous PDE operator L = ∂t − K∂xx .
I We denote the discrete operator L∆t,∆x = D+t − KD0
xx .
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Stability and Convergence
1. Convergence: the discrete solution converges to thecontinuous solution
lim∆t,∆x→0
‖u − U‖ .
in some norm.
2. Consistency: the discrete difference operator converges to thecontinuous
lim∆t,∆x→0
‖L(u)− L∆t,∆x(u) ‖ = 0
for every bounded u.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Stability
3. Numerical stability: noise from initial conditions, numericalerrors, etc. is not amplified.The numerical scheme is un+1 = Nun
Formally, stability means
∀N > 0 ∃C (N) > 0 s.t. ∀n ≤ N ‖N n‖ ≤ C (N).
If C is a constant for all N, this is known as unconditionalstability.
Lax’s equivalence theorem: Given a well-posed initial valueproblem and a finite-difference approximation to it that satisfiesthe consistency condition, stability is the necessary and sufficientcondition for convergence.
consistency: stability ⇔ convergence
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
The CFL condition for the wave equation
I The (Courant-Friedrichs-Lewy (CFL) condition) is a necessarycondition for advection problems limiting the maximum timestep allowed in the difference equations.
I Specifically, this condition states that the domain ofdependence of the differential equation must be included inthe domain of dependence of the difference equation.
I Note: This does not apply to equations where informationflows “instantly” such as the heat equation – but the termCFL condition is often still used to denote stability limitationson step size in these cases.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
A condition for stability of the discrete heat equation
Denote r = K ∆t(∆x)2
. The finite difference scheme is
un+1m = unm + r
(unm+1 − 2unm + unm−1
)= (1− 2r)unm + runm+1 + runm−1
If 1− 2r > 0, we can write:
un+1m ≤ (1− 2r)max
munm + r max
munm + r max
munm
Then un+1m ≤ max
munm(1− 2r + 2r) = max
munm.
(maximum principle property)
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Observation: Maximum principle implies stability of the scheme.
Lemma(Maximum principle)If K ∆t
(∆x)2≤ 1/2, then
minm
u0m ≤ unm ≤ maxm
u0m.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
0 10 20 30 40 50 60 70−8
−6
−4
−2
0
2
4
6
8Initial data
0 10 20 30 40 50 60 70−5
−4
−3
−2
−1
0
1
2
3
4
5 Solution to 1−D heat equation
dt=0.0008
0 10 20 30 40 50 60 70−5
−4
−3
−2
−1
0
1
2
3
4
5Solurion to 1−D heat eq
dt=0.004
0 10 20 30 40 50 60 70−4
−3
−2
−1
0
1
2
3
4x 10
70
dt=0.006
Figure: Computation of the discrete heat equation with Neumannboundary conditions. (h = 0.1) Top left: Initial data. Top right: stablesolution of the heat equation conditions ∆t = 0.0008. Bottom left:stable solution of the heat equation for ∆t = 0.004. Bottom right:unstable solution (∆t = 0.006).
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
The 2D heat equation. Numerical scheme.
ut = K (uxx + uyy )
un+1m,p = unm,p + K
( ∆t
(∆x)2(unm+1,p − 2unm,p + unm−1,p)+
+∆t
(∆y)2(unm,p+1 − 2unm,p + unm,p−1)
).
A stabilty condition suggested by Courant, Friedrichs and Lewy:
K( ∆t
(∆x)2+
∆t
(∆y)2
)≤ 1
2.
If ∆x = ∆y , then the condition is:
K∆t
(∆x)2≤ 1
4.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Stability condition by Fourier analysis.
Another way to check the stability of a numerical scheme is byFourier analysis. Looking again at the heat equation:
ut = uxx
using the linearity of the equation we assume a solution of theform unm = λne iξm∆x , where for stability we would like ‖λ‖ < 1.For the usual explicit scheme, we obtain:
un+1m − unm
∆t=
unm+1 − 2unm + unm−1
(∆x)2−→
λn(λ− 1)e iξm∆x =∆t
(∆x)2λn(e iξ∆x + e−iξ∆x − 2)e iξm∆x −→
λ = 1 +∆t
(∆x)2(e iξ∆x + e−iξ∆x − 2)
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Stability condition by Fourier analysis.
finally arriving at
‖λ‖ = ‖1 + ∆t
(∆x)2(2cos(ξ∆x)− 2)‖ ≤? 1.
Again we see that the condition ∆t(∆x)2
< 12 guarantees stability.
The same method extends to higher dimensions.In the case of a finite time interval T , we can settle for a weakerform of stability, by demanding merely
‖λ‖ < 1 + K̃T
.This condition is known as the von-Neumann condition. Theanalysis is often referred to as von-Neumann analysis.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
The 2D heat equation. Numerical example
Figure: Application of the heat eq. on a gray-scale image.
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes
Implicit finite difference schemes for the heat equation
un+1m − unm
∆t= K
un+1m+1 − 2un+1
m + un+1m−1
(∆x)2
I Main advantage of implicit schemes: are unconditionallystable (i.e. stable for all time-steps and thus we can take largestep times).(I −∆tA)Un+1 = Un.
I Requires solving a linear system of equations. The matrix(I −∆tA) is tridiagonal and can to be inverted using theThomas algorithm (we will describe it in detail later on).
I Note: Assuring stability does not prevent the accuracy fromdeteriorating..
Guy Rosman CS 236861 - Tutorial 3 - Introduction to numerical schemes