1

Numerical Methods for Partial Differential Equations

1. Introduction

2. Finite difference method for first order hyperbolic PDEs

3. Method of characteristics for first order hyperbolic PDEs

4. Method of lines approach for first order hyperbolic PDEs

5. Finite difference method for second order elliptic PDEs

6. Finite element method for second order elliptic PDEs

7. Weighted residuals method for second order elliptic PDEs

8. Finite difference method for second order parabolic PDEs

Slides adapted from Prof. Shang-Xu. Hu of ZJU, “Applied Numerical Computation Methods”. 2000.5

2

1. Introduction to PDEs

• number of variables: at least 2

,,,,

0,,,,,,,,

,,

2

2

2

yx

uu

x

uu

y

uu

x

uu

uuuuuuyxF

yxuu

xyxxyx

yyxyxxyx

• order : the highest order of derivative

0

0

03

3

yyyx

yxx

yx

uu

uu

ubu

order third

order second

orderfirst

• characteristic

using the first order PDE as an example

0 cubua yx

3

yyxyxxyx

yx

yxyyxyxx

yx

yx

yx

yx

uuuuuuyx

uuuyx

yx

CBA

FuEuDuCuBuA

uu

xuuu

buu

uuuyx

uyx

yx

yxccyxbbyxaa

cba

,,,,,,,:

,,,,:

,:

：:,:,:

0:::

PDEsorder secondFor

0

0

0

:examplesfor

,,,,

,,

,

,,,,,

constants are,,

2

2

Nonlinear

rQuasilinea

Linear

...

...

...

Nonlinear

rQuasilinea

Linear

...

...

...

Nonlinear

rQuasilinea

Linear

...

...

...

4

• types

first order linear PDE

xt

yx

uvu

cubua

0

Advection Equation(AE)

second order linear PDE

0 GCuBuAu yyxyxx

ellipticACB 042

eq. conductionheat eq., 0

,

Laplaceuu

equationPoissonyxfuu

yyxx

yyxx

parabolicACB 042

equation

equation diffusion

BurgeruKuuu

uu

yyyx

yyx

hyperbolicACB 042

equation waveyyxx uu

5

• solution methods:

Method of Finite Differences (MFD)

Method of Characteristics (MOC)

Method of Lines (MOL)

Method of Finite Elements (MFE)

Method of Weighled Residuals (MWR)

• Numerical questions:

Convergence

When the steps approach to infinitely small, will the numerical results coincide the theoretical results?

Stability

When the error is introduced at a certain step, will this error be amplified or attenuated after several steps of numerical computation?

dx

dy

x

y

k

k

ey

y

ey

y

1

1

6

2. Finite difference method for first order hyperbolic PDEs

0

x

uv

t

u

known as Advection Equation (AE)

v is the flow speed

The analytical solution is:

vtxfu

dw

du

x

w

dw

du

x

udw

duv

t

w

dw

du

t

u

vtxwwfu

, inces

To find a specific solution, we need two auxiliary conditions:

vtxftxu

vtxftxu

00

00

,

,

7

assuming the forcing function is a Rump

The solution of is shown below.

sW

Wss

W

W

Wf

,0.0

0,1

0,0.1

0 xt vuu

wf

W0s

0.1

0.0

u

t

x

0tt 0xx

jx

nt

ntt

jxx tvx

txu j ,

txu ,0

0, txu

ntxu ,

Fig. 1. Propagation of the Wave Front

8

2.1 The simplest finite difference format:

t

x0x 1jx jx 1jxx

1nt

nt

0t

t

tbxtu

xaxtux

uv

t

u

0

0

,

,xjxx j 0

tnttn 0

211

,

1

,

2xO

x

uu

x

u

tOt

uu

t

u

nj

nj

nj

nj

nj

nj

x

uuv

t

uu nj

nj

nj

nj

211

1

nj

nj

nj

nj uu

x

tvuu 11

1

2

1 nn

Fig. 2

9

The above method is called Forward Time Centered Space

FTCS representation

In fact, this method is not practical since it is an unstable method

Consider the numerical error r

Because the original PDE is linear, the error propagation is by

which is identical to the original equation

Independent Solutions of Difference Equations

nj

nj

nj

nj

nj

nj

nj

nj ruru

x

tvruru 1111

11

2

nj

nj

nj

nj rr

x

tvrr 11

1

2

xjkir

xjkirnn

j

nnj

exp

exp11

FactorionAmplificat called is

notor sequence increasing an is,,,,,,

if determine toneed We1110 n

jnj

njjj rrrrr

10

ODElinear heConsider t

xFQydx

dyP

dx

yd

2

2

xFQyDyPyD 2

solution=auxiliary solution and specific solution

Auxiliary solution:

0

0

0

21

2

2

yPDPD

yQDPD

QyDyPyD

equationauxiliary

QPxxPP 0 of roots twoare , 221

00 21 yPDyPD

Auxiliary solution composes of two independent solutions:

xPAyxPAy 222111 expexp

number complex a is4 when

exp2

111111

iPQP

yPxPPADy

pyyyy 21

.by determined is

solution specific theand solutionst independen

two withsolutionauxiliary theis 21

xFy

yy

p

11

Finite difference solution

Let us use Operator Calculus to derive with the Difference Operator

jjjjj

jjj

xfxfyyy

xxx

11

1

jjjjj Eyyyyy 11

,1 E

22

21

1

jj

jj

jj

yyE

yEy

yEy

It is the same as the differential operator.It is a linear operator. When applied to the linear second order difference equation

xFQyPyy nnn 12

Similar to the differential equation, its auxiliary solution can be obtained as follows:

0

0

0

21

2

12

n

n

nnn

yPEPE

yQPEE

QyPyy

12

0

0

2

1

n

n

yPE

yPE

Two independent solutions aren

nn

n PAyPAy 2211 21

nnnn

n yPEyyPPAy , : Since 1

1

nnn PAPAy 2211

ixiPQP exp，4 When 2 xniAy n

n exp （ Eigenmode ）

nn

nn

nnn

n

PyEy

inxAixpy

xniAyEy

aaai

aa

aiaaiaia

n

aaaa

expexp

1exp

!5!3!4!21

!4!3!21exp

!!21exp

11

5342

432

2

So, from

13

The general form of the independent solution of difference equation is given by:

inxAy nn exp

In our numerical error analysis problem of PDE solution, clearly,

xikjr

xikjrnn

j

nnj

exp

exp11

FactorionAmplificat called is Therefore,

case. unstable theisit ，1 when

Substitute the independent solution into the difference expression, we have,

nj

nj

nj

nj rr

x

tvrr 11

1

2

So, we can get

xkix

tv

xikxikx

tv

xikj

xjikxjik

x

tv

xjikxjikx

tv

xikj

nn

n

sin22

expexp2

exp

1exp1exp

21

1exp1exp2

exp1

14

2.2 Improved finite difference format

that is

1 ensure tocondition The

sincos

:isfactor ionamplificat

ingcorrespond thecase, thisIn22

1

:becomesformat difference

finitenew our Then,2

1

use uslet

using of instead

2

11111

11

i

Xkx

tviXk

uux

tvuuu

uu

u

MethodLax

nj

nj

nj

nj

nj

nj

nj

nj

1x

tv

Courant Condition

t

x0x 1jx 1jxjx

nt

0t

1nt

I

Rtk

1x

tv

Fig. 3

Fig. 4

15

The physical meaning

Of Courant condition

Waveform travels along

The line x=vt

t knot selection

• when knot is on the line:

• when knot is outside the line:

• when knot is inside the line:

The Lax finite difference format can also be written as

This can be regarded as the FTCS difference format for the following PDE:

1,2

x

tv

v

xt

unstable1,3 x

tv

v

xt

stable1,1 x

tv

v

xt

t

uuu

x

uuv

t

uu nj

nj

nj

nj

nj

nj

nj 1111

1 2

2

1

2

2

22

x

u

t

x

x

uv

t

u

dissipative term

Numerical Viscosity

t

x1jx jx 1jx

x

nt1t

3t2t

Fig. 5

16

3. Method of characteristics (MOC) for 1st order hyperbolic PDEs

dyy

udx

x

udu

yxu

Ayu

BCx

u

yxCBA

, of aldifferenti wholeThe

, of functions be can ,, where

dyy

udx

Ayu

BCdu

Cy

uB

x

uA

0

AduCdxy

uBdxAdy

0 BdxAdy

yxFA

B

dx

dy,

So, this has the same solution as the original PDE.

is called the MOC equation.

On the characteristic curve, those satisfying

Are the solution of the original PDE.

0 AduCdx

17

3.1 Method of Characteristics (MOC)

yxFdx

dy,

yxGdx

du,

,,,,,,,

,,,,,,,

,,,,,,,

get can we,, ODE theintegrate

,,,,,,,

conditions initial given theon Based

jectoriescurves/tra sticcharacteri offamily

aget can we,ODE the

integrate ,conditions initial theas

,,,,ly respective

using，from starting

lanep thenO

10

11110

00100

01000

10

0

njnn

j

j

n

n

yxuyxuyxu

yxuyxuyxu

yxuyxuyxu

yxGdx

du

yxuyxuyxu

yxFdx

dy

yyyy

xx

yx

y

x

ny

1y

0yy

x0,0u

yxu ,

0x jx1x

Fig. 6

18

3.1 Method of Characteristics (MOC)

yxFdx

dy,

yxGdx

du,

y

x

ky0

01y

00yy

x0,0u yxu ,

0x jx1x

ky1

11y

10y

jky

0jy

1jy

kL

0L

1L

,2,1,0, ies trajectoroffamily aget can we

, ODE integrate ,,,,, conditions

initial ly therespective use , from starting plane, nO

00100

0

kL

yxFdx

dyyyy

xxyx

k

k

,,,,,,,,,:

,,,,,,,,,:

,,,,,,,,,:

221100

12121110101

02021010000

jkjkkkk

jj

jj

yxyxyxyxL

yxyxyxyxL

yxyxyxyxL

Fig. 7

19

get can weODE, above theintegrate So,

,

also satisfiesit curve, sticcharacteri each on Since

,,,,,,,

compute can we,, condition, initial Given

00010000

0

yxGdx

du

yxuyxuyxu

yxu

k

,,,,,,,

,,,,,,,

,,,,,,,

1100

1111010

0101000

jkjkk

jj

jj

yxuyxuyxu

yxuyxuyxu

yxuyxuyxu

required. are

points discreateregular if method ioninterpolat

use torequired isit So, direction. y alone defined ,

points discreteirregular is , method based MOC

by solution numerical theknow that weTherefore,

jkj

jkj

yx

yxu

20

4 Method of lines (MOL) approach for first order hyperbolic PDEsFinite Difference: PDE is completely discretized as a set of difference equations. Using linear algebra to solve.

Method of lines: PDE is partly discretized as a set of ODEs and using ODE numerical solution method to solve.

Cx

uB

t

uA

nix

u

A

B

A

C

dt

du

nixx

tgtxu

xftxux

u

A

B

A

C

t

u

tx

i

i

i

,,1,0

:ODEs ofset aget can we

,,1,0, as axis ngDiscretizi

,

,

,

0

0

21

Method of Lines (MOL)

nitutxu i ,,1,0, as , ngRepresenti

method tiondiscretiza partial a

t

x0x ix nxx

1t

0tt

Line space

Integration step

ODEs. theintegratey numericall

toapplied be can method numericalany Then,

... splines ,difference finite e.g., eapproximat to

method ationdifferenti numericalany use can We

known. areor, condition Initial 00

x

u

tutxu

i

ii

nix

u

A

B

A

C

t

u ii ,,1,0

Fig. 8

22

5. Finite difference method for second order elliptic PDEs

02

2

2

2

y

u

x

u

known as the steady-state heat conduction equation and general form is

• Dirichlet problem

• Neumann problem

operator

equation 0

2

22

2

Laplacex

Laplaceu

k k

02

2

2

2

y

u

x

u

02

2

2

2

y

u

x

u

yfyxu

yfyxu

yfyxu

yfyxu

n

m

41

301

2

10

,

,

,

,

ygxu

ygxu

ygxu

ygxu

n

m

y

y

x

x

4

3

2

1

0

0

23

y

x

ny

0y

jy

0x ix mx

xfyxu n 4,

xfyxu 30,

02

2

2

2

y

u

x

u

yxQ ,

u(x m

,y)=

f 2(y)

u(x 0,y

)=f 1(

y)

xgy

u

y

3

0

xgy

u

ny

4

xgy

u

mx

2

xgy

u

x

1

0

Laplace equation: Dirichlet boundary condition and Neumann boundary condition.

Poisson equation

yxy

u

x

u,

2

2

2

2

Four (4) boundary conditions required. There are 3 types of boundary conditions:

• Dirichlet boundary condition

• Neumann boundary condition

•Mixed or hybrid boundary condition.

Fig. 9

24

5.1 Finite difference of Laplace operator

02

2

2

22

y

u

x

uu

:form concise a in expressed be can This12

2

2462

2462

expansion seriesTaylor thefrom form difference finite in

expressed be can derivativeorder second The

24

''2

44

3'''

2''

'

44

3'''

2''

'

hf

xfh

hxfxfhxf

hf

hxf

hxf

hxfxfhxf

hf

hxf

hxf

hxfxfhxf

iiii

iiiiii

iiiiii

22

11'' 2hO

h

ffff iii

i

Apply the above for Laplace operator

2

11

2

112

,,2,

,,2,,

y

yxuyxuyxu

x

yxuyxuyxuyxu

jijiji

jijijiii

following theusesimply can we, When hyx

jijijijijiij uuuuuh

u ,1,1,,1,122 4

1

25

1ix ix 1ix

1jy

jy

1jy

1, jiu

jiu , jiu ,1jiu ,1

1, jiu

0

11

161

111

space 3D in If

0

1

141

11

,

format difference finite equation

2

2

2

2

2

2

22

22

ijk

ij

uh

z

u

y

u

x

uu

uh

yxu

Laplace

Fig. 10

26

Example: Laplace equation with Dirichlet boundary

co0

co0co0co0

co0 co0 co0

co100

y

x

cm

10

cm20

5yh

5xh

is solution the

100

0

0

410

141

014

or,

0400100

0400

04000

equationlinear a writecan weknot, interval eachFor

3

2

1

32

231

12

T

T

T

TT

TTT

TT

C786.26

C143.7

C786.1

3

2

1

o

o

o

T

T

T

Fig. 11

27

To increase the accuracy, we should use a denser grid:

1 8 15 5 12

2 9 16 6 13

3 10 17 7 14

4 11 18 1 8 15

5 12 2 9

6 13 3 10

7 14 4 11

987654321 151413121110

9

8

7

6

5

4

3

2

1

100000141000001

100000141000001

10000004100000

1000001410000

100000141000

10000014100

1000001410

100000141

10000014

5:elements nonzero,15121 width

h

a

Fig. 12

28

Laplace equation with Dirichlet boundary condition – numerical solutions

• elimination method:

• Direct iteration Liebmann method

• S.O.R. method

• Alternating direction iteration (A.D.I.) method

04, ,1,1,,1,12 jijijijijiij uuuuuu

411,1,,1,1,

kjijijijikji uuuuu

1,1,,

1,11,1,,1,1

1, 4

kjikjikji

kjikjijijiji

kji

uRuu

uuuuu

uR

1,1,1,

,,1,1,1,

1,1,1,

,,1,11,,

24

24

24

24

kjijiji

kjijijikjikji

kjijiji

kjijijikjikji

uuuP

uuuP

uu

uuuP

uuuP

uu

29

6 Finite element method for second order elliptic PDEsFinite Elements Method (FEM)Let us use Laplace equation Dirichlet problem as an illustrative example

Based on Variational

Principles

Equivalence theorem

The solution of the above PDE

will minimize the following functional

yxyxgyxu

Dyxy

u

x

u

,,,,

,,02

2

2

2

dxdyy

u

x

uuJ

D

22

2

1

S

D

meD

Fig. 13

30

Discretize D, usually using trangulation method:

For each element

use bivariate function to approximate

At three vertices, we can get

Then,

SDeDm

m ,

yaxaayxW e321,

kkk

jjj

iii

Wyaxaa

Wyaxaa

Wyaxaa

321

321

321

kk

jj

ii

kk

jj

ii

kkk

jjj

iii

wx

wx

wx

ea

yw

yw

yw

ea

yxW

yxW

yxW

ea

1

1

1

2

1

1

1

1

2

1

2

1

3

21

where

kk

jj

ii

yx

yx

yx

e

1

1

1

2

31

X

Y

u

ixjx

kxiy

ky

jy

yxu ,e yxW ,

i

j

ke

Ui=Wi

Uk=Wk

Uj=Wj

Fig. 14

32

Therefore,

where

Now that the vertices coordinates are specified, one can get

Moreover,

kkkk

jjjj

iiiie

wydxcb

wydxcb

wydxcbe

yxW

2

1,

ijkjikijjik

kijikjkijkj

jkikjijkkji

xxdyycyxyxb

xxdyycyxyxb

xxdyycyxyxb

,,

,,

,,

kjie WWWyxW ,,,

kkjjii

eey

kkjjii

eex

wdwdwdey

WyxW

wcwcwcex

WyxW

2

1,

2

1,

33

Functional minimization problem amounts to

with the following approximation:

The minimum solution is from

Therefore, we can get

That is

e e

ey

ex

yx

dxdyww

dxdyuuuJ

22

22

2

1

2

1

dxdye

wdwdwd

e

wcwcwcwJ

kkjjii

e e

kkjjii

2

2

2

22

1

nmwJWm

,,2,1,0

rWA nmuw mm ,,2,1,

n: the number of inner knots

W is given on the boundary

34

2

2

121

1

,

,

,,cos,cos,

,,,

,

,

,,,,

yx

yxh

yxuyxhy

uyxq

x

uyxp

yxyxgyxu

Dyx

yxf

yxuyxry

uyxq

yx

uyxp

x

For a more general situation

The functional to be minimized is

We can similarly do

the discretization and

get the finite element

solution

2

212

2

22

2

1,

,,,2

1

dsuhuhdxdyuyxf

uyxry

uyxq

x

uyxpuJ

y

x

D

1

2sx

syS

normal

genttan

12

Fig. 15

35

8 Finite difference method for second order parabolic PDEsDynamic diffusion equation:

For one dimensional space

When using finite difference to replace the differentiation, there are many options, e.g.,

So,

We need

We have some other easy methods:

t

C

DC

12

xgtxC

xgtxC

xgtxC

x

CD

t

C

n 3

20

10

2

2

,

,

,

,

iixx

jijii

tx

jjtjiji

tx

xxhh

CCC

t

C

tthht

CC

t

C

ji

ji

12

,1,1

2

2

11,1,

,2

,2

jijijix

tjiji CCC

h

hDCC ,1,,121,1, 22

ji tt and at valuesnumerical 1

36

Explicit method:

We get

or,

Then, we have

22

,1,,1

,

2

2

,1,

,

2x

x

jijiji

tx

tt

jiji

tx

hOh

ccc

x

c

hOh

cc

t

c

ji

ji

jijijix

tjiji ccc

Dh

hcc ,1,,12,1, 2

jijijiji crccrc ,,1,11, 21

2

1，2 when， where 22 rDhh

h

Dhr tx

x

t

jijiji ccc ,1,11, 2

1

0x ix 1ix1ix nx x

t

1jt

jt

0t

Fig. 16

37

Illustrative example:

where

Compare the numerical result with the following analytical result:

2.0,20

0,0

04.00,

119.02

2

tc

tc

xcx

c

t

c

sec2.67

119.0

42

sec119.042

2

t

x

h

cmDcmh

1

2

1

2

20

12sin1200294.0exp

12

10sin01175.0exp

20

2,

n

n

xntn

xntn

xtxC

Saturated steam

C2H5OH

32.0,20

cm

mgtc 304.00, cmmgxc

sec11904.0 2cmD

cm2020 0 x

30.0,0

cm

mgtc

air

2cmA

Fig. 17

38

0

1

2

3

4

5

6

840 24201612

%c

cmx 4

cmx 12

Number of time steps

Analytical Solutions

Numerical Solutions

steptimepert

cmxset

cmDx

tDr

sec6.334119.0

25.0

0.4

sec119.0,25.0

2

22

Analytical versus Numerical Solutions

Diffusion Dynamics

r0.25

Fig. 18

39

0

1

2

3

4

5

6

420 121086

%c

cmx 4

cmx 12

Number of time steps

Analytical Solutions

Numerical Solutions

steptimepert

cmxset

cmDx

tDr

sec2.674119.0

5.0

0.4

sec119.0,5.0

2

22

Analytical versus Numerical Solutions

Diffusion Dynamics

r0.5

Fig. 19

40

Stability analysis of the explicit method

Therefore,jijiji

ji

Wce

tx

,,,

as denoted is ,at error The

have we,2

1 When

0,et L

, from and equation, previous the toSubstitute

,

,

2

,

2

expansion series

21

21

2

,1,

22

22

,,,1

21

22

,,,1

1,,,,1,1

,,1,11,

r

MDEe

Dt

xr

t

xctww

x

tcx

x

cxww

x

tcx

x

cxww

Taylor

wwrwwr

ereere

jj

ijiji

jijiji

jijiji

jjjijiji

jijijiji

2

2

,,1,11,

,,

21

x

tcD

t

xct

ereere

ji

jijijiji

41

Therefore, we have

tME

tMErrEE

j

jjj

2121

0100

11

1

2

ttMEtMjE

tMEtMEE

j

jjj

then,,0,0 if ,2

1 When

so, ,0 ,0 When

11

00

txr

MtE

Et

jj

0

,,

,

2

2

,

2

2

jiji

ji

x

cD

t

c

x

tcD

t

xcM

is conditionstability The

stable is algorithm theand0 isThat 1 jE

2

12

x

tDr