Computational Computational Methods in PhysicsMethods in Physics
PHYS 3437 PHYS 3437Dr Rob ThackerDr Rob Thacker
Dept of Astronomy & Physics Dept of Astronomy & Physics (MM-301C)(MM-301C)
[email protected]@ap.smu.ca
Today’s LectureToday’s Lecture Notes on projects & talksNotes on projects & talks Issues with adaptive step size Issues with adaptive step size
selectionselection Second order ODEsSecond order ODEs nth order ODEsnth order ODEs Algorithm outline for handling nth Algorithm outline for handling nth
order ODEsorder ODEsAdvanced warning – no class on March 31st
Project time-lineProject time-line By now you should have approximately ½ By now you should have approximately ½
of the project coding & development of the project coding & development donedone
There are only 4 weeks until the first talkThere are only 4 weeks until the first talk I would advise you to have at least some I would advise you to have at least some
outline of how your write-up will look outline of how your write-up will look alreadyalready
Project reports are due on the last day of Project reports are due on the last day of termterm
Talk FormatTalk Format Allowing for changing over presenters, Allowing for changing over presenters,
connecting/disconnecting etc, we can connecting/disconnecting etc, we can allow up to 60 minutes per lecture allow up to 60 minutes per lecture periodperiod
3 people per lecture means 20 minutes 3 people per lecture means 20 minutes per presenterper presenter
15 minute presentation15 minute presentation 5 minutes for questions5 minutes for questions Remember your presentation will be Remember your presentation will be
marked by your peersmarked by your peers
Issues with the adaptive Issues with the adaptive step size algorithmstep size algorithm
Step (8) If err < 0.1Step (8) If err < 0.1h is too small. Increase by 1.5 h is too small. Increase by 1.5 for next stepfor next step
If err > If err > h is too big. Halve h and repeat h is too big. Halve h and repeat iterationiteration
If 0.1 If 0.1 ≤ err ≤ ≤ err ≤ h is OK. h is OK. Clearly if there is problems with convergence this step will Clearly if there is problems with convergence this step will
continue to keep dividing h forever. You need to set up a limit continue to keep dividing h forever. You need to set up a limit here, either by not allowing h to get smaller than some preset here, either by not allowing h to get smaller than some preset limit or counting the number of times you halve hlimit or counting the number of times you halve h
Because of rounding error as you increment x it is very Because of rounding error as you increment x it is very unlikely that you will precisely hit xunlikely that you will precisely hit xmaxmax with the final with the final stepstep Therefore you should choose h=min(h,xTherefore you should choose h=min(h,xmaxmax-x) where x is -x) where x is
current positioncurrent position
Issues with the adaptive Issues with the adaptive step size algorithm: more step size algorithm: more
“gotcha’s”“gotcha’s” Step (7) Estimate errorStep (7) Estimate error
Should add a very small (“tiny”) number to the absolute Should add a very small (“tiny”) number to the absolute value of the denominator to avoid a divide by zero (when the value of the denominator to avoid a divide by zero (when the two estimates are very close to zero you might - in incredibly two estimates are very close to zero you might - in incredibly rare situations - get the two numbers adding to zero).rare situations - get the two numbers adding to zero).
Since you store values of y and x in an array it is Since you store values of y and x in an array it is possible that you may run out of storage (because possible that you may run out of storage (because you need too many points). You should do some you need too many points). You should do some check of the number of positions stored versus the check of the number of positions stored versus the maximum possible allowed in your decleration of the maximum possible allowed in your decleration of the x & y arrays . This will avoid the possibility of a x & y arrays . This will avoid the possibility of a segmentation fault.segmentation fault.
ˆ21
ˆ
1*1
1*1
yy
yyerr
Second order ODEsSecond order ODEs Consider the general second order ODEConsider the general second order ODE
We now require two initial values be We now require two initial values be provided, namely the yprovided, namely the y00 value and the value and the derivative y´(xderivative y´(x00)) These are called the Cauchy conditionsThese are called the Cauchy conditions
If we let z=y´, then z´=y´´ and we have If we let z=y´, then z´=y´´ and we have
)',,()('' yyxgxy
00
00
)( ; ))(),(,(''')( ; 'zxzxzxyxgyzyxyzy
(1)
Second order ODEs contSecond order ODEs cont
We have thus turned a second order ODE into a We have thus turned a second order ODE into a first order ODE for a vectorfirst order ODE for a vector
Can apply the R-K solver to the system but you Can apply the R-K solver to the system but you now have two components to integratenow have two components to integrate
At each step must update all x, y and z valuesAt each step must update all x, y and z values
0
00
0
00
with '''
as system thecan write then we
and , so Let
zy
YFYFzy
gz
Fzy
Yzy
Y
DiagrammaticallyDiagrammatically
x0 x1
y(x)
y0
x0 x1
z(x)
z0
Remember z’=g(x,y,y’)Remember y’=z
y´0=z0 z1z´0=g0
y1
nth order ODEsnth order ODEs Systems with higher than second order Systems with higher than second order
derivatives are actually quite rare in derivatives are actually quite rare in physicsphysics
Nonetheless we can adapt the idea for 2Nonetheless we can adapt the idea for 2ndnd order systems to nth orderorder systems to nth order
Suppose we have a system specified bySuppose we have a system specified by
Such an equation requires n initial values Such an equation requires n initial values for the derivatives, suppose again we have for the derivatives, suppose again we have the Cauchy conds.the Cauchy conds.
),...,'',',,()( )1()( nn yyyyxgxy
0)1(
0)1(
000000 )(,...,'')('',')(',)( nn yxyyxyyxyyxy
Definitions for the nth Definitions for the nth order systemorder system
(n)(n))(nn-n
n-n
yyy...y''y'y'y
y'yy'yyy
1
121
1
23
12
1
Let
format. vector ain writing suggests thisand Thus 1 )'',...,yg(x,y,y',yy' )(n-
n
Definitions for the nth Definitions for the nth order systemorder system
),(
),..,,(''
'
''
'Y
gives sderivative taking
'
''
YLet
1
1
3
2
1
2
1
1
2
1
3
2
1
YxF
yyxgy
y
yy
yy
y
yy
y
yyy
y
yyy
n
n
i
n
n
i
nn
So another system of coupled first order equations that can be solved via R-K.
Algorithm for solving such Algorithm for solving such a systema system
Useful parameters to set:Useful parameters to set:imax=number of points at which y(x) is to be imax=number of points at which y(x) is to be
evaluated (1000 is reasonable)evaluated (1000 is reasonable)nmax=highest order of ODE to be solved (say 9)nmax=highest order of ODE to be solved (say 9)errmax=highest tolerated error (say 1.0×10errmax=highest tolerated error (say 1.0×10-9-9))
Declare x(imax),y(imax,nmax),y0(nmax),yl(nmax), Declare x(imax),y(imax,nmax),y0(nmax),yl(nmax), ym(nmax),yr(nmax),ytilde(nmax),ystar(nmax)ym(nmax),yr(nmax),ytilde(nmax),ystar(nmax)
Note that the definitions used here are not quite consistent with the variable definitions used in the discussion of the single function case.
User inputsUser inputs Need to get from the user (not Need to get from the user (not
necessarily at run time)necessarily at run time) The g(x,y,y’,…) functionThe g(x,y,y’,…) function Domain of x, i.e. what are xmin and xmaxDomain of x, i.e. what are xmin and xmax What is the order of the ODE to be solved, What is the order of the ODE to be solved,
stored in variable nordstored in variable nord What are the initial values for y and the What are the initial values for y and the
derivatives (the Cauchy conditions), these derivatives (the Cauchy conditions), these are stored in y0(nmax)are stored in y0(nmax)
Correspondence of arrays Correspondence of arrays to variablesto variables
The arrays y(imax,nmax), y0(nmax) The arrays y(imax,nmax), y0(nmax) corresponds to the ycorresponds to the y derivatives as follows:derivatives as follows: y(1:imax,1)≡y vals with y(1:imax,1)≡y vals with
y(1,1)=yy(1,1)=y00=y0(1)=y0(1) y(1:imax,2)≡y´ vals with y(1,2)=yy(1:imax,2)≡y´ vals with y(1,2)=y
´́00=y0(2)=y0(2) y(1:imax,3)≡y´´ vals with y(1,3)=yy(1:imax,3)≡y´´ vals with y(1,3)=y
´´´´00=y0(3)=y0(3)
y(1:imax,nord)≡yy(1:imax,nord)≡y(nord-1)(nord-1) vals with y(1,nord)=y vals with y(1,nord)=y(n-1)(n-1)0 0 = =
y0(nord)y0(nord)
| |
Choose initial step size & Choose initial step size & initialize yl valuesinitialize yl values
Apply same criterion as standard Runge-Apply same criterion as standard Runge-KuttaKutta dx=0.1×errmax×(xmax-xmin)dx=0.1×errmax×(xmax-xmin) dxmin=10dxmin=10-3-3×dx×dx
We can use this value to ensure that adaptive step size We can use this value to ensure that adaptive step size is never less than 1/1000is never less than 1/1000thth of the initial guess of the initial guess
x(1)=xl=xminx(1)=xl=xmin Set initial position for solverSet initial position for solver
Initialize yl (left y-values for the first interval)Initialize yl (left y-values for the first interval) do n=1,nord yl(n)=y0(n)=y(1,n)do n=1,nord yl(n)=y0(n)=y(1,n)
Start adaptive loopStart adaptive loop Set i=1 this will count number of x positions Set i=1 this will count number of x positions
evaluatedevaluated dxh=dx/2 --- half width of zonedxh=dx/2 --- half width of zone xr=xl+dx --- right hand boundaryxr=xl+dx --- right hand boundary xm=xl+dxh --- mid point of zonexm=xl+dxh --- mid point of zone
Perform R-K calculations on this zone for all yPerform R-K calculations on this zone for all ynn Need to calculate all y values on right boundary Need to calculate all y values on right boundary
using a single R-K step (stored in ytilde(nmax))using a single R-K step (stored in ytilde(nmax)) Need to calculate all y values on right boundary Need to calculate all y values on right boundary
using two half R-K steps (stored in ystar(nmax)) – using two half R-K steps (stored in ystar(nmax)) – for examplefor examplecall rk(xl,yl,ytilde,nord,dx)
call rk(xl,yl,ym ,nord,dxh)call rk(xm,ym,ystar ,nord,dxh)
Now evaluate R.E.-esque Now evaluate R.E.-esque value yr(n)value yr(n)
err=0.err=0. do n=1,norddo n=1,nord
If err< 0.1×errmax then increase dx: dx=dx*1.5If err< 0.1×errmax then increase dx: dx=dx*1.5 If err > errmaxIf err > errmaxdx is too big: dx=max(dxh,dxmin) and dx is too big: dx=max(dxh,dxmin) and
repeat evaluation of this zonerepeat evaluation of this zone If 0.1 errmaxIf 0.1 errmax≤ err ≤ errmax ≤ err ≤ errmax dx is OK & we need to dx is OK & we need to
store all resultsstore all results
),max()()()()(2
.15/))()(*.16()(
errnerrerrnystarnyrnystarnyrerrn
nytildenystarnyr
Error is now set over all thefunctions being integrated
Update values and prepare Update values and prepare for next stepfor next step
Increment i: i=i+1 (check that it doesn’t Increment i: i=i+1 (check that it doesn’t exceed imax)exceed imax)
x(i)=xrx(i)=xr xl=xr (note should check xl hasn’t gone xl=xr (note should check xl hasn’t gone
past xmax & that h value is chosen past xmax & that h value is chosen appropriately)appropriately)
do n=1,norddo n=1,nord y(i,n)=yr(n)y(i,n)=yr(n) yl(n)=yr(n)yl(n)=yr(n)
Then return to top of loop and do next stepThen return to top of loop and do next step
Runge-Kutta routineRunge-Kutta routine Subroutine rk(xl,yl,yr,nord,dx)Subroutine rk(xl,yl,yr,nord,dx) yl and yr will be arrays of size nordyl and yr will be arrays of size nord Set useful variables:Set useful variables:
dxh=dx/2.dxh=dx/2. xr=xl+dxxr=xl+dx xm=xl+dxhxm=xl+dxh
Recall, given y’=f(x,y), (xl,yl), and dx then Recall, given y’=f(x,y), (xl,yl), and dx then
),(~
),(
~ )~,(~
~ ),( where
)~
2~
2(6
11
2/112/12/1
2/12/12/12/1
02/10
12/12/10
yxff
fdxyyyxfffdxhyyyxff
fdxhyyyxff
ffffhyy
r
lm
lm
lll
lr
Steps in algorithmSteps in algorithm Complications: have a vector of functions and vector Complications: have a vector of functions and vector
of yl valuesof yl values call derivs(xl,yl,f0,nord) (sets all function f0 values)call derivs(xl,yl,f0,nord) (sets all function f0 values) do n=1,nord do n=1,nord call derivs(xm, ,nord) (sets function call derivs(xm, ,nord) (sets function
values)values) do n=1,norddo n=1,nord call derivs(xm, ,nord) (sets function values)call derivs(xm, ,nord) (sets function values) do n=1,norddo n=1,nord call derivs(xr, ,nord) (sets function values)call derivs(xr, ,nord) (sets function values)
)()()(~02/1 nfdxhnyny l
)(~)()( 2/12/1 nfdxhnyny l 2/12/1
~,~ fy 2/1
~f
2/12/1 , fy 2/1f
)()()( 2/11 nfdxnyny l
11~
, fy 1~f
Calculate R-K formula & Calculate R-K formula & derivs subroutinederivs subroutine
do i=1,norddo i=1,nord
That’s the end of the R-K routineThat’s the end of the R-K routine
subroutine derivs(x,y,yp,nord)subroutine derivs(x,y,yp,nord) do n=1,nord-1: yp(n)=y(n+1)do n=1,nord-1: yp(n)=y(n+1) Lastly set yp(nord)=g(x,y,y´,…,yLastly set yp(nord)=g(x,y,y´,…,y(nord-1)(nord-1)))
)(~)()(~2)(6
)()( 12/12/10 nfnfnfnfdxnyny lr
SummarySummary There are a few issues with the adaptive There are a few issues with the adaptive
step size algorithm you need to be step size algorithm you need to be concerned about (avoiding divide by zeroconcerned about (avoiding divide by zero etc.etc.))
Second order systems can be turned into Second order systems can be turned into coupled first order systemscoupled first order systems
Nth order systems can be turned into n Nth order systems can be turned into n coupled first order systemscoupled first order systems
The adaptive R-K algorithm for vectors is in The adaptive R-K algorithm for vectors is in principle similar to that for a single functionprinciple similar to that for a single function However, must loop over vectorsHowever, must loop over vectors
Implicit Methods: Implicit Methods: Backward Euler methodBackward Euler method
Recall in the Forward Euler methodRecall in the Forward Euler method
Rather than predicting forward, we can predict Rather than predicting forward, we can predict backward using the value of backward using the value of f(tf(tnn,y,ynn))
Rewriting in terms of n+1 and nRewriting in terms of n+1 and n
Replacing Replacing yyn+1n+1=r=r we need to use a root finding we need to use a root finding method to solve method to solve
),(1nn
nn ytfhyy
),(),( 11 nn
nnnnnn ytf
hyyythfyy
),(),( 111111
nnnnnn
nn ythfyyytfhyy
0),( 1 rthfyr nn
NON-EXAMINABLE BUT USEFUL TO KNOW
Notes on implicit Notes on implicit MethodsMethods
Implicit methods tend to be more stable, and for the same Implicit methods tend to be more stable, and for the same step size more accuratestep size more accurate The trade-off is the increased expense of using an interative The trade-off is the increased expense of using an interative
procedure to find the rootsprocedure to find the roots This can become very expensive in coupled systemsThis can become very expensive in coupled systems
Richardson Extrapolation can also be applied to implicit Richardson Extrapolation can also be applied to implicit ODEs, results in higher order schemes with good ODEs, results in higher order schemes with good convergence propertiesconvergence properties Use exactly the same procedure as outline in previous lecture, Use exactly the same procedure as outline in previous lecture,
compare expansions at h and h/2compare expansions at h and h/2 Crank-Nicholson is another popular implicit methodCrank-Nicholson is another popular implicit method
Relies upon the derivative at both the start and end points of the Relies upon the derivative at both the start and end points of the intervalinterval
Second order accurate solutionSecond order accurate solution ),(),(21
111
nnnnnn ytfytf
hyy
Multistep methodsMultistep methods Thus far we’ve consider self starting Thus far we’ve consider self starting
methods that use only values from xmethods that use only values from xnn and x and xn+1n+1
Alternatively, accuracy can be improved by Alternatively, accuracy can be improved by using a linear combination of additional using a linear combination of additional pointspoints
Utilize Utilize yyn-s+1n-s+1,y,yn-s+2n-s+2,…,y,…,ynn to construct to construct approximations to derivatives of order up to approximations to derivatives of order up to s, at ts, at tnn Example for s=2Example for s=2
hytfytfy
ytfy
nnnnn
nnn
),(),(''
),('
11
Comparison of single and Comparison of single and multistep methodsmultistep methods
Second order methodSecond order method We can utilize this relationship to describe a multistep We can utilize this relationship to describe a multistep
second order methodsecond order method
Generalized to higher orders, these methods are Generalized to higher orders, these methods are known as Adams-Bashforth (predictor) methodsknown as Adams-Bashforth (predictor) methods s=1 recovers the Euler method s=1 recovers the Euler method
Implicit methodologies are possible as wellImplicit methodologies are possible as well Adams-Moulton (predictor-corrector) methods Adams-Moulton (predictor-corrector) methods
Since these methods rely on multisteps the first few Since these methods rely on multisteps the first few values of y must be calculated by another method, e.g. values of y must be calculated by another method, e.g. RKRK Starting method needs to be as accurate as the multistep Starting method needs to be as accurate as the multistep
methodmethod
),(),(32
''2
'
11
2
1
nnnnn
nnnn
ytfytfhy
yhhyyy
““Stiff” problemsStiff” problems Definition of stiffness:Definition of stiffness:
““Loosely speaking, the initial value problem is referred to as Loosely speaking, the initial value problem is referred to as being being stiffstiff if the absolute stability requirement dictates a if the absolute stability requirement dictates a much smaller time step than is needed to satisfy much smaller time step than is needed to satisfy approximation requirements alone.”approximation requirements alone.”
Fomally: An IVP is stiff in some interval [0,b] if the step size Fomally: An IVP is stiff in some interval [0,b] if the step size needed to maintain stability of the forward Euler method is much needed to maintain stability of the forward Euler method is much smaller than the step size required to represent the solution smaller than the step size required to represent the solution accurately.accurately.
Stability requirements are overriding accuracy Stability requirements are overriding accuracy requirementsrequirements
Why does this happen?Why does this happen? Trying to integrate smooth solutions that are surrounded by Trying to integrate smooth solutions that are surrounded by
strongly divergent or oscillatory solutionsstrongly divergent or oscillatory solutions Small deviations away from the true solution lead to forward Small deviations away from the true solution lead to forward
terms being very inaccurateterms being very inaccurate
ExampleExample Consider: y’(t)=-
15y(t), t≥0, y(0)=1 Exact solution: y(t)=e-
15t, so y(t)→0 as t→0 If we examine the
forward Euler method, strong oscillatory behaviour forces us to take very small steps even though the function looks quite smooth
Implicit methods in stiff Implicit methods in stiff problems problems
Because implicit methods can use longer Because implicit methods can use longer timesteps, they are strongly favoured in timesteps, they are strongly favoured in integrations of stiff systemsintegrations of stiff systems
Consider a two-stage Adams-Moulton Consider a two-stage Adams-Moulton integrator:integrator:
nn
nn
nnnnnn
yh
h
y
yhyh
ytfytfhyy
2151
2151
2151
2151
),(),(2
1
1
111
Adams-Moulton solution for Adams-Moulton solution for h=0.125h=0.125
Much better behaviour and convergence
Choosing stiff solversChoosing stiff solvers This isn’t as easy as you might thinkThis isn’t as easy as you might think
Performance of different algorithms can be quite Performance of different algorithms can be quite dependent upon the specific problemdependent upon the specific problem
Researchers often write papers comparing the Researchers often write papers comparing the performance of different solvers on a given performance of different solvers on a given problem and then advise on which one to useproblem and then advise on which one to use
This is a sensible way to do things This is a sensible way to do things I recommend you do the same if you have a stiff I recommend you do the same if you have a stiff
problemproblem Try solvers from library packages like ODEPACK or Try solvers from library packages like ODEPACK or
the Numerical recipes routinesthe Numerical recipes routines
Stability of the Forward Stability of the Forward Euler methodEuler method
Stability is more important than the Stability is more important than the truncation errortruncation error Consider y’=Consider y’=y for some complex y for some complex Provided Re Provided Re < 0 solution is bounded < 0 solution is bounded Substitute into the Euler methodSubstitute into the Euler method
For For yynn to remain bounded we must have to remain bounded we must have
Thus a poorly chosen Thus a poorly chosen t that breaks the above t that breaks the above inequality will lead to inequality will lead to yynn increasing without limit increasing without limit
01
12
1 )1()1()1( ytytyty nnnn
1|1| t
||2
t
Behaviour of small errorsBehaviour of small errors We considered the previous y’=We considered the previous y’=y y
equation because it describes the equation because it describes the behaviour of small changesbehaviour of small changes
Suppose we have a solution ySuppose we have a solution yss=y+=y+ where where is the small error. is the small error. Substitute into y’=f(t,y) and use a Taylor Substitute into y’=f(t,y) and use a Taylor
expansionexpansion
dydf
Odydfytf
ytfytfy ss
'
)(),('
),('),('
2
To leading order in
Next lectureNext lecture Monte Carlo methodsMonte Carlo methods
