IEEE TRANSACTIONS ON ELECTRONIC COMPUTERS, VOL. EC-15, NO. 6, DECEMBER, 1966 849 A Spectral Stability Analysis of Finite Difference Operators M. C. GILLILAND, SENIOR MEMBER, IEEE Abstract-This paper outlines a simple technique for determining pled-data character; the value of the function is speci- stability criteria for numerical integration procedures. Use is made fied only for particular values of the independent vari- of the Z-transform and the method is presented in engineering terms. able. Thus, f(t) becomes INTRODUCTION f(n T), n = 0, 1, . . IGITAL SIM\ULATION programs and software IGITAL SIMULA O p. where T is the increment of the independent variable. are receiving much attention in the engi- If we define an incremental operator neering field today. Because the engineer does not have a broad enough selection of previously written problem-oriented programs to solve his problems, it is Ef (nt) f(nT + T), frequently necessary for him to write his own program. it follows that One difficulty he faces immediately is the numerical stability characteristics of his program. This paper de- Emf(nT) f(nT + mT). velops stability analysis from an engineering point of With this notation view, and provides the engineer with a convenient and d2 simple tool for determining stability criteria. dy [E + E-l - 2]y(nT) Much has been written [1]-[3] about the problem of dt2 T2 stability of integration formulas, particularly as applied to partial differential equations. A concise statement of the operator approach to the problem [4] was published d2y in 1957. This method provided insight into stability cri- -dt2 + Y f(t) teria, but was not easy to apply to day-to-day problems. A convenient empirical approach to the problem was has the finite difference representation presented by W. J. Karplus [5], who examined the cir- (E + E-1-2)y(nT) + T2y(nT) = T2f(nT) cuit stability of electrical analogs for finite difference equations. or In this paper a generalized method is presented which [E2 (2-T2)E + 1]y(nT) = T2f(n T). makes use of the Z-transform [6] and gives a spectral E2 - T2)E to b T = Tife nt)e representation of stability criteria. The results are much F2- (2-T2)E-+-1 is said to be the differential operator. the same as those in references [2], [4] but are ex- If this equation is solved for y(nT), then pressed in terms of the Z-transform method. T2 STABILITY OF DIFFERENTIAL OPERATORS y(nT) = E2 - (2 - T2)E A f(nT), The numerical solution of differential equations re- and quires that finite difference approximations be made to T2 derivatives. For example, d2y y(t + T) + y(t- T) - 2y(t) -(2- T2)E+ 1 dt2 T-0 T 2 is said to be the integration operator. In what follows we will only be interested in stability, as determined by and the roots of the denominator or, equivalently, the dif- d2y 1 ferential operator. i\'Iost differential operators have the dt2 T2 [y(t A- T) + y(t -T) -2y(t)] form o- = amEm is a good approximation for suficiently small T.m The digital representation of a function has a sam- whr sasial ne etadteasaecntns The Z-transform of the differential operator is Manuscript received February 1, 1966; revised August 20, 1966., The author is with Computer Research, Inc., Lafayette, Califv . Zgo } = Ea,,zm = 0(Z).
A Spectral Stability Analysis of FiniteDifference Operators
M. C. GILLILAND, SENIOR MEMBER, IEEE
Abstract-This paper outlines a simple technique for determining pled-data character; the value of the function is speci-stability criteria for numerical integration procedures. Use is made fied only for particular values of the independent vari-of the Z-transform and the method is presented in engineering terms. able. Thus, f(t) becomes
INTRODUCTION f(n T), n = 0, 1, . .
IGITAL SIM\ULATION programs and softwareIGITAL SIMULA O p. where T is the increment of the independent variable.are receiving much attention in the engi- If we define an incremental operatorneering field today. Because the engineer does
not have a broad enough selection of previously writtenproblem-oriented programs to solve his problems, it is Ef(nt) f(nT + T),frequently necessary for him to write his own program. it follows thatOne difficulty he faces immediately is the numericalstability characteristics of his program. This paper de- Emf(nT) f(nT + mT).velops stability analysis from an engineering point of With this notationview, and provides the engineer with a convenient and d2simple tool for determining stability criteria. dy [E + E-l - 2]y(nT)Much has been written [1]-[3] about the problem of dt2 T2
stability of integration formulas, particularly as appliedto partial differential equations. A concise statement ofthe operator approach to the problem [4] was published d2yin 1957. This method provided insight into stability cri- -dt2 + Y f(t)teria, but was not easy to apply to day-to-day problems.A convenient empirical approach to the problem was has the finite difference representationpresented by W. J. Karplus [5], who examined the cir- (E + E-1-2)y(nT) + T2y(nT) = T2f(nT)cuit stability of electrical analogs for finite differenceequations. or
In this paper a generalized method is presented which [E2 (2-T2)E + 1]y(nT) = T2f(n T).makes use of the Z-transform [6] and gives a spectral E2- T2)E
tob
T =Tife nt)erepresentation of stability criteria. The results are much F2- (2-T2)E-+-1 is said to be the differential operator.the same as those in references [2], [4] but are ex- If this equation is solved for y(nT), thenpressed in terms of the Z-transform method. T2
STABILITY OF DIFFERENTIAL OPERATORS y(nT) = E2 - (2 - T2)E A f(nT),
The numerical solution of differential equations re- andquires that finite difference approximations be made to T2derivatives. For example,
d2y y(t + T) + y(t- T) - 2y(t) -(2- T2)E+ 1
dt2 T-0 T2 is said to be the integration operator. In what followswe will only be interested in stability, as determined by
and the roots of the denominator or, equivalently, the dif-
d2y 1 ferential operator. i\'Iost differential operators have thedt2 T2 [y(t A- T) + y(t -T) -2y(t)] form
o-= amEmis a good approximation for suficiently small T.mThe digital representation of a function has a sam- whr sasial ne etadteasaecntns
The Z-transform of the differential operator isManuscript received February 1, 1966; revised August 20, 1966.,The author is with Computer Research, Inc., Lafayette, Califv. Zgo } = Ea,,zm = 0(Z).
850 IEEE TRANSACTIONS ON ELECTRONIC COMPUTERS DECEMBER
The stability of the operator depends on the location of equations.) There is a round-off error, which is a func-its zeros in the z-plane [6]. If 0 has no zeros in the ex- tion of n, for each member of the set. The set of func-terior of the unit circle, IzI = 1, and only zeros of unit tions can be represented by one function of two vari-multiplicity on the unit circle, then 0 is said to be stable ables, u(k, n), where(i.e. the inverse transform of u(z)/O(z) will be bounded
. ..
for any finite "impulse" function, u). Suppose R is the u(k, n) = fk(nT), k = 1, M.
response of the differential operation, 0, to the input, I. The round-off error function associated with u(k, n)Then can be denoted e(k, n). Then, for any fixed k, E(k, n) is
the round-off error of fk(nT) as a function of n. E(k, n)O(z)R(z) = I(z). is generally a bounded random function of both k, n.
In other words, at each calculation step, n, E(k, n) willIn a digital calculation the input, I, usually is a com- be a random function of k, e,,(k), and there will be nonebination of the "true" or exact input, I'(z), and round- other than a statistical relationship between e&(k) andofterror, e(z). (i.e., the exact value of I' is I-e because En+l(k) as functions of k.Of limited precision of the calculation). Thus, When analyzing the stability of a system of finite dif-
(z.)R(z) = I'(z) + e(2.)- ference equations, it is not sufficient to determine thestability of any one equation (as in the first section).
The response, R, can be represented The entire system of equations must be consideredbecause of coupling terms between the equations.
R = R' + RfSPECTRAL STABILITY
where It is the sensitivity to E(k, n) of the finite-differencesystem equations which generate u(k, n) that determines
O(z)R'(z) = I'(z) the stability of the numerical procedure. The evaluation
O(z)R,(z) = e(z). of the response of the system equations to the randomvariable E(k, n) seems difficult. However, the stability
The stability of the calculation then depends on whether analysis can be simplified easily. For fixed n, E(k, n)or not R, is constant or tends to zero for a spurious =En(k). Suppose spectral representation is found forinput, E. If 0 is not stable, then E will produce an Re En(k), say Sn(w) where w is the spectral variable. Now,which is unbounded. It is important to note that fre- if the response of the system equation is evaluated on aquently R' is bounded while R, is not. In this event R spectral basis, the response to each frequency, w, mustwill be unbounded and will not represent R' due to be stable. (The spectral representation, Sn, cannot bespurious inputs f. In the event R' is unbounded by in- assigned any particular character because it will dependtent (the differential equation corresponding to the dif- on the type of software and hardware which composeferential operator has an unstable solution) the question the computer system as well as perhaps previous calcu-of how well R represents R' is answered by a comparison lations executed by the program.) Thus, the essence ofof the growth of R' and Rf. the stability analysis is to determine the stability ofSometimes a differential operator will have roots other the system equations as a function of frequency within
than those which generate the desired response, R'. the spectrum.These are called secondary or parasitic roots. If theseroots are unstable then R may be unstable without the SPCRL IERSEAN OFENUMERIpresence of E. But the point is that these roots will also oN a OPeraTORsbe excited by an arbitrary e. The Z-transform of a numerical (sampled-data func-
Thus, the problem of stability is reduced to algebraic tion), f(nT), whereT( lei ncrement of the inde-form; namely, the solution of the polynomial equation, pendent variable (sample interval) is given by
O(z) = O. F(z) -Z{f(nT)} Ef(mT)z-m.m=o
ROUND-OFF ERROR FUNCTIONS The spectral representation of f(nT) is obtained by the
As noted above, the value of f(nT) at any step, n, in a substitution [7 ] z=ei, where 6=wcT is said to be thenumerical calculation, is not exact due to the limited sample angle. (This corresponds to the substitutionprecision used by the digital computer. The round-off s =jc in the Laplace transform of a continuous func-error is then a function of n1 and can be denoted by e(n). ticon.) Thus, F, can be expressed as a function of 6 ande is generally a random function. consequently, the spectral variable, co. The range of
Suppose the values of a set of functions fk(nT), 6 to be used is restricted so that 101 <wr, and more thank= 1, . , m, are to be calculated for each n. (This is twro values per cycle are known for every frequelncythe case for the digital solution of partial differential component.
1966 GILLILAND: ANALYSIS OF FINITE DIFFERENCE OPERATORS 851
Since any bounded numerical function has a spectral Applying the Z-transform again (with respect to n)representation, it follows that in particular so does the leads toround-off error en(k). L(0 0( z) = 0The application of the finite difference operator ' )
where L(6, z) is a polynomial in z with 0 as a parameter.o(E) = E a,E- The stability of LS=0 depends on the location of the
m=1 zeros of
where all the am's are constants, to some function, f, has L(0, z) = 0the multiplicative property [6] which is said to be the stability equation. To ensure sta-
O(z)F(z) + Initial or boundary conditions forf, bility for the system of finite difference equations,
under Z-transform where L(k, n)u(k, n) = f(n),n L must be stable for all I01 <7r. This is required because
0(z) = Eaz. Sn(O) is an arbitrary function of 0.m=l
Consequently, the operation 0 corresponds to spectral EXAMPLESmultiplication by The results above are illustrated with several ex-
n amples. The following notation will be adopted:O(eil) =Eame=e Un,m = u(nAt, mAx) or Un,m = u(nT, mb)
Now suppose the system of finite-difference equations where T=At and 8 =Ax. (Note that Un, m can be regardedwhich generates u(k, n) is given by as a set of functions fm(nT).) The finite-difference op-
erators for both x, t are defined byL(k, n)u(k,n) f(k) E5kUn"m = u(nT; mb + k8)
where L is a differential operator. Applying the Z-trans- ETkUn,m = u(nT + kT, mb).form with the substitution z=eiO,
L(O, n) U(6, n) = F(O) + boundary conditions for u. ExampleConsider the finite difference diffusion equation with aNow in a computer, the equation which really represents forward difference representation of the time derivative
(It can be assumed that F and the boundary conditions Under the transformation U (nT, z) = Z { UnMare represented precisely, for our purpose.) Here
U'(O, n) = U(O, n) + Sn(6) Un+1= 1+k
(z+ z-1-2) U, + F(z)82
where U is the exact (theoretical) solution, and Sn(O) isthe Z-transform of the round-off error Ef(k). Then where F(z) is a transformed boundary condition. Sub-
and the stability of the computer solution is determined Thus stability for the system is determined byby the solution of
L(0, nf)S (0) = 0. Sn+1(0) = + 2T (COs0 - 1)]Sn(0).Again, as in the first section, U(G, n) can be stable whileSn(O9) is not. The converse is not true since the class of Now under the transformation S(6, :)=Z {Sn(0) },functions U(0, n) is a subset of the functions Sn(O). Note r 2kT 1that stability is not influenced by the specification of i-L 0-2(° -lis =oboundary conditions. In other words, changing theboundary conditions will not change the system re- The stability equation issponse due to round-off error. The solution of LSn=0is found by using the procedure outlined in the first z-kT+(o -1) =Osection. Z-L 82 (cs ) 0
852 IEEE TRANSACTIONS ON ELECTRONIC COMPUTERS DECEMBER
In order that the root of the stability equation be inside thenthe unit circle zl =1, it follows S(0) =K, 0=ir
2kT-1 < 1 + (cos 0 -1) < 1, -0 0 z i
and the stability inequality isor kT 1
-2 -2kT 62 2-2< - 2 (1- cos 0) < 0.6 Thus it is seen, loosely speaking, that the stability ofThis inequality will be satisfied on the right for any this numerical system is determined by the sensitivityT/62>0. Consequently stability is determined by the of the system to the highest frequency round-off errorleft inequality which leads to mode with respect to m.
kT 1 Example62 1-cos Consider the diffusion equation using the Crank-
Nicolson formulaSince Sn(O) is an arbitrary function, this inequality mustbe satisfied for all 0. The worst case is 0=wr. Thus, when /kT [Un+l,m±l + un+1,m-l - 2Un±l,m]
kT 1 6262<K-; (I1- )kT ~ 2nmZj2 2 +-[Un,m+l + un qn_-2Un,m+nm] i Un+1,m Un,m
62the numerical solution to the diffusion equation will bestable, and this agrees with the classical result. O< .1.Now to illustrate the spectral nature of stability, sup- Under the transformation
pose that Sn(0) is not arbitrary. Then, different stability =bounds for the parameter kT/62 will result. For example.if with the substitution ei0, and after some simplification,
E(n, m) = g(n) the round-off error response is given byF2kT
so that for fixed n, e=constant for all m, then - (1-cos 0)32
Sn(0) =K, 0 =0 =1- 2kT sn
-2 Cos0)
and therefore the stability inequality becomes The stability equation is
kT 2kT
62 62 (1-cos0)
Z-1 + =0.Suppose, 2kT
1+ C(1-cos0)-(n, m) = g(n)f(m), 62
where f(m) is the sequence The stability inequality is (-1 <z<1),o,i,o,-i,0,1,* ~~~~~2kT**,O, 1, O, -1 6, 2 (1-cos0)
with ascending m. Then -1<- + <12kT
Sn (0) =K, 0 = r/2 1 -- 4(1 - cos0)62
7r= 0, 0 Z -7 which can be simplified to
kT 1and the stability inequality is ^2(1-24) < 1c
kT- K 1. Since Sn(69) must be arbitrary, 6=wr is the worst case
62~ so that
If kT 1
E(n,m) = (-...)mg(n), 6(12 ) 2-
1966 GILLILAND: ANALYSIS OF FINITE DIFFERENCE OPERATORS 853
Thus the numerical process is unconditionally stable for EIT20> 2 and for )<' M84 (Un,m+2 -4Un,m+l + 6fUn,m - 4un,m-±+Un,m-2)
kT 1 = Unl,m + Un-i,m - 2Un,m.a2 2(1- 2) Under transformation with respect to m and the substi-
which agrees with the classical result. tution z =ei-2EIT2Example [M i- (os 20 2 cos0 + 3)+ 2Un =Un+1+ Unl.
The standard finite difference form of the wave equa-tion is The equation for Sn is
c2T2 - EIT2_2 (un,m+1 +±n,m-l-2Un,m) = Un+l m + un--,m-2 n,M. Sn+2-2 1 + EI (cos 20- 2 cos 0 + 3)1 Sn+ja2 L Ma4i
Under transformation with respect to m and the sub- +Sn = 0.stitution z = ei0 Under transformation with respect to n, the stability
2c2T2 equation isa (Cos 6 - 1) Un = U,+-+ U,_1 - 2 Un, - 2[ } + 1 = 0,
and the equation for round-off error is and the stability inequality is
The stability inequality (all roots in z <1) is will satisfy the inequality for all 0. The numerical pro-cess is unstable for any T2/I4.
c2T2-1<1 - (1-cos 0) < 1 Example
a2Consider the partial differential equation
so that02u 1 au au
c2T2 2 - + -0 < < . x2 1 dx at
a2 1-Cos0If forward and central differences are used for the first
Since the worst case is 6=7r, the stability parameter, and second order derivatives respectivelyc2T2/a2 must satisfy T T
c2T2 -(Un,m+1 + Un,m-i 2Un,m) + (Un,m+1 - Un,m)
62= Un+l,m -Un,m.
which agrees with the classical result.whchageewthth casicl eslt Under transformation with respect to m and the sub-
Example stitution z = iConsider the equation for a vibrating uniform beam 2T T
-(cos 6 - 1)Un + - (e10 - 1)Un = U,+l - n
d4u M d2u a aidx4 El dt2 so that the equation for the round-off error is
where u is displacement. If standard central differences sn = [F1--1-co2T--T-i,are used to approximate the derivatives L 2 ai
854 IEEE TRANSACTIONS ON ELECTRONIC COMPUTERS
Under transformation with respect to n, the stability F N kTequation is L - 2 (Ex + Ex- 2)] Un,ml. Mn
2T T = Un+1.. ',n-Un,. "tz = 1 --(1 -cos0) --(1 -eil), n
82 81 where t=nT and xx =m8x.or Under N Z-transformations on the variables, M1,
T T,MN with the substitutions zx=ei9x, the equationz = 1 -- (1 - cos 0) - -(1 - cos -j sin 0). for the round-off error is
62 61 F N 2kT 1
For stability the zero of this equation must lie in Iz| < t. L E (cos Oj-t) + I Sn = S1+1.This condition will be satisfied if
Under the Z-transformation on n, the stability equa-z2 < 1, tion is
where z denotes the conjugate of z. Thus, the stability N 2kTinequality is z =t- E (1-Cos Ox),
[i (a'2 + a2 (1 COS 0)] + [Lsin]2 <1. and the stability inequality is(1 cos 0) + -sinO F<N1- co Oxl
Let X =2T/82, M=8/21 so that T < Lk E - x2
[- (1 + )( - COS 0)6])2 + X2A2 sin2 0 < 1, The worst case results from Ox =w for all X and
and T < [ 1x-T < E_ a_ ~ ~ ~ ~~20 2k _ x=1 6X2_
1 -2X(1 + ju) sin2 - + 4X2,U2 sin2 -COS2 - <[_ - 2(1±i)sn2 -±42 sn2y os22 2 If bx=8for all X, then
After some simplification kT 1
82 2NX M2 + (1 + 2A) sin2 i< 1 +C c
2 CONCLUSION
and The method presented here is simple from a manipula-tive standpoint and also provides insight into the
< + spectral behavior of the stability of numerical processes./1 1-+ 2,A It can be applied readily to more general cases of con-\ 2 / stant coefficient operators. The use of the method for
the general linear operator depends on finding a suitableThe worst case corresponds to 0=7r and the stability spectral representation for the operator. Once this hasinequality is been done the philosophy of the method presented here
1 applies. Nonlinear operators must be examined on an
x < individual basis.
REFERENCESor [1] J. Douglas, "On the relation between stability and convergence in
the numerical solution of linear parabolic and hyperbolic differen-T 1 tial equations," J. Soc. for Ind. and Appl. Math., vol. 3, no. 1,-< March 1956.82 6 [2] P. D. Lax and R. D. Richtmyer, "Survey of stability of linear
2 + - finite difference equations," Commun. on Pure and Appl. Math.,I vol. 9, pp. 267-293, May 1956.
[3] A. R. Mitchell, "Round-off errors in implicit finite differenceIf I= cc the result agrees with that of the first example. methods," Quart. J. Mechanics and Appl. Math., vol. 9, pt. 1,I pp. 111-121, 1956.
[4] Arnold N. Lowan, "The operator approach to problems of sta-Example bility and convergence of solutions of differential equations andConsider heN-dimnsional iffusionequationthe convergence of various iteration procedures," Scripta Mathe-Cons1der heN-dlmnslonal lffuslonequatlonmatica Studies, no. 8, Scripta Mathematica, Yeshiva University,
N 02 ~~~~~~~~~~~~~NewYork, N. Y., 1957.N 82 ~1 OU [5] W. J. Karplus, "An electric circuit theory approach to finite dif-z = , u = u(x1, X2, * , Xn, f). ference stability," Comm. and Electronics, May 1958.
x=i 0Xx2 k 01 [6] E. I. Jury, Theory and Application of the Z-Transform MethodNew York: Wiley, 1964.
If forward and central differences are used for the first [7] M- C. Gilliland, "The spectral evaluation of iterative differentialanalyzer integration techniques," 1961 Proc. W.J.C.C., vol. 12.2,
