Methods for Ordinary Differential Equations
Lecture 10
Alessandra Nardi
Thanks to Prof. Jacob White, Deepak Ramaswamy Jaime Peraire, Michal Rewienski, and Karen Veroy
Outline
• Transient Analysis of dynamical circuits– i.e., circuits containing C and/or L
• Examples
• Solution of Ordinary Differential Equations (Initial Value Problems – IVP)– Forward Euler (FE), Backward Euler (BE) and
Trapezoidal Rule (TR)– Multistep methods– Convergence
Ground Plane
Signal Wire
LogicGate
LogicGate
• Metal Wires carry signals from gate to gate.• How long is the signal delayed?
Wire and ground plane form a capacitor
Wire has resistance
Application ProblemsSignal Transmission in an Integrated Circuit
capacitor
resistor
• Model wire resistance with resistors.• Model wire-plane capacitance with capacitors.
Constructing the Model• Cut the wire into sections.
Application ProblemsSignal Transmission in an IC – Circuit Model
Nodal Equations Yields 2x2 System
C1
R2
R1 R3 C2
Constitutive Equations
cc
dvi C
dt
1R Ri v
R
Conservation Laws
1 1 20C R Ri i i
2 3 20C R Ri i i
1
1 2 21 1
2 22
2 3 2
1 1 1
0
0 1 1 1
dvR R RC vdt
C vdv
R R Rdt
1Ri
1Ci
2Ri
2Ci
3Ri
1v 2v
Application ProblemsSignal Transmission in an IC – 2x2 example
eigenvectors
1
1 2 21 1
2 22
2 3 2
1 1 1
0
0 1 1 1
dvR R RC vdt
C vdv
R R Rdt
1 2 1 3 2Let 1, 10, 1C C R R R 1.1 1.0
1.0 1.1
A
dxx
dt
11 1 0.1 0 1 1
1 1 0 2.1 1 1A
Eigenvalues and Eigenvectors
Eigenvalues
Application ProblemsSignal Transmission in an IC – 2x2 example
1Change of variab (le ) ( ) ( ) (s ): Ey t x t y t E x t
1
1 2 1 2
10 0
0 0
0 0n n
n
A E E E E E
E
E
Eigendecomposition:
0
( )Substituting: ( ), (0)
dEy tAEy t Ey x
dt
11Multiply by ( ) : ( )dy t
E AE tdt
E y 0 0
10 0
0 0
( )n
y t
0Consider an ODE( )
( ), (0): dx t
Ax t x xdt
An Aside on Eigenanalysis
( )Decoupling: ( ) ( ) (0)iti
i i i
dy ty t y t e y
dt
1 0 0
0 0
0 0
From last slide:( ) ( )
n
dy tdt
y t
DecoupledEquations!
1) Determine , E 1 0 0
3) Compute ( ) 0 0 (0)
0 0 n
t
t
e
y t y
e
102) Compute (0) y E x
4) ( ) ( ) x t Ey t
0Steps for solvi( )
( ), (0)ng dx t
Ax t x xdt
An Aside on Eigenanalysis
1(0) 1v
2 (0) 0v
Notice two time scale behavior• v1 and v2 come together quickly (fast eigenmode).• v1 and v2 decay to zero slowly (slow eigenmode).
Application ProblemsSignal Transmission in an IC – 2x2 example
Circuit Equation Formulation
• For dynamical circuits the Sparse Tableau equations can be written compactly:
• For sake of simplicity, we shall discuss first order ODEs in the form:
riablescircuit va of vector theis where
)0(
0),,)(
(
0
x
xx
txdt
tdxF
),()(
txfdt
tdx
Ordinary Differential EquationsInitial Value Problems (IVP)
Typically analytic solutions are not available
solve it numerically
.condition initial given the intervalan in
)(
),()(
:(IVP) Problem Value Initial Solve
00
00
x,T][t
xtx
txfdt
tdx
• Not necessarily a solution exists and is unique for:
• It turns out that, under rather mild conditions on the continuity and differentiability of F, it can be proven that there exists a unique solution.
• Also, for sake of simplicity only consider
linear case:
0),,( tydt
dyF
We shall assume that has a unique solution 0),,( tydt
dyF
00 )(
)()(
xtx
tAxdt
tdx
Ordinary Differential Equations Assumptions and Simplifications
First - Discretize Time
Second - Represent x(t) using values at ti
ˆ ( )llx x t
Approx. sol’n
Exact sol’n
Third - Approximate using the discrete ( )ldx t
dtˆ 'slx
1 1
1
ˆ ˆ ˆ ˆExample: ( )
l l l l
ll l
d x x x xx t or
dt t t
Lt T1t 2t 1Lt 0
t t t
1t 2t 3t Lt0
3x̂ 4x̂1x̂
2x̂
Finite Difference MethodsBasic Concepts
lt 1lt t
x
1( ) ( )slope l lx t x t
t
slope ( )ldx t
dt
1( ) ( ) ( )l l lx t x t t A x t
1
1
( ) ( )( ) ( )
or
( ) ( ) ( )
l ll l
l l l
x t x tdx t A x t
dt t
x t x t t A x t
Finite Difference Methods Forward Euler Approximation
1t 2t t
x
(0)tAx
11 ˆ( ) (0) 0x t x x tAx
3t
2 1 12 ˆ ˆ ˆ( )x t x x tAx
1ˆtAx
1 1ˆ ˆ ˆ( ) L L LLx t x x tAx
Finite Difference Methods Forward Euler Algorithm
lt 1lt t
x 1slope ( )l
dx t
dt
1( ) ( )slope l lx t x t
t
1 1( ) ( ) ( )l l lx t x t t A x t
11 1
1 1
( ) ( )( ) ( )
or
( ) ( ) ( )
l ll l
l l l
x t x tdx t A x t
dt t
x t x t t A x t
Finite Difference Methods Backward Euler Approximation
1t 2t t
x
1ˆtAx
2ˆtAx
1 11 ˆ ˆ( ) (0)x t x x tAx
Solve with Gaussian Elimination
1ˆ[ ] (0)I tA x x
1 1ˆ ˆ( ) [ ]L LLx t x I tA x
2 1 12 ˆ ˆ( ) [ ]x t x I tA x
Finite Difference Methods Backward Euler Algorithm
1
1
1
1 1
1( ( ) ( ))
21
( ( ) ( ))2
( ) ( )
1( ) ( ) ( ( ) ( ))
2
l l
l l
l l
l l l l
d dx t x t
dt dt
Ax t Ax t
x t x t
t
x t x t tA x t x t
t
x
1( ) ( )slope l lx t x t
t
slope ( )ldx t
dt
1slope ( )l
dx t
dt
1 1
1 1( ( ) ( )) ( ( ) ( ))
2 2l l l lx t tAx t x t tAx t
Finite Difference Methods Trapezoidal Rule Approximation
1t 2t t
x
1ˆ (0)2 2
t tI A x I A x
Solve with Gaussian Elimination
1 11 ˆ ˆ( ) (0) (0)
2
tx t x x Ax Ax
12 1
2
11
ˆ ˆ( )2 2
ˆ ˆ( )2 2
L LL
t tx t x I A I A x
t tx t x I A I A x
Finite Difference Methods Trapezoidal Rule Algorithm
1
1( ) (( ) )) )( (l
l
t
l l t
dx t x t x t A dx t xA
dt
lt 1lt
1
( )l
l
t
tAx d
1( )ltAx t BE
( )ltAx t FE
( ) ( )2 l l
tAx t Ax t
Trap
Finite Difference Methods Numerical Integration View
Trap Rule, Forward-Euler, Backward-Euler Are all one-step methods
Forward-Euler is simplest No equation solution explicit method. Boxcar approximation to integralBackward-Euler is more expensive Equation solution each step implicit method Trapezoidal Rule might be more accurate
Equation solution each step implicit method
Trapezoidal approximation to integral
1 2 3ˆ ˆ ˆ ˆ is computed using only , not , , etc.l l l lx x x x
Finite Difference Methods Summary of Basic Concepts
( ) ( ( ), ( ))dx t f x t u t
dtNonlinear Differential Equation:
k-Step Multistep Approach: 0 0
ˆ ˆ ,k k
l j l jj j l j
j j
x t f x u t
Solution at discrete points
Time discretization
Multistep coefficients
2ˆ lx
lt1lt 2lt 3lt l kt
ˆ lx1ˆ lx
ˆ l kx
Multistep Methods Basic Equations
Multistep Equation:
1 1 1,l l l lx t x t t f x t u t
FE Discrete Equation: 1 11ˆ ˆ ˆ ,l l l
lx x t f x u t
0 1 0 11, 1, 1, 0, 1k
Forward-Euler Approximation:
Multistep Coefficients:
Multistep Coefficients:
BE Discrete Equation:
0 1 0 11, 1, 1, 1, 0k 1ˆ ˆ ˆ ,l l l
lx x t f x u t
Trap Discrete Equation: 1 11ˆ ˆ ˆ ˆ, ,
2l l l l
l l
tx x f x u t f x u t
0 1 0 1
1 11, 1, 1, ,
2 2k Multistep Coefficients:
0 0
ˆ ˆ ,k k
l j l jj j l j
j j
x t f x u t
Multistep Methods – Common AlgorithmsTR, BE, FE are one-step methods
Multistep Equation:
01) If 0 the multistep method is implicit 2) A step multistep method uses previous ' and 'k k x s f s
03) A normalization is needed, 1 is common
4) A -step method has 2 1 free coefficientsk k
How does one pick good coefficients?
Want the highest accuracy
0 0
ˆ ˆ ,k k
l j l jj j l j
j j
x t f x u t
Multistep Methods Definition and Observations
Definition: A finite-difference method for solving initial value problems on [0,T] is said to be
convergent if given any A and any initial condition
0,
ˆmax 0 as t 0lT
lt
x x l t
exactx
tˆ computed with
2lx
ˆ computed with tlx
Multistep Methods – Convergence Analysis
Convergence Definition
Definition: A multi-step method for solving initial value problems on [0,T] is said to be order p
convergent if given any A and any initial condition
0,
ˆmaxpl
Tl
t
x x l t C t
0for all less than a given t t
Forward- and Backward-Euler are order 1 convergentTrapezoidal Rule is order 2 convergent
Multistep Methods – Convergence Analysis
Order-p Convergence
Multistep Methods – Convergence Analysis
Two types of error
made.been haserror previous no assuming ,solution theof value
exact theand ˆ valuecomputed ebetween th difference theis
at methodn integratioan of (LTE)Error Truncation Local The
1
1
1
)x(t
x
t
l
l
l
exactly.known iscondition
initial only the that assuming ,solution theof value
exact theand ˆ valuecomputed ebetween th difference theis
at methodn integratioan of (GTE)Error Truncation Global The
1
1
1
)x(t
x
t
l
l
l
• For convergence we need to look at max error over the whole time interval [0,T]– We look at GTE
• Not enough to look at LTE, in fact:– As I take smaller and smaller timesteps t, I would
like my solution to approach exact solution better and better over the whole time interval, even though I have to add up LTE from more timesteps.
Multistep Methods – Convergence Analysis
Two conditions for Convergence
1) Local Condition: One step errors are small (consistency)
2) Global Condition: The single step errors do not grow too quickly (stability)
Typically verified using Taylor Series
All one-step methods are stable in this sense.
Multistep Methods – Convergence Analysis
Two conditions for Convergence
Definition: A one-step method for solving initial value problems on an interval [0,T] is said to be consistent if for any A and any initial condition
1ˆ0 as t 0
x x t
t
One-step Methods – Convergence Analysis
Consistency definition
Forward-Euler definition
22
2
0()
2 0
dxtdxxtxt
dtdt
Expanding in about zero yields t
1ˆ00 xxtAx
Noting that (0)(0) and subtractingdxAx
dt
22
12 ˆ
2
tdxxxt
dt
Proves the theorem if
derivatives of x are bounded
0,t
One-step Methods – Convergence Analysis
Consistency for Forward Euler
Forward-Euler definition
1l
xltxlttAxlte
Expanding in about yields tlt 1
ˆˆ ˆlllxxtAx
where is the "one-step" error bounded byle
2
2
[0,]2 , where 0.5maxl
T
dxeCtC
dt
One-step Methods – Convergence Analysis
Convergence Analysis for Forward Euler
ˆ Define the "Global" errorll
Exxlt
1ˆˆ 1lllxxltItAxxlte
Subtracting the previous slide equations
Taking norms and using the bound on le
1 lll
EItAEe
2 1
21
ll
l
EItAECt
tAECt
One-step Methods – Convergence Analysis
Convergence Analysis for Forward Euler
10
1,0, If 0ll
uubu
To prove, first write as a power series and sum l
u
1
0
111
11
l lj l
j
ubb
Then l
leub
A lemma bounding difference equation solutions
One-step Methods – Convergence Analysis
A helpful bound on difference equations
To finish, note (1)(1) ll
ee
1111
11
lllle
ubbb
2 1
1ll
EtAECt
b
Mapping the global error equation to the lemma
One-step Methods – Convergence Analysis
A helpful bound on difference equations
2 1
1ll
EtAECt
b
Applying the lemma and cancelling terms
2
ltAe
CttA
Finally noting that , ltT
0, maxAT l
lL
CEet
A
One-step Methods – Convergence Analysis
Back to Convergence Analysis for Forward Euler
0, maxAT l
lL
CEet
A
• Forward-Euler is order 1 convergent• The bound grows exponentially with time interval• C is related to the solution second derivative• The bound grows exponentially fast with norm(A).
One-step Methods – Convergence Analysis
Observations about Convergence Analysis for FE
Summary
• Transient Analysis of dynamical circuits– i.e., circuits containing C and/or L
• Examples
• Solution of Ordinary Differential Equations (Initial Value Problems – IVP)– Forward Euler (FE), Backward Euler (BE) and
Trapezoidal Rule (TR)– Multistep methods– Convergence