7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 1/16
Applications of Stochastic Differential Equations (SDE)
• Modelling with SDE: Ito vs. Stratonovich (6.1 & 6.2)
• Parameter Estimation (6.4)
• Optimal Stochastic Control (6.5)
• Filtering (6.6)
1
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 2/16
Ito SDE or Stratonovich SDE? (6.1 + 6.2)
• From a purely mathematical viewpoint both the Ito and
Stratonovich calculi are correct;
• Ito or Stratonovich? This question can only be discussed
in the context of a particular application;
• Ito SDE is appropriate when the continuous approximationof a discrete system is concerned (many examples in the
biological sciences);
• Stratonovich SDE is appropriate when the idealization of asmooth real noise process is concerned (many examples in
engineering and the physical sciences).
2
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 3/16
Idealization of smooth noise processesExample: a random differential equation
dX (n)t = aX
(n)t dt + bX
(n)t dR
(n)t
where R(n)t is the piecewise differentiable linear interpolation of a
Wiener process W t on a partition 0 = t(n)0 < t
(n)1 < ... < t
(n)n = T
−50
−40
−30
−20
−10
0
10
R(n)t
Wt
δn
t1
(n)t2
(n)tn+1
(n)tn
(n)
3
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 4/16
−→ The solution by classical calculus
X (n)t = X t0 expa(t − t0) + b(R
(n)t − R
(n)t0 )
When n → ∞ and max1≤j≤n
t(n)j+1 − t
(n)j
−→ 0
• The noise process −→ W t
• The solution −→ X t = X t0 exp (a(t − t0) + b(W t − W t0));
• The random DE
−→ a Stratonovich SDE
dX t = aX tdt + bX t ◦ dW t
4
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 5/16
Continuous approximation of discrete systems
• Example 1: a random walk
X (N )k+1 = X
(N )k + 1√
N ξ k
at times t(N )k = k
N , k = 0, 1, ..., N , where ξ k take +1 or −1
with probability 12
N →∞−→ a standard Wiener process in [0, 1]
(Central Limit Theorem);
• Example 2: a random walk
X (N )k+1 = X
(N )k + aN
X
(N )k
1
N + bN
X
(N )k
1√ N
ξ k
5
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 6/16
with consistency conditions such as
limn→∞
N E X (N )k+1 − X (N )
kX (N )
k = x = limN →∞
E (aN (x)) = a(x)
limn→∞
N E
X
(N )k+1 − X
(N )k
2
X
(N )k = x
= lim
N →∞E (b2N (x)) = b2(x)
limn→∞
N E X (N )k+1 − X (N )
k3X (N )
k = x = 0
N →∞−→ an Ito stochastic differential equation
dX t = a(X t)dt + b(X t)dW t
6
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 7/16
• Example 3: a population of 2N genes with two alleles a and A.
– Suppose that there are i genes of type a and 2N − i of
type A at the k-th generation;
– The probability of j genes of type a at the k + 1-th generation
is B(2N, i2N );
– Further all time intervals between two successive generationsare of length 1
N 2 ;
– Let’s define X (N )k = i
2N . When N goes infinity, we obtain a
process which is a solution of the Ito SDE
dX t = X t(1 − X t)dW t.
7
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 8/16
Parameter Estimation (6.4): An example
dX t = α · a(X t)dt + dW t
To determine a maximum likelihood estimate of α when the
trajectory X (t) of a solution process over the time interval
[0, T ] is given.
• the likelihood ratio
L(α, T ) = exp+1
2 α2 T 0
α2(X t)dt − α T 0
α(X t)dX t– the Euler scheme:
X i+1 = X i + αa(X i)∆ + ∆W i
for i = 0, 1, ..., N - 1 where ∆ = T N ;
– ∆W 0, ..., ∆W N −1 are i.i.d. and ∆W i ∼ N (0; ∆) ∀i
– ∆X 0, ..., ∆X N −1 are i.i.d. and ∆X i ∼ N (0; ∆) ∀i
8
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 9/16
– the Radon-Nikodyn derivation of the process X t w.r.t. W t
P (∆X 0, ..., ∆X N −1)
P (∆W 0, ..., ∆W N −1)
N →∞−→ L(α, T )
• the maximum likelihood estimator
α̂(T ) =
T 0 a(X t)dt T 0 a2(X t)dt
If E T 0 a2(X
t)dt <
∞, the SDE above has a stationary
solution with density ¯ p −→
α̂(T ) − α = T
0 a(X t)dW t
T 0 a2(X
t)dt
· 1/T
1/T T →∞−→ 0
a2(x)¯ p(x)dx = 0
the Central Limit Theorem T →∞−→
T 1/2(α̂(T )
−α)
∼ N 0,
1
a2(x)¯ p(x)dx9
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 10/16
Optimal Stochastic Control
• problem formulation
– state X ∈ d:
dX t = a(t, X t, u)dt + b(t, X t, u)dW t
– control parameter u ∈ k:
∗ u = u(t, X t) (Markov feedback control)∗ u = u(t, ω) (open-loop control)
– the cost functional for Markov feedback controls
J (s, x; u) = E K (τ, X τ ) + τ s
F (t, X t, u)dtX s = xwhere K and F are given functions and τ is a specified
Markov time
10
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 11/16
• the Hamilton-Jacobi-Bellman (HJB) equation
The minimum cost functional
H (s, x) = minu(·) J (s,x,u(·))
The HJB equation
minu∈k{F (s,x,u) + LuH (s, x)} = 0
with the final time condition H (T, x) = K (T, x)
where τ = T and
Lu = ∂
∂s +
di=1
ai(s,x,u) ∂
∂xi+
1
2
di,j=1
Di,j(s,x,u) ∂ 2
∂xi∂xj
with D = bb
11
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 12/16
• the linear-quadratic regulator problem– K (T, X T ) = X T RX T ( R ∈
d×d, symmetric and positive
semi-definite)
– F (t, X t, u) = X
t C (t)X
t + uG(t)u ( C ∈
d×d, symmetric and
positive semi-definite, G ∈ k×k, symmetric and positive definite)
– a(t, X t, u) = A(t)X t + M (t)u ( A ∈ d×d
M ∈ d×k)
– b(t, X t, u) = σ(t) ( σ ∈ d×m)
→ a guess solution
H (s, x) = xS (s)x + a(s)
with the final time condition
S (T ) = R and a(T ) = 0;
→ the left side of HJB
12
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 13/16
x
S
(s)x + a
(s) + x
C (s)x + u
G(s)u
+(A(s)x + M (s)u)(S (s)x + S (s)x) +ij
(σσ)ijS ij
→ the minimizer of the left side of HJB
u = −G−1(s)M S (s)x
→ 0 = a
(s) + tr(σσ
S ) =0
+
xS (s) + A(s)S + SA(s)
−SM (s)G(s)−1M (s)S
−C (s)
=0 a Riccati type equationx
• when situations of partial information occur, linear stochastic
control = linear filtering + deterministic control
13
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 14/16
Filtering• linear filtering
dX t = AX tdt + BdW t
dY t = HX tdt + ΓdW ∗t
−→ the Kalman-Bucy filter
The estimate X̂ t = E (X t|Y t) satisfies the SDE
d X̂ t =
A − SH (ΓΓ)−1H
X̂ tdt + SH (ΓΓ)−1dY t
where the error covariance S (t) satisfies the matrix Riccati
equation
dS
dt = AS + SA + BB − SH (ΓΓ)−1HS
14
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 15/16
• non-linear filtering
the Fokker-Planck equation in operator form
∂p
∂t
=
L∗ p
and the observation process
dY t = h(X t)dt + dW ∗t
−→the conditional probability densities of X t given Y tare given by
¯ p(t, x) =
Qt(x) Qt(x)dx
where the unnormalized densities Qt(x) satisfy
the Wong-Zakai equation
dQt(x) = LQt(x)dt + h(x)Qt(x)dY t
15
7/21/2019 Applications of Stochastic Differential Equations
http://slidepdf.com/reader/full/applications-of-stochastic-differential-equations 16/16
Further Reading: Venkatarama Krishnan, Nonlinear Filtering and
Smoothing – An Introduction to Martingales, Stochastic Integrals
and Estimation, Dover 2005.
16