BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 1
BEA400 Microeconomics
Lecture 11
Module 5: Choice Over Time with Uncertainty
Lecture 11: Stochastic Processes, Ito’s Lemma and Stochastic Optimal Control
• Stochastic Processes
The Pure Weiner Process or Brownian Motion Scaling the variance of the Process Weiner Process or Brownian Motion with Drift Markovian Process General Weiner Process or Brownian Motions
• Stochastic Integration • Ito’s Lemma
Demonstration of Ito’s Lemma Some Properties Ito’s Lemma Derivation of Ito’s Lemma
• Applications of Ito’s Lemma
A Stochastic Rate of Inflation Real Rate of Return with Stochastic Inflation Black-Scholes Option Pricing
• Stochastic Optimal Control Theory
Hamilton-Jacobi-Bellman (HJB) Equation Optimal Extraction of an Uncertain Non-Renewable Resource Consumption-Savings Decision with Risky Income Consumption-Savings Decision with optional Risky Asset
• Logarithmic Utility • HARA Utility
Reading: I haven’t found a text that I’m happy with for this part of the course, so lecture notes should suffice. You could try: Malliaris, A.G. and W.A Brock, Stochastic Methods in Economics and Finance, North-Holland, 1982. Chapters 2, 3 and 4.
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 2
Stochastic Processes 1. The Pure Weiner Process or Brownian Motion A Pure or Basic Weiner Process or Brownian Motion refers to a continuous time stochastic process where a random variable at time t, ( ) = tx t x which evolves over a small interval of time dt , according to a stochastic differential equation, =dx z dt
where z is a standard normal with mean zero and standard deviation of 1.
Since z is normally distributed it follows that dx will be normally distributed with an
expected value of zero and standard deviation of dt . Thus x is changing randomly
by dx since it depends on z dt
( )~ 0,dx N dt
That is dx is be normally distributed with ( ) 0=E dx and ( ) =Var dx dt (and
( ) =SD dx dt )
The reason that dx is scaled with dt is that any other choice for the magnitude of
dx would lead to a problem that is either meaningless or trivial when we consider
what happens at the limit when 0dt → .
Also if dx were not scaled in this way, the variance of the random walk would have a
limiting value of 0 or ∞ .
Now let us consider dx over two very small but consecutive time periods.
Consider two time periods 0t and 1t , such that 0 1= +T t t with corresponding values
of x, ( )0 0=x t x and ( )1 1=x t x and changes ( )0 0=dx t dx and ( )1 1=dx t dx
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 3
With the change over both periods as 0 1= +Tdx dx dx then the expected value of dx
over both periods is
( ) ( ) ( )0 1 0 0 0= + = + =TE dx E dx E dx
and variance
( ) ( ) ( ) ( )0 1 0 12 ,= + +TVar dx Var dx Var dx Cov dx dx
If we assume that the values of z are independent over time then dx in any period of
time is independent of dx in all other periods and the variance is
( ) ( ) ( )0 1
0 1
= +
= +
=
TVar dx Var dx Var dxdt dtdT
More generally over a long time period T,
( ) 0=TE dx and variance ( ) = = ∑TVar dx dT dt and ( ) = ∑SD dx dt
that is the variance of the Weiner process over the time period T, is equal to the sum
of the changes in time periods.
This allows us to consider the value x over discrete changes in time 1=dt
Consider a starting point 0s = then at time T, after T changes of 1=dt then the
variance of Tdx
( )0 0
1= =
= = = =∑ ∑T T
Ts s
Var dx dT dt T
At the starting point 0s = , so with 0=sx x the change in x over T, Tdx is
0= −T Tdx x x
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 4
and
0= +T Tx x dx
If the process starts at zero, 0 0= =sx x , then the value of x in an period T > s > 0, is
given by
0= +
=T T
T
x x dxdx
Thus from
( )~ 0,dx N dt
( ) 0=E dx and variance ( ) =Var dx dt ( ) =SD dx dt
with the assumptions of 0s = , 0 0= =sx x and 1=dt
we obtain that x in any future period T is
( )~ 0,Tx N T
( ) 0=TE x and variance ( ) =TVar x T ( ) =TSD x T
and for any time period in between s and T, t ( )0,tx N t
2. Scaling the variance of the Basic Weiner Process The Basic Weiner Process can be enhanced to scale the standard deviation of the
random variable tx by σ
σ=dx z dt .
In which case ( )~ 0,tx N tσ
Note that σ can be modelled to depend on time, t, and the value of tx such that
( ),σ σ= tt x
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 5
3. Weiner Process or Brownian Motion with Drift The Basic Weiner Process can also be enhanced by adding a drift or growth term, µ
to the stochastic differential equation.
µ σ= +dx dt z dt In which case ( )~ ,tx N t tµ σ
Note that as with σ , µ can be modelled to depend on time, t, and the value of tx
such that ( ),µ µ= tt x
4. Markovian Process A Markovian process is one where the probability values of future values of x
conditional on being at time t, only depend upon the current value of x and no other
information.
( )( ) ( ) ( )( ), , tP x t t P x t t x t x= =
While this might seem rather restrictive it can be modified to allow a fixed amount of
past information. The General Weiner Process described below is an example of a
Markovian process.
5. General Weiner Processes or Brownian Motions General Weiner Process or Brownian Motion refers to a continuous time stochastic
process where a random variable ( ) = tx t x evolves over time, t, according to some
stochastic differential equation:
( ) ( ), ,µ σ= +t tdx t x dt t x z dt
This also called an Ito Process.
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 6
Stochastic Integration Stochastic Integration was developed by Ito(1944) who generalised the stochastic
integral first introduced by Weiner (1923)
Consider for which a stochastic process ( ), ,dx t x z with deterministic component and
a random component which follows a Standardised Weiner process z .
( ) ( ), ,µ σ= +t tdx t x dt t x z dt
Stochastic Integration transforming the above into an integral equation we get,
( ) ( ) ( )= + +∫ ∫0 0
0 , ,t t
t s sx x s x ds s x z dsµ σ
We have encountered integrals of the form ( )0
,µ∫t
ss x ds before but how do we cope
with ( )0
,σ∫t
ss x z dt which does not exist? Ito’s Lemma!
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 7
Ito’s Lemma Suppose x follows a Brownian Motion with
( ) ( ), ,µ σ= +t tdx t x dt t x z dt and
( )22 2,σ= tdx t x z dt as ( )2 0dt = and 0× =dz dt and ( )2dz dt=
If the dynamics of ( )x t can be written by an Ito Process, then the dynamics of well-
behaved function of ( )x t that describe its distribution, ( )= ,t ty F t x will also be described by an Ito Process. Ito’s Lemma gives: 21
2x t xxdy F dx F dt F dx= + +
( )212t x xx xdy F F F dt F dzµ σ σ= + + +
and expectation
[ ] ( ) [ ]( )
212
212
/ /t x xx x
t x xx
E dy dt F F F F E dz dt
F F F
µ σ σ
µ σ
= + + +
= + + as [ ] 0E dz =
[ ] ( )
( )( ) ( )
( )
212
212
22 2 2 2 2 2 21 12 2
22 2 2 2 212
t x xx x
t x xx
t x xx x t x xx x
t x xx x
E dy E F F F dt F dz
F F F dt
E dy E F F F dt F F F F dzdt F dz
F F F dt F E dz
µ σ σ
µ σ
µ σ σ µ σ σ
µ σ σ
⎡ ⎤= + + +⎣ ⎦
= + +
⎡ ⎤⎡ ⎤ = + + + + + +⎣ ⎦ ⎢ ⎥⎣ ⎦
⎡ ⎤= + + + ⎣ ⎦
[ ] 2 2 2
2 2
x
x
Var dy F E dz
F
σ
σ
⎡ ⎤= ⎣ ⎦=
as 2 1E dz⎡ ⎤ =⎣ ⎦
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 8
Demonstration of Ito’s Lemma Suppose x follows a Brownian Motion with
0 0dx xdt xdzµ σ= + and 2 2 2 2 2 2
0 0dx x dz x dtσ σ= = as ( )2 0dt = , 0dz dt× = and ( )2dz dt= Consider ( )lny x t= Ito’s Lemma 21
2x t xxdy F dx F dt F dx= + +
212 2
1 1dy dx dxx x
⎛ ⎞= + −⎜ ⎟⎝ ⎠
Note that 2 2 2 2 2 2
0 0dx x dz x dtσ σ= = ,
( ) 2 2
0 0 02
20 0 0
1 12
12
dy xdt xdz x dtx x
dt dz
µ σ σ
µ σ σ
= + −
⎛ ⎞= − +⎜ ⎟⎝ ⎠
which can be written
( ) ( ) ( )210 0 02
0 0
0t t
y t y dt dzµ σ σ= + − +∫ ∫
which gives
( ) ( ) ( ) ( )210 0 020y t y t z tµ σ σ= + − +
Now substituting back ( )lny x t= by using ( )ye x t= and ( ) ( )00 yx e= gives
( ) ( ) ( ) ( )210 0 020 expx t x t z tµ σ σ⎡ ⎤= − +⎣ ⎦
Some Properties Ito’s Lemma
( )1 1 2 1 1 1 2 20 0 0
T T T
a a dz a dz a dzσ σ σ σ+ = +∫ ∫ ∫
( ) ( ) ( )2
2
0 0
T T
E t dz t E t dtσ σ⎡ ⎤
⎡ ⎤=⎢ ⎥ ⎣ ⎦⎣ ⎦∫ ∫ if ( ) 2
0
T
E s dsσ⎡ ⎤ < ∞⎣ ⎦∫
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 9
Derivation of Ito’s Lemma (Not examinable) A Taylor series expansion of ( ) ( )( ),y t F t x t= around 0y gives dy The expected value of dy can be computed (noting that [ ] 0E dz = ) as [ ] ( ) ( ) ( ) ( )21
0 0 0 0 02 higher order termst t tE y y F x E x x F x E x x⎡ ⎤′ ′′⎡ ⎤− = − + − +⎣ ⎦ ⎣ ⎦
since ( )0tE x x tµ⎡ ⎤− =⎣ ⎦ and ( ) [ ]⎡ ⎤− = − + = +⎣ ⎦
2 2 2 2 2 20 0t tE x x V x x t t tµ σ µ
[ ] ( ) ( )( )2 2 21
0 0 02 higher order termstE y y F x t F x t tµ σ µ′ ′′− = + + + with the variance of 0ty y− ignoring higher order terms and using ( )2 0dt = ,
( )2dz dt= and 0dz dt× =
so by setting t =1 allows the change in y to be given by
( ) ( ) ( )2
0 0 02dy F x F x dt F x dzσµ σ
⎛ ⎞′ ′′ ′= + +⎜ ⎟
⎝ ⎠
or if F also depends on t then
( )21
2t x xx xdy F F F dt F dzµ σ σ= + + +
( ) ( ) ( )212t x xx xy t F F F t F z tµ σ σ= + + +
and finally dividing by dt and taking the expectation (noting [ ] 0E dz = ) gives
[ ] ( ) [ ]( )
212
212
/ /t x xx x
t x xx
E dy dt F F F F E dz dt
F F F
µ σ σ
µ σ
= + + +
= + +
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 10
Applications of Ito’s Lemma A Stochastic Rate of Inflation
Suppose the rate of inflation is given by a basic Brownian Motion 0 0dP dt dzP
µ σ= +
We can easily find the mean and variance of such a series,
[ ] [ ]µ σ
µ
⎡ ⎤ = +⎢ ⎥⎣ ⎦⎡ ⎤ =⎢ ⎥⎣ ⎦
0 0
0
dPE E dt E dzP
dPE dtP
The average proportionate change in prices is expected to be µ0 multiplied by the time period over which it is considered. This can result can be re-expressed as the
[ ]
µ
µ
π µ
⎡ ⎤ =⎢ ⎥⎣ ⎦⎡ ⎤
=⎢ ⎥⎣ ⎦
=
&
&
0
0
0
/dP dtEP
dPEP
E
Thus the expected continuos rate of change in inflation is µ0
[ ] [ ]{ }
[ ] [ ]
µ µ σ σ µ
σ
⎡ ⎤ ⎧ ⎫⎡ ⎤ ⎛ ⎞ ⎡ ⎤= −⎢ ⎥ ⎨ ⎬⎜ ⎟⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎝ ⎠ ⎣ ⎦⎩ ⎭⎢ ⎥⎣ ⎦⎡ ⎤ ⎡ ⎤ ⎡ ⎤= + + −⎢ ⎥ ⎣ ⎦ ⎣ ⎦⎣ ⎦⎡ ⎤ = = = =⎢ ⎥⎣ ⎦
22
22 2 2 20 0 0 0 0
2 20
as E 0, E 0 and
dP dP dPVar E EP P P
dPVar E dt E dt dz E dz E dtP
dPVar dt dz dt dz dt dzP
The variance in the proportionate change in prices is σ 20 multiplied by the time period
over which it is considered.
[ ]
[ ]
σ
σπ
σπ
⎡ ⎤ =⎢ ⎥⎣ ⎦
=
= =
&
&
20
20
0
/
as
dP PVardt dt
Vardt
SD dz dtdz
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 11
While the Brownian Motion can readily provide the mean and variance for the
proportionate change in prices if we want to know something about the price level in
any particular period we will need to solve the differential equation
µ σ= +0 0dP Pdt Pdz . The presence of P, which depends on time, in the drift term is
no problem as we have learnt to solve such differential equations previously, but its
presence with the dz term requires the use Ito’s Lemma in order to solve for P(t).
Let ( ) ( )lny t P t= then by Ito’s Lemma 21
2x t xxdy F dx F dt F dx= + +
µ σ σ
µ σ σ
⎛ ⎞= + −⎜ ⎟⎝ ⎠
= + −
⎛ ⎞= − +⎜ ⎟⎝ ⎠
212 2
2 20 2
20 2
1 1
1 1 12
12
dy dP dPP P
Pdt Pdz P dtP P P
dt dz
All terms are independent of time and so can easily be integrated to provide y(t)
( ) 20 2
12
y t t zµ σ σ⎛ ⎞= − +⎜ ⎟⎝ ⎠
( ) ( ) ( ) ( ) ( )21
0 0 020 expy tP t e P t z tµ σ σ⎡ ⎤= = − +⎣ ⎦
Real Rate of Return with Stochastic Inflation If we also have an asset returning a nominal return of Q period that is growing at r then
dP Pdt Pdzµ σ= + and dQ rQdt=
Then defining QqP
= the real return then Ito’s lemma gives
2 2 2
2 22 2
1 22
q q q q q qdq dt dP dQ dP dQ dQdPt P Q P Q Q P
⎛ ⎞∂ ∂ ∂ ∂ ∂ ∂= + + + + +⎜ ⎟∂ ∂ ∂ ∂ ∂ ∂ ∂⎝ ⎠
Since ( )2 0dt = , ( )2dz dt= and , dQdP and 2dQ equal zero. Which allows
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 12
( )
( )
⎛ ⎞= − + + ⎜ ⎟⎝ ⎠
⎛ ⎞= − + + + ⎜ ⎟⎝ ⎠
= − + −
2 22 3
22
2
1 1 22
1
Q Qdq dP dQ P dtP P PQ QPdt Pdz rQdt dtP P P
dq r dt dzq
σ
µ σ σ
µ σ σ
since =QqP
Black-Scholes Option Pricing Suppose there are three assets. The first with mean rate of return
1pµ and SD 1pσ ,
the second with a mean rate of return 2pµ and SD
2pσ and the third a risk free asset that earns a rate of return r per period. The nominal value of the portfolio is
( ) ( ) ( ) ( ) ( ) ( )1 1 2 2 3v t n t p t n t p t np t= + +
( ) ( ) ( )1 11 1 1p pdp t p t dt p t dzµ σ= + and ( ) ( ) ( )
2 22 2 2p pdp t p t dt p t dzµ σ= +
( ) ( )( ) ( )
1 1 2 2
1 1 2 2
1 1 2 2 3
1 1 2 2 3
31 1 2 2
p p p p
p p p p
dv n dp n dp dnp
n p dt dz n p dt dz r np dt
npn p n pdv dt dz dt dz r dtv v v v
µ σ µ σ
µ σ µ σ
= + +
= + + + +
= + + + +
Defining 1w , 2w and 3w as the shares the three assets from of the portfolio’s value
so that 1 11
n pwv
= , 2 22
n pwv
= and 33 1 21npw w w
v= = − − so that the above equation
can be written
( ) ( ) ( )= + + + + −1 1 2 21 2 1 21-w p p p p
dv w dt dz w dt dz w r dtv
µ σ µ σ
Defining 1w , 2w for a riskless portfolio
1 21 2var 0p pdv w dz w dzv
σ σ⎡ ⎤ = + =⎢ ⎥⎣ ⎦
then 1 21 2 p pw dz w dzσ σ= − 2
1
1
2
p
p
ww
σσ
⇒ = −
( )1 21 2 1 2 1-wp p
dv w dt w dt r w dtv
µ µ= + + −
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 13
since dv rdtv
= for the riskless portfolio
( ) ( )( )1 21 2 0p pw r w r dtµ µ− + − = ( )( )
2
1
1
2
p
p
rww r
µ
µ
−⇒ = −
−
Combining the two
( )( )
−= − = −
−22
1 1
1
2
pp
p p
rww r
µσσ µ
and so
( ) ( )− −
=1 2
1 2
p p
p p
r rµ µ
σ σ
Thus the rate of return per unit of risk must be equal for the two risk assets. If asset 2 is stock option whose price is determined by ( )2 1,p F p t= and
1 11 p pdp dt dzµ σ= + Using Ito’s Lemma and that the rate of return per unit of risk must be equalised
( )= + +
= + + +
= + − +
1 1 1
1 1 1 1 1 1 1
1 1 1 1
212 1 12
2 212 12
2 212 1 12
t p p p
t p p p p p p p
t p p p p
dp F dt F dp F dp
dp F F F p dt F
dp F rp F rF F p
µ σ σ
σ
If the option can only be exercised at terminal time T with exercise price Tp the boundary condition ( )0, , -F s s s T t= = and ( ) [ ]1 1, max 0, TF p T p p= − Then Black and Scholes (1973) and Merton (1973) demonstrated that
( ) ( ) ( )
( )
−
−
−∞
= +
⎡ ⎤⎛ ⎞= + +⎢ ⎥⎜ ⎟⎜ ⎟⎢ ⎥⎝ ⎠⎣ ⎦
= −
= ∫
1
1
1
1
2
12
22 1 1
21
/ 2
, , , 1 e 2
11 ln2
2 1
12
rsp
p
T p
p
ys
p s p r p d d
pd r sp s
d d s
y e ds
σ φ φ
σ
σ
σ
φπ
Note that ( )yφ is the cumulative normal distribution
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 14
Stochastic Optimal Control Theory Previously our equation of motion was written,
( ), ,y f u y t•
=
Now consider a stochastic differential equation where ( ), ,y f u y t•
= is the
deterministic (non-random) component and ( ), ,t y u dzσ is the stochastic component
with ( ), ,t y uσ being a deterministic function and dz being a Brownian motion
increment.
The problem then becomes
( )( ) ( ) ( )( )u
0
, Max E , ,T
V y t T F t y t u t dt⎡ ⎤
= ⎢ ⎥⎣ ⎦∫
subject to
( ) ( ), , , ,dy f t y u dt t y u dzσ= +
The optimal value of the maximised problem can be written as
( ) ( ) ( )
( )( ) ( )
0
0 0
0 0 0 0u
0 0u
, Max , , , ,
Max , , ,
t t T
t t t
V y t E F t y u dt F t y u dt
E F t y u t t V y y t t
+∆
+∆
⎡ ⎤= +⎢ ⎥
⎢ ⎥⎣ ⎦⎡ ⎤= ∆ + + ∆ + ∆⎣ ⎦
∫ ∫
( ) ( ) ( ){ }u
, Max , , V y+ , tV y t E F t y u t y t≅ ∆ + ∆ + ∆
Note that following our previous section on Ito’s Lemma the above stochastic
differential equation can be rewritten as,
( ) ( ) ( )
212
212, , , , , ,
t y yy
t y yy y
dV V V dy V dy
V V f t y u dt V t y u dt V t y u dzσ σ
= + +
= + + +
Assuming that ( ),V y t is twice differentiable we expand the function on the right
around ( ),y t by Talyor Series expansion:
( ) ( ) ( ) ( )2 21 12 2V y+ , t V y, . .y yy t tt yty t t V y V y V t V t V y t h o t∆ + ∆ = + ∆ + ∆ + ∆ + ∆ + ∆ ∆ +
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 15
Inserting ( ) ( ), , , ,y f t y u t t y u zσ∆ = ∆ + ∆ , using ( )2 0t∆ = , ( )2z t∆ = ∆ and 0z t∆ ∆ = and then simplifying gives.
( ) ( ) ( )( ) ( ) ( ) ( ){ }212u
, Max , , V y, , , , ,t y yy yV y t E F t y t u t t t V V f t y u V t y u t V zσ σ⎡ ⎤≅ ∆ + + + + ∆ + ∆⎣ ⎦
Now tale the expectation of the above, the only stochastic term is z∆ and its
expectation is ( ) 0E z∆ = . Then subtract ( ),V y t from both sides and divide through
by t∆ and let 0t∆ → .
( ) ( )( ) ( ) ( ){ }212u
0 Max , , , , , ,t y yyF t y t u t V V f t y u V t y uσ= + + +
( ) ( )( ) ( ) ( )212u
Max , , , , , ,t y yyV F t y t u t V f t y u V t y uσ⎡ ⎤− = + +⎣ ⎦
Hamilton-Jacobi-Bellman (HJB) Equation This is the Hamilton-Jacobi-Bellman (HJB) equation of stochastic control theory
( ) ( )( ) ( ) ( )212u
Max , , , , , ,t y yyV F t y t u t V f t y u V t y uσ⎡ ⎤− = + +⎣ ⎦
Note that the co-state variable λ is yV so that ( )( ) ( ), ,
y
V y t TV t
y tλ
∂= =
∂, differentiating
with respect to the state variable gives ( )( )( )
( )( )
2
2
, ,yy y
V y t T tV
y ty t
λλ
∂ ∂= = =
∂∂ so that the HJB
equation can be written ( ) ( )( ) ( ) ( )212u
Max , , , , , ,t yV F t y t u t f t y u t y uλ λ σ⎡ ⎤− = + +⎣ ⎦ .
If the transformed Hamiltonian function ( ), , , yH H u y λ λ=% % is
( ) ( )( ) ( ) ( )212, , , , , ,yH F t y t u t f t y u t y uλ λ σ= + +%
Then u
MaxtV H− = %
Assuming 0Hu
∂=
∂
% can be solved for the optimal choice ( )* * , , yu u y λ λ= then
inserting into the HJB
( )* , ,t yV H y λ λ− = % .
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 16
since ( )* * , , yu u y λ λ=
( ) ( ), , , , , ,y ydy f t y dt t y dzλ λ σ λ λ= + Then since ( )* , , , yH f t yλ λ λ=%
( )λ σ= +% *dy H dt dz
It can also be shown by using the definition of yVλ = , the stochastic equation of motion for y the state variable and Ito’s lemma that
( )λ σ λ= − +% *y yd H dt dz then optimal HJB conditions are:
0Hu
∂=
∂
% Equation for optimal choice u.
( )λ σ λ= − +% *y yd H dt dz Equation of motion for the co-state variable.
( )λ σ= +% *dy H dt dz Equation of motion for the state variable y.
( )( ), 0T y Tλ = Endpoint restriction.
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 17
Consumption-Savings Decision with Risky Income
Suppose that each period t the consumer receives an income ( )y t per period t it
grows at constant rate of Ydy µ= but with random component Y ydzσ where dz is a
standardised Weiner process.
y ydy ydt ydzµ σ= +
We could include y as a state variable, with a corresponding co-state variable and
apply the rules of Stochastic Optimal Control. However it will be much simpler for us
to use Ito’s Lemma on dy first (see stochastic rate of inflation example) and replacing
y with:
( ) ( ) ( ) ( )2120 exp y y yy t y t z tµ σ σ⎡ ⎤= − +⎣ ⎦
so that the wealth evolution now explicitly includes the variance of income.
( ) ( ) ( )( ) ( ) ( ) ( ) ( )
( ) ( ) ( ) ( )•
= + −
⎡ ⎤= + − + −⎣ ⎦⎡ ⎤ ⎡ ⎤= + − −⎢ ⎥ ⎣ ⎦⎣ ⎦
212
212
0 exp
0 exp
y y y
y y
dw r w t dt y t dt c t dt
r w t dt y t z t dt c t dt
E w r w t y t c t
µ σ σ
µ σ
since ( )⎡ ⎤ =⎣ ⎦ 0E z t
with terminal conditions ( ) ( )00 , w 0w w T= = .
Proceeding in this manner we can simply use the non-stochastic version of the
Hamiltonian, which for this problem is
( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( )δλ λ µ σ σ− ⎡ ⎤⎡ ⎤ = + + − + −⎣ ⎦ ⎣ ⎦21
2, , , ln 0 expty y yH t w t c t e c t t rw t y t z t c t
with Hamiltonian Conditions
H1: ( ) ( ) ( )−∂
= − =∂
1 0tH e tc t c t
δ λ ⇒ ( ) ( )−=
1 tc t et
δ
λ
H2: ( ) ( ) ( ) dw tH w r w t c t
dtλ∂
= = = −∂
& ⇒
( ) ( ) ( )21
2
0 0y y r trt rtw t e w y e dt e c t dt
µ σ⎡ ⎤− −− −⎣ ⎦= + −∫ ∫
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 18
H3: ( )( ) ( )d t H r t
dt w tλ
λ λ∂= = − = −
∂& ⇒ ( ) ( )0 rtt eλ λ −=
( )( ) ( )
( )
2120
0 212
0
1
1
y y r T rT
y y
T
yw e w T er
ce
µ σ
δ
δµ σ
⎡ ⎤− − −⎣ ⎦
−
⎛ ⎞⎛ ⎞⎜ ⎟+ − −⎜ ⎟⎜ ⎟− − ⎝ ⎠⎝ ⎠=−
to go with ( ) ( )−= 0r tc t c e δ and
( ) ( ) ( )( )21
20 00 21
2
1 1 y y r trt rt t rt
y y
c yw t w e e e e er
µ σδ
δ µ σ
⎡ ⎤− −− ⎣ ⎦⎛ ⎞= − − + −⎜ ⎟− − ⎝ ⎠
to give the solution
for the control variable, consumption and state variable wealth.
We can see that income growth adds to initial consumption in that it effectively
reduces the rate of discounting on future income by its growth.
While income uncertainty reduces initial consumption in that it effectively increases
the rate of discounting on present value of future income due to its uncertainty.
If we had proceeded without solving for dy first, we would have had to use the
stochastic version of the Hamiltonian, which in this problem would be
( ) ( ) ( ) ( ) ( ) ( )( ) 2 212, , , , lnt
w y w yH t w t c t e c t t rw t c t yδλ λ λ µ λ σ−⎡ ⎤ = + + − +⎣ ⎦
and its first order conditions.
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 19
Consumption-Savings Decision with Optional Risky Asset Logarithmic Utility
Ignoring income for the moment, assume that there is a risk asset with rate of return
ar and SD of aσ then evolution of wealth becomes.
( ) ( )( ) ( )( ) ( ) ( )
1 a a
a a
dw w t a r ar c t dt aw dz
rw t a r r w c t dt aw dz
σ
σ
⎡ ⎤= − + − +⎣ ⎦⎡ ⎤= + − − +⎣ ⎦
The Maximum Value Function is
( )( )( )
( )0
, lnT t
c tV y t T Max e c t dtδ−= ∫
so that ( ) ( )( ) ( ) ( )212u
Max , , , , , ,t y yyV F t y t u t V f t y u V t y uσ⎡ ⎤− = + +⎣ ⎦ the stochastic optimal
control problem is,
( ) ( )( ) ( )( ) ( ) ( ) ( ){ }− ⎡ ⎤− = + + − − +⎣ ⎦
212,
Max lntt w a a wwc t a t
V e c t V r a r r w t c t aw Vδ σ
Or replacing wV λ= and defining the stochastic Hamiltonian as
( ) ( ) ( ) ( ) ( )( )( ) ( ) ( ) ( ) ( )( )212, , , , , lnt
w a w aH t w t c t a t e c t r a t r r w t c t a t w tδλ λ λ λ σ− ⎡ ⎤⎡ ⎤ = + + − − +⎣ ⎦ ⎣ ⎦%
( ) ( )[ ]{ }= %
,Max , , , , , wc t a t
V H t w c aδ λ λ
Examine the Hamiltonian Conditions (removing time (t) from the variables for clarity).
H1A: 2 2 0a w aH r w rw awa
λ λ λ σ∂= − + =
∂
% ⇒
( )2* a
w a
r ra
wλλ σ
−= −
H1: 1 0tH ec c
δ λ−∂= − =
∂
% ⇒ 1 tc e δ
λ−=
H1 implies 1 0tH ec c
δ λ−∂= − =
∂
%
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 20
////
c w
c w
c ww
cww
c ww
c w
cc ww
c w
F V
F Vw w
c F Vw c
F wVc c
F Vc w wwc F V c c
F V w wc wF V c c
λ= =
∂ ∂=
∂ ∂∂ ∂
=∂ ∂
∂ ∂=
∂ ∂∂ ∂
− = −∂ ∂
∂− = −
∂
where ww w
w
Vw w
Vλλ
− = − is the Arrow-Pratt Measure Relative Risk Aversion RRA of
wealth
//
w wc c
∂∂
the elasticity of wealth for consumption
cc c
c c
F F ccF c F
∂− = −
∂ the intertemporal elasticity of substitution
Thus the intertemporal elasticity of substitution is equal to the product of Arrow-Pratt
Measure Relative Risk Aversion for wealth and the elasticity of wealth for
consumption.
Or
Thus the Arrow-Pratt Measure Relative Risk Aversion is equal to the product of and
the elasticity of consumption for wealth and the intertemporal elasticity of substitution.
Since ww w
w
Vw w
Vλλ
− = − the Arrow-Pratt Measure Relative Risk Aversion the optimal
proportion invested in the risky asset is
( )( )
( )2 2
1* a a
w a a
r r r ra
RRA wwλλ σ σ
− −= − =
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 21
Thus the proportion invested in the risky asset is inversely related to the Relative
Risk Aversion of wealth and positively related to assets return over the risk free asset
and negatively to its variance. Thus if RRA is constant then a* will also be constant
over life.
/ /
cww
cc c ww w
F wVc c
F F V V
∂ ∂=
∂ ∂= −
For ( )lnc t the intertemporal elasticity of substitution is
2
1 1cc c
c c
F F c cc cF c F c
−
−
∂ −− = − = − × =
∂
( ) /1/
w wRRA wc c
∂=
∂ since 1w
c∂
= −∂
and so ( )1 c RRA ww
= − thus the Arrow-
Pratt Measure Relative Risk Aversion for logarithmic utility is ( ) wRRA wc
= − and
Returning to the other Hamiltonian Conditions
H2: ( ) ( ) ( )( ) ( ) ( )a
dw tH w rw t a t r r w t c tdtλ
∂= = = + − −
∂&
H3: ( )
( ) ( ) ( )( )( ) ( ) ( )2 2a w a
d t H t r a t r r a t w tdt w tλ
λ λ λ σ∂= = − = − + − −
∂&
Inserting H1A:( )
2* a
w a
r ra
wλλ σ
−= − into H2 and H3
H2: ( ) ( )( )
( ) ( ) ( ) ( )
( ) ( )( ) ( )
2
2
2
aa
w a
a
w a
t r rw rw t r r w t c t
w t
t r rrw t c t
λλ σ
λλ σ
−= − − −
−= − −
&
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 22
Since
( ) ww
w
w
w
VRRA w w
vw wc
c
λλ
λλ
= −
− = −
=
( ) ( ) ( )2
2 1a
a
r rw rw t c t
σ
⎛ ⎞−⎜ ⎟= − +⎜ ⎟⎝ ⎠
&
Now H3:
H3:
( ) ( )( )( ) ( ) ( )( )
( ) ( )
( ) ( ) ( )( )
( )( )
( )
( )
2
22 2
2 2 2 2
2 2
a aa w a
w a w a
a a
ww a a
t r r t r rt r r r w t
w t w t
t r r t r rt r
w tw t
r t
λ λλ λ λ σ
λ σ λ σ
λ λλ
λλ σ σ
λ λ
⎛ ⎞ ⎛ ⎞− −= − + − − −⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟
⎝ ⎠ ⎝ ⎠
− −= − + −
= −
&
&
Thus ( ) ( )0 rtt eλ λ −= and H1 is now ( ) ( )−=0
1 r tc t e δ
λ so ( ) ( )
100
cλ
= thus
( ) ( ) ( )0 r tc t c e δ−=
( ) ( ) ( ) ( )2
20 1r t a
a
r rw rw t c e δ
σ−
⎛ ⎞−⎜ ⎟= − +⎜ ⎟⎝ ⎠
&
Denote ( )2
2 1a
a
r rσ
⎛ ⎞−⎜ ⎟= +⎜ ⎟⎝ ⎠
l
( ) ( )0
r tw rw t c e δ−− = −& l
( )( ) 0rt te w rw t c e δ− −− = −& l
Using ( )( )
( )= +∫ '
f xf x ee dx A
f x to integrate the RHS 1
0
1ts t
s
e ds e Aδ δ
δ− −
=
= − +∫ gives
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 23
( )( )
( )
( )
00
0 1 2
0
1
1
trt s
s
rt t
rt t
e w rw t c e ds
w t e c e A A
w t e A c e
δ
δ
δ
δ
δ
− −
=
− −
− −
− = −
= + −
= +
∫ ∫& l
l
l
setting t=0
( ) 010A w cδ
= − l
( ) ( ) ( )010 1rt tw t e w c e δ
δ− −= − −l
setting t = T
( ) ( ) ( )( )( )
( )
0
00
10 1
1 1
rT T
rT
T
w T e w c e
w w T ec
e
δ
δ
δ
δ
− −
−
−
= − −
−=
−
l
l
Note 20, 0a ar σ
∂ ∂> <
∂ ∂l l thus initial consumption 0 00, 0
a a
c cr σ
∂ ∂< >
∂ ∂
Consumption growth is the same ( ) ( ) ( )0 r tc t c e δ−=
While the PV of wealth ( ) ( ) ( )010 1rt tw t e w c e δ
δ− −= − −l will be higher for higher ar
and lower 2aσ .
Since the intertemporal elasticity of substitution (Arrow-Pratt degree of relative risk
aversion for consumption) is constant and equal to one. In ( ) wRRA wc
= − there is
no parameter to describe degree of degree of risk aversion for wealth with the
logarithmic utility function other than the wealth and consumption.
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 24
Hyperbolic Absolute Risk Aversion (HARA) class of utility function
However if we choose a different functional form for utility the picture is different. If
we consider a Hyperbolic Absolute Risk Aversion (HARA) class of utility function.
( )( ) ( )1u c t c t θ
θ=
For this HARA utility function the intertemporal elasticity of substitution is
( ) 2
1
11c cc
c c
cF Fc c cc F F c
θ
θ
θθ
−
−
−∂− = − = − × = −∂
( ) /1/
w wRRA wc c
θ ∂− =
∂ since 1w
c∂
= −∂
and so ( )1 c RRA ww
θ− = − thus the
Arrow-Pratt Measure Relative Risk Aversion for simple HARA utility is
( ) ( )1 wRRA wc
θ= − −
The corresponding stochastic Hamiltonian is
( ) ( ) ( ) ( ) ( ) ( )( )( ) ( ) ( ) ( ) ( )( )212
1, , , , , tw a w aH t w t c t a t e c t t r a t r r w t c t a t w tθδλ λ λ λ σ
θ− ⎡ ⎤⎡ ⎤ = + + − − +⎣ ⎦ ⎣ ⎦
%
H1A: 2 2 0a w aH r w rw awa
λ λ λ σ∂= − + =
∂
% ⇒
( )2* a
w a
r ra
wλλ σ
−= −
H1: 1 0tH e cc
δ θ λ− −∂= − =
∂
% ⇒
11
1 tc eθ
δ
λ
−
−⎛ ⎞= ⎜ ⎟⎝ ⎠
H3:
( ) ( )( )( ) ( ) ( )( )
( ) ( )
( ) ( ) ( )( )
( )( )
( )
( )
2
22 2
2 2 2 2
2 2
a aa w a
w a w a
a a
ww a a
t r r t r rt r r r w t
w t w t
t r r t r rt r
w tw t
r t
λ λλ λ λ σ
λ σ λ σ
λ λλ
λλ σ σ
λ λ
⎛ ⎞ ⎛ ⎞− −= − + − − −⎜ ⎟ ⎜ ⎟⎜ ⎟ ⎜ ⎟
⎝ ⎠ ⎝ ⎠
− −= − + −
= −
&
&
Thus ( ) ( )0 rtt eλ λ −= and H1 is now ( ) ( )( )
11
10
r tc t eθ
δ
λ
−
−⎛ ⎞= ⎜ ⎟⎜ ⎟⎝ ⎠
so ( ) ( )
11
100
cθ
λ
−
⎛ ⎞= ⎜ ⎟⎜ ⎟⎝ ⎠
thus
( ) ( ) ( )( )1
1
0 r tc t c eθ
δ−
−=
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 25
Since
( )
( )
( )
1
1
ww
w
w
w
VRRA w
vww
cc
λθ
λλ
θ λ
= −
− − = −
=−
Then substituting ( )1w
cλλ θ
=−
into w& to give
( ) ( ) ( ) ( )2
21a
w a
c t r rw rw t c t
θ λ σ−
= − −−
&
( ) ( ) ( )( )
2
2
1 11
a
a
r rw rw t c t
θσ
⎛ ⎞−⎜ ⎟= − +⎜ ⎟−⎝ ⎠
&
And substituting ( ) ( ) ( )( )1
1
0 r tc t c eθ
δ−
−=
( ) ( ) ( )( ) ( )( )
11
2
2
10 11
r t a
a
r rw rw t c e
θδ
θσ
−−
⎛ ⎞−⎜ ⎟= − +⎜ ⎟−⎝ ⎠
&
Denote ( )
( )
2
2
1 11
a
a
r rθσ
⎛ ⎞−⎜ ⎟= +⎜ ⎟−⎝ ⎠
l
( )( )1
0
rt
w rw t c eδθ−
−− = −& l
( )( )( )1
0
rr t
rte w rw t c eδθ
⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠− = −& l
Using ( )( )
( )= +∫ '
f xf x ee dx A
f x to integrate the RHS
( ) ( )1 1
10
1r rt s t
s
e ds e Ar
δ δθ θθ
δ
− −
− −
=
−= +
−∫ gives
BEA400 Microeconomics – Lecture 11
© University of Tasmania - CRICOS PROVIDER CODE 00586B 26
( )( )( )
( )( )
( )( )
10
0
10 1 2
10
1
1
rt r srt
s
rr t
rt
rr t
rt
e w rw t c e ds
w t e c e A Ar
w t e A c er
δθ
δθ
δθ
θθ δ
θθ δ
⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠
=
⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠
⎛ ⎞−−⎜ ⎟⎜ ⎟−− ⎝ ⎠
− = −
−= − + −
−
−= −
−
∫ ∫& l
l
l
setting t=0
( ) 010A w c
rθ
θ δ−
= +−
l
( ) ( ) 10
10 1r trtw t e w c e
r
θ δθθ
θ δ
−− −
⎛ ⎞−= + −⎜ ⎟
− ⎝ ⎠l
Note that if θ=0 the logarithmic case
( ) ( ) ( )010 1rt tw t e w c e δ
δ− −= − −l
If t=T then we can use the terminal condition for wealth to remove 0c from the
problem. If there was no terminal condition on wealth we would need to use the
transversality conditions, (such as = 0Tλ if the terminal state is free) together with
the equation of motion of the co-state variable to provide 0λ and so 0c .
( ) ( )
( )( )
10
00
1
10 1
1 1
r TrT
rT
r T
w T e w c er
w w T ec
er
θ δθ
θ δθ
θθ δ
θθ δ
−− −
−
−−
⎛ ⎞−= + −⎜ ⎟
− ⎝ ⎠
−=
⎛ ⎞−−⎜ ⎟
− ⎝ ⎠
l
l