STOCHASTIC MODELSLECTURE 1

MARKOV CHAINSNanChen

MScPrograminFinancialEngineeringTheChineseUniversityofHongKong

(ShenZhen)Sept.6,2016

Outline1. Introduction2. Chapman-Kolmogrov

Equations3. TheGambler’sruinProblem

1.1INTRODUCTION

WhatisaStochasticProcess?

• Astochasticprocessisacollectionofrandomvariablesthatareindexedbytime.Usually,wedenoteitby– or–

• Examples:– DailyaveragetemperatureonCUHK-SZcampus– Real-timestockpriceofGoogle

(X1,X2,!,Xn,!)(Xt, t ≥ 0).

MotivationofMarkovChains

• Stochasticprocessesarewidelyusedtocharacterizethetemporalrelationshipbetweenrandomvariables.

• Thesimplestmodelshouldbethatareindependentofeachother.

• But,theabovemodelmaynotbeabletoprovideareasonableapproximationtofinancialmarkets.

Xn

WhatisaMarkovChain?

• Letbeastochasticprocessthattakesonafinite/countablenumberofpossiblestates.WecallitbyaMarkovchain,iftheconditionaldistributionofdependsonthepastobservationsonlythrough.Namely,

forall

(Xn,n = 0,1, 2,!)

Xn+1

(X1,X2,!,Xn )Xn

P(Xn+1 = j | X1 = i1,X2 = i2,!,Xn = in ) = P(Xn+1 = j | Xn = in )

n.

TheMarkovian Property

• ItcanbeshownthatthedefinitionofMarkovchainsisequivalenttostatingthat

• Inwords,giventhecurrentstateoftheprocess,itsfutureandhistoricalmovementsareindependent.

P(Xn+1 = j,X1 = i1,X2 = i2,!,Xn−1 = in−1 | Xn = in )= P(Xn+1 = j | Xn = in )P(X1 = i1,X2 = i2,!,Xn−1 = in−1 | Xn = in )

FinancialRationale:EfficientMarketHypothesis• TheMarkovian propertyturnsouttobehighlyrelevanttofinancialmodelinginlightofoneofthemostprofoundtheoryinthehistoryofmodernfinance --- efficientmarkethypothesis.

• Itstates–Marketinformation,suchastheinformationreflectedinthepastrecordortheinformationpublishedinfinancialpress,mustbeabsorbedandreflectedquicklyinthestockprice.

MoreaboutEMH:aThoughtExperiment• Letusstartwiththefollowingthoughtexperiment:AssumethatProf.ChenhadinventedanformulawhichwecouldusetopredictthemovementsofGooglestockpriceveryaccurately.Whatwouldhappenifthisformulawasdisclosedtothepublic?

MoreaboutEMH:aThoughtExperiment• SupposethatitpredictedthatGoogle’sstockpricewouldrisedramaticallyinthreedaystoUS$700fromUS$650.– Thepredictionwouldinduceagreatwaveofimmediatebuyorders.

– HugedemandsonGoogle’sstockswouldpushitspricetojumpto$700immediately.

– Theformulafails!• AtruestoryofEdwardThorpandtheBlack-Scholesformula

ImplicationofEfficientMarketHypothesis• OneimplicationofEMHisthatgiventhecurrentstockprice,knowingitshistorywillhelpverylittleinpredictingitsfuture.

• Therefore,weshoulduseMarkovprocessestomodelthedynamicoffinancialvariables.

TransitionMatrix

• Inthislecture,weonlyconsidertime-homogenousMarkovchains;thatis,thetransitionprobabilitiesareindependentoftime

• DenoteWethencanusethefollowingmatrixtocharacterizetheprocess.

P(Xn+1 = j | Xn = in )n.

pij := P(Xn+1 = j | Xn = i).

P :=

p11 p12 ! p1np21 p22 ! p2n! ! ! !pn1 pn2 ! pnn

!

"

#####

$

%

&&&&&

TransitionMatrix

• ThetransitionmatrixofaMarkovchainmustbeastochasticmatrix:––

pij ≥ 0.

pijj=1

n

∑ =1.

ExampleI:ForecastingtheWeather

• SupposethatthechanceofraintomorrowinShenzhendependsonpreviousweatherconditionsonlythroughwhetherornotitisrainingtoday.

• Assumethatifitrainstoday,thenitwillraintomorrowwithprobability70%;andifitdoesnotraintoday,thenitwillraintomorrowwithprob.50%.

• HowcanweuseaMarkovchaintomodelit?

ExampleII:1-dimensionalRandomWalk• AMarkovchainwhosestatespaceisgivenbytheintegersissaidtobearandomwalkif,forsomenumber

• Wesaytherandomwalkissymmetricif;asymmetricif

0 < p <1,0,±1,±2,...

Pi,i+1 = p =1−Pi,i−1, i = 0,±1,±2,...

p =1/ 2 p ≠1/ 2.

1.2CHAPMAN-KOLMOGOROV EQUATIONS

TheChapman-KolmogorovEquations

• TheCKequationsprovideamethodforcomputingthe-steptransitionprobabilitiesofaMarkovchain.

•

orinamatrixform,

n

Pijn+m = P Xn+m = j | X0 = i( ) = Pik

nPkjm

k∑

Pn+m = PnPm.

ExampleIII:RainProbability

• ReconsiderthesituationinExampleI.Giventhatitisrainingtoday,whatistheprobabilitythatitwillrainfourdaysfromtoday?

ExampleIV:UrnandBalls

• Anurnalwayscontains2balls.Ballcolorsareredandblue.Ateachstageaballisrandomlychosenandthenreplacedbyanewball,whichwithprob.80%isthesamecolor,andwithprob.20%istheoppositecolor.

• Ifinitiallybothballsarered,findtheprobabilitythatthefifthballselectedisred.

1.3THE GAMBLER’S RUINPROBLEM

TheGambler’sRuinProblem

• Consideragamblerwhoateachplayofthegamehasprobabilityofwinningonedollarandprobabilityoflosingonedollar.Assuming thatsuccessiveplaysofthegameareindependent,whatistheprobabilitythat,startingwithdollars,thegamblerfortunewillwindollarsbeforeheruins(i.e.,hisfortunereaches0)?

p

q =1− p

iN

MarkovDescriptionoftheModel

• Ifweletdenotetheplayer’sfortuneattime,thentheprocessisaMarkovchainwithtransitionprobabilities–– ,

• TheMarkovchainhasthreeclasses:

{Xn,n =1,2,...}

Xn

n

P00 = PNN =1Pi,i+1 = p =1−Pi,i−1 i =1,2,...,N −1.

{0},{1, 2,...,N −1},{N}

Solution

• Letbetheprobabilitythat,startingwithdollars,thegamblerfortunewilleventuallyreach.

• Byconditioningontheoutcomeoftheinitialplay

,and

Pi = pPi+1 + qPi−1

Pi i

N

i =1,2,...,N −1.

P0 = 0, PN =1.

Solution(Continued)

• Hence,weobtainfromtheprecedingslidethat––

– ……–

P2 −P1 =qp(P1 −P0 ) =

qpP1;

P3 −P2 =qp(P2 −P1) =

qp

"

#$

%

&'

2

P1;

PN −PN−1 =qp(PN−1 −PN−2 ) =

qp

"

#$

%

&'

N−1

P1.

Solution(Continued)

• Addingalltheequalitiesup,weobtain–

–

P1 =1/ N,

1− (q / p)1− (q / p)N

,

"

#$

%$

p =1/ 2;

p ≠1/ 2.

Pi =i / N,

1− (q / p)i

1− (q / p)N,

"

#$

%$

p =1/ 2;

p ≠1/ 2.

Solution(Continued)

• Notethat,as

• Thus,if,thereisapositiveprobabilitythatthegambler’sfortunewillincreaseindefinitely;whileif,thegamblerwill,withprobability1,goruinagainstaninfinitelyrichadversary(say,acasino).

N→+∞,

Pi →0,

1− qp

#

$%

&

'(

i

,

)

*++

,++

p ≤1/ 2;

p >1/ 2.

p >1/ 2p ≤1/ 2

HomeworkAssignments

• ReadRossChapter4.1,4.2,and4.5.1.• AnswerQuestions:– Exercises5,6(Page261,Ross)– Exercises13,14(Page262,Ross)– Exercises56,57(Page270,Ross)

• DueonSept.13,Wed.