MATH180B: Introduction to Stochastic Processes I
www.math.ucsd.edu/~ynemish/180b
This week:
Midterm #2 next Wednesday
HW 5 due today 11:59 pm
Today: Limit theorems for MC's with infinite state spaces
Next: Review of Markov chains
:
Recurrentandtransientstaks-criterion-na.TT.
The following dichotony holds !ffiii-lif.IE#/Li)Iffii=I,then i is récurrent and Ê
.
Pinto
Iii) if fiis 1,then I is transirent and ÊPiÎ' ca .
Define Vi -_ Ê.
#Anais (m, courts the # of visit to i )
E (Vi IX. = i ) = Ê.EC#gxn=islXo--i)--E.PCXn--ilXo--i)--ÊPÎ'"
"
By Markov property , ← ( feomctfii) on fair - - t)K
P ( Yi = K IX. = i ) = f-id - fii ) ,or équivalentty P ( Vista IX. - i ) = fit ,In particulier '
Edi IX.= =
Prooffcont.ci) If fii =L ,then
(ii) If fi ici ,then
Corotlary If i is récurrent and i ,then j is récurrent .
Récurrence ( like period icity ) is a class proparty :the whale class is either récurrent or transirent .
lnterestingfactsconsiderone -
'two - and three - dimensional symmetric
random walk.All three models are irreducible (one class )
. Y-
÷:÷Ë÷÷Z
Z
récurrent / 7L -23
transirent récurrents récurrentstransirent transirent
Recurrenceoft-Drandomwa.lkThe model is described by a stationary infinite t.p.in
.
Enough to considéranty one (any ) point ( fake O) .
pff"" - o ( need even member of steps to come back )# left = # right
Using Stirling 's formula n ! nn""êÆ
Basic limit therem of Markov chains-
(on {0 , 1,2 , - - - b)
Define - r-v.,first returns time
Then fiim-PCXn-i.Xj-ittjc-hh-n.in - t } IX. - i ) = PKRi-nIX.ci)[ as a function of in fi? gives a p.m -
f. of Ri
Now mi - E ( Ri IX.ai ) gives the mean du rationbetween visit
Thx.
Let #ns.obearecurrentirreducibleaperiodic MCOn { 0,42 ,
_ . . } (not necessarily finit ) .
Then (a)
(b)
Stationarydistributionlnthe setting of the thorens :if for some call ) i , then the chain (class )
if for all i,then the chain class ) is
(then (Ti) is called the
of MC #a)ns.o .Thin
.
In a positive récurrent irreducibleaperiodic Mcthe stationary distribution is unique ly détermined bg( i )
,
Cii ) ( iii )
ExamptesuccessrunsofbinomialtrialsthfC Oh { 0 , 1,2 ,- - - } " Po i- pr t - pa t -pz - - -
Po t - po o o o - -- .
F-Q : Récurrent or transirent ?§; : IPËÏÏ ) m dûimiitingaasss⇒ enoughtostudy récurrenceof one Carry ) stateftakeo)
Define Ro - min hnzl : Xn --0f .
Example-isuccessrunsofbinomialtrialsontheotherhand.PL¥ > n IX. = n) =,so
n- t (K)[ f00 =
K=o
Thus,
O is récurrent
Fact. Hospice ,
then jÏ t- pj ) → o , nanif Êpi = - .
Conclusion 1.State O is récurrent IA
- litt) a
E-g. : If pi = 2 ,then Épi = l
⇒ siamois manient . / ?"
oi:ùË÷?au states are visitedonly all skates are visited
finit member of times infinite member of times
Example-isuccessrunsofbinomialtrialspositivere.aerreut or null récurrent ?
For positive récurrence need
Compute E (Ro 1×0=0)
E ( Ro Ho -- o ) -
If then O is positive récurrent ,
otherwise null récurrent
E.g. H pj-
_ petit ) , then If pk -- ¥
a- rit 1¥.it ¥,f- o
Ois positive récurrent
Examptesuccessrunsofbinomialtrialswhatabout the stationary distribution ?
From the limit theorem To =
Po t - po O O O - -- .
¥,write tuera""
* §; : ": :p . :)Tk =
E.g. 1f pjpso ,
then (dénote g-- t - p )
( is stationany for thisMC
