| Date post: | 18-Dec-2015 |
| Category: |
Documents |
| Upload: | sheena-spencer |
| View: | 225 times |
| Download: | 2 times |
1
Part IIIMarkov Chains & Queueing Systems
10.Discrete-Time Markov Chains
11.Stationary Distributions & Limiting
Probabilities
12.State Classification
13.Limiting Theorems For Markov Chains
14.Continuous-Time Markov Chains
2
10. Discrete-Time Markov Chain Definition of Markov Processes A random process is said to be a Markov Process
if, for any set of n time points t1<t2<…<tn in the index set or time range of the process, the conditional distribution of X(tn), given the values of X(t1), X(t2), …, X(tn-1), depends only on the immediately preceding value; that is, for any real numbers x1, x2, …, xn,
})(|)(Pr{
})(,...,)(,)(|)(Pr{
11
112211
nnnn
nnnn
xtXxtX
xtXxtXxtXxtX
3
Classification of Markov Processes
Type of Parameter State Space Discrete Continuous Discrete-Time Continuous-TimeDiscrete Markov Chain Markov Chain
Continuous Discrete-Time Continuous-Time Markov Process Markov Process
4
Discrete-Time Markov Chain Definition ： A discrete-time Markov chain {Xn| n=0, 1, 2, …}, is
a discrete-time, discrete-value random sequence such that given X0, X1, …, Xn, the next random variable depends only on Xn through the transition probability
ij
nn
nnnn
p
iXjXP
iXiXiXjXP
]|[
],...,,|[
1
00111
5
Theorem 10.1 The transition probabilities pij of a Markov chain
satisfy
Pf：
0
.1 ,0j
ijij pp
6
Example 10.1 The two-state Markov chain can be used to model a
wide variety of systems that alternate between ON and OFF states. After each unit of time in the OFF state, the system turns ON with probability p. After each unit of time in the ON state, the system turns OFF with probability q. Using 0 and 1 to denote the OFF and ON states, what is the Markov chain for the system?
Sol：
7
Example 10.2 A packet voice communications is in talkspurts (state 1)
or silent (state 0) states. The system decides whether the speaker is talking or silent every 10 ms (slot time). If the speaker is silent in a slot, then the speaker will be talking in the next slot with probability p= 1/140. If the speaker is talking in a slot, the speaker will be silent in the next slot with probability q = 1/100. Sketch the Markov chain of this system.
Sol：
8
Example 10.3 A computer disk drive can be in one of 3
possible states: 0 (IDLE), 1 (READ), or 2 (WRITE) . When a unit of time is required to read or write a sector on the disk, sketch the Markov chain.
Sol：
9
Example 10.4 In a discrete random walk, a person’s position is
marked by an integer on the real line. Each unit of time, the person randomly moves one step, either to the right (with probability p) or to the left. Sketch the Markov chain.
Sol：
10
Example 10.5 What is the transition matrix of the two-state
ON-OFF Markov chain of Example 10.1 ? Sol：
11
n-Step Transition Probabilities Definition For a finite Markov chain, the n-step transition
probabilities are given by the matrix P(n) which has i, j-th element
.
)()()(
)()()(
)()()(
)(
],|[)(
10
11110
00100
npnpnp
npnpnp
npnpnp
n
iXjXPnp
KKKK
K
K
mmnij
P
12
Theorem 10.2 : Chapman-Kolmogorov Eq.
For a finite Markov chain with K states, the n-step transition probabilities satisfy
Pf：).()()( ,)()()(
0
mnmnmpnpmnpK
kkjikij PPP
13
Theorem 10.3 For a finite Markov chain with transition matrix
P, the n-step transition matrix is
Pf：.)( nn PP
14
Example 10.6 For the two-state Markov chain described in
Example 10.1, find the n-step transition matrix P(n). Given the system is OFF at time 0, what is the probability the system is OFF at time n = 33 ?
Pf：
15
State Probability Vector Definition A vector p=[p0 p1 … pK] is a state probability
vector if , and each element is nonnegative.
10
K
j jp
16
Theorem 10.4 The state probabilities pj(n) at time n can be defined by
either one iteration with the n-step transition probabilities:
or n iterations with the one-step transition probabilities:
Pf：
.)1()( ,)1()(0
K
iijij nnpnpnp Ppp
,)0()( ,)()0()(0
nK
iijij nnppnp Ppp
17
Example 10.7 For the two-state Markov chain described in
Example 10.1 with initial state probabilities p(0)=[p0 p1], find the state probability vector p(n).
Sol：
18
Part IIIMarkov Chains & Queueing Systems
10.Discrete-Time Markov Chains
11.Stationary Distributions & Limiting
Probabilities
12.State Classification
13.Limiting Theorems For Markov Chains
14.Continuous-Time Markov Chains
19
11. Stationary Distributions & Limiting
Probabilities Definition of Limiting State Probabilities For a finite Markov chain with initial state
probability vector p(0) , the limiting state probabilities, when they exist, are defined to be the vector
).(lim nn
pπ
20
Example 11.1For a two-state packet voice system of
Example 10.2, what is the limiting state probability
vector
Sol：
? )(lim] [ 10 nn
p
0 11/140
1/10099/100139/140
21
Theorem 11.1If a finite Markov chain with transition matrix
P and initial state probability p(0) has limiting probability vector , thenPf：
)(lim nn
pπ
.πPπ
22
Stationary Probability VectorDefinitionFor a finite Markov chain with transition
matrix P, a state probability vector is stationary if .πPπ
23
Theorem 11.2If a finite Markov chain Xn with transition
matrix P is initialized with stationary probability
vector p(0) = , then p(n) = for all n and the stochastic process Xn is stationary.
Pf：)|()|()(),,( 1|12|11,,
11211
mmXXXXXmXX xxpxxpxpxxpmnmnnnnmnn
)()( 112 1211 mmxxxxx nnpnnp
mm
)|())()(()(
,)(
1|11
1
111
11
jjXXjjxxjjxx
xX
xxpknknpnnp
xp
kjnkjnjjjj
kn
StationaryXxxpxxp nmXXmXX kmnknmnn ),,,(),,( 1,,1,,
11
24
Example 11.2A queueing system is described by a Markov chain in which that state Xn is the number of customers in the
queue at time n. The Markov chain has a unique stationary
distribution . The following questions are all equivalent. (1) What is the steady-state probability of at least 10
customers in the system?(2) If we inspect the queue in the distant future, what is
the probability of at least 10 customers in the system?
(3) What is the stationary probability of at least 10 customers in the system?
(4) What is the limiting probability of at least 10 customers in the system?
25
Example 11.3Consider the two-state Markov chain of Example
10.1 and Example 10.6. For what values of p and q does (a) exist, independent of the initial state probability vector p(0); (b) exist, but depend on p(0); (c) or not exist?
Sol：
)(lim nn
p
26
Part IIIMarkov Chains & Queueing Systems
10.Discrete-Time Markov Chains
11.Stationary Distributions & Limiting
Probabilities
12.State Classification
13.Limiting Theorems For Markov Chains
14.Continuous-Time Markov Chains
27
12. State Classification
Accessibility State j is accessible from state i, written i j,
if pij(n) > 0 for some n > 0.
Communicating States State i and j communicates, written i j, if i
j and j i.Communicating Class A communicating class is a nonempty subset of
states C such that if i C , then j C if and only if i j.
28
State ClassificationPeriodic and Aperiodic State i has period d if d is the largest integer such
that pii(n) = 0 whenever n is not divisible by d (pii(n) > 0 whenever n is divisible by d). If d = 1, then state i is called aperiodic.
Transient and Recurrent States In a finite Markov chain, a state i is transient if there
exists a state j such that i j but j i; otherwise, if no such state j exists, then state i is recurrent.
Irreducible Markov chain A Markov chain is irreducible if there is only one
communicating class.
29
Example 12.1The Markov chain is with each branch pij > 0.
How many communicating classes?Sol：
0
216
5
4
3
30
Example 12.2
Consider the five-state discrete random walk with Markov chain
What is the period of each state i ? Sol：
2 4310p
1p
p
1p
p
1
1
1p
31
Theorem 12.1
Communicating states have the same period. Pf：
jijijdid where, and state of periods the:)( ),(
. and somefor to from
path hop-an and to frompath hop-an is there,
nmji
mijnji
mnjdmnp jj divides )( ,0)(
. divides )( then ,0)(such that any For kidkpk ii
, divides )( ,0)( mknjdmknp jj kjd divides )( . have weAnd hops hops hops jiij mkn
)( )( idjd that showcan we, and of labels thereverse Similarly, ji
).()( jdid ).()( jdid
32
Example 12.3
The Markov chain is with each branch pij > 0.
Find the periodicity of each communicating class.
Sol：
0
216
5
4
3
33
Theorem 12.2
If i is recurrent and i j, then j is recurrent.Sol：
. show toneed we, that stateany for Assume jkkjk
,ji
ki
. via to frompath a exists there , and jkikjji
. recurrent, is iki
.jk
34
Example 12.4
The Markov chain is with each branch pij > 0.
Identify each communicating class and indicate
whether it is transient or recurrent.Sol：
2 4310 5
35
Theorem 12.3If state i is transient, then Ni, the number of
visits to state i over all time, has expected value
E[Ni] < .
Pf： ici
i
EE
iE
ofevent ary complement the:
state togoes eventually system theevent that the:
)(]|[)(]|[][ iiici
ciii EPENEEPENENE
.0]|[ ,0 then occurs,not is, that occurs, If ciiii
ci ENENEE
.occurs given , state toreturnsnever system the
event that theis where],[/1]|[ And
on.distributi geometric is ,given then ,occurs If
i
cii
ciiii
iii
Ei
EEPENE
NEE
)(]|[][ iiii EPENENE
36
Theorem 12.4A finite-state Markov chain always has a recurrent communicating class.Pf：
states, ofnumber the:
timeallover state to visitsofnumber the:
K
iN i
have we, any timeat then exist, states recurrent no Assume n
ion.contradict a isit infinite,approach can But n
,][ ,11
K
ki
K
ki NEnNn
classs. ingcommunicat
recurrent a has alwayschain Markov finite a Hence,
37
Example 12.5The Markov chain is with each branch pij > 0.
If the system starts in state j C1 = {0, 1, 2}, the system never leaves C1.
If the system starts in communicating class C3 = {4, 5}, the system never leaves C3.
If the system starts in the transient state 3, then in the first step there is a random transition to either state 2 or to state 4 and the system then remains forever in the corresponding communicating class.
2 4310 5
38
Part IIIMarkov Chains & Queueing Systems
10.Discrete-Time Markov Chains
11.Stationary Distributions & Limiting
Probabilities
12.State Classification
13.Limiting Theorems For Markov Chains
14.Continuous-Time Markov Chains
39
13. Limiting Theorems For Markov Chains
For Markov chains with multiple recurrent classes, the limiting state probabilities depend on the initial state distribution.
For understanding a system with multiple communicating classes, we need to examine each recurrent class separately as an irreducible system consisting of just that class.
In this part, we first focus on irreducible, aperiodic chains and their limiting state probabilities.
40
Theorem 13.1
,lim
10
10
10
K
K
K
n
n
πP 1
For an irreducible, aperiodic, finite Markov chain with states {0, 1, 2, …, K}, the limiting n-step transition matrix is
where is the column vector and is the unique vector satisfying
] [ 10 K π1 T1] 1[
.1 , 1ππPπ
41
Theorem 13.1 (cont’d)Pf：
42
Theorem 13.2
.)(lim πp
nn
For an irreducible, aperiodic, finite Markov chain with transition matrix P and initial state probability vector p(0) , Pf：
nn Pppp )0()( ,1)0( and1
πππp
πpPpp
1)0(
)0(lim )0()(lim
1
1n
nnn
43
Example 13.1For the packet voice communications system of Example 12.8, use Theorem 13.2 to calculate the stationary probabilities . Sol：
] [ 10 0 1
1/140
1/10099/100139/140
44
Example 13.2A digital mobile phone transmits one packet in every 20-ms time slot over a wireless connection. A packet is received error with probability p = 0.1, independent of other packets. Whenever 5 consecutive packets are received in error, the transmitter enters a timeout state. During the timeout state, the mobile terminal performs an independent Bernoulli trial with success probability q = 0.01 in every slot. When a success occurs, the mobile terminal starts transmitting in the next slot as though no packets had been in error. Construct a Markov chain for this system. What are the limiting state probabilities?
45
Example 13.2 (Cont’d)Sol：
2 4310p
1p
p
1p
p
1p
p
1p
p
5
q
1p1q
otherwise.0
,51
)1(
,4,3,2,1,01
)1)(1(
65
5
65
npqpq
pp
npqpq
ppq n
n
46
Theorem 13.3Consider an irreducible, aperiodic, finite Markov chain with transition probabilities {pij} and stationary probabilities {i}. For any partition of the state space into disjoint subsets S and S’,
Pf：
.
Sj Si
jijSi Sj
iji pp
otherwise.0
from occurs transition1)(: at time from stransition
SSmImSS SS
.]1)([)]([
Si Sj
ijiSSSS pmIPmIE
, time toup from crossings of no.:)( nSSnN SS
1
0
1
0
]1)([)]([n
m Si Sjiji
n
mSSSS pmIEnNE
Si Sjiji
n
mi
nSi Sj
ijSS
np
np
n
nNE 1
0
1lim
)]([lim
1)()(1)( nNnNnN SSSSSS
.
Sj Si
jijSi Sj
iji pp
47
Example 13.3 In each time slot, a router can either store an arriving data packet in its buffer or forward a stored packet (and remove that packet from its buffer). In each time slot, a new packet arrives with probability p, independent of arrivals in all other slots. This packets is stored as long as the router is storing fewer than c packets. If c packets are already buffered, then the new packet is discarded by the router. If no new packet arrives and n > 0 packets are buffered by the router, then the router will forward one buffered packet. That packet is then removed from the router. Let Xn denote the number of buffered packets at time n. Sketch the Markov chain for Xn and find the stationary probabilities.
48
Example 13.3 Sol：
i i+110p
1p
p
1p
p
1p…1p c
p
1p…
pS S’
.,,2,1,0 ,)()(1
)(111
1
1 ciip
pc
pp
pp
i
49
Periodic States and Multiple Communicating Classes
Theorem 13.4：For an irreducible, recurrent, periodic, finite Markov chain with transition matrix P, the stationary probability vector is the unique nonnegative solution of
Pf：
π
.1 , 1ππPπ
50
Example 13.4
Find the stationary probabilities for the Markov
chain shown to the right. Sol：
2101 1
1
51
Theorem 13.5 For a Markov chain with recurrent communicating classes C1, … Cm, let denote the limiting state probabilities associated with class k. Given that the system starts in a transient state i, the limiting probability of state j is
where P[Bik] is the conditional probability that the system enters class Ck.
Pf：
],[][)(lim )(1
)1(im
mjijij
nBPBPnp
)(kπ
].[]|[][]|[]|[ 110 imimniinn BPBjXPBPBjXPiXjXP
. if 0 where,]|[lim )()(k
kj
kjikn
nCjBjXP
].[][]|[lim)(lim )(1
)1(0 im
mjijn
nij
nBPBPiXjXPnp
52
Example 13.5
For each possible starting state i{0,1,…,4}, find
the limiting state probabilities for the following
Markov chain. Sol：
2103/4 1/2
1/4
431/2 1/2
1/4
1/4 3/4 1/2 3/4
3
1
6
1 0
8
3
8
1π
53
Countably Infinite Chains
Countably infinite Markov chains has infinite set of states {0, 1, 2, …}.
We will consider only a single communicating class here. (Irreducible Markov chains)
Multiple communicating classes represent distinct system modes that are coupled only through an initial transient phase that results in the system landing in one of the communicating classes.
54
Example 13.6Suppose that the router in Example 13.3 has unlimited buffer space. In each time slot, a router either store an arriving data packet in its buffer or forward a stored packet ( and remove that packet from its buffer). In each time slot, a new packet is stored with probability p, independent of arrivals in all other slots. If no new packet arrives, then one packet will be removed from
the buffer and forwarded. Sketch the Markov chain for Xn,
the number of buffered packets at time n. Sol：
210p
1p
p
1p
p
1p…
1p
55
Theorem 13.6 ： Chapman-Kolmogorov Eqs.
The n-step transition probabilities satisfy
Pf ： Omitted.
).()()(0
mpnpmnp kjk
ikij
56
Theorem 13.7The state probabilities pj(n) at time n can be found by either one iteration with the n-step transition probabilities
or n iterations with the one-step transition probabilities
Pf ： Omitted.
).()0()(0
nppnp iji
ij
.)1()(0
i
ijij pnpnp
57
Visitation, First Return Time, No. of Returns
Definitions ： Given that the system is in state i at
an arbitrary time,(a) Eii is the event that the system eventually
returns to visit state i. (b) Tii is the time (number of transitions) until
the system first returns to state i, (c) Nii is the number of times (in the future)
that the system returns to state i.
Definition ： For a countably infinite Markov chain, state i is recurrent if P[Eii]=1; otherwise state i is transient.
58
Example 13.7A system with states {0, 1, 2, …} has Markov chain
Note that for any state i > 0, pi,0 = 1/(i+1) and pi,i+1= i/(i+1).
Is state 0 transient or recurrent?
Sol：
2 43101/2
1/3
2/3
1/4
3/4
1/5
1
1/2
4/5…
01
1limlim
01,
01,
n
ppEPn
n
iii
nn
nncii
,1)(1 ciiii EPEP recurrent. is 0 state
59
Theorem 13.8The expected number of visits to state i over all time is
Pf：.)(][
1
n
iiii npNE
otherwise.0
, at time state toreturns system the1)( Define
ninI ii
,)(1
n
iiii nIN
.)(]1)([1][11
n
iin
iiii npnIPNE
60
Theorem 13.9State i is recurrent if and only if E[Nii]= .Pf：
. at time return tofirst ofy probabilit theis )( where
,)()()(1
rirf
rnprfnp
ii
n
riiiiii
)()()()()(11 11
r rn
iiiin
n
riiii
nii rnprfrnprfnp
1 11 0
)()0()( )()(r n
iiiiiir n
iiii npprfnprf
1 1
)(1)( r n
iiii nprf )(1
)()(
1
1
1
r ii
r ii
nii
rf
rfnp
.1)( recurrent, is state1
r ii rfi .)(][1
niiii npNE
61
Example 13.8The discrete random walk introduced in Example 10.4 has state space {…, 1, 0, 1, …} and Markov chain
Is state 0 recurrent?Sol：
0 2112p
1p
p
1p
p
1p
p
1p
p
1p
p
1p……
,)1()2(][1
2
10000
nn
n
nn
n
ppCnpNE
,2
!!
!2 22
nπnn
nC
nn
n ,)]1(4[
][ 00 n
ppNE
n
staterecurrent diverges 1
][ ,2
1 If
100
n nNEp
state.transient converges ][ ,2
1 If
100
n
n
nNEp
62
Positive Recurrence and Null Recurrence
Definition：
A recurrent state i is positive recurrent if E[Tii] < ; otherwise, state i is null recurrent.
Example 13.9 ： In Example 13.7, we found that state 0 is recurrent. Is state 0 positive recurrent or null recurrent?Sol：
2 43101/2
1/3
2/3
1/4
3/4
1/5
1
1/2
4/5…
)1(
1][]1[][ 000000
nnnTPnTPnTP
100
100 )1(
1][][
nn nnTPnTE
63
Theorem 13.10For a communicating class of a Markov chain,
one of the following must be true：(a)All states are transient.(b)All states are null recurrent.(c)All states are positive recurrent.
64
Example 13.10In a Markov chain of Example 13.7 and 13.9,
is state 33 positive recurrent, null recurrent, or transient?Sol：
65
Stationary Probabilities of Infinite Chains
Theorem 13.11：
For an irreducible, aperiodic, positive recurrent
Markov chain with states {0, 1, …}, the limiting n-step transition probabilities are where are the unique state probabilities satisfying
jijn
np
)(lim
},2,1,0|{ jj
,2,1,0 , ,100
jpi
ijijj
j
66
Example 13.11For the stationary probabilities of the router buffer described in Example 13.6. Make sure
to identify for what values of p that the
stationary probabilities exist. Sol：
210p
1p
p
1p
p
1p…
1p
67
Part IIIMarkov Chains & Queueing Systems
10.Discrete-Time Markov Chains
11.Stationary Distributions & Limiting
Probabilities
12.State Classification
13.Limiting Theorems For Markov Chains
14.Continuous-Time Markov Chains
68
14. Continuous-Time Markov ChainsDefinition：A continuous-time Markov chain {X(t)| t 0} is a continuous-time, discrete-value random process such that for an infinitesimal time step of size ,
Continuous-time Markov chain is closely related to the Poisson process. The time until the next transition is an exponential R.V. with parameter
ijij
ij
qitXitXP
qitXjtXP
1])(|)([
])(|)([
. , iqij
iji state of rate departure the
69
Embedded Discrete-Time Markov ChainDefinition：For a continuous-time Markov chain with transition rates qij and state i departure rates
vi, the embedded discrete-time Markov chain
has transition probabilities pij=qij /i for states i with i > 0 and pii = 1 for states i with i = 0.
Definition：The Communicating Classes of a Continuous-
Time Markov Chain are given by the
communicating classes of its embedded discrete-time Markov chain.
70
Irreducible Continuous-Time Markov ChainDefinition：A continuous-time Markov chain is irreducible
if the embedded discrete-time Markov chain is irreducible. Definition：An irreducible continuous-time Markov chain
is positive recurrent if for all states i, the time Tii to return to state i satisfying E[Tii] < .
71
Example 14.1In a continuous-time ON-OFF process,
alternating OFF and ON (state 0 and 1) periods have independent exponential durations. The
average ON period lasts 1/ seconds, while the average OFF period lasts 1/ seconds. Sketch the continuous-time Markov chain. Sol： 10
72
Example 14.2Air conditioner is in one of 3 states ： OFF(0),
Low (1), or High(2). The transitions from OFF to Low occur after an exponential with mean time 3
mins. The transitions from Low to OFF or High are equally likely and transitions out of the Low
state occur at rate 0.5 per min. When the system is
in High state, it makes a transition to Low with probability 2/3 or to the OFF state with probability 1/3. The time spent in the high
state is an exponential (1/2) R.V. Model this air conditioning system using a continuous-time Markov chain.
73
Example 14.2 (Cont’d)Sol：
2101/4
1/3
1/3
1/4
1/6
74
Theorem 14.1For a continuous-time Markov chain, the state probabilities pj(t) evolve according to the differential equations
where
Pf：
.)()(
, ,,2,1,0 ,)()(
Rpp
tdt
tdjtpr
dt
tdp
iiij
j form vector in or
. if
if
ji
jiqr
i
ijij
i
j itXPitXjtXPjtXPtp ])([])(|)([])([)(
ji
itXPitXjtXPjtXPjtXjtXP ])([])(|)([])([])(|)([
ji
iijjj tpqtp )()()1(
jiiijjj
jj tpqtptptp
)()()()(
i
iij tpr )(
75
Theorem 14.2For an irreducible, positive recurrent
continuous-time Markov chain, the state probabilities satisfying
where the limiting state probabilities are the unique solution to
Pf：
,)(lim , ,)(lim pp
tptpt
jjt
form vector in or
.1 , ,1
, , ,0
1
0
p
pR
form vector in or
form vector in or
jj
iiij
p
pr
,)(lim)(
lim
ii
tij
j
ttpr
dt
tdp 0
iiij pr ij
j
qij
Flow Conservation Law ： ave. rate in = ave. rate out
76
Example 14.3Calculate the limiting state probabilities for the ON-OFF system of Example 14.1. Sol： 10
77
Example 14.4Find the stationary distribution for the Markov chain describing the air conditioning system of Example 14.2. Sol： 210
1/4
1/3
1/3
1/4
1/6
.5
1 ,
5
2 : 210 pppAns
78
Birth-Death ProcessDefinition:A continuous-time Markov chain is a birth-death process if the transition rates satisfy qij = 0 for |ij| > 1.
2101
2
2
3
0
1
34
…3
79
Theorem 14.3For a birth-death queue with arrival rate i and service rate i , the stationary probabilities pi satisfy
Pf：
.1 ,0
11
iiiiii ppp
80
Theorem 14.4For a birth-death queue with arrival rate i and service rate i , let i =i /i+1 . The limiting state probabilities, if they exist, are
Pf：
.1
1
1
0
1
0
k
k
j j
i
j j
ip
81
The M/M/1 QueueThe arrivals are a Poisson process of rate , independent of the service requirements of the customers. The service time of a customer is an
exponential () R.V., independent of the system state.
M/M/1
Markovian arrival(Poisson arrivals)
Markovian service time(Exponential R.V.)
One server
82
Theorem 14.5The M/M/1 queue with arrival rate > 0 and service rate , > , has limiting state probabilities,
where = / . Pf：
,2,1,0 ,)1( np nn
83
Example 14.5Cars arrive at an isolated toll booth as a
Poisson process with arrival rate = 0.6 cars per minute. The service required by a customer is an exponential R.V. with expected value 1/ = 0.3 minutes. What are the limiting state
probabilities for N, the number of cars at the toll booth?
What is the probability that the toll booth has zero
cars some time in the distant future? Sol：
