7/24/2019 Bright 1995

1/30

This article was downloaded by: [UQ Library]On: 24 April 2015, At: 10:20Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK

Communications in Statistics. Stochastic ModelsPublication details, including instructions for authors and subscription information:

http://www.tandfonline.com/loi/lstm19

Calculating the equilibrium distribution in level

dependent quasi-birth-and-death processesL. Bright

a& P.G. Taylor

b

aDepartment of Applied Mathematics , The University of Adelaide , South Australia,

Australia, 5005 E-mail:bDepartment of Applied Mathematics , The University of Adelaide , South Australia,

Australia, 5005 E-mail:

Published online: 21 Mar 2007.

To cite this article:L. Bright & P.G. Taylor (1995) Calculating the equilibrium distribution in level dependent quasi-birth-and-death processes, Communications in Statistics. Stochastic Models, 11:3, 497-525, DOI: 10.1080/15326349508807357

To link to this article: http://dx.doi.org/10.1080/15326349508807357

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditionshttp://dx.doi.org/10.1080/15326349508807357http://www.tandfonline.com/page/terms-and-conditionshttp://www.tandfonline.com/page/terms-and-conditionshttp://dx.doi.org/10.1080/15326349508807357http://www.tandfonline.com/action/showCitFormats?doi=10.1080/15326349508807357http://www.tandfonline.com/loi/lstm19

7/24/2019 Bright 1995

2/30

COMMU N. STATIST.-STOCHASTIC MODELS

11 3), 497-525 1995)

Calculating the Equilibrium Distribution in Level

Dependent Quasi-Birt h-and-Death Processes

L. BRIGHT

and

P.G. TAYLOR

Department of Applied h/Iathematics

Th e University of Adelaide, South Austra ia 5005, Aus tra lia

email:

lbright@maths.adelaide.edu.au

taylor@maths adelaide edu au

Abstract

Quasi-Birth-and-Death (QBD) processes have been analysed in

detail

by

many authors. This paper considers the calculation of equilibrium

distributions in level dependent QBD processes which are an extension of

the classical QBD process.

In

addition to the general case, we consider

a number of special cases of level dependent

QBD

processes, presenting

algorithms for computing the equilibrium distribution.

Keywords: level dependent Quasi-Birth-and-Death process, dominating

process, level, phase.

1 INTRODUCTION

Consider a regular continuous time two-dimensional Markov chain X t )

on the state space {(k,

j

0 , l j

M k .

In the state (k, , k is

referred to as the level of the s ta te and j is referred to as the phase of the

sta te. Suppose the generator m atr ix for this Marltov chain is of the block

partitioned form

Copyright 1995 by Marcel Dekker

Inc

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

3/30

BRIGHT ND

T YLOR

where Qik ), 1, Q?), k 2 0, Q r ) ,k 0 are matric es of order M k Mk-1,

M k x M k , and M k Mk+1 espectively. A Markov chain with this structure

is referred to as a level depende nt q uasi-bir th-and-death process (hereby

abbreviated to L DQ BD ). We will assume tha t the process is i rreducible.

I f w e p u t M k = M ~ k Z ~ a n d Q f ) = Q o~ > o , Q ~ ) = Q ~k L 1 ,

QP Q 1 and Q f ) Q z Vk 2 1 then X (t ) becomes a level independent

QBD proc ess. Level inde pend ent QBD s a re a powerful mod elling tool.

Th ey can be used to model, for examp le, queues in a rando m environm ent,

high speed commun ication systems, databa se systems and m ult iprogram-

ming system s (see

[ 6 ] [ lo] ,

[ l l ]

19], [20] an d [9]). Since LDQB D s are a

generalisation of level independent processes, they enable us to model a

wider class of problems. For example, the P H I M I 1 queue (i.e the single

server queue with a phase-type arrival process) can be analysed using a

level independent QBD whereas the PH/M/oo can be analysed us ing a

LDQ BD. S imilarly, LDQ BD s en able us to m odel infinite server queues in

a random environment rather tha n just single server queues. LDQB D s

also enable us to m odel more com plex communication sy stems and m ult i-

programming sys tems.

Level indepen dent QBD s have been dealt with widely in the l i terature.

Neuts [12] has developed m eth od s for determ ining positive recurrence of

X ( t ) and for calculating the equil ibrium distr ibution. Neuts also con-

sidered level ind ependen t QB D7 s where there is some non-homogeneous

boundary behaviour bu t th e process is eventually level independe nt. Re-

cently Latouc he and R amasw ami [7] have developed a logarithm ic reduc-

t ion algori thm that can be used to calculate the equil ibrium distr ibution

in the level independent case. The computational complexity of this algo-

rith m is significantly less than t ha t for previously existing algorithm s. T he

me tho ds we present for LDQBD s involve an extension of th is logarith mic

reduction algori thm to the level depende nt case.

Th e special case where X t ) has a finite sta te space ha s also received sig-

nificant at te ntion in the l i terature. Th is occurs when for some I? QiK) 0

a n d Q f ) Q ) Q r ) 0 k

2 l

1. Hajeli [4] initially considered

level independent processes on a f inite state space an d Gaver, Jaco bs a nd

Latouche [3] developed a general metho d for the level depe nde nt case o n

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

4/30

LEVEL DEPEND ENT Q D PROCESSES 99

a finite sta te space. Ye and Li [18] have developed a co m puta tion ally effi-

cient algorith m for the case where the gener ator is piecewise homog eneous.

In th e level dependen t case on an infinite sta te space, as far as the au thors

know, there is no general method for determining positive recurrence an d

the problem of calculating the equilibrium distribution has not been ad-

dressed.

In order to calculate the ecpilibrium distribution for a

LDQBD,

i t is

necessary to evaluate the family of matrices { R k , 2

0)

which are the

min imal non-negative solutions to the system of eq uations

Th e matr ices { R k , 0 ) have the following physical inte rpre tatio n. Rk) i , j

is the expected sojourn t ime in the s ta t e k + 1 , ) per un i t so journ in the

s t a t e

k ,

) before returning to level

k ,

given the process s tarted in s ta te

k ,

.

(1)

Define

mo

to be

a

positive left eigenvector of the matrix

Q P)

+

RoQz

and define m k by m k mo l ;j Re. Then i t is eas ily seen that

m

mo , l ,

.

. satisfies mQ 0 From Theorem

4.5

in Anderson [ I ] X t )

is positive recurrent if and only if m e < where e is

a

colum n vector of

ones. In this case, the equilibrium distribution x x o , I , .

.

. of X t ) s

given by xk m k / m e .

Thus to show that a particular process is positive recurrent we must

show that the equation

[Q O +

R ~ Q ~ ) ]o

has a positive solution xo such tha t

and then x is given by

where

xo

satisfies

1.3)

a n d

We can only calculate the infinite sum in

l . G )

if we know

{ R k ,

2

0).

Th is is not often the case since generally only numerical solutions can be

found for the

Rk

matr ices. Hence we m ust calculate a trunca ted sum . If

we define x ~ I < * ) ) ~ ,5

k 5 Ii

to be the s ta t ionary p robab i l i ty tha t

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

5/30

5 BRIGHT

AND TAYLOR

X 4) i s in the s ta te

( k , )

cgnditional on

X ( t )

being in the set { i, l O

5

k ,

5 j 5

M i ) then it

s

clear that

x k ( I < * ) , k 5 I(*

is given y

where

x o ( I < * )

atisfies

( 1 .3 )

and

Note tha t we are not trunc ating X ( t ) to a finite state space consisting

of all states in and below level

I

7/24/2019 Bright 1995

6/30

LEVEL DEPEND ENT QBD PROCESSES 5 1

where

Uf

and

D

are Adk-l

x

Mk-1+2t nd Mk-1

x

Mk-l-2c matrices respec-

tively and are defined recursively by

roof

In Theorem 2.2 of [14], Ramaswami and Taylor considered discrete time

LDQBD s. In discrete time LDQBD s the generator matrix

Q

in (1.1) is

replaced by a stochastic matrix

P

which has the same block-partitioned

form as Q. Suppose for the jump chain of

X t )

this matrix P consists

of the sub matrices ALk), AIk) and AF). In discrete time LDQBD7s it is

necessary to calculate the family of matrices {Rk,

3 k

which are the

minimal non-negative solutions of the equations

By Theorem 2.2 of [14] an explicit expression for

for the case where the

jump chain has an invariant measure is

where

u

and

D

re defined recursively by a set of equations similar to

equations (2.2) to (2.5). Since

S t )

s positive recurrent, from results

in [I] we know that the jump chain must be recurrent and so have an

invariant measure. Ramaswami and Taylor showed that Rk can be written

as

Rk Qk~kJQ;l,where RkJ is the matrix Rk for the jump chain and k is

defined by Qk -diag[(Q(,k))ii]. Using this expression together with (2.7j,

it is a simple matter to derive equation (2.1).

In order to use Lemma 1 to calculate Rk, we need to truncate the infinite

sum in (2.1).

A

simple way to do this is to define Rk (n) as the sum of

the first

n

1 terms in (2.1) and then to take Rk Rk(L) such that

(R k(L) Rk (L I)),,,, for some tolerance where (M),,, is the

maximum entry of the matrix

Ad

To calculate {Rk, 5 k 5 K * ) , t is not necessary to use equation (2.1)

repeatedly. By re-arranging equation (1.2),

we

have

provided the inverse exists. So a possible method for calculating {Rk,0

k 5

I

7/24/2019 Bright 1995

7/30

5 2

BRIGHT

ND T YLOR

equation (2.8) o calculate RIp-2,RIi--3, RO. n order for the recursion

to be st abl e we require th at the inverse contain non negative elements. W e

prove this is the case and th at t he inverse does exist in the appe ndix. Note

that the truncation rule given above does not guarantee that

(Rk (n )

Rk(n ,,,

< E

holds

V

n

>

L which is what we want. An algo rithm for

calculating

{xn.(I

7/24/2019 Bright 1995

8/30

LEVEL DEPENDENT Q D PROCESSES

In Algorithm 2 we calculate

U

and D:+zol a t the same s tep. Th is is

because both quantities involve the same inverse and so it is sensible to

com put e the m to gether t o avoid calculating the sam e inverse twice. To see

this we resta te equatio ns 2.2) to 2.5) for

U

and

D:+~ +~.

Note th at we calculate

in A lgorithm

2

but do not use it. However,

i t is no e xtr a effort to com pute this quan ti ty because we already have the

necessary inverse from calculating U

We refer to i and D:+2t+l as the U D-pair UD C, k) . At the end of Al-

gorithm 2 we have calculated the UD-pairs UD e, k) for for 0 , l , . . L

an d fixed k. To do this we need to calculate other UD-pairs U D n , m) for

n < e. We consider a way to com pute these quantitie s so tha t U D-pairs are

not com puted mo re than once. B y storing various UD-pairs it is possible

to develop an efficient meth od for calculating UD e, k) , 0 , .

.

L.

To

determ ine which UD -pairs to calculate and which to store, it is useful to

represent the recursive relationships expressed by equations 3.3) an d 3.4)

y diagram. Figure 1 shows that to calculate UD e, k), we need to know

the UD-pa irs UD C- 1 , k ) , U D e- 1 , k+2e -1) , and UD C- 1 , k +2.2e-1).

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

9/30

BRIGHT ND T YLOR

1 - 1 .

k

k + 2e -1 I ~ + 2 . 2 ~ - l

F ig u re

1:

Recursive re la t ionsh ip be tween UD-pa irs .

By using Figure 1 repeatedly, we see tha t t o c alculate U D 2, k) we

need first to calculate UD 1, k m for n 0 , 2 , 4 an d U D 0 , k

m

for

m 0 , 1 , .

.

6 . Similarly, to c alculate UD 3, k) we need first to calculate

U D 2 , k

m

for

m

0 , 4 , 8 , U D 1 , k

rn

for m 0 , 2 , 4 , . . 1 2 a n d

U D 0 , k

m

for

n

0 , 1 , 2 , .

.

,14. Figure 2 represents the last two

statem ents. To calculate U D 1, k) we need to calculate al l the U D-pairs

in the second smallest triangle in Figure 2. To c alculate U D 2, k) we need

to calculate al l the UD-pairs in the third smallest t r iangle an d to calculate

U D 3 , k) we need to calculate all the U D-pairs in the largest trian gle.

Figure 2 can be extended to any arbitrary value of From Figure

2

we

see that if we calculate UD l, k ) f o r 0 , 1 , 2 , 3 by directly implem enting

equa tions 3.3) an d 3.4) then we will be computing some UD-pairs over

an d over again. However, it is possible to calculate UD -pairs in an order

th at will result in not compu ting any UD-pair more than once. Th e UD-

pairs needed for Algorithm 2 can be calculated and used) in the following

order.

A l g o r i t h m

3 :

E v a l u a t i n g U D -p a ir s f o r A l g o r i t h m

compu te U D 0, k) , use i t in A lgorithm 2 and store i t

do

f o r o

f o r j k 2e -i+1 1 )2 ~o

j k + 2 2e-i+1 1) 2i in steps of size 2i

com pute U D i , j and store i t

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

10/30

LEVEL DEPENDENT QBD PROCESSES

Figure

2:

Dependencies between

UD pairs

c om p u te U D ( t , k ) , use it in Algorithm

2

and s to re i t

remove all UD -pairs from stor age except the pairs

U D ( j , k ( 2e -j 1 ) 2 j + l ) , 0 , 1 , .

.

until (Algorithm 2 s tops)

W hen Algorithm 2 stop s, we have calculated the U D-pairs UD (-t, k)

for t 0 , 1 , . . .

L

and fised I; If these UD-pairs are com puted according

to Algorithm 3 then a total of 4.2L

L

inverses must be calculated.

If the UD -pairs were calculated by directly implementing the recursive

equatio ns (3.3) and (3.4) the n a tota l of approxim ately i .5 L inverses mu st

be calculate d. Note th at w ith Algorithm 3 the number of inverses still

increases exponentially with L. However, for many practical examples

only a sm all value of L (say four or five) is needed. To see this, we give a

physical interpretation of equation (2.1).

Recall tha t (R k) i , j s interpreted as the expected sojourn t ime in the

s ta te (k

1

j

per unit sojourn in th e s ta t e ( k , i) before re turning t o level

k , given the process s tarted in s ta te (k , i) . In order t o calculate

Rk

e

need to consider a ll sample paths th at s ta rt and finish a t level k a nd d on t

visit any levels below level k 1 in between. For each of these sam ple

pa ths we have to calculate the expected sojou rn times in each of th e states

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

11/30

5 6 BRIGHT AND T YLOR

in level

k

1. Adding together the sojourn times for all the sam ple pa ths

will give us R k. R amaswam i and Taylor [14] observed th at the 1t h ter m

in (2.1) calculates the expected sojourn times in level k

1

for sample

paths that go at least as high as level

k

1 2e and no higher th an level

1 2 + . So when we tru nc ate th e sum in (2.1) to

L

1 erm s, we only

consider the samp le paths th at remain below level k

1

2L+2. Th us even

for relatively sm all

L

(say four or five) we consider pa th s which m ove up 6

or 128 levels before returning. For mo st physical processes this acc oun ts

for most of the probability mass.

Storage is an issue in Algorithm

3

In i ts present form the Algorithm

requires the storage of many U D-pairs. However, it is no t difficult to alt er

the alg orithm to decrease the am ount of storage space required. In fact it

is possible to alter Algorithm 3 so that a t any t ime only 1

1

UD-pairs

need to be in storage.

4 CHOOSING

A VALUE

FOR

I

7/24/2019 Bright 1995

12/30

LEVEL DEPEND ENT QBD PROCESSES

5 7

kl This requires approximately

4.27

=

512

inverses. Now suppose in order

to calculate we required only

L

4 .

This requires approximately

4.24

=

64 inverses. If we use the recursion in ( 2 . 8 ) to calculate

Rk

e

need to calculate a further 40 inverses giving a total of 104 inverses. The

second method is obviously the more efficient.

A

simple approach to finding a value of

I

7/24/2019 Bright 1995

13/30

BRIGHT

ND

T YLOR

until ( x ~ ; , , ( I ( ~ ~ ~ ) e

E

Set I I? which is what we

obtain from equation (1.5). To find xk,

0

5 k 5

I?

Neuts solved

Th is does not highlight the m at ri s analytic struc ture of xk for k 5 I?.

nice property of our metho d is tha t i t shows that xa has a matrix analytic

form for all

0

5 k 5 I?

In th is particular case, it is possible to d eterm ine if X ( t ) is positive

recurrent and to determine the equilibrium distribution exactly. Neuts

[12] has shown tha t X ( t ) is posit ive recurrent if and only if x Q o e x Q z e

where Q Q1+ Q o + Q 2 and rr is the solution to rrQ

o

su ch t h a t x e 1 .

Because of th e physical interpretation of R k , t is clear tha t R k RI; or

k E . If X ( t ) i s posi tive recurren t then

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

14/30

LEVEL DEPENDENT QBD PROCESSES

and the equil ibrium dis tribution can be calculated exactly us ing equation

(1.5). T hi s is one of the few cases where th is is possible. Since X ( t ) is

a level independent QBD for levels I and above, when we calculate RK,

we are in fact dealing with a level independent QBD. In this case, the

expression in

(2 .1)

simplifies to the case in Latouche and Ramaswami [7].

An algorith m for thi s case is given below.

Algorithm

5

: Calculating the equilibrium distribution

for a large number of boundary states

Calcu late RR

Recurs ively calculate RRWl, Rv2,

,

R

Solve X ~ ( Q ( ~ ~ )R ~ Q ~ ) )o s . t .

xo [E =~ n s , , n i o m ( I RE)- ] e

=

Calculate xl: k according to equat,ion

(1.5)

4.3 Dominating Process Exists

Suppose X ( t ) is such th at the following condition holds.

Condition

1 V k 3 s . t ( ~ f ) ) i , ~

0.

If condition

1

holcls, then at any phase a t any level, X ( t ) moves down t o

a

state in the level below according to some positive rate. Note that this

class of processes is a non-trivia l class. Th e PH M co queue modelled

a s a LDQ BD ha s this property. W hen condition 1 holds, it is possible to

construct a second LDQBD that s tochastically dom inates th e firs t process .

This second process a lso has the property that i ts marginal dis tribution

over the levels is very easy to calculate. It is the n po ssible to calc ulate a

value of I{ using this marginal distribu tion. Before we present o ur results,

we define what it means for a process X ( t ) on a partia l ly ordered s ta te

space S , to stochastically dominate S ( t ) .

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

15/30

51 BRIGHT AND TAYLOR

Definition : Let and j be probability density functions on a partially

ordered set

S

Then T stocl~astically ominates

y

writ ten y 5 j j if for

all monotone functions

f

.) : S R

Definition

:

Th e process ~ ( t )i th s tate space

S

and density function

~ ( t )tochastically dominates the process X ( t ) with s tate space S and

density function

x ( t )

f

~ ( 0 )

s

a : O )

x ( t )

5

2 ( t )

for all

t. (4.2)

If

X ( t )

stochastically dominates

X ( t )

hen equation

(4.2)

holds

V t 0

so we can take the limit as t - m of both sides of the inequality and

obta in

x

0 ,

g ( 1 , O) 0 .

q i, 1 ) (Q;- e), ,, , ,,

i

>

0 .

So {j,,,

n

1 ) is given by

Note that a sufficient condition for G-'

< m

is tha t ~ i ,

+ ) / i j ( i+ 1 , i

5

Y < 1 V i I for some I since then the terms in the infinite sum will be

Downloadedby[UQLibrary]at10:2024April2015

7/24/2019 Bright 1995

17/30

5 2

BRIGHT

ND T YLOR

bounded above by a geometric series. If { I n ,n > 1 ) exists, must exist

and therefore x must exist since ~ ( t )tochastically dominates X ( t ) .

Now we want to find I 1 SO it is sufficient to find K*

such that

03

C j n < ~ . (4 .11 )

n = K *

k)

Often it is the case that we have an analytic form for the matrices

Qi

and hence an analytic form for the rates q ( i , ) . In this case it is possible

-

to compute C: . C n analytically, using (4 .10 ) and hence we can find

K*

from (4 .11) .

When the rates q ( i , j ) are not l ino~ l l nalytically, we can still compute

the quantity { ( N ) , n 5 N ) given by

where

j 1 ( N )

s found using c=~ j n ( N ) = 1. It is clear that for any N

>

1 ,

we have ( N ) > V 1

n

N and (m) = jn. Hence we have

f n ~ ) C n V l

5

n 5

N N 21.

We can then choose I i * such that TIi.(lr'*)

< E.

The reasoning behind

this choice of I i * is as follows. Firstly, .(I

7/24/2019 Bright 1995

18/30

LEVEL DEPEND ENT QBD PROCESSES 513

I * is sufficiently large for x k I < * ) o approximate xk . At the very least ,

x k I < * ) an always be interpreted as an upper bound for the equilibr ium

distr ibution.

We now give an algorithm for determining I

7/24/2019 Bright 1995

19/30

4

BRIGHT

AND TAYLOR

Using a proof similar to that of Theorem 1 it can be shown that ~ ( t )

stochastically dominates X ( t ) provided condition 2 holds for X ( t ) . Note

that ~ ( t )s irreducible only on the levels I , +

1 ,

+ 2 .

.

In the

same way that we defined in(^) we can define { j , ~ ) , n N and

this will have a similar form to defined in 4.12). Then we choose I *

such that

i r c w ( I < * )

E SO when condition 2 holds, we have the following

algorithm.

Algor i thm 7 Calculating { x k ( I < * ) , O k I

7/24/2019 Bright 1995

20/30

LEVEL DEPENDENT QBD PROCESSES

515

where D v is a diagonal matrix with diagonal elements equal to the

elements of the vector v.

Condition 1 holds provided pl,p2 > 0 so we can use Algorithm 6 to

compute the equilibrium distribution. It is easy to show th at the domi-

nating process is positive recurrent for all values of and so th e original

process is always positive recurrent. Consider the case where

Q

is a 2-stage

Markov process and Q is given by

and the parameter values are pl 2, ~ 2 1,

X

40 and X 5. For this

and the other examples below, we took R k R k n ) s.t R k n ) R k n

I)),,, 10-12. In Algorithm 6 we took 6 10-lo which gave a value of 88

for I< .n Table 1 we give the marginal probabilities

l k

n order to see if

the value

I