CS352 - Introduction to Queuing Theory

Rutgers University

CS352 Fall,2005 2

Queuing theory definitions (Bose) “the basic phenomenon of queueing arises

whenever a shared facility needs to be accessed for service by a large number of jobs or customers.”

(Wolff) “The primary tool for studying these problems [of congestions] is known as queueing theory.”

(Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queueing Theory." "Any system in which arrivals place demands upon a finite capacity resource may be termed a queueing system.”

(Mathworld) “The study of the waiting times, lengths, and other properties of queues.”

http://www2.uwindsor.ca/~hlynka/queue.html

CS352 Fall,2005 3

Applications of Queuing Theory Telecommunications Traffic control Determining the sequence of computer

operations Predicting computer performance Health services (eg. control of hospital bed

assignments) Airport traffic, airline ticket sales Layout of manufacturing systems.

http://www2.uwindsor.ca/~hlynka/queue.html

CS352 Fall,2005 4

Example application of queuing theory

In many retail stores and banks multiple line/multiple checkout system a

queuing system where customers wait for the next available cashier

We can prove using queuing theory that : throughput improves increases when queues are used instead of separate lines

http://www.andrews.edu/~calkins/math/webtexts/prod10.htm#QT

CS352 Fall,2005 5

Example application of queuing theory

http://www.bsbpa.umkc.edu/classes/ashley/Chaptr14/sld006.htm

CS352 Fall,2005 6

Queuing theory for studying networks

View network as collections of queues FIFO data-structures

Queuing theory provides probabilistic analysis of these queues

Examples: Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost

CS352 Fall,2005 7

Little’s Law

Little’s Law: Mean number tasks in system = mean arrival rate x mean response time Observed before, Little was first to prove

Applies to any system in equilibrium, as long as nothing in black box is creating or destroying tasks

Arrivals Departures

System

CS352 Fall,2005 8

Proving Little’s Law

J = Shaded area = 9

Same in all cases!

1 2 3 4 5 6 7 8

Packet #

Time

123

1 2 3 4 5 6 7 8

# in System

123

Time

1 2 3

Time inSystem

Packet #

123

Arrivals

Departures

CS352 Fall,2005 9

Definitions J: “Area” from previous slide N: Number of jobs (packets) T: Total time l: Average arrival rate

N/T W: Average time job is in the system

= J/N L: Average number of jobs in the system

= J/T

CS352 Fall,2005 10

1 2 3 4 5 6 7 8

# in System(L) 1

23

Proof: Method 1: Definition

Time (T) 1 2 3

Time inSystem(W)

Packet # (N)

123

=

WL TN )(

NWTLJ

WL )(

CS352 Fall,2005 11

Proof: Method 2: Substitution

WL TN )(

WL )(

))(( NJ

TN

TJ

TJ

TJ Tautology

CS352 Fall,2005 12

Model Queuing System

Server System Queuing System

Queue Server

Queuing System

Use Queuing models to Describe the behavior of queuing systems Evaluate system performance

CS352 Fall,2005 13

Characteristics of queuing systems

Arrival Process The distribution that determines how the tasks

arrives in the system. Service Process

The distribution that determines the task processing time

Number of Servers Total number of servers available to process the

tasks

CS352 Fall,2005 14

Kendall Notation 1/2/3(/4/5/6)

Six parameters in shorthand First three typically used, unless specified

1. Arrival Distribution

2. Service Distribution

3. Number of servers

4. Total Capacity (infinite if not specified)

5. Population Size (infinite)

6. Service Discipline (FCFS/FIFO)

CS352 Fall,2005 15

Distributions

M: stands for "Markovian", implying exponential distribution for service times or inter-arrival times.

D: Deterministic (e.g. fixed constant) Ek: Erlang with parameter k

Hk: Hyperexponential with param. k G: General (anything)

CS352 Fall,2005 16

Kendall Notation Examples

M/M/1: Poisson arrivals and exponential service, 1 server, infinite

capacity and population, FCFS (FIFO) the simplest ‘realistic’ queue

M/M/m Same, but M servers

G/G/3/20/1500/SPF General arrival and service distributions, 3 servers, 17

queue slots (20-3), 1500 total jobs, Shortest Packet First

CS352 Fall,2005 17

Poisson Process

For a poisson process with average arrival rate , the probability of seeing n arrivals in time interval delta t

0...)2Pr(

)1Pr()(...]!2

)(1[)1Pr(

1)0Pr()(1...!2

)(1)0Pr(

)(!

)()Pr(

2

2

ttott

ttte

ttott

te

tnEn

ten

t

t

nt

CS352 Fall,2005 18

Poisson process & exponential distribution

Inter-arrival time t (time between arrivals) in a Poisson process follows exponential distribution with parameter

1)(

)Pr(

tE

et t

CS352 Fall,2005 19

Analysis of M/M/1 queue

Given: • l: Arrival rate of jobs (packets on input link) • m: Service rate of the server (output link)

Solve: L: average number in queuing system Lq average number in the queue W: average waiting time in whole system Wq average waiting time in the queue

CS352 Fall,2005 20

M/M/1 queue model

l m

1

Wq

W

L

Lq

CS352 Fall,2005 21

Solving queuing systems

4 unknowns: L, Lq W, Wq

Relationships: L=lW Lq=lWq (steady-state argument) W = Wq + (1/m)

If we know any 1, can find the others Finding L is hard or easy depending on the type of

system. In general:

0

n

nnPL

CS352 Fall,2005 22

Analysis of M/M/1 queue

Goal: A closed form expression of the probability of the number of jobs in the queue (Pi) given only l and m

CS352 Fall,2005 23

Equilibrium conditions

n+1nn-1

l l ll

m mm m

)(tPnDefine to be the probability of having n tasks in the system at time t

0)()(

lim,)(lim, when Stablize

)()()()()()(

)()()()(

)]1)()[(()]1)()[((])1)(1)[(()(

)]1)()[((])1)(1)[(()(

11

1000

11

100

t

tPttPPtP

tPtPtPt

tPttP

tPtPt

tPttP

tttPtttPtttttPttP

tttPtttttPttP

nn

tnn

t

nnnnn

nnnn

CS352 Fall,2005 24

Equilibrium conditions

11

10

)(

nnn PPP

PP

n+1nn-1

l l ll

m mm m

CS352 Fall,2005 25

Solving for P0 and Pn

Step 1

Step 2

0,0

2

201 , PPPPPPn

n

0

0

000

1,1,1

n

n

n

n

nn PPthenP

CS352 Fall,2005 26

Solving for P0 and Pn

Step 3

Step 4

1ρρ1

1

ρ1

ρ1ρ,ρ

00

n

n

n

n

then

ρ1ρandρ1ρ

1

0

0

nn

n

n

PP

CS352 Fall,2005 27

Solving for L

0

n

nnPL )1(0

n

nn )1(1

1

n

nn

11)1( d

d

0

)1(n

ndd

2)1(1)1(

)1(

CS352 Fall,2005 28

Solving W, Wq and Lq

11LW

)(11

WWq

)()(

2

qq WL

CS352 Fall,2005 29

Online M/M/1 animation

http://www.dcs.ed.ac.uk/home/jeh/Simjava/queueing/mm1_q/mm1_q.html

CS352 Fall,2005 30

Response Time vs. Arrivals

1W

Waiting vs. Utilization

0

0.05

0.1

0.15

0.2

0.25

0 0.2 0.4 0.6 0.8 1 1.2

W(s

ec)

CS352 Fall,2005 31

Stable Region

Waiting vs. Utilization

0

0.005

0.01

0.015

0.02

0.025

0 0.2 0.4 0.6 0.8 1

W(s

ec)

linear region

CS352 Fall,2005 32

Example

On a network gateway, measurements show that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model, what is the probability of buffer overflow if the gateway had only 13 buffers. How many buffers are needed to keep packet loss below one packet per million?

CS352 Fall,2005 33

Example

Measurement of a network gateway: mean arrival rate (l): 125 Packets/s mean response time (m): 2 ms

Assuming exponential arrivals: What is the gateway’s utilization? What is the probability of n packets in the gateway? mean number of packets in the gateway? The number of buffers so P(overflow) is <10-6?

CS352 Fall,2005 34

Example

Arrival rate λ = Service rate μ = Gateway utilization ρ = λ/μ = Prob. of n packets in gateway =

Mean number of packets in gateway =

CS352 Fall,2005 35

Example Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps Gateway utilization ρ = λ/μ = 0.25 Prob. of n packets in gateway =

Mean number of packets in gateway =

nn )25.0(75.0ρ)ρ1(

33.057.0

25.0

ρ1

ρ

CS352 Fall,2005 36

Example

Probability of buffer overflow:

To limit the probability of loss to less than 10-

6:

CS352 Fall,2005 37

Example Probability of buffer overflow:

= P(more than 13 packets in gateway)

To limit the probability of loss to less than 10-6:

CS352 Fall,2005 38

Example Probability of buffer overflow:

= P(more than 13 packets in gateway) = ρ13 = 0.2513 = 1.49x10-8

= 15 packets per billion packets To limit the probability of loss to

less than 10-6:

CS352 Fall,2005 39

Example Probability of buffer overflow:

= P(more than 13 packets in gateway) = ρ13 = 0.2513 = 1.49x10-8

= 15 packets per billion packets To limit the probability of loss to

less than 10-6:

610ρ n

CS352 Fall,2005 40

Example To limit the probability of loss to

less than 10-6:

or

25.0log/10log 6n

610ρ n

CS352 Fall,2005 41

Example To limit the probability of loss to

less than 10-6:

or

= 9.96

25.0log/10log 6n

610ρ n

CS352 Fall,2005 42

Empirical Example

M/M/msystem

http://www.powershow.com/view.php?id=2858801&t=queuing-theory