Introduction to Queuing Theory
2
Queuing theory definitions
(Kleinrock) “We study the phenomena of standing, waiting, and serving, and we call this study Queueing Theory."
(Mathworld) “The study of the waiting times, lengths, and other properties of queues.”
http://www2.uwindsor.ca/~hlynka/queue.html
3
Queuing theory for studying networks View network as collections of queues
FIFO data-structures Queuing theory provides probabilistic
analysis of these queues Examples:
Average length Average waiting time Probability queue is at a certain length Probability a packet will be lost
4
Model Queuing System
Server System Queuing System Queue Server
Queuing System
Use Queuing models to Describe the behavior of queuing systems Evaluate system performance
5
Characteristics of queuing systems Arrival Process
The distribution that determines how the tasks arrives in the system.
Service Process The distribution that determines the task
processing time Number of Servers
Total number of servers available to process the tasks
6
Kendall Notation 1/2/3(/4/5/6) Six parameters in shorthand
• First three typically used, unless specified 1. Arrival Distribution2. Service Distribution3. Number of servers 4. Total Capacity (infinite if not specified) 5. Population Size (infinite) 6. Service Discipline (FCFS/FIFO)
7
Arrival time distributions M: stands for "Markovian“, Poisson
arrivials D: Deterministic (e.g. fixed constant) G: General (anything)
8
Kendall Notation Examples M/M/1:
Poisson arrivals and exponential service, 1 server, infinite capacity and population, FCFS (FIFO)
the simplest ‘realistic’ queue M/M/m
Same, but M servers G/G/3/20/1500/SPF
General arrival and service distributions, 3 servers, 17 queue slots (20-3), 1500 total jobs, Shortest Packet First
9
Analysis of M/M/1 queue Given:
• : Arrival rate of jobs (packets on input link) • : Service rate of the server (output link)
Solve: L: average number in queuing system Lq average number in the queue W: average waiting time in whole system Wq average waiting time in the queue
10
M/M/1 queue model
Wq
W
LLq
11
Little’s Law
Little’s Law: Average number of jobs in system = average arrival rate of jobs × average service time Observed before, Little was first to prove
Applies to any system in equilibrium, as long as nothing in black box is creating or destroying jobs
Arrivals Departures
System
12
Formally, Little’s Law is
L = W
: Average arrival rate W: Average time job is in the system (average service
time) L: Average number of jobs in the system
Average number of jobs in system = average arrival rate × average service time for each job
13
Proof of Little’s Law:Let J be the sum of all service time, that is, the sum of time spent
on each job. E.g., J = j1+j2+j3…., where j1 is the service time of job 1, and so on.
Let N be the number of jobs (packets).Let T be the total service time of the system on the jobs. Note
that T ≤ J. We know is the average arrival rate, W is the average service
time of jobs, and L is the average number of jobs in the system.
On the right-hand side, by definitions, in a stable system we have
W)( WTN )( ))(( N
JTN T
JBy definition, J is the sum of service time for all jobs, T is the total service time of
the system. Thus, J/T is the average number of jobs in the system, which is L. Therefore, the Little’s Law holds.
14
An example of J
J = Shaded area = 9
Same in all cases!
1 2 3 4 5 6 7 8
Packet #
Time
123
1 2 3 4 5 6 7 8
# in System
123
Time
1 2 3
Time inSystem
Packet #
123
Arrivals
Departures
15
M/M/1 queue model
Wq
W
LLq
L=λWLq=λWq
• : Arrival rate of jobs (packets on input link)
• : Service rate of the server (output link)
L: average number in queuing system
Lq average number in the queue
W: average waiting time in whole system
Wq average waiting time in the queue
16
Poisson process Used to model random events in time
that are largely independent of one another
E.g., a event may be that a customer arrives at the ice cream shop
E.g., or you see a roadkill when driving E.g., or a packet arrives at the router
17
Poisson Process For a poisson process with average
arrival rate , the probability of seeing n arrivals in time interval Δt
0...)2Pr(
)1Pr()(...]!2)(1[)1Pr(
1)0Pr()(1...!2)(1)0Pr(
)(!
)()Pr(
2
2
ttottttte
ttottte
tnEnten
t
t
nt
18
Poisson process & exponential distribution Inter-arrival time t (time between
arrivals) in a Poisson process follows exponential distribution with parameter
1)(
)Pr(
tE
et t
19
M/M/1 queue model
1
Wq
W
LLq
L=λWLq=λWq
W = Wq + (1/μ)
• : Arrival rate of jobs (packets on input link)
• : Service rate of the server (output link)
L: average number in queuing system
Lq average number in the queue
W: average waiting time in whole system
Wq average waiting time in the queue
20
Solving queuing systems 4 unknowns: L, Lq W, Wq Relationships:
L=W Lq=Wq W = Wq + (1/)
If we know any 1, can find the others
0
n
nnPL
21
Analysis of M/M/1 queue
Goal: A closed form expression of the probability of the number of jobs in the queue (Pi) given only and
Online M/M/1 animation http://www.dcs.ed.ac.uk/home/jeh/Simjava/
queueing/mm1_q/mm1_q.html
22
Equilibrium conditions
10
11
10
)(
nnn PPPPP
Pn is the probability that in equilibrium the system has n number of jobs
n+1nn-1
23
Solving for P0 and Pn at equilibrium Step 1
Step 20,0
2
201 , PPPPPPn
n
0
0
000
1,1,1n
n
n
n
nn PPthenP
Verify this after class
24
Step 3
Step 4
, then
n
n0
n 1
1 n0
1
1 1
P0 1
nn0
1 and Pn n 1
Solving for P0 and Pn at equilibrium
Pn: probability of n jobs in the system
25
Solving for L
0
n
nnPL )1(0
n
nn )1(1
1
n
nn
(1 ) dd
11
0
)1(n
ndd
(1 ) 1(1 )2
)1(
26
Solving W, Wq and Lq
W L
1 1
Wq W 1
1
( )
Lq Wq ( )
2
( )
L
27
Response time vs. utilization (ρ)
1W
Waiting vs. Utilization
0
0.05
0.1
0.15
0.2
0.25
0 0.2 0.4 0.6 0.8 1 1.2
W(s
ec)
28
Stable Region
Waiting vs. Utilization
0
0.005
0.01
0.015
0.02
0.025
0 0.2 0.4 0.6 0.8 1
W(s
ec)
linear region
29
An example On a network gateway, measurements show
that the packets arrive at a mean rate of 125 packets per second (pps) and the gateway takes about 2 millisecs to forward them. Assuming an M/M/1 model.
Measurement of a network gateway: mean arrival rate (l): 125 Packets/s mean response time (m): 2 ms
Assuming exponential arrivals: What is the gateway’s utilization? What is the probability of n packets in the gateway? mean number of packets in the gateway?
30
Example cont’d Arrival rate λ = 125 pps Service rate μ = 1/0.002 = 500 pps Gateway utilization ρ = λ/μ = 0.25 Prob. of n packets in gateway = Pn =
Mean number of packets in gateway = L =
nn )25.0(75.0ρ)ρ1(
33.057.025.0
ρ1ρ