1

Chapters 8Chapters 8

Overview ofOverview ofQueuing Queuing AnalysisAnalysis

Chapter 8 Overview of Queuing Analysis2

Projected vs. Actual Response Projected vs. Actual Response TimeTime

Chapter 8 Overview of Queuing Analysis3

Introduction- MotivationIntroduction- Motivation How to analyze changes in network How to analyze changes in network

workloads? (i.e., a helpful workloads? (i.e., a helpful tooltool to use) to use) Analysis of system (network) load and Analysis of system (network) load and

performance characteristicsperformance characteristics– response timeresponse time– throughputthroughput

Performance tradeoffs are Performance tradeoffs are often not often not intuitiveintuitive

Queuing theory, although Queuing theory, although mathematically complex, often mathematically complex, often makes analysis very straightforwardmakes analysis very straightforward

Chapter 8 Overview of Queuing Analysis4

Single-Server Queuing SystemSingle-Server Queuing System

QueuingSystem

(Delay Box)

Items ArrivingItems Arriving

(message, packet, cell)(message, packet, cell)

Items Lost Items Lost

Items DepartingItems Departing

Chapter 8 Overview of Queuing Analysis5

Parameters for Single-Server Parameters for Single-Server Queuing SystemQueuing System

Comments, assuming queue has infinite capacity:1. At = 1, server is working 100% of the time (saturated), so items are

queued (delayed) until they can be served. Departures remain constant (for same L).

2. Traffic intensity, u = L/R. Note that Ts = L/R, so:max = 1 / Ts = 1 / (L/R) is the theoretical maximum arrival rate,

and that

Lmax/R = u = 1 at the theoretical maximum arrival rate

Chapter 8 Overview of Queuing Analysis6

The Fundamental Task of The Fundamental Task of Queuing AnalysisQueuing Analysis

Given:Given:• Arrival rate, Arrival rate, • Service time, Service time, TTss

• Number of servers, Number of servers, NN

Determine:Determine:• Items waiting, Items waiting, ww• Waiting time, Waiting time, TTww

• Items queued, Items queued, rr• Residence time, Residence time, TTrr

Chapter 8 Overview of Queuing Analysis7

Queuing Process - ExampleQueuing Process - Example

General Expression:General Expression:TTRn+1Rn+1 = T = TSn+1Sn+1 + MAX[0, D + MAX[0, Dnn – A – An+1n+1]]

Depth of the Queue

Chapter 8 Overview of Queuing Analysis8

General Characteristics of General Characteristics of Network Queuing ModelsNetwork Queuing Models

Item populationItem population– generally assumed to be generally assumed to be infiniteinfinite therefore, therefore,

arrival rate is persistentarrival rate is persistent Queue sizeQueue size

– infiniteinfinite, therefore no loss, therefore no loss– finite, more practical, but often immaterialfinite, more practical, but often immaterial

Dispatching discipline Dispatching discipline – FIFOFIFO, typical, typical– LIFOLIFO– Relative/Preferential, based on QoSRelative/Preferential, based on QoS

Chapter 8 Overview of Queuing Analysis9

Multiserver Queuing SystemMultiserver Queuing System

Comments:1. Assuming N identical servers, and is the utilization of each server. 2. Then, N is the utilization of the entire system, and the maximum

utilization is N x 100%.3. Therefore, the maximum supportable arrival rate that the system can

handle is: max = N / Ts

Chapter 8 Overview of Queuing Analysis10

Multiple Single-Server Queuing Multiple Single-Server Queuing SystemsSystems

Chapter 8 Overview of Queuing Analysis11

Basic Queuing RelationshipsBasic Queuing Relationships

GeneralGeneral Single Single ServerServer MultiserverMultiserver

rr = = TTrr Little’s Little’s FormulaFormula = = TTss

= =

ww = = TTww Little’s Little’s FormulaFormula rr = = ww + + uu = = TTss = = NN

TTrr = = TTww + + TTss r = w + Nr = w + N

TTs s

NN

Chapter 8 Overview of Queuing Analysis12

Kendall’s notationKendall’s notation Notation is Notation is X/Y/NX/Y/N, where:, where:

X is distribution of interarrival X is distribution of interarrival timestimes

Y is distribution of service timesY is distribution of service timesN is the number of serversN is the number of servers

Common distributionsCommon distributions G = general distribution if interarrival times G = general distribution if interarrival times

or service timesor service times GI = general distribution of interarrival time GI = general distribution of interarrival time

with the restriction that they are independentwith the restriction that they are independent M = exponential distribution of interarrival M = exponential distribution of interarrival

times (Poisson arrivals – p. 167) and service times (Poisson arrivals – p. 167) and service timestimes

D = deterministic arrivals or fixed length D = deterministic arrivals or fixed length serviceservice

M/M/1? M/D/1?M/M/1? M/D/1?

Chapter 8 Overview of Queuing Analysis13

Important Formulas for Important Formulas for Single-Server Queuing Single-Server Queuing SystemsSystems

Note Coefficient of variation: if Ts = Ts => exponential if Ts = 0 => constant

Chapter 8 Overview of Queuing Analysis14

Mean Number of Items in Mean Number of Items in System (System (rr)- Single-Server )- Single-Server QueuingQueuing

Ts/Ts = Coefficient of variation

M/M/1

M/D/1

Chapter 8 Overview of Queuing Analysis15

Mean Residence Time – (Mean Residence Time – (TTrr) ) Single-Server QueuingSingle-Server Queuing

M/M/1

M/D/1

Chapter 8 Overview of Queuing Analysis16

Multiple Server Queuing Multiple Server Queuing SystemsSystems

Multiple Multiple Single-Single-Server Server Queuing Queuing SystemSystem

Multiserver Multiserver Queuing Queuing SystemSystem

Chapter 8 Overview of Queuing Analysis17

Important Formulas for Important Formulas for Multiserver QueuingMultiserver Queuing

Note:Note:Useful only inUseful only inM/M/N case,M/M/N case,with equal with equal service times service times at all N at all N servers.servers.

Chapter 8 Overview of Queuing Analysis18

Multiple Server Queuing Multiple Server Queuing Example Example (p. 203)(p. 203)

Single serverM/M/1 (2nd Floor)

MultiserverM/M/? (2nd Floor)

Multiple Single server

M/M/1 (1st Floor)

M/M/1 (2nd Floor)

M/M/1 (3rd Floor)

Chapter 8 Overview of Queuing Analysis19

MultiServer vs. Multiple Single-MultiServer vs. Multiple Single-Server Queuing System Server Queuing System Comparison Comparison (from example problem, pp. 203-(from example problem, pp. 203-204)204)

Single server case (M/M/1):Single server case (M/M/1):Single server utilization: Single server utilization: = 10 engineers x 0.5 hours each / 8 = 10 engineers x 0.5 hours each / 8

hour work dayhour work day

= 5/8 = .625= 5/8 = .625

Average time waiting: TAverage time waiting: Tww = = TTss / 1 - / 1 - = 0.625 x 30 / .375 = 50 = 0.625 x 30 / .375 = 50

minutesminutes

Arrival rate: Arrival rate: = 10 engineers per 8 hours = 10/480 = 0.021 = 10 engineers per 8 hours = 10/480 = 0.021

engineers/minuteengineers/minute

9090thth percentile waiting time: m percentile waiting time: mTTww(90) = T(90) = Tww// x ln(10 x ln(10) = 146.6 minutes) = 146.6 minutes

Average number of engineers waiting: w = Average number of engineers waiting: w = TTww = 0.021 x 50 = 1.0416 = 0.021 x 50 = 1.0416

engineersengineers

Chapter 8 Overview of Queuing Analysis20

Example: Router QueuingExample: Router Queuing

InternetInternet ……96009600bpsbps

= 5 packets/sec= 5 packets/secL = 144 octetsL = 144 octets

From data provided:From data provided:• TTs s = L/R = (144x8)/9600 = .12sec= L/R = (144x8)/9600 = .12sec = = TTs s = 5 packets/sec x .12sec = = 5 packets/sec x .12sec =

.6.6

Determine:Determine:1.1. TTrr= T= Tss / (1- / (1-) = .12sec/.4 = .3 sec) = .12sec/.4 = .3 sec2.2. r = r = / (1- / (1-) = .6/.4 = 1.5 ) = .6/.4 = 1.5

packetspackets

3. m3. mrr(90) = - 1 = 3.5 (90) = - 1 = 3.5 packetspackets

4.4. mmrr(95) = - 1 = 4.8 (95) = - 1 = 4.8 packetspackets

ln(1-.90)ln(1-.90)ln (.6)ln (.6)

ln(1-.95)ln(1-.95)ln (.6)ln (.6)

For 3 & 4, use:For 3 & 4, use:

mmrr(y) = - (y) = - 1 1

ln(1 – ln(1 – y/100)y/100)ln ln

Chapter 8 Overview of Queuing Analysis21

Priorities in Queues – Two Priorities in Queues – Two priority classespriority classes

r

Chapter 8 Overview of Queuing Analysis22

Priorities in Queues – Priorities in Queues – ExampleExample

Router queue services two packet Router queue services two packet sizes:sizes:• Long = 800 octetsLong = 800 octets• Short = 80 octetsShort = 80 octets• Lengths exponentially distributedLengths exponentially distributed• Arrival rates are equal, 8packets/secArrival rates are equal, 8packets/sec• Link transmission rate is 64KbpsLink transmission rate is 64Kbps• Short packets are priority 1,Short packets are priority 1,• Longer packets are priority 2Longer packets are priority 2From data above, calculate:From data above, calculate:TTs 1s 1 = L = Lshortshort/R = (80 x 8) / 64000 = .01 /R = (80 x 8) / 64000 = .01 secsecTTs 2s 2 = L = Llonglong/R = (800 x 8) / 64000 = .1 /R = (800 x 8) / 64000 = .1 secsec11 = = TTs 1 s 1 = 8 x 0.01 = 0.08= 8 x 0.01 = 0.08

22 = = TTs 2 s 2 = 8 x 0.1 = 0.8= 8 x 0.1 = 0.8

= = 1 1 ++ 2 2 = 0.88= 0.88

Find the average Queuing Delay (Find the average Queuing Delay (TTrr) ) through the router:through the router:

TTr1r1 = T = Ts1 s1 + +

= .01 + = 0.098 = .01 + = 0.098 sec sec

TTr2r2 = T = Ts2 s2 + +

= .1 + = 0. 833 sec = .1 + = 0. 833 sec

TTrr = T = Tr1 r1 + + TTr2 r2

= .5 x .098 + .5 x .833 = 0.4655 = .5 x .098 + .5 x .833 = 0.4655 secsec

11 TTs 1 s 1 + + 2 2 TTs 2s 2

1 - 1 - 11.08 x .01 .08 x .01 + + .8 .8 x .1x .1

1-.081-.08

TTr 1 r 1 -- TTs 1s 1

1 - 1 - .098.098 -- .01 .01

1 - .881 - .88

11

22

64Kbps64Kbps

TTrr

Chapter 8 Overview of Queuing Analysis23

Network of QueuesNetwork of Queues

Chapter 8 Overview of Queuing Analysis24

Elements of Queuing NetworksElements of Queuing Networks

Chapter 8 Overview of Queuing Analysis25

Queuing NetworksQueuing Networks

Chapter 8 Overview of Queuing Analysis26

Jackson’s Theorem and Jackson’s Theorem and Queuing NetworksQueuing Networks Assumptions:Assumptions:

– the queuing network has m nodes, each providing the queuing network has m nodes, each providing exponential serviceexponential service

– items arriving from outside the system at any node items arriving from outside the system at any node arrive with a Poisson ratearrive with a Poisson rate

– once served at a node, an item moves immediately once served at a node, an item moves immediately to another with a fixed probability, or leaves the to another with a fixed probability, or leaves the networknetwork

Jackson’s Theorem states: Jackson’s Theorem states: – each node is an independent queuing system with each node is an independent queuing system with

Poisson inputs determined by partitioning, merging Poisson inputs determined by partitioning, merging and tandem queuing principlesand tandem queuing principles

– each node can be analyzed separately using the each node can be analyzed separately using the M/M/1 or M/M/N modelsM/M/1 or M/M/N models

– mean delays at each node can be added to mean delays at each node can be added to determine mean system (network) delaysdetermine mean system (network) delays

Chapter 8 Overview of Queuing Analysis27

Jackson’s Theorem - Application Jackson’s Theorem - Application in Packet Switched Networksin Packet Switched Networks

Packet SwitchedPacket SwitchedNetworkNetwork

External load, offered to network:External load, offered to network: = = jkjk

where:where: = = total workload in total workload in packets/secpackets/sec jk jk = = workload between source j workload between source j

and destination kand destination k N = total number of (external) N = total number of (external) sources and destinationssources and destinations

N NN N

j=1 j=1 k=2k=2

Internal load:Internal load:

= = ii

where:where: = = total on all links in networktotal on all links in network i i = = load on link iload on link i

L = total number of linksL = total number of links

L L

i=i=11

Note:Note:• Internal > offered loadInternal > offered load• Average length for all paths:Average length for all paths: E[number of links in path] = E[number of links in path] = //• Average number of item waiting Average number of item waiting and being served in link i: rand being served in link i: rii = = i i

TTriri

• Average delay of packets sent Average delay of packets sent through the network is:through the network is:

T = T =

where: M is average packet length where: M is average packet length andand RRi i is the data rate on link iis the data rate on link i

11

L L

i=i=11

MMii

RRii - - MMii

Chapter 8 Overview of Queuing Analysis28

Estimating Model Estimating Model ParametersParametersTo enable queuing analysis using To enable queuing analysis using

these models, we must these models, we must estimate estimate certain parameterscertain parameters::– Mean and standard deviation of Mean and standard deviation of

arrival ratearrival rate– Mean and standard deviation of Mean and standard deviation of

service timeservice time (or, packet size) (or, packet size)Typically, these estimates use Typically, these estimates use

sample measurementssample measurements taken from taken from an existing systeman existing system

Chapter 8 Overview of Queuing Analysis29

Sample Means for Exponential Sample Means for Exponential DistributionDistribution

Sampling:• The mean is

generally the most important quantity to estimate:

() = Xi

• Sample mean is itself a random variable

• Central Limit Theorem: the probability distribution tends to normal as sample size, N, increases for virtually all underlying distributions

• The mean and variance of X can be calculated as:

E[]= E[X] = Var[]= 2

x/N

N

i = 1

1N