CSCI1600: Embedded and Real Time SoftwareLecture 19: Queuing Theory

Steven Reiss, Fall 2015

Queuing Theory History

Agner Krarup Erlang (1878-1929)

Analysis of telephone networks (1908-1922) Why does this involve queues?

Analysis of Telelphone Networks

Issues How many operators are needed

How many circuits are needed

Queuing Theory A branch of probability theory studying queues

Applications Computer systems

Communication networks

Call centers

Transportation

…

Arriving Jobs

ServerDeparting Jobs

Parameters: Arrivals

Arrival rate λ How fast jobs are arriving

Regular or random Random = Poisson process

With some distribution

Arriving Jobs

ServerDeparting Jobs

Parameters: service

Service demand S – amount of service the job needs Measured in various terms (time, size, …)

Regular or random (with a given distribution)

Service capacity C – how fast server process jobs Service rate μ = C/S

Service time = S/C = 1/μ

Arriving Jobs

ServerDeparting Jobs

Parameter: Service

Multiple symmetric servers m parallel servers constitute a single serving station

Each has capacity C

Overall capacity depends on number of serviced jobs C(N) = min(N,m)C

Arriving Jobs

Server 1Departing Jobs

Server m

Parameters: Queue

Scheduling discipline First come, first served (FCFS, FIFO)

Priority queue (Shortest in, first out)

OS I/O scheduler

Preemptive vs. nonpreemptive servicing Preemptive servicing can be interrupted by higher priority job

Linux real-time thread scheduling

Arriving Jobs

ServerDeparting Jobs

Statistical Basis for Queuing Theory

Arrival probability distribution

Service probability distribution

Determine various properties of the system Average queue size

Maximum expected queue size

Average wait time

Maximum expected wait time

Poisson Arrival A completely random arrival process

The number of jobs is very large

Jobs are independent of each other

Job arrivals are not simultaneous

λ: the average arrival rate

Examples

Phone calls to a switchboard

Radioactive decay

Customers going to checkout

Trains needing switches thrown

Poisson Process

Represented by Poisson distribution Probablity of seeing i jobs during the time interval T

P[n(t) = i] = (λT)i/i! * e-λT

PDF CDF

Poisson Process Properties

Memoryless The number of arrivals in the time interval (T;T+Δt)

Does not depend on the number before T

Superposition

Combination of m independent Poisson processes with λ1,λ2,…λm is a Poisson process with λ= λ1+λ2+…+λm

Interval between job arrivals

Is distributed exponentially fx(x) = λe-λx

Exponential Service Distribution

PDF: fs(t) = μe-μt where μ is the service rate

CDF: Fs(t) = 1 - μe-μt

Mean = Std. Deviation = 1/μ = service time

Other Distributions

Erlang The job visits an exponential server k times

Hyperexponential

k types of jobs, each has probability πi

Each type of job has its own service time

Each type of jobs passed to its own server

General With known mean and standard deviation

A/S/n Notation Proposed by D. Kendeall in 1953

Arrival process (A) – message arrival distribution

Server process (S) – servicing time distribution

Number of servers (n)

Distribution lables M : exponential (memoryless, Poisson)

D : deterministic (constant arrival/processing times)

G : general

Ek : k-stage Erlang

Hk : k-stage Hyperexponential

Examples

M/M/1

One server for jobs that arrive and are serviced by Poisson process

System with many independent jobs sources can be approximated this way

M/D/n

Arrival process is Poisson, service is deterministic

There are n servers

G/G/n

General system with arbitrary arrival and service times

Goals of Queuing Theory

Predict the performance of the system in steady state Not startup or shut down

Assumptions The population of jobs is infinite

The size of the queue is infinite (can be bounded)

Specific distributions for A and S known

Performance Metrics W: waiting time (queuing delay)

Time between receipt of a job and start of processing

Depends on the load on the server

S: service time (processing delay)

The actual processing time of the job

Depends on the serving capacity

T: response time

T = W + S

Throughput

Average number of jobs per time unit

Little’s Law

Number of jobs in the system N(t)

N = λT λ : the mean arrival rate

T : the mean service time

Queue size Q Q = max(0,N – m)

Waiting time in the Queue W Q = λW

Little’s Law Example

Clothing store Customers arrive 10/hour and stay 30 minutes

λ = 10, T = 0.5

Average number of customers in the store N = 10*0.5 = 5

Register line subsystem in the store Average register queue is 2 people

The wait time W = Q/λ = 2/10 = 0.2 hours

Other Metrics

Traffic intensity p Single server: p = arrival rate * service time = λ/μ

m servers: p = λ/(mμ)

p is dimensionless, measured in Erlangs

If p > 1, then more jobs arrive than the system can handle

p < 1 needed to prevent overload

Utilization factor U = min(p,1)

Analytic Solutions

Some queuing models can be solved analytically M/M/1 for example

Other variants

M/M/1 Performance

Average # jobs in the system N = p/(1-p)

Average queue size Q = p2/(1-p)

Average wait time W =

Average response time T =

M/M/1 Example Post office

Customers arrive 10 per hour (λ=10)

Clerk can handle 16 customer per hour (μ = 16)

Result Average number of customers N = 1.667

Average queue size Q = 1.042

Average wait time W = 0.1042 hours (6.25 minutes)

Average response time T = 10 minutes

How to Fix Overload

How can we reduce job delays

Obvious: Shorten μ (speed up the code, profile, optimize)

Lengthen λ (decrease the load, make less bursty)

Move to M/M/m How many servers do you want

This can be solved as well

Problems

Classical queuing theory is too restrictive Infinite number of jobs

Only certain distributions are handled

Job routine is not allowed

Some limitations can be alleviated But calculations and the math involved are complex

Provides insufficient information

Solving Queuing Problems

Use a computer representation of the system Instead of a mathematical model

Use simulation to learn the properties

This is what is generally used MATLAB SimEvents

OMNET

Can construct arbitrary models and run them

Homework

For next week (10/26) Present you project to the class

Plans, progress to date

Project model

Tasks and model(s) for each task

Hand this in

Can be part of presentation