Bridging Router Performance and Queuing Theory

www.intel.com/research

Bridging Router Performance and Queuing Theory

Dina Papagiannaki,Dina Papagiannaki,

Intel Research CambridgeIntel Research Cambridge

withwith

Nicolas Hohn, Darryl Veitch and Christophe DiotNicolas Hohn, Darryl Veitch and Christophe Diot

2www.intel.com/research

• Intel Research •

Motivation End-to-end packet delay is an important metric for End-to-end packet delay is an important metric for

performance and SLAsperformance and SLAs

Building block of end-to-end delay is through router Building block of end-to-end delay is through router delaydelay

We measure the delays incurred by We measure the delays incurred by allall packets packets crossing a single routercrossing a single router



Overview Full Router MonitoringFull Router Monitoring

Delay AnalysisDelay Analysis

ModelingModeling

Delay Performance: Understanding and ReportingDelay Performance: Understanding and Reporting

Causes of microcongestionCauses of microcongestion



Measurement Environment



Full Router Monitoring Gateway routerGateway router

2 backbone links (OC-48), 2 domestic customer links (OC-2 backbone links (OC-48), 2 domestic customer links (OC-3, OC-12), 2 Asian customer links (OC-3)3, OC-12), 2 Asian customer links (OC-3)

13 hours of trace collection on Aug. 14, 200313 hours of trace collection on Aug. 14, 2003

7.3 billion packets – 3 TeraBytes of IP traffic7.3 billion packets – 3 TeraBytes of IP traffic

Monitor more than 99.9% of all through trafficMonitor more than 99.9% of all through traffic

μμs timestamp precisions timestamp precision



Packet matching

SetSet LinkLink Matched pktsMatched pkts % traffic C2-out% traffic C2-outC4C4 InIn 215987215987 0.03%0.03%C1C1 InIn 7037670376 0.01%0.01%BB1BB1 InIn 345796622 47.00%47.00%BB2BB2 InIn 389153772 52.89%52.89%C2C2 outout 735236757 99.9399.93%%



Packet matching (cntd)





ModelingModeling





Store & Forward Datapath Store: storage in input linecard’s Store: storage in input linecard’s

memorymemory Forwarding decisionForwarding decision Storage in dedicated Virtual Output Storage in dedicated Virtual Output

Queue (VOQ)Queue (VOQ) Decomposition into fixed-size cellsDecomposition into fixed-size cells Transmission through switch fabric cell Transmission through switch fabric cell

by cellby cell Packet reconstructionPacket reconstruction Forward: Output link schedulerForward: Output link scheduler

Not part of the system



Delays: 1 minute summary



Minimum Transit Time

Packet size dependent minimum delay Δ(L), specific to router architecture and linecard technology



Store & Forward Datapath Store: storage in input linecard’s Store: storage in input linecard’s

memorymemory Forwarding decisionForwarding decision Storage in dedicated Virtual Output Storage in dedicated Virtual Output

Queue (VOQ)Queue (VOQ) Decomposition into fixed-size cellsDecomposition into fixed-size cells Transmission through switch fabric cell Transmission through switch fabric cell

by cellby cell Packet reconstructionPacket reconstruction Forward: Output link schedulerForward: Output link scheduler

Not part of the system

Δ(L)

FIFO queue





ModelingModeling





Modeling



Modeling



Model Validation



Model validation



Error as a function of time



Modeling results Our crude model performs wellOur crude model performs well

Use effective link bandwidth (account for Use effective link bandwidth (account for encapsulation)encapsulation)

The front end The front end ΔΔ only matters when the output queue only matters when the output queue is emptyis empty

The model defines Busy Periods: The model defines Busy Periods: time between the arrival of a packet to the empty system and the time when the system becomes empty again.





ModelingModeling





Delay Performance Packet delays Packet delays cannotcannot be inferred from output link be inferred from output link

utilizationutilization

Source of large delays: queue build-ups in output Source of large delays: queue build-ups in output bufferbuffer

Busy Period structures contain Busy Period structures contain allall delaydelay information information

Busy Period durations and idle duration contain Busy Period durations and idle duration contain all all utilization utilization informationinformation



Reporting BP Amplitude



Reporting BP Duration



Report BP joint distribution



Busy periods have a common shape



Reporting Busy Periods Answer performance related questions Answer performance related questions directlydirectly

How long will a given level of congestion last?How long will a given level of congestion last?

Method:Method: Report Report partialpartial busy period statistics A and D busy period statistics A and D

Use “triangular shape”Use “triangular shape”



Understanding Busy Periods

LAifALDd T

DAL ),1()(,,



Reporting Busy Periods



Summary of modeling part ResultsResults

Full router empirical studyFull router empirical study

Delay modelingDelay modeling

Reporting performance metricsReporting performance metrics





ModelingModeling





Causes of microcongestion1.1. Reduction in link bandwidth from core to the Reduction in link bandwidth from core to the

access.access.

2. Multiplexing of multiple input traffic streams toward Multiplexing of multiple input traffic streams toward a single output stream.a single output stream.

3. Degree and nature of burstiness of input traffic Degree and nature of burstiness of input traffic stream(s).stream(s).



Stretching and merging

QueueBuildup!



Causes of microcongestion1. Reduction in link bandwidth from core to the Reduction in link bandwidth from core to the

access.access.

2.2. Multiplexing of multiple input traffic streams toward Multiplexing of multiple input traffic streams toward a single output stream.a single output stream.

3. Degree and nature of burstiness of input traffic Degree and nature of burstiness of input traffic stream(s).stream(s).



Multiplexing



Causes of microcongestion1. Reduction in link bandwidth from core to the Reduction in link bandwidth from core to the

access.access.

2. Multiplexing of multiple input traffic streams toward Multiplexing of multiple input traffic streams toward a single output stream.a single output stream.

3.3. Degree and nature of burstiness of input traffic Degree and nature of burstiness of input traffic stream(s).stream(s).



Traffic Burstiness Duration and amplitude of busy periods depends on Duration and amplitude of busy periods depends on

the spacing of packets at the input.the spacing of packets at the input.

Highly clustered packets at the input are more likely Highly clustered packets at the input are more likely to form busy periods.to form busy periods.



Busy periods

Maximum amplitude: 5 msMaximum duration: 15 ms

120,000 busy periods > 1 ms

ts

D

A

tA



Methodology Run semi-experimentsRun semi-experiments

Simulate busy periods and measure their amplitude A(S, Simulate busy periods and measure their amplitude A(S, μμ) under two different traffic scenarios, one that contains ) under two different traffic scenarios, one that contains the effect studied and one that does notthe effect studied and one that does not

Define a metric to quantitatively capture the studied Define a metric to quantitatively capture the studied effecteffect



Reduction in Bandwidth



Amplification factor Reference stream:Reference stream:

SSTT: traffic from a single OC-48 link: traffic from a single OC-48 link Output link rate: Output link rate: μμii

Test stream:Test stream: SSss: traffic from a single OC-48 link: traffic from a single OC-48 link Output link rate: Output link rate: μμoo

),(max)( ,

iTkk

oT

SASA

AF



Amplification factor (2)



Link multiplexing



Link multiplexing Reference stream:Reference stream:

SSTT: output link traffic: output link traffic Output link rate: Output link rate: μμoo

Test stream:Test stream: SSii: traffic from a single OC-48 link: traffic from a single OC-48 link Output link rate: Output link rate: μμoo

),()(max ,

oT

oikk

SASA

LM



Link multiplexing (2)



Flow burstiness

Non-bursty flow

Bursty flow



Flow Burstiness Reference stream:Reference stream:

SSTT: input traffic stream from a single OC-48 link: input traffic stream from a single OC-48 link Output link rate: Output link rate: μμoo

Test stream:Test stream: SSjj: top 5-tuple flow OR the set of ALL bursty flows: top 5-tuple flow OR the set of ALL bursty flows Output link rate: Output link rate: μμoo

),()(max ,

oT

ojkk

SASA

FB



Flow burstiness



Summary Methodology (and metrics) to investigate impact of Methodology (and metrics) to investigate impact of

different congestion mechanismsdifferent congestion mechanisms In today’s access networks:In today’s access networks:

Reduction in link bandwidth plays a significant roleReduction in link bandwidth plays a significant role Multiplexing has a definite impact since individual links Multiplexing has a definite impact since individual links

would not have led to similar delayswould not have led to similar delays Flow burstiness does NOT significantly impact delay Flow burstiness does NOT significantly impact delay

(bottleneck bandwidths too small to dominate the (bottleneck bandwidths too small to dominate the backbone)backbone)

Congestion may be the outcome of network design!Congestion may be the outcome of network design!

www.intel.com/research

Thank you!



References K. Papagiannaki, S. Moon, C. Fraleigh, P.Thiran, F. Tobagi, K. Papagiannaki, S. Moon, C. Fraleigh, P.Thiran, F. Tobagi,

C. Diot.C. Diot.Analysis of Measured Single-Hop Delay from an Analysis of Measured Single-Hop Delay from an Operational Backbone Network.Operational Backbone Network.In In IEEE InfocomIEEE Infocom, New York, U.S.A., June, 2002., New York, U.S.A., June, 2002.

N. Hohn, D. Veitch, K. Papagiannaki, C. Diot.N. Hohn, D. Veitch, K. Papagiannaki, C. Diot.Bridging router performance and queuing theory.Bridging router performance and queuing theory.To appear in To appear in ACM SigmetricsACM Sigmetrics, New York, U.S.A., June, 2004., New York, U.S.A., June, 2004.

K. Papagiannaki, D. Veitch, and N. Hohn.K. Papagiannaki, D. Veitch, and N. Hohn.Origins of Microcongestion in an Access Router.Origins of Microcongestion in an Access Router.In In Passive & Active Measurement WorkshopPassive & Active Measurement Workshop, Antibes, , Antibes, France, April, 2004.France, April, 2004.



Busy Period Construction

Date post:	21-Mar-2016
Category:	Documents
Upload:	mahola
View:	28 times
Download:	0 times

Bridging Router Performance and Queuing Theory

Documents