User-level Internet Path User-level Internet Path DiagnosisDiagnosis
R. Mahajan, N. Spring, D. Wetherall and T. AndersonR. Mahajan, N. Spring, D. Wetherall and T. Anderson
The network is a black The network is a black box…box…
...so what can I do...so what can I do1.We want the
users to be able to diagnose their paths
2.Communicate information to ISP or NOC to improve the network
TULIP: User-level path TULIP: User-level path diagnosisdiagnosis
Objectives: Objectives: Detect performance faults that Detect performance faults that
affect a user’s flows. This involves a affect a user’s flows. This involves a measure of the magnitude of the measure of the magnitude of the fault (queuing delay, loss) and the fault (queuing delay, loss) and the localization of the faulty link.localization of the faulty link.
How TULIP does itHow TULIP does it Ideal Architecture – Packet based Ideal Architecture – Packet based
solutionssolutions Each router the packet traverses adds a certain Each router the packet traverses adds a certain
number of information to the packet: timestamp, number of information to the packet: timestamp, global address of the router’s input interface.global address of the router’s input interface.
Issue: Packet size increases at each hop. A packet loss Issue: Packet size increases at each hop. A packet loss involves a loss of all the information. Corruption involves a loss of all the information. Corruption of a packet might yield to incorrect diagnosis data of a packet might yield to incorrect diagnosis data (allthough most corruption are treated as losses)(allthough most corruption are treated as losses)
Because things are never Because things are never idealideal
Basic architecture sufficient for data collectionBasic architecture sufficient for data collectionAssets: Fixed packet size and sufficient information…Assets: Fixed packet size and sufficient information…Assuming : stationarity of paths (paths between source and Assuming : stationarity of paths (paths between source and
destination don’t change too often)destination don’t change too often)
Diagnosis tools in use in Diagnosis tools in use in TULIPTULIP
Out-of-band measurement probes (or Out-of-band measurement probes (or TTL based search) TTL based search) obtain the Sample TTL and Interface IDobtain the Sample TTL and Interface ID
ICMPICMP Router timestamp Router timestamp
IP identifiers IP identifiers Approximation of the per-flow counterApproximation of the per-flow counter
How to detect path How to detect path loss/reorderingloss/reordering
Sending two probes to determine the behavior of the remote router
Packet queuingPacket queuing
An ICMP timestamp is used to determine the queuing delays within a router (median)
The TULIP methodsThe TULIP methods To perform the measurement, TULIP To perform the measurement, TULIP
uses two “scanning” methods. uses two “scanning” methods. Binary search (reduces diagnostic Binary search (reduces diagnostic
traffic but at a cost of diagnosis time)traffic but at a cost of diagnosis time) Parrallel search (interleaves Parrallel search (interleaves
measurements to different routers by measurements to different routers by cycling through them in nodes)cycling through them in nodes)
Network Load and Network Load and Diagnosis TimeDiagnosis Time
Because of the relative stationary Because of the relative stationary behavior of a router, with an behavior of a router, with an approximative diagnosis time of 10/30 approximative diagnosis time of 10/30 min, TULIP can provide accurate results. min, TULIP can provide accurate results.
The load for Binary search is B/W and for The load for Binary search is B/W and for parrallel LB/W (lower bound)parrallel LB/W (lower bound)L: # of measurable routersL: # of measurable routersB: Bandwitdth cost of the probesB: Bandwitdth cost of the probesW: Wait time (usually 1s)W: Wait time (usually 1s)
Diagnosing granularityDiagnosing granularity The granularity is the weighted The granularity is the weighted
average of the lengths of its average of the lengths of its diagnosable segments. diagnosable segments.
1
2 31’ 2’
Rank(G)=2
1 2
10
01
11
G
Various granularity for Various granularity for different measurementsdifferent measurements
•50 % of the paths have a granularity less than 3 hops (75% <4)•TULIP matches ideal tomography implementation
ValidationValidation Compared results with Planet Lab Compared results with Planet Lab
coupled with a tomography systemcoupled with a tomography system Use a measure “rate delta” that Use a measure “rate delta” that
computes the difference between the computes the difference between the rate at the far end minus that at the rate at the far end minus that at the near end of a segment. near end of a segment. Negative valuesNegative values implies a lack of implies a lack of consistency (values spawn a range too consistency (values spawn a range too large)large)
Reordering ResultsReordering Results
85 % of the results are consistent for forward path75 % for round trip (due to the asymmetric nature of some paths)
Loss resultsLoss results
•85% again of non negative deltas•Round trip counterpart less affected by asymmetry than the Reordering diagnosis (because loss usually occurs close to the destination)
Queuing ResultsQueuing Results
•ICMP message generation has a poor timestamp resolution (the two median within 2ms of each other – One from TCPDump on planet lab and one from TULIP).•Forward path shows that queuing delay is consistent (very few negative values)•Round trip reflects the variability in the return path
The last mile…The last mile… First hops from user is the bottleneckFirst hops from user is the bottleneck
Persistance of a faultPersistance of a fault
•We check for how many iterations, TULIP yields similar results•80% of the path show faults persisting long enough for TULIP to diagnose them (typical time a binary search takes to locate a fault : 6 runs)
ConclusionsConclusions Network Operators would be able to Network Operators would be able to
diagnose links efficientlydiagnose links efficiently And a user too … if the world was And a user too … if the world was
populated entirely by Computer populated entirely by Computer nerds.nerds.
Issues…Issues… Multiple TULIP users could reduce Multiple TULIP users could reduce
the accuracy of the probing method, the accuracy of the probing method, the per flow counterthe per flow counter
An application doesn’t experience An application doesn’t experience the network the same way an active the network the same way an active measurement does. (TCP, measurement does. (TCP, application dependant as well as application dependant as well as flags)flags)
……and possible and possible improvementsimprovements
Per flow counter at the router level Per flow counter at the router level (unrealistic)(unrealistic)
Hash source address and IPID (for Hash source address and IPID (for flow)flow)
ICMP timestamp have reception time ICMP timestamp have reception time as well as transmission time (allows as well as transmission time (allows the calculation of the delay the the calculation of the delay the packet is processed at the router)packet is processed at the router)