Internet Measurement5.4 State of the art
ECE Department, University of Tehran
Fall 2009
Outline Equipment Properties Topology Properties
Static properties of AS graph Static properties of Router graph Dynamic aspects of topology Geographic Location
Interaction of Traffic and Network Packet Delay Packet Loss Packet Reordering, Duplication and Jitter Bandwidth and Throughput
2
Equipment Properties Different types of equipment:
Links and other communication devices that are highly predictable
Routers that are more complex NATs and Firewalls: delay on the order of 100s of
miliseconds
3
Equipment Properties - Routers Normally, routers has very small
delay (order of tens of micro seconds).
But if heavily loaded, delay is on the order of miliseconds.
Additional sources of router delay Periodic processes within router software Packets carrying IP options Packets leaving on different interfaces
4
Equipment Properties - Routers Delay that accounts for traffic
production and consumption of routers LSA processing is on the order of 100
microseconds. Mostly for data copying within the router
OSPF packet processing vary linearly with number of LSAs
Shortest path calculation scales quadratic with the number of nodes in a fully connected topology
5
Outline Equipment Properties Topology Properties
Static properties of AS graph Static properties of Router graph Dynamic aspects of topology Geographic Location
Interaction of Traffic and Network Packet Delay Packet Loss Packet Reordering, Duplication and Jitter Bandwidth and Throughput
6
Static properties of AS graph Highest level of internet topology AS graph is a pair G=(V,E)
V = Autonomous Systems E = Existence of direct traffic
G is highly variable in degree distribution that can often be approximated by power law
Most ASes have low degree (<5), but a few ASes have thousands degree.
7
AS graph – Power Distribution Here is a sample created by fusion of different topology
measurements (e.g. BGP-based, routing registries…)
It was only an approximation in 2001
8
AS graph – Power Distribution High variability in degree distribution
Some ASes are very highly connected Different ASes have dramatically different roles in the
network Node degree seems to be highly correlated with AS size
Generative models of AS graph “Rich get richer” model Newly added nodes connect to existing nodes in a way that
tends to simultaneously minimize the physical length of the new connection, as well as the average number of hops to other nodes
New ASes appear at an exponentially increasing rate, and each AS grows exponentially as well
9
AS Graph is a Small World Graph Graphs with high clustering degree and low
diameter AS graph taken in Jan 2002 containing
12,709 ASes and 27,384 edges Average path length is 3.6 Clustering coefficient is 0.46 (0.0014 in random
graph) It appears that individual clusters can contain
ASes with similar geographic location or business interests
ASes of high degree are likely top-tier ASes.
10
AS Traffic Exchange Policies Four relationships
Customer-provider Peering
Exchange only non-transit traffic Mutual transit
typically between two administrative domains such as small ISPs who are located close to each other
Enforced by BGP route advertisement AS Hierarchical structure?
11
Router Graph
Router graph is a pair G=(V,E) V = Routers E = Existence of direct connection Impossible to obtain a complete Internet
topology. But we can Focus on a single AS subgraph
12
ARPANET Router graph, 1972
IMP: Interface Message Processors, TIP:Terminal IMP
The first message ever sent over the ARPANET; it took place at 10:30PM on October 29, 1969
13
Router-level Topology
Abilene Network, Research and educational backbone of USA14
Static Properties of Router Graph High variability in degree distribution Most nodes have degree less than 5 but
some can have degrees greater than 100 Sampling bias (e.g. by traceroute)
Small set of sources with much larger set of destinations
Nodes and links closest to the sources are explored much more thoroughly
Majority of edges are far and so undersampled Artificially increase the proportion of low-
degree nodes in the sampled graph
15
Generative Router-level Topology
Based on network robustness & technology constrains
Network edge: High degree nodes Serve many users with
low bandwidth. Network cores:
More likely to be meshes for robustness.
High bandwidth is essential.
16
Static Properties of Router Graph AS vs. Router Graphs
AS graph can be measured passively (Using BGP tables and traffic)
Router graph needs active measurement (Using traceroute that can be slow in practice)
They have different graph structures In AS graph highly connected central nodes
but in router graph they are edges. AS Path properties
Average length around 16, rare paths longer than 30 hops
Path inflation, because of AS-AS policies and interdomain routings
17
Dynamic Aspects of Topology Internet Growth
Number of unique AS numbers advertised within the BGP system
18
Dynamic Aspects of Topology Difficult to measure the number of routers
DNS is decentralized Router-level graph changes rapidly
Different ways of measuring number of end systems and routers: Regional Internet registeries (RIR) BGP system Use ping Query DNS
19
Measuring Number of Routers - RIR Counting number of addresses in RIRs:
Is a serious overestimate as many addresses are not in use.
20
Measuring Number of Routers - RIR
Pinging IP addresses is a serious underestimate Intermittent connection (e.g. Dial up, Wireless) Network Address Translation (NAT) & Firewalls
Query DNS is also an underestimate All connected hosts do not have DNS name
No. of addresses in the BGP system is an overestimate Prefixes contain addresses not in use
21
Registered Hosts in DNS During the 1990s Internet growing exponentially Slowed down somewhat today Rapid growth means more difficulty for measuring router
graph
22
Stability of Internet
Sources of instability: Failures, restarts and reconfigurations of
network infrastructure changes network topology: Graph structure (nodes, edges) will be
altered Routes become unstable
Router misconfiguration and policy changes May need long sequence of BGP updates,
packet delay and loss
23
Stability of Internet Router level instability
Some routes exhibit significant fluctuation Consistent with the behavior AS-level paths High variability of route stability
Majority of routes going days or weeks without change
High variability of route unavailable duration Causes of instability of the router graph
Failure of links Majority of link failures are concentrated on a small subset of
the links Marjority of link failures are short-lived (<10min)
Router failure
24
Geographic Location Relation of Net. Infrastructure with population, social
organization and economic activity Online per interface is a good measure
IP addresses
25
Outline Equipment Properties Topology Properties
Static properties of AS graph Static properties of Router graph Dynamic aspects of topology Geographic Location
Interaction of Traffic and Network Packet Delay Packet Loss Packet Reordering, Duplication and Jitter Bandwidth and Throughput
26
Traffic and Network - Delay Packet delay (RTT): high variable distribution
Deterministic delays: Propagation delay & Transmission delay
Stochastic Delays: Forwarding delay (queue) Transmission delay
Only significant on slow access links. e.g. 1500B takes 1.2 mic.sec on OC192 and 200 milisec. for
a 56k modem Propagation delay
Influenced by geographic distance minRTT = Propagation delay + Transmission
delay, so it is deterministic Is strongly affected by topology can be estimated by Euclidean space
27
Traffic and Network - Delay Forwarding delay and congestion events
Can dominate the deterministic delays Often minRTT<50 mil.sec, maximum 200 mil.sec Stochastic can cause RTT=100s mil.sec or 10s
sec. Often cause high ‘spikes’ in RTTs as a result of
Routing changes Temporary queues within routers (Weibull or
Pareto distribution) It is not clear which link is responsible for
congested links
28
Traffic and Network - Loss Main sources of packet loss
Congestion within routers For wireless: packet corruption, radio interference and
multipath fading Often occur in ‘bursts’
Because of congestion in routers Packet loss show correlation over time scales up to
1000 mili.sec Can be modeled using 2,3-state Markov model Generally <0.1% over wired paths but 2% for
wireless paths
29
Traffic and Network - Reordering
Reordering: inverting order of 2 packets Parallelism within a router Load sharing between network paths Route changes
Out of sequence arrival reasons: True packet reordering Packet duplication Often for retransmission by TCP sender
Accounts for 1-2% or less of a long-lived TCP connection.
30
Packet Duplication and Jitter Packet duplication rarely happen. sources:
Measurement point is inside a router Network actually duplicates packets
Packet jitter: Non-uniformity of inter-packet gaps Queuing: can be 100s – 1000s mili.sec.
31
Bandwidth and Throughput Much less known compared to other
metrics But declining loss and RTT shows
thoughput growth Throughput is a steady metric that
changes slowly (e.g. factor of 3 over an hour)
32
Conclusion Internet users in the middle east
Internet World Stats: http://www.internetworldstats.com – Nov 2010
33
Any Question
?
34
Internet Infrastructure measurement