Internet Measurement5.4 State of the art

ECE Department, University of Tehran

Fall 2009

Outline Equipment Properties Topology Properties

Static properties of AS graph Static properties of Router graph Dynamic aspects of topology Geographic Location

Interaction of Traffic and Network Packet Delay Packet Loss Packet Reordering, Duplication and Jitter Bandwidth and Throughput

2

Equipment Properties Different types of equipment:

Links and other communication devices that are highly predictable

Routers that are more complex NATs and Firewalls: delay on the order of 100s of

miliseconds

3

Equipment Properties - Routers Normally, routers has very small

delay (order of tens of micro seconds).

But if heavily loaded, delay is on the order of miliseconds.

Additional sources of router delay Periodic processes within router software Packets carrying IP options Packets leaving on different interfaces

4

Equipment Properties - Routers Delay that accounts for traffic

production and consumption of routers LSA processing is on the order of 100

microseconds. Mostly for data copying within the router

OSPF packet processing vary linearly with number of LSAs

Shortest path calculation scales quadratic with the number of nodes in a fully connected topology

5

Outline Equipment Properties Topology Properties

Static properties of AS graph Static properties of Router graph Dynamic aspects of topology Geographic Location

Interaction of Traffic and Network Packet Delay Packet Loss Packet Reordering, Duplication and Jitter Bandwidth and Throughput

6

Static properties of AS graph Highest level of internet topology AS graph is a pair G=(V,E)

V = Autonomous Systems E = Existence of direct traffic

G is highly variable in degree distribution that can often be approximated by power law

Most ASes have low degree (<5), but a few ASes have thousands degree.

7

AS graph – Power Distribution Here is a sample created by fusion of different topology

measurements (e.g. BGP-based, routing registries…)

It was only an approximation in 2001

8

AS graph – Power Distribution High variability in degree distribution

Some ASes are very highly connected Different ASes have dramatically different roles in the

network Node degree seems to be highly correlated with AS size

Generative models of AS graph “Rich get richer” model Newly added nodes connect to existing nodes in a way that

tends to simultaneously minimize the physical length of the new connection, as well as the average number of hops to other nodes

New ASes appear at an exponentially increasing rate, and each AS grows exponentially as well

9

AS Graph is a Small World Graph Graphs with high clustering degree and low

diameter AS graph taken in Jan 2002 containing

12,709 ASes and 27,384 edges Average path length is 3.6 Clustering coefficient is 0.46 (0.0014 in random

graph) It appears that individual clusters can contain

ASes with similar geographic location or business interests

ASes of high degree are likely top-tier ASes.

10

AS Traffic Exchange Policies Four relationships

Customer-provider Peering

Exchange only non-transit traffic Mutual transit

typically between two administrative domains such as small ISPs who are located close to each other

Enforced by BGP route advertisement AS Hierarchical structure?

11

Router Graph

Router graph is a pair G=(V,E) V = Routers E = Existence of direct connection Impossible to obtain a complete Internet

topology. But we can Focus on a single AS subgraph

12

ARPANET Router graph, 1972

IMP: Interface Message Processors, TIP:Terminal IMP

The first message ever sent over the ARPANET; it took place at 10:30PM on October 29, 1969

13

Router-level Topology

Abilene Network, Research and educational backbone of USA14

Static Properties of Router Graph High variability in degree distribution Most nodes have degree less than 5 but

some can have degrees greater than 100 Sampling bias (e.g. by traceroute)

Small set of sources with much larger set of destinations

Nodes and links closest to the sources are explored much more thoroughly

Majority of edges are far and so undersampled Artificially increase the proportion of low-

degree nodes in the sampled graph

15

Generative Router-level Topology

Based on network robustness & technology constrains

Network edge: High degree nodes Serve many users with

low bandwidth. Network cores:

More likely to be meshes for robustness.

High bandwidth is essential.

16

Static Properties of Router Graph AS vs. Router Graphs

AS graph can be measured passively (Using BGP tables and traffic)

Router graph needs active measurement (Using traceroute that can be slow in practice)

They have different graph structures In AS graph highly connected central nodes

but in router graph they are edges. AS Path properties

Average length around 16, rare paths longer than 30 hops

Path inflation, because of AS-AS policies and interdomain routings

17

Dynamic Aspects of Topology Internet Growth

Number of unique AS numbers advertised within the BGP system

18

Dynamic Aspects of Topology Difficult to measure the number of routers

DNS is decentralized Router-level graph changes rapidly

Different ways of measuring number of end systems and routers: Regional Internet registeries (RIR) BGP system Use ping Query DNS

19

Measuring Number of Routers - RIR Counting number of addresses in RIRs:

Is a serious overestimate as many addresses are not in use.

20

Measuring Number of Routers - RIR

Pinging IP addresses is a serious underestimate Intermittent connection (e.g. Dial up, Wireless) Network Address Translation (NAT) & Firewalls

Query DNS is also an underestimate All connected hosts do not have DNS name

No. of addresses in the BGP system is an overestimate Prefixes contain addresses not in use

21

Registered Hosts in DNS During the 1990s Internet growing exponentially Slowed down somewhat today Rapid growth means more difficulty for measuring router

graph

22

Stability of Internet

Sources of instability: Failures, restarts and reconfigurations of

network infrastructure changes network topology: Graph structure (nodes, edges) will be

altered Routes become unstable

Router misconfiguration and policy changes May need long sequence of BGP updates,

packet delay and loss

23

Stability of Internet Router level instability

Some routes exhibit significant fluctuation Consistent with the behavior AS-level paths High variability of route stability

Majority of routes going days or weeks without change

High variability of route unavailable duration Causes of instability of the router graph

Failure of links Majority of link failures are concentrated on a small subset of

the links Marjority of link failures are short-lived (<10min)

Router failure

24

Geographic Location Relation of Net. Infrastructure with population, social

organization and economic activity Online per interface is a good measure

IP addresses

25

Outline Equipment Properties Topology Properties

Static properties of AS graph Static properties of Router graph Dynamic aspects of topology Geographic Location

Interaction of Traffic and Network Packet Delay Packet Loss Packet Reordering, Duplication and Jitter Bandwidth and Throughput

26

Traffic and Network - Delay Packet delay (RTT): high variable distribution

Deterministic delays: Propagation delay & Transmission delay

Stochastic Delays: Forwarding delay (queue) Transmission delay

Only significant on slow access links. e.g. 1500B takes 1.2 mic.sec on OC192 and 200 milisec. for

a 56k modem Propagation delay

Influenced by geographic distance minRTT = Propagation delay + Transmission

delay, so it is deterministic Is strongly affected by topology can be estimated by Euclidean space

27

Traffic and Network - Delay Forwarding delay and congestion events

Can dominate the deterministic delays Often minRTT<50 mil.sec, maximum 200 mil.sec Stochastic can cause RTT=100s mil.sec or 10s

sec. Often cause high ‘spikes’ in RTTs as a result of

Routing changes Temporary queues within routers (Weibull or

Pareto distribution) It is not clear which link is responsible for

congested links

28

Traffic and Network - Loss Main sources of packet loss

Congestion within routers For wireless: packet corruption, radio interference and

multipath fading Often occur in ‘bursts’

Because of congestion in routers Packet loss show correlation over time scales up to

1000 mili.sec Can be modeled using 2,3-state Markov model Generally <0.1% over wired paths but 2% for

wireless paths

29

Traffic and Network - Reordering

Reordering: inverting order of 2 packets Parallelism within a router Load sharing between network paths Route changes

Out of sequence arrival reasons: True packet reordering Packet duplication Often for retransmission by TCP sender

Accounts for 1-2% or less of a long-lived TCP connection.

30

Packet Duplication and Jitter Packet duplication rarely happen. sources:

Measurement point is inside a router Network actually duplicates packets

Packet jitter: Non-uniformity of inter-packet gaps Queuing: can be 100s – 1000s mili.sec.

31

Bandwidth and Throughput Much less known compared to other

metrics But declining loss and RTT shows

thoughput growth Throughput is a steady metric that

changes slowly (e.g. factor of 3 over an hour)

32

Conclusion Internet users in the middle east

Internet World Stats: http://www.internetworldstats.com – Nov 2010

33

Any Question

?

34

Internet Infrastructure measurement