On the Impact of Layer-2 on Node Degree Distribution
Pascal Mérindol, Benoit Donnet, Jean-Jacques Pansiot, Olivier Bonaventure
Melbourne - 2th November 2010
Internet Measurement Conference 2010
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
Agenda
‣ Topology Discovery Background‣ Data Collection : mrinfo-rec‣ Problem Statement‣ Revisiting the node degree distribution ‣ Domain Specificities‣ Model key distributions
Topology Discovery
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣ Internet seen as a dynamic graph✓ of IP interfaces
➡ traceroute, route_record✓ of routers
➡ alias resolution : ally, iffinder, ...✓ of Autonomous Systems
➡ IP to AS mapping (routeview project), router to AS mappping ?
‣ Goals✓ IP network models & simulations✓ ground truth input for topology generation
R1
R0
R2switch
1.1.0.2
1.1.1.1
1.1.1.2
1.1.0.1
1.1.2.3
1.1.2.1
R5
R6
1.1.2.2
1.1.3.1
1.1.0.2 [version 12.4]1.1.0.2 → 1.1.0.1 [1/0/pim/querier]1.1.2.3 → 1.1.2.1 [1/0/pim/querier]1.1.2.3 → 1.1.2.2 [1/0/pim/querier]1.1.3.1 → 0.0.0.0 [1/0/pim/leaf]
Topology Discovery
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
mrinfo‣ Topology discovery using mrinfo
➡ Uses IGMP messages✓ ASK_NEIGHBORS✓ NEIGHBORS_REPLY
➡ Output✓ All multicast interfaces of a given router✓ All multicast neighbors/links
‣ mrinfo applied recursively➡ mrinfo-rec✓ probe all neighbors✓ daily based
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
mrinfo-rec
Pajek
Sprint - 2006
while (--ncount >= 0 && p < ep ) {
/*NEIGHBOR ADDRESS*/ register u_int32 neighbor = *(u_int32*)p; p += 4;
‣ Limitations➡ multicast scope➡ IGMP filtering➡ non compliant routers
‣ Advantages ➡ network friendly probing : 1 probe injected per router➡ aliasing : no need to gather IP interfaces➡ forwarding independent : backup links visible [IMC2009]➡ layer-2 vision : distinguish the IP logical layer over MAC
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
The dataset‣ More than 4 years of daily collected data
➡ ~10000 routers➡ ~100000 IPs➡ ~300-800 AS
‣ Raw data towards IP network graphs➡ Cleaning process (tunnels, private IP, disabled IP, ...)➡ Router to AS mapping [PAM2010]➡ Layer-2 Inference➡ Graph selection : one per AS per month
✓ dimensions (|V=R+S|,|E|)✓ connectivity✓ influence of L2 hardware✓ ...
V = set of vertex/nodes E = set of edgesR = set of routersS = set of layer-2 nodes
Num
ber o
f nod
es w
ith k
link
s
Number of links (k)
Many nodes witha few links
A few hubs withlarge number of links
Topology Discovery
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
Problem : MAC vs. IP layerR1
R2
R3
R4
R1
R2
R3
R4
mrinfo, layer-2 view: L2 graph traceroute, layer-3 view: IP graph
‣ The node degree distribution follows a heavy tailed power law ?
‣ What is the impact of L2 on scale free & preferential attachment models ?
Ethernet Switches & Shared Risk Link Groups
(SRLG)
IP logical view
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
Problem : Global Overview
LAN
MAN
WAN
0.0.0.0 ← 10.0.0.1
0.0.0.0 ← 192.168.0.1
← point-to-point connection
← point-to-multipoint connection
IP-only layer vision
L3 routerL2 switch
L1 hub
L1 bus
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣ Using our dataset, we can infer several kinds of L2 deviceL2 inference validation
‣ Focusing on Ethernet Broadcast networks, we check our inference validity using three rules :
➡ symmetry, querier election and subnet mask.‣ Three states: coherent - incoherente - incomplete
‣ Most of L2 inferred devices seems to reveal Ethernet broadcast devices such as switches
minimal covering prefix for p2p and p2mp connections proportion of Ethernet switches
Revisiting the node degree distribution‣ We construct two graphs of the same network
➡ the L2 inferred graph (L2-aware) and the L3 graph (L2-agnostic)
t(x): degree distribution of L2 and L3 nodes in the L2 graphr’(x): degree distribution of L3 nodes in the L3 graph
Overall networks analysis
‣The power law distribution is much less heavy tailed considering an L2 view !
R1
R2
R3
R4
R1
R2
R3
R4
t(1)=4, t(4)=1
r’(3)=4
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣What impact on other graph/routing characteritics ?
Revisiting the node degree distribution
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
✓α : proportion of routers connected to at least one L2 device ✓β - L2 : proportion of p2p connections going through an L2 device in the L2 aware network✓β - L3 : proportion of p2p connections going through an L2 device in the L2 agnostic network
‣ ~10-30 % of L2 nodes...
...but resulting in two strongly different graphs :
➡ The number of edges/connections in the L3 graph is much more larger than in the L2 graph,
➡ Between 40% and 75% of routers are connected to an L2 device,
➡ Most of the L3 connections rely on L2 Ethernet switches... ...even in the L2 graph, almost half of router-to-router connections goes through an L2 node !
Telecom Italia
Per AS analysis: a uniform observation ?
Sprint IUNET
Level 3
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
t(x): degree distribution of L2 and L3 nodes in the L2 graphr’(x): degree distribution of L3 nodes in the L3 graph
R1
R2
R3
R4
R1
R2
R3
R4
Why such a great tail shift ?
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣How a small (L2-aware) degree router can become a large L3 degree router ?
L2 aware degree distributionL3 degree distribution
‣ Must look at specific distributions on the L2-aware graph
➡ r : router degree distribution
➡ s : switch degree distribution
➡ b : L2 degree distribution of routers (#L2 neighbors)
➡ p : degree distribution of routers connected to L2 devices («subdistribution» of r)
multi-graph problem
R
RR
3 7
A closer look at key distributionsr - L3 node degree
b - #L2 neighbors
p - routers connected to L2 neighbors
s- L2 node degree
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
Sprint IUNET
Telecom ItaliaLevel 3
Practical IP networks design
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣ MAN access distributes IP connectivity at low cost
➡ quite typical accross our dataset ...
‣ Redundancy may exacerbate this phenomenon
‣ VLAN can introduce an additional MAC logical layer (lower-bound analysis)
‣ Access/core networks ?Level 3 - AS3356
FRANKFURT MAN ACCESSASBR
SWITCH
ROUTER
NEIGHBOR AS
VLAN ?
Sprint
Telecom Italia
IUNET
A large degree router has more L2 neighbors
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
p: degree distribution of L3 nodes connected to b switches in the L2 graphb: number of L2 neighbors
Level 3
Towards a model ?
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣How to model this shift ?‣What are key factors ?
L2 aware degree distributionL3 degree distribution
RR
‣ «Replay» the collected data using key distributions... ...and one of their possible correlations
➡ r (router degree distribution), s (switch degree distribution)
➡ b (L2 degree distribution of routers = #L2 neighbors)
➡ p×b : correlation between p and b distributions
‣ Is this sufficient ? What about AS specificities ?
Towards a model ?
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
Sprint
Level 3 Telecom Italia
data: degree distribution of L3 nodes in the L2-agnostic graphmodel: degree distribution of L3 nodes in the generated graph
IUNET
Conclusion
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣ mrinfo-rec is a useful tool for three reasons➡ describe a connected multicast topology at the router level (no need for alias resolution)
➡ can discover backup links (no forwarding dependence)
➡ able to natively infer L2 devices (hybrid bipartite graph)
‣ Layer-2 switches strongly impact the node degree distribution➡ most of IP router-to-router connections go through Broadcast Ethernet switches !
➡ such switches are generally connected to a large number of routers
➡ correlation between the L2-aware degree of a router and its number of L2 neighbors
Ongoing Work
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
‣ Improve Active Topology Discovery➡ using the best of mrinfo, traceroute and alias resolution techniques
➡ MERLIN : A new measurement platform to MEasure the Router Level of the INternet
‣ Various Analysis on other graph/routing metrics➡ clustering coefficient/assortativity : understand the dynamic of attachement preferences
➡ path diversity : measure the physical resilience of IP networks
Questions ?
IMC 2010 - On the Impact of Layer-2 on Node Degree Distribution
✓ Pascal Mérindol, Benoit Donnet, Jean-Jacques Pansiot, Matthew Luckie, Young Huyn. MERLIN: MEasure the Router Level of the INternet
Université catholique de Louvain, Technical Report 2010-3, September 2010.
✓ Jean-Jacques Pansiot, Pascal Mérindol, Benoit Donnet, and Olivier Bonaventure. Extracting Intra-Domain Topology from mrinfo ProbingIn Proc. Passive and Active Measurement Conference (PAM), April 2010.
✓ Pascal Mérindol, Virginie Van den Schrieck, Benoit Donnet, Olivier Bonaventure and Jean-Jacques Pansiot. Quantifying ASes Multiconnectivity using Multicast InformationIn Proc. ACM/USENIX Internet Measurement Conference (IMC), November 2009.
‣http://inl.info.ucl.ac.be/content/mrinfo‣http://svnet.u-strasbg.fr/mrinfo/