In VINI VeritasRealistic and Controlled Network Experimentation

Andy Bavier Nick Feamster* Mark Huang

Larry Peterson Jennifer Rexford

Princeton University *Georgia Tech

Modified and presented by Arjumand Younus and Moonyoung Chung

Outline

Motivation Scientific Value and

Importance of VINI Fundamental

Philosophy of VINI Design Requirements

Strategy for building VINI

PL-VINI: prototype on PlanetLab

Experimental results Conclusion

2

Motivation - How to Validate an Idea?

Fixed, shared among many experiments Runs real routing software Exposes realistic network conditions Gives control over network events Carries traffic on behalf of real users

Simulation

Emulation

Small-scaleexperiment

Livedeployment

VINI

3

Realism has Scientific Value

The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ (I found it!) but ‘That’s funny …’ -- Isaac Asimov

Discover unanticipated interactionsCan only simulate / emulate known propertiesBuild richer models

Test robustness and scalability of ideas

4

Fundamental Philosophy “Controlled Realism”

Control: Reproduce results Methodically change or

relax constraints

Realism: Long-running services

attract real “customers” Forward high traffic

volumes (Gb/s) Robustly handle

unexpected events

Topology

Actual network

Arbitrary, emulated

Traffic

Real clients, servers

Synthetic or traces

Network Events

Observed in operational network

Inject faults, anomalies

5

Overview

VINI requirementsFixed, shared infrastructureFlexible network topologyExpose/inject network eventsExternal connectivity and routing adjacencies

PL-VINI: prototype on PlanetLab Experimental results Conclusion

6

Fixed Infrastructure

VINI nodes embedded in Abilene7

Shared Infrastructure

Experiments given illusion of dedicated hardware8

Shared Infrastructure

Experiments given illusion of dedicated hardware9

Flexible Topology

VINI supports arbitrary virtual topologies10

Flexible Topology

VINI supports arbitrary virtual topologies11

Network Events

VINI can inject network failures12

Network Events

VINI can expose network failures (coming soon)

real link fails

virtual link fails

13

External Connectivity

s

c

Experiments can carry traffic for real end-users14

External Routing Adjacencies

s

c

BGP

BGP

BGP

BGP

Experiments can participate in Internet routing15

PlanetLab VINI

Build VINI from PlanetLab, a global testbed for distributed servicesBegun in 2002700 nodes at 336 sites in 35 countries 600 projects and 2500 researchersServes 3-4 TB/day to ~1M clients

MyPLC: PlanetLab software distributionAnyone can run their own private PlanetLab

16

PL-VINI: Prototype on PlanetLab

PlanetLab: testbed for planetary-scale services Simultaneous experiments in separate VMs

Each has “root” in its own VM, can customize

Can reserve CPU, network capacity per VM

Virtual Machine Monitor (VMM)(Linux++)

NodeMgr

LocalAdmin

VM1 VM2 VMn…PlanetLab node

17

Resource Isolation

Issue: Forwarding packets in user space PlanetLab sees heavy use CPU load affects virtual network performance

Property Depends On Solution

Throughput CPU% received PlanetLab provides CPU reservations

Latency CPU scheduling delay

PL-VINI: boost priority of packet forward process

18

PL-VINI: Prototype on PlanetLab

Enable experiment: Internet In A Slice XORP open-source routing protocol suite (NSDI ’05) Click modular router (TOCS ’00, SOSP ’99)

Clarify issues that a VINI must address Unmodified routing software on a virtual topology Forwarding packets at line speed Illusion of dedicated hardware Injection of faults and other events

19

Synthesis of Software Components to enable PL-VINI XORP – Open Source Routing Protocol Suite especially

developed for network research community. [9, NSDI ’05]

Click – Software Architecture for Building Flexible and Configurable routers [10]

UML – Linux-Kernel running as regular Linux-User-Process without Root-Privileges.

OpenVPN – Open source SSL VPN solution.

20

XORP: Control Plane

BGP, OSPF, RIP, PIM-SM, IGMP/MLD

Goal: run real routing protocols on virtual network topologies

XORP(routing protocols)

21

User-Mode Linux: Environment

Interface ≈ network PlanetLab limitation:

Slice cannot create new interfaces

Run routing software in UML environment

Create virtual network interfaces in UML

XORP(routing protocols)

UML

eth1 eth3eth2eth0

22

Click: Data Plane

Performance Avoid UML overhead Move to kernel, FPGA

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

Data

23

Click: Data Plane

Performance Avoid UML overhead Move to kernel, FPGA

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

24

Click: Data Plane

Performance Avoid UML overhead Move to kernel, FPGA

Interfaces tunnels Click UDP tunnels

correspond to UML network interfaces

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

Tunnel table

25

Injecting Link Failures

XORP(routing protocols)

UML

eth1 eth3eth2eth0

Click

PacketForwardEngine

Control

DataUmlSwitch

element

Tunnel table

Filters

Performance Avoid UML overhead Move to kernel, FPGA

Interfaces tunnels Click UDP tunnels

correspond to UML network interfaces

Filters “Fail a link” by blocking

packets at tunnel26

Life of a packet in IIAS

27

Experiments

MicrobenchmarksCapacityBehavior

Intra domain Route changesFailure and recovery

28

Overlay Efficiency IIAS’s user-space Click forwader vs. in-

kernel forwarding

29

Performance is bad

User-space Click: ~200Mb/s forwarding Can do a lot with 200Mb/s

20 experiments can have dedicated 10Mb/s nationwide networks

Improving performance is ongoing workAllow experiments to load custom Click

modules into the VINI kernel30

Overlay on PlanetLab on dedicated hardware (DETER) vs.

shared platform (PlanetLab) and PL-VINI

31

PL-VINI and IIAS together provide a close approximation of the underlying network’s behaviors.

Overlay on PlanetLab

32

Intra-domain Route Changess

c

1176

587 846

260

700

6391295

2095

902

548

233

1893

366

Watch OSPF route convergence on Abilene

856

33

Intra-domain Route Changess

c

1176

587 846

260

700

6391295

2095

902

548

233

1893

366

Watch OSPF route convergence on Abilene

856

34

Intra-domain Route Changess

c

1176

587 846

260

700

6391295

2095

902

548

233

1893

366

Watch OSPF route convergence on Abilene

856

35

Intra-domain Route Changess

c

1176

587 846

260

700

1295

2095

902

548

233

1893

366

Watch OSPF route convergence on Abilene

856

36

Intra-domain Route Changess

c

1176

587 846

260

700

6391295

2095

902

548

233

1893

366

Watch OSPF route convergence on Abilene

856

37

Ping During Link Failure

70

80

90

100

110

120

0 10 20 30 40 50

Pin

g R

TT

(m

s)

Seconds

Link down

38

Ping During Link Failure

70

80

90

100

110

120

0 10 20 30 40 50

Pin

g R

TT

(m

s)

Seconds

Link down

Link up

39

Ping During Link Failure

70

80

90

100

110

120

0 10 20 30 40 50

Pin

g R

TT

(m

s)

Seconds

Link down

Link up

Routes converging

40

TCP Throughput

0

2

4

6

8

10

12

0 10 20 30 40 50

Meg

abyt

es t

rans

ferr

ed

Seconds

Packet receiv ed

Link down

41

TCP Throughput

0

2

4

6

8

10

12

0 10 20 30 40 50

Meg

abyt

es t

rans

ferr

ed

Seconds

Packet receiv ed

Link down

Link up

42

TCP Throughput

0

2

4

6

8

10

12

0 10 20 30 40 50

Meg

abyt

es t

rans

ferr

ed

Seconds

Packet receiv ed

Link down

Link up

Zoom in

43

Arriving TCP Packets

2.1

2.15

2.2

2.25

2.3

2.35

2.4

2.45

17.5 18 18.5 19 19.5 20

Meg

abyt

es in

str

eam

Seconds

Packet receiv ed

Slow start

44

Arriving TCP Packets

Slow start

Retransmitlost packet

2.1

2.15

2.2

2.25

2.3

2.35

2.4

2.45

17.5 18 18.5 19 19.5 20

Meg

abyt

es in

str

eam

Seconds

Packet received

45

Arriving TCP Packets

2.1

2.15

2.2

2.25

2.3

2.35

2.4

2.45

17.5 18 18.5 19 19.5 20

Meg

abyt

es in

str

eam

Seconds

Packet receiv ed

Slow start

Retransmitlost packet

PL-VINI enables a user-space virtual networkto behave like a real network on PlanetLab

46

Conclusion

VINI: Virtual Network Infrastructure Controlled and realistic experimentation New routing, forwarding, and addressing

Initial experiment Intradomain routing convergence in XORP Later, perhaps run commercial router code?

Long term goal Shared platform for networking experiments Real user traffic and upstream ISP connections

47