VIOLIN: A Network Virtualization Middleware for Virtual Networked Computing

Dongyan XuLab FRIENDS

(For Research In Emerging Network and Distributed Services)Department of Computer Sciences

Center for Education and Research in Information Assurance and Security (CERIAS)

Purdue University

The Team

Lab FRIENDS Xuxian Jiang (Ph.D. student) Paul Ruth (Ph.D. student) Dongyan Xu (faculty)

Supported in part by NSF Middleware Initiative (NMI)

Outline

Motivations and goals Architecture of VIOLIN Applications of VIOLIN

Network system emulation Scientific computing Honeyfarm (network attack aggregation)

On-going work

Motivations

Formation of wide-area shared cyber-infrastructure Multiple domains Heterogeneous platforms Large number of users

Need for mutually isolated distributed environments Customized system administration and configuration Consistent and binary-compatible runtime support Un-trusted or malfunctioning applications

Known vulnerabilities in SETI@Home, KaZaa, and Condor Un-trusted network traffic control

Potential Applications

Multi-institutional collaboratories Large-scale distributed emulations

Cyber-systems Real-world systems

Parallel/distributed scientific applications Philanthropic (volunteer) computing

services Content distribution networks

VM (Virtual Machine): a Solution?

Achieves single node isolation (SODA*) Administration Resource Runtime services/libraries Fault/attack impact

However, does not achieve network isolation VMs addressable from/to any Internet hosts Cannot control traffic volume between VMs Cannot have overlapping address spaces* X. Jiang, D. Xu, “SODA: Service-on-Demand Architecture for Service

Hosting Utility Platforms”, IEEE HPDC-12, 2003.

VIOLIN: Proposed Solution

VIOLIN: A VN (Virtual Network) for VMs * Independent IP address space Invisible from Internet and vice versa Un-tamperable topology and traffic control Value-added network services (e.g., IP

multicast) Binary and IP compatible runtime

environment * X. Jiang, D. Xu, “VIOLIN: Virtual Internetworking on OverLay INfrastructure”, Springer LNCS Vol. 3358 (ISPA 2004).

* D. Xu, X. Jiang, “Towards an Integrated Multimedia Service Hosting Overlay”, ACM Multimedia 2004.

VIOLIN: the Big Picture

Internet

NMI

NMI

NMI N

MI

NMIN

MI

NMI

Physicalinfrastructure

NMI-based Gridinfrastructure

Two mutuallyIsolated VIOLINs VM

Key Ideas in VIOLIN

One level of indirection between VIOLIN and real Internet “All problems in Computer Science can be

solved by another level of indirection ” – Butler Lampson

A middleware-level underlay network serving as “intelligent carrier” of a VIOLIN Traffic tunneling Topology control Traffic volume control Traffic encryption Network service virtualization

VIOLIN Architecture

Host OS

Guest OS

App1

Guest OS

App2

VIOLIN daemon

…

ExistingNMI

Middleware

VMs

Physical host

VIOLIN Architecture

Host OS

Guest OS

App1

VIOLIN daemon

Virtual NIC

Host OS

Guest OS

App1

VIOLIN daemon

Virtual NIC

Message (e.g.,MPI)

TCP, UDP, …

IP

Ethernet frame via UDP tunneling

Between two VIOLIN nodes (VMs)

planetlab6.csail.mit.edu planetlab6.millennium.berkeley.edu

196.128.1.2 196.128.1.3

VIOLIN Network Performance

0

0.5

1

1.5

2

2.5

3

2 12 22 32 42 52

TCP Window Size (KBytes)

Th

rou

gh

pu

t (M

bp

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

0

0.5

1

1.5

2

2.5

3

2 12 22 32 42 52

TCP Window Size (KBytes)

Th

rou

gh

pu

t (M

bp

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

TCP throughput measurement on PlanetLabplanetlab6.csail.mit.edu → planetlab6.millennium.berkeley.edu

VIOLIN Network Performance

ICMP latency measurement on PlanetLabplanetlab6.csail.mit.edu → planetlab6.millennium.berkeley.edu

60

70

80

90

100

110

120

130

30 530 1030 1530 2030 2530

ICMP Data Length (bytes)

La

ten

cy

(m

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

60

70

80

90

100

110

120

130

30 530 1030 1530 2030 2530

ICMP Data Length (bytes)

La

ten

cy

(m

s)

w/o VIOLIN

w/ VIOLIN

w/ VIOLIN + encryption

Application I: Network System Emulation

vBET: an education toolkit for network emulation * “Create your own IP network ” on a shared platform

IP address space and network topology Routers, switches, firewalls, end-hosts, links Real-world network software (OSPF, BGP…)

Strict confinement (network security experiments) Flexible configuration

Not constrained by device/port availability No manual cable re-wiring or hardware setup

* X. Jiang, D. Xu, “vBET: a VM-Based Emulation Testbed”, ACM SIGCOMM Workshop on Models, Methods, and Tools for Reproducible Network Research (ACM MoMeTools), 2003

vBET GUI

Sample Emulation: OSPF Routing

Emulation of OSPF RoutingDemo video clip:

Sample Emulation: Critical Server Protection

Screenshot: Distributed Firewall

Sample Emulation: Chord P2P Network

Screenshot

Sample Emulation: Internet Worms

* X. Jiang, D. Xu, H. J. Wang, E. H. Spafford, “Virtual Playgrounds for Worm Behavior Investigation”, 8th International Symposium on RecentAdvances in Intrusion Detection (RAID’05), 2005.

A shared infrastructure (e.g. PlanetLab)

A worm playground

Virtual

Physical

Application II: Scientific Computing*

Virtual clusters leveraging idle CPU cycles Long running parallel/distributed jobs Complicated communication patterns

between nodes (different from SETI@Home, Condor)

Runtime adaptation Resource re-allocation Migration/re-location Scale adjustment

* P. Ruth, X. Jiang, D. Xu, S. Goasguen, “Towards Virtual DistributedEnvironments in a Shared Infrastructure”, IEEE Computer, May 2005.

Experiment Setup

Physical Cluster(ITaP)

Two mutuallyisolated virtual clusters

VM

Physical Switch

VS VS

VIOLIN vs. Physical Hosts (running HPL benchmark)

Physical host: dual processor 1.2 GHz Athlon, 1GB memory VM: running one per host, ≤512MB memory

Performance of VIOLIN vs. Physical Cluster

0

5

10

15

20

25

30

35

2 4 8 16 32 64Number of Processors or VMs

GF

lops VIOLIN

Physical Cluster

Multiple VIOLINs Sharing Physical Hosts(running HPL benchmark)

Aggregate performance remains stable (up to 16 VIOLINs) In this example, 16 VIOLINs exhaust memory

0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

GF

lops

1 2 4 8 16

Number of VIOLINs

Performance of Multiple VIOLINS Sharing Physical Hosts

VM Communication Pattern

7MB/s

6MB/s

172

3640

5

6MB/s

4MB/s

6MB/s

6MB/s

5MB/s

5MB/s

7MB/s

5MB/s

3MB/s

Application III: Honeyfarm

Collapsar: a network attack aggregation center * Achieving two (seemingly) conflicting goals

Distributed honeypot presence Centralized honeypot operation

Key ideas Leveraging unused IP addresses in each network Diverting corresponding traffic to a “detention”

center (transparently), by VIOLIN Creating VM-based honeypots in the center

* X. Jiang, D. Xu, “Collapsar: a VM-Based Architecture for Network Attack Detention Center”, 13th USENIX Security Symposium (Security’04), 2004.

Collapsar Architecture

VM-based Honeypot

Collapsar Architecture

Redirector

Redirector Redirector

Correlation Engine

Management Station

Production Network

Production Network

Production Network

Collapsar Center

Attacker

Front-End

Real-Time Worm Alert

* X. Jiang, D. Xu, R. Eigenmann, “Protection Mechanisms for Application Service Hosting Platforms”, IEEE/ACM CCGrid’04, 2004.

Log Correlation: Stepping StoneLog Correlation: Stepping Stoneiii.jjj.kkk.11 compromised a honeypot & installed a rootkit, which contained

an ssh backdoor

iii.jjj.kkk.11 compromised a honeypot & installed a rootkit, which contained

an ssh backdoor

xx.yyy.zzz.3 connected to the ssh backdoor using the same passwd

xx.yyy.zzz.3 connected to the ssh backdoor using the same passwd

Log Correlation: Network ScanningLog Correlation: Network Scanning

On-going Work

VIOLIN-based virtual distributed environments on shared cyber-infrastructure Self-management (making them smart

entities) Missing role of VIOLIN administrator Automatic customization and bootstrapping Enforcement of application-specific policies

Self-provisioning (application-driven) Resource scaling Scale adaptation Topology evolution

Thank you.

For more information:

Email: dxu@cs.purdue.eduURL: http://www.cs.purdue.edu/~dxuGoogle: “Purdue SODA Friends”