Post on 16-Aug-2019
transcript
libVNF: building VNFs made easyPriyanka Naik, Akash Kanase, Trishal Patel, Mythili Vutukuru
Dept. of Computer Science and EngineeringIndian Institute of Technology, Bombay
SoCC’18 11th October, 2018
NFV ecosystem
Network address translator
Router
Load balancer
Firewall
VM
VNF
VM
VNF
VM
VNF
VM
VNF
Hypervisor
Orchestrator
NFV: Network Function Virtualization VNF: Virtual Network Function �2
NFV ecosystem
Network address translator
Router
Load balancer
Firewall
VM
VNF
VM
VNF
VM
VNF
VM
VNF
Hypervisor
● Will they give good performance ?
● Is it easy to build them?
�3NFV: Network Function Virtualization VNF: Virtual Network Function
Orchestrator
How to build VNF?
VNF code developed by VNF developer
�4
How to build VNF?
VNF code developed by VNF developer
�4
38% EPC code → read/write packets
CORD Intel EPC: https://gerrit.opencord.org/ngic
How to build VNF?
VNF code developed by VNF developer
VNF Processing logic
VNF Framework
�4
38% EPC code → read/write packets
CORD Intel EPC: https://gerrit.opencord.org/ngic
How to build VNF?
VNF Processing logic
VNF Frameworknetbricks
OpenNF VPP
StatelessNF
�4
How to build VNF?
VNF Processing logic
VNF Frameworknetbricks
OpenNF VPP
StatelessNF
�4
What is missing in these frameworks?
What is required from VNF frameworks?● Requirement 1: Support for both L3 and Transport VNF
● Requirement 2: Flexibility of network stack
● Requirement 3: Support for distributed state management
�5
What is required from VNF frameworks?● Requirement 1: Support for both L3 and Transport VNF
● Requirement 2: Flexibility of network stack
● Requirement 3: Support for distributed state management
�6
Support for L3 and transport VNFsLayer 3 VNFs
Network address translator
Layer 3 Load balancer
�7
Support for L3 and transport VNFsLayer 3 VNFs
N/W layer
Data link layer
Header manipulations
Network address translator
Layer 3 Load balancer
�7
Support for L3 and transport VNFsLayer 3 VNFs
N/W layer
Data link layer
Header manipulations
Network address translator
Layer 3 Load balancer
Frameworks: netbricks, YANFF
�7
Support for L3 and transport VNFsLayer 3 VNFs
N/W layer
Data link layer
Header manipulations
Network address translator
Layer 3 Load balancer
Frameworks: netbricks, YANFF
vEPC internet
Transport Layer VNFs
�7
Support for L3 and transport VNFsLayer 3 VNFs
N/W layer
Data link layer
Header manipulations
Network address translator
Layer 3 Load balancer
Frameworks: netbricks, YANFF
vEPC internet
Transport Layer VNFs
N/W layer
Transport Layer
Data link layer
Request processing
Connection termination
Connection initiation
�7
Support for L3 and transport VNFsLayer 3 VNFs
N/W layer
Data link layer
Header manipulations
Network address translator
Layer 3 Load balancer
Frameworks: netbricks, YANFF
vEPC internet
Transport Layer VNFs
N/W layer
Transport Layer
Data link layer
Request processing
Connection termination
Connection initiation
Frameworks: mTCP, TLDK
�7Netbricks: Taking the v out of nfv. In Proc. of OSDI’16 YANFF: https://www.openhub.net/p/yanff mTCP: A highly scalable user-level tcp stack for multicore systems. In Proc. of NSDI’14 TLDK: https://wiki.fd.io/view/TLDK
Support for L3 and transport VNFsLayer 3 VNFs
N/W layer
Data link layer
Header manipulations
Network address translator
Layer 3 Load balancer
Frameworks: netbricks, YANFF
vEPC internet
Transport Layer VNFs
N/W layer
Transport Layer
Data link layer
Request processing
Connection termination
Connection initiation
Frameworks: mTCP, TLDK
�7Netbricks: Taking the v out of nfv. In Proc. of OSDI’16 YANFF: https://www.openhub.net/p/yanff mTCP: A highly scalable user-level tcp stack for multicore systems. In Proc. of NSDI’14 TLDK: https://wiki.fd.io/view/TLDK
Are these frameworks enough?
Event driven I/O
�8
Existing transport-layer frameworks are event-driven
Event driven I/O
Pros:
�8
Existing transport-layer frameworks are event-driven
Event driven I/O
Pros:
● Efficient for multi-core scalability
�8
Existing transport-layer frameworks are event-driven
Event driven I/O
Pros:
● Efficient for multi-core scalability
�8
Existing transport-layer frameworks are event-driven
Event driven I/O
Pros:
● Efficient for multi-core scalability
Cons:
�8
Existing transport-layer frameworks are event-driven
Event driven I/O
Pros:
● Efficient for multi-core scalability
Cons:
● Needs explicit request state storage
�8
Existing transport-layer frameworks are event-driven
Need to maintain request state
A B C1 2
34
�9
Need to maintain request state
State at B to process A’s request
A B C1 2
34
A’s request
C’s reply
Connection identifiers
�9
Need to maintain request state
DPDK and netmap layer (packet)
network stack (mTCP) (connection)
VNF processing layer (abstraction ?)
State at B to process A’s request
A B C1 2
34
A’s request
C’s reply
Connection identifiers
�9
Need to maintain request state
DPDK and netmap layer (packet)
network stack (mTCP) (connection)
VNF processing layer (abstraction ?)
State at B to process A’s request
A B C1 2
34
A’s request
C’s reply
Connection identifiers
�9
Existing frameworks do not provide this support
What is required from VNF frameworks?● Requirement 1: Support for both Layer 3 and Transport VNF
● Requirement 2: Flexibility of network stack
● Requirement 3: Support for distributed state management
�10
Flexibility of network stack
vNIC
Kernel network stack
Application VNF
Kernel Stack
�11
Flexibility of network stack
vNIC
DPDK/netmap
Application VNF + userspace stack
vNIC
Kernel network stack
Application VNF
Kernel Stack Kernel Bypass Stack
�11
Flexibility of network stack
vNIC
DPDK/netmap
Application VNF + userspace stack
vNIC
Kernel network stack
Application VNF
Kernel Stack Kernel Bypass Stack
�11
Easy switch between stacks
What is required from VNF frameworks?● Requirement 1: Support for both L3 and Transport VNF
● Requirement 2: Flexibility of network stack
● Requirement 3: Support for distributed state management
�12
Support for distributed state management
VM
VNF 1
VM
VNF 2
VM
VNF 2
VM
VNF 3
Hypervisor
Orchestrator
�13
Support for distributed state management
VNF 2
VNF 2
�13
State Synchronization
Support for distributed state management
VNF 2
VNF 2
State Migration openNF, split/merge
�13
State Synchronization
Support for distributed state management
VNF 2
VNF 2
State Migration openNF, split/merge
Data StoreRemote store statelessNF
�13
State Synchronization
Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI’17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI’13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM’14
Support for distributed state management
VNF 2
VNF 2
State Migration openNF, split/merge
Data StoreRemote store statelessNF
�13
State Synchronization
None of above support transport layer VNFs
Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI’17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI’13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM’14
Summary of VNF FrameworksRequirement/
Framework netbricks Flick StatelessNF Split-Merge/
OpenNFlibVNF
Layer 3 + App-layer support
no yes no no yes
Flexibility of network stack
no no no no yes
Distributed State
Management
no no yes yes yes
�14
Netbricks: Taking the v out of nfv. In Proc. of OSDI’16 Flick: Developing and running application-specific network services. In Proc. of USENIX ATC’16 Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI’17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI’13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM’14
libVNF Design Goals
Flexibility of network stack
Support for network and transport layer VNF
Distributed State Management
VNF processing logic Handled by VNF developer
Handled by libVNF
R3
R2
R1
R: Requirement�15
libVNF overviewVNF code
libVNF APIAPI Calls
�16
libVNF overview
mTCP+ netmap/DPDK initialization
Kernel stack initialization
VNF code
libVNF APIAPI Calls
Stack initialization
�16
libVNF overview
mTCP+ netmap/DPDK initialization
Kernel stack initialization
VNF code
libVNF APIAPI Calls
Stack initialization
�16
Per-core threads
libVNF overview
mTCP+ netmap/DPDK initialization
Kernel stack initialization
VNF code
libVNF APIAPI Calls
Per-core data structures
Stack initialization
�16
Per-core threads
● Lock-free ● Cache optimized
libVNF API
Communication State ManagementRequest state
libVNF API
�17
libVNF API
State ManagementRequest state
libVNF API
�18
libVNF API
State ManagementRequest state
libVNF API
�18
Communication
Communication APIVNF code
libVNF API
Per-core packet pool
Per-core data structures
�19
Pre-allocated memory pools (Per-core packet pools)
Communication APIVNF code
libVNF APIregisterCallback(socket, fn)
Per-core packet pool
Per-core data structures
Store mapping
�19
Pre-allocated memory pools (Per-core packet pools)
Communication APIVNF code
libVNF APIregisterCallback(socket, fn)
Per-core packet pool
Per-core data structures
Store mapping
�19
Packet arrives on socket
fn(packet)
Pre-allocated memory pools (Per-core packet pools)
Communication APIVNF code
libVNF API
Per-core packet pool
Per-core data structures
getPktBuf
�19
Pre-allocated memory pools (Per-core packet pools)
Communication APIVNF code
libVNF API
Per-core packet pool
Per-core data structures
getPktBuf
Buffer to write packet
�19
Pre-allocated memory pools (Per-core packet pools)
VNF Design Requirements
Communication State Management
libVNF API
�20
VNF Design Requirements
Communication State Management
libVNF API
�20
Request state
(abstraction ?)
Need for request state
DPDK and netmap layer (packet)
network stack (mTCP) (connection)
VNF processing layer (abstraction ?)
State at B to process A’s request
A B C1 2
34
A’s request
C’s reply
Connection identifiers
�21
(abstraction ?)
Need for request state
DPDK and netmap layer (packet)
network stack (mTCP) (connection)
VNF processing layer REQUEST OBJECT
State at B to process A’s request
A B C1 2
34
A’s request
C’s reply
Connection identifiers
�22
Request object
Request Object API
A B C1 2
34
libVNF API
Per-core request pool
�23
Request Object API
A B C1 2
34
libVNF API
Per-core request pool
allocReqObj(A connection_id)
Allocate request object block for A’s
request
�23
A’s request
C’s reply
Connection identifiers
Request Object API
A B C1 2
34
libVNF API
Per-core request pool
allocReqObj(A connection_id)
Allocate request object block for A’s
request
�23
A’s request
C’s reply
Connection identifiers
Per-core packet pool
Request Object API
A B C1 2
34
libVNF API
Per-core request pool
linkReqObj(C connection_id)
Link to the existing A request object
�23
A’s request
C’s reply
Connection identifiers
Per-core packet pool
VNF Design Requirements
Communication Request state
libVNF API
�24
VNF Design Requirements
Communication Request state
libVNF API
�24
State Management
State Management APIVNF code
libVNF API
�25
State Management APIVNF code
libVNF API
Local data store pool
�25
State Management APIVNF code
libVNF API
Local data store pool
setData( )LOCAL
Store in local datastore
�25
State Management APIVNF code
libVNF API
Local data store pool
libVNF data store wrapper
Redis KV storeRemote Data store
setData( )LOCAL
Store in local datastore
�25
State Management APIVNF code
libVNF API
Local data store pool
libVNF data store wrapper
Redis KV storeRemote Data store
setData( )REMOTE
Cache locally
Store in remote data store
�25
Evaluation
● Overhead of libVNF
● Scalability with cores
● Benefits of libVNF
�26
Setup
�27
A B C1 2
34
Setup
�27
VNF A VNF C
S/W switch (on kernel)
A B C1 2
34
Setup
�27
VNF A VNF C
S/W switch (on kernel)
VNF B
S/W switch (like netmap-vale)
Physical NIC
NIC Queue
A B C1 2
34
Setup
�27
VNF A VNF C
S/W switch (on kernel)
VNF B
S/W switch (like netmap-vale)
Physical NIC
NIC Queue
A B C1 2
34
VNF A, C: 4 core, 4GB RAM VNF B: 4 GB RAM, cores varied
Evaluation
● Overhead of libVNF
● Scalability with cores
● Benefits of libVNF
�28
Overhead check
�29
Overhead check
�29
<5% overhead of libVNF DPDK~ netmap performance
Evaluation
● Overhead of libVNF
● Scalability with cores
● Benefits of libVNF
�30
Core scalability
�31
Core scalability
�31scales linearly with cores
Evaluation
● Overhead of libVNF
● Scalability with cores
● Benefits of libVNF
�32
Building VNFs
VNF Performance Overhead of libVNF
LoC Saved
IMS (IP Multimedia Subsystem)
3.4% 42%
EPC (LTE-Evolved Packet Core )
5.5% 38%
Layer 3 Load Balancer 14% 52%
�33
Building VNFs
VNF Performance Overhead of libVNF
LoC Saved
IMS (IP Multimedia Subsystem)
3.4% 42%
EPC (LTE-Evolved Packet Core )
5.5% 38%
Layer 3 Load Balancer 14% 52%
�33
Low overhead in app-layer VNF Higher overhead in L3 VNF
Summary● Library to ease building of VNFs
● Expressive to build L3 and App-layer VNF
● Supports multiple network stacks
● Low performance overhead
�34
https://github.com/networkedsystemsIITB/libVNF ppnaik@cse.iitb.ac.in
�35
Thank You
Setup
�36
VNF A VNF C
S/W switch (on kernel)
LB VNF VNF B VNF B Data store VM
S/W switch (vale on netmap)
Physical NIC
NIC Queue
A B C1 2
34
Setup
�36
VNF A VNF C
S/W switch (on kernel)
LB VNF VNF B VNF B Data store VM
S/W switch (vale on netmap)
Physical NIC
NIC Queue
A B C1 2
34
VNF A, C: 4 core, 4GB RAM VNF B: 4 GB RAM, cores varied Data Store VM: 6 core, 16GB RAM LB: 1 core, 4GB RAM