What do we do at Axsh?

● Virtualization● Infrastructure as a Service (IaaS)● Software Defined Networking (SDN)● DevOps, Continuous Integration/Delivery● Free open source software

Our two main FOSS projects

● Wakame-vdc

Full featured IaaS solution

(virtual data center)

● OpenVNet

Full featured SDN solution

(virtual network)

● http://axsh.jp

What is OpenVNet

● Software defined networking (SDN)● Free open source software● Written in Ruby

● http://axsh.jp/openvnet/● https://github.com/axsh/openvnet

Agenda

● Quick introduction to SDN● Simple OpenVNet demo● Detailed look at integration test

● Goal:Learn how to set up advanced OpenVNet environments by reading the integration test

The concept of SDN

● First look at vitualization

Hardware

Operating SystemVS

Physical machine

Hardware

Operating System Operating System

Hypervisor

Virtual machines

SDN with OpenVNet

Hypervisor host Hypervisor host Hypervisor hostPhysicalNetwork

Virtual(softwaredefined)Network

VM VM VM VM VM VM

NAT DNS DHCP Routers Switches Firewalls

The concept of SDN

● A physical network

NIC NIC NIC NIC NIC NIC

172.16.90.0/24

172.16.91.0/24

Router

The concept of SDN

● A physical network

NIC NIC NIC NIC NIC NIC

Switch Switch

Switch Switch

Router

The concept of SDN

● A virtual network

NIC NIC NIC NIC NIC NIC

Switch Switch

Switch Switch

Router

The concept of SDN

● Create any network without changing hardware

NIC NIC NIC NIC NIC NIC

Switch Switch

Switch Switch

Router

Virtual network: 10.100.0.0/24

The concept of SDN

● Create any network without changing hardware

NIC NIC NIC NIC NIC NIC

Switch Switch

Switch Switch

Router

Virtual network: 10.100.0.0/24 Virtual network: 192.168.100.0/24

The concept of SDN

● Create any network without changing hardware

NIC NIC NIC NIC NIC NIC

Switch Switch

Switch Switch

Router

Virtual network: 172.16.0.0/16

Virtual network: 10.100.0.0/24

The concept of SDN● How is this magic possible?● Because OpenVNet controls the switches

(using OpenFlow)

NIC NIC NIC NIC NIC NIC

Switch Switch

Switch Switch

RouterOpenVNet OpenVNet

OpenVNet

OpenVNet's inner workings

User laptop

Hypervisor Host

Open vSwitchVNA

VM VM VM VM VM

Vnctl

Web (REST) API

HTTP

Database Vnmgr

VNet Edge

Other networks

Physical network

A quick demo

● Try it yourself http://openvnet.org/installation/

Vnctl

WebAPI

Vnmgr

VNA Open vSwitch

VM (inst1)VM (inst1) VM (inst2)Database

Demo: 1 network

● http://openvnet.org/creating-virtual-networks/single-network/

VM (inst1)VM (inst1) VM (inst2)

10.100.0.0/24

10.100.0.10 10.100.0.11

Demo 2: DHCP server

● http://openvnet.org/creating-virtual-networks/single-network-dhcp/● DHCP is simulated by OpenVNet

VM (inst1)VM (inst1) VM (inst2)

10.100.0.0/24

10.100.0.10 10.100.0.11

DHCP

10.100.0.100

Demo 3: Two virtual networks

● http://openvnet.org/creating-virtual-networks/two-networks/

VM (inst1)VM (inst1)

172.16.0.10

DHCP

172.16.0.100

172.16.0.0/24

VM (inst1)VM (inst2)

192.168.100.10

DHCP

192.168.100.100

192.168.100.0/24

Demo 4: Router

● http://openvnet.org/creating-virtual-networks/two-networks-router/● Like DHCP, routers are simulated

VM (inst1)VM (inst1)

172.16.0.10

DHCP

172.16.0.100

172.16.0.0/24

VM (inst1)VM (inst2)

192.168.100.10

DHCP

192.168.100.100

192.168.100.0/24

172.16.0.1

192.168.100.1

The integration test

● Shows how to set up OpenVNet on multiple hosts

● Included in the OpenVNet source code● https://github.com/axsh/openvnet

● OpenVNet's physical and virtual network setup is integration_test/dataset directory.

● Test scenarios are in the integration_test/lib/vnspec/spec directory.

The test code

● The yaml files get translated to vnctl commands

datapaths:

- uuid: dp-1 node_id: vna1 display_name: node1 dpid: "0x0000aaaaaaaaaaaa"

vnctl datapaths add --uuid "dp-1" --node_id "vna1" --display_name "node1" --dpid "0x0000aaaaaaaaaaaa"

Becomes

The Integration Test

The Integration Test

Let's take it step by step

● OpenVNet's simplest physical network● Two VMs in 1 host● This allows us to test many scenarios already

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Add one host

● Test scenarios with multiple VNA on multiple hosts

● The red network is for OpenVNet's internal communication

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Itest2

VNA Open vSwitch

VM 3 VM 4

Eth0 Eth0Eth1 Eth1

Add another host in another subnet

● Test scenarios with multiple VNA/hosts on multiple subnets

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Eth0Eth1

Itest2

VNA Open vSwitch

VM 3 VM 4

Eth0Eth1

Itest3

VNA Open vSwitch

VM 5 VM 6

Eth0Eth1

A quick review

● Remember this whole setup is OpenVNet's physical network

● This topology will never change over the course of the tests

● We can create any virtual network topology we want using VM 1 ~ VM 6

One more thing we need

● We can now test everything except VNet Edge

● VNet Edge connects non-OpenVNet managed networks to virtual networks

Add VNet Edge● Simulate non-OpenVNet (Legacy) network using Legacy1 VM

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Eth0Eth1

Itest2

VNA Open vSwitch

VM 3 VM 4

Eth0Eth1

Itest3

VNA Open vSwitch

VM 5 VM 6

Eth0Eth1

Itest-Edge

Eth1VNA

Open vSwitch

Eth0 Eth2

Legacy1

Eth0

We're done

● We can now test all of OpenVNet's features

● We run this entire environment in one host

● That means we use nested virtualization

Host (server or laptop)

Nested VMs

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Eth0Eth1

Itest2

VNA Open vSwitch

VM 3 VM 4

Eth0Eth1

Itest3

VNA Open vSwitch

VM 5 VM 6

Eth0Eth1

Itest-Edge

Eth1VNA

Open vSwitch

Eth0 Eth2

Legacy1

Eth0

= HOST = VM = VM in VM

Building this environment

● The build scripts are available on github

● https://github.com/axsh/wakame-ci-cluster/tree/master/kvm-guests/90-vteskins

● These scripts build the environment but do not run any test code

Physical network setup

● OpenVNet needs to be made aware of the physical network layout

● This is the same for every test scenario

● integration_test/dataset/base.yml+base_dp.yml OR base_topology.yml

base_dp Vs. base_tp

● A datapath roughly means a single Open vSwitch

● OpenVNet uses internal datapath_network structures to keep track of which network is on which datapath.

Open vSwitch = datapath

base_dp Vs. base_tp

● Topology is a new feature that generates datapath_networks automatically

● base_dp sets datapath_networks manually

● base_tp uses Topology

● Which one to use depends on the test scenario

Other yaml files are test scenarios

● simple.yml is the simplest scenario with manual datapath_network

● simple_tp.yml is the same scenario with Topology

simple.yml / simple_tp.yml

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Eth0Eth1

Itest2

VNA Open vSwitch

VM 3 VM 4

Eth0Eth1

Itest3

VNA Open vSwitch

VM 5 VM 6

Eth0Eth1

Itest-Edge

Eth1

VNA

Open vSwitch

Eth0 Eth2

Legacy1

Eth0DHCP

nw-vnet1

DHCP

nw-vnet2

simple.yml / simple_tp.yml

VM 1 VM 2 VM 3 VM 4 VM 5 VM 6

DHCP

nw-vnet1 (10.101.0.0/24)

DHCP

nw-vnet2 (10.101.0.0/24)

● Both networks can have the same IP addresses

Itest1

VNAWebAPI

Database

Vnmgr

Open vSwitch

VM 1 VM 2

Eth0Eth1

Itest2

VNA Open vSwitch

VM 3 VM 4

Eth0Eth1

Itest3

VNA Open vSwitch

VM 5 VM 6

Eth0Eth1

Itest-Edge

Eth1

VNA

Open vSwitch

Eth0 Eth2

Legacy1

Eth0

router_v2v / router_v2v_tp

VM 1 VM 2 VM 3 VM 4 VM 5 VM 6

DHCP

nw-vnet1

DHCP

nw-vnet2

router_v2v / router_v2v_tp

VM 1 VM 2 VM 3 VM 4 VM 5 VM 6

DHCP

nw-vnet1 (10.101.0.0/24)

DHCP

nw-vnet2 (10.102.0.0/24)

● Two networks with a router● IP addresses can't be the same with

a router between networks

Service.yml

VM 1 VM 2 VM 3 VM 4 VM 5 VM 6

DHCP

nw-vnet1 (10.101.0.0/24)

DHCP

nw-vnet2 (10.101.0.0/24)

● Adds DNS service

DNS

DNS

Other scenarios

● edge.yml: L2 connectivity between virtual network and non-OpenVNet managed network

● event.ymlChanging virtual network layout on the fly

● secg.yml / secg_reference.ymlSecurity groups and connection tracking

● edge_esxi.ymlDeprecated esxi support

Other scenarios

● filter.ymlSimple firewall rules

● router_p2v.ymlRouting between virtual and physical networks

● WanEdgeNAT and routing to global network

OpenVNet feature list

● Two protocols to support virtual networking.

- **MAC2MAC** (Axsh original protocol for physical L2 tunneling)

- **GRE** (protocol for L3 tunneling)● Simulated DHCP service● Simulated DNS service● L3 routing between virtual networks.● Single hop L3 routing between physical and virtual networks.● Firewall● Connection tracking● Integration with Wakame-vdc.● VNet Edge feature (connect virtual and physical networks)

Thank you for listeninghttp://axsh.jp