Bridges and Tunnels: A Drive Through OpenStack Networking

Mark McClaintwitter: @gtwmm

Why Create Neutron?

• New Technology

• Tenant Isolation

• Rich Topologies

• Advance Services Support

• Load Balancing, VPN, Firewall

The Basics

What does the user see?

Compute API

Network API

Storage APIGUI, CLI, API Libs

KVM

ML2 Plugin

Ceph

Abstractions

Net110.0.0.0/24

Nova

Neutron

L2 virtual network

virtual port

virtual server

virtual interface (VIF)

virtual subnet

VM110.0.0.2

VM210.0.0.2

Architecture

Design Goals

Open vSwitch / Linux Bridge

Ryu OpenFlow Controller

• Unified API

• Small Core

• Pluggable Open Architecture

• Multiple Vendor Support

• Extensible

OpenStack The Operator View

Basic Deployment

neutron-server

L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent

L3 AgentL3 Agent

L3 AgentL3 Agent

Database

L3 Agent

DHCP Agent

L2 Agent

Message Queue

Adv Services

neutron-server

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

REST API SERVICE RPC SERVICE

• REST API

• HTTP(S) Python WSGI Application

• Customary TCP port is 9696

• Exposes logical resources

• networks, subnets, ports, etc

• Request/Response Serialization

• RPC Service

• AMQP via Oslo messaging modules

• Enables bidirectional agent communication

• Optional

neutron-server

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

• PLUGIN

• Written in Python

• Only one active

• Must implement V2 API calls

• Optional database access

• Optional extension support

The Plugin

ML2Plugin

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

The Plugin

NeutronPluginBaseV2

NeutronDbPluginV2

ML2Plugin

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

Plugin Extensions

• Add logical resources to the REST API

• Discovered by server at startup

• REST: /v2.0/extensions

• Common Extensions

• Binding, DHCP, L3, Provider, Quota, Security Group

• Other Extensions

• Allowed Addresses, Extra Routes, Metering

Monolithic Plugin

• Full implementation of core resources

• Two types:

• Proxy

• Direct control PLUGIN

ML2: Modular Layer 2 Plugin

• Full V2 Plugin Implementation

• Delegates calls to proper L2 drivers

• Two kinds of drivers

• Type Driver

• Mechanism Driver Mech Mgr

PLUGIN

Type MgrPLUGIN

Basic Deployment

neutron-server

L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent

L3 AgentL3 Agent

L3 AgentL3 Agent

Database

L3 Agent

DHCP Agent

L2 Agent

Message Queue

Adv Services

L2 Agent

L2 Agent

• Runs on hypervisor

• Watch and notify when devices added/removed

• Wires new devices to bridge

• Proper network segment

• Security Group Rules

Isolation

VLAN

• 802.1Q

• limited

• underlay must support

GRE/VXLAN

• L2 encapsulated in L3

• routable

• overlay independence

Tunneling

A

D

CB

Tunneling with L2 Population

A

D

CB

• Open Source Virtual Switch

• http://openvswitch.org

• Can manage

• VLAN, GRE, VXLAN OVSDB

Open vSwitch

Neutron Server

OVS Agent

OVS

RPC

Basic Deployment

neutron-server

L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent

L3 AgentL3 Agent

L3 AgentL3 Agent

Database

L3 Agent

DHCP Agent

L2 Agent

Message Queue

Adv Services

Linux Network Namespace

• Isolated copy of network stack

• private loopback

• scope limited to namespace

• can reuse addresses

• Explicit configuration needed to connect

• Processes can spawn within namespace

lo

eth1

eth0

lo

eth1

eth0

lo

eth1

eth0

Host A B

br-int

Network Node

L3 Agent

• Run on Network Node

• Uses Namespaces

• Metadata Agent (if enabled)

Network Node

Core

Hypervisor Hypervisor Hypervisor

L3 Agent How it’s implemented

• Manages Collection of Network Namespaces

• Isolated IP Stacks

• Forwarding Enabled

• net.ipv4.ip_forward=1

• Static Routing

• Metadata Proxy

lo

eth1

eth0

lo

qg-2

qr-1

lo

qg-b

qr-e

Host A B

br-ex

New in Icehouse

• 4 New Plugins/Drivers

• Stability Improvements

• Externally Tested Vendor Plugins

• Performance Improvements

Looking ahead to Juno

• Nova-Network Migration

• Distributed Virtual Routers

• IPv6

• Multi Vendor Service Support

More Information

• Cloud Administrator Guide

• http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html

• Network v2.0 API

• http://developer.openstack.org/api-ref-networking-v2.html

Thank You

Mark McClaintwitter: @gtwmm

Extra Info

OSI Model

Physical LayerData Link LayerNetwork Layer

Transport LayerSession Layer

Presentation LayerApplication Layer

123

4567

TCP, UDP

IPv4,IPv6, ICMP

HTTP, DNS, etc

ARP, Ethernet, VLAN

Configuration Agents

Configuration Agents: DHCP

• RPC based notifications

• dnsmasq

• Isolation Support via Network Namespaces

• Multiple copies for HA

Configuration Agents: Metadata Proxy

• Proxies Metadata requests to Nova

• Routed Networks

• process embedded in router

• Non-routed Networks

• static route redirects traffic running in DHCP namespace

Configuration Agents: Metadata Proxy

curl http://169.254.169.254/openstack/latest/meta_data.json

Tenant

VM

Nova Metadata Service

Meta NS Proxy

Metadata Agent

Unix Domain Socket

Management Network

X-Router-Id: 2bc7c882-d612-438c-a334-0047f2b5c2d7 X-Forwarded-For: 10.0.0.1 X-Instance-ID: aaaaaaaa-aaaa-aaaa-aaaaaaaaaaaa