OPNFV Upstreaming -Kubernetes ODL IntegrationPREM SANKAR GOPANNAN, ERICSSON

HTTP://TWITTER.COM/PREMSANKAR

Demo objective

Microservices – Quick intro

CNCF Landscape

Introduce to Kuryr

ODL architecture

Highlight issues of VM and Container Internetworking

Hand-on

Pod Creation

Pinging PODS

Issue OVS commands

Business Requirements

- 5G- IOT- Digital Transformation

Microservices – Quick Intro

• Reference – Martin Fowler, http://martinfowler.com

Principles that drive Microservices architecture

• Services must be loosely coupled so that they can be developed, deployed and scaled independently

• Organized around business capabilities

• API Focused

• Smart endpoints and dumb pipes

• Decentralized Governance

• Decentralized Data Management

• Infrastructure Automation (infrastructure as code)

• Design for failure

• Evolutionary Design

Microservices Design Patterns

• Reference – Chris Richardson http://microservices.io/patterns/microservices.html

Microservices Design Patterns

Decomposition Pattern

Subdomain context

Service discovery Pattern

Database per service Pattern

Keep each microservice’s persistent data private to that service and accessible only via its API

RDBMS design approaches

Private-tables-per-service – each service owns a set of tables that must only be accessed by that service

Schema-per-service – each service has a database schema that’s private to that service

Database-server-per-service – each service has it’s own database server.

Circuit Breaker

API Gateway Pattern

CNCF Landscape

Lifecycle

Requirements

Service Orchestration

Infrastructure Orchestration

Business Process mapping

Audit and Validation

Security

WorkflowManager and Orchestrator

Continuous Integration

ContinuousDeployment

Monitoring

Design and Creation

Deployment and Execution

Operation, Analytics and Maintenance

Opensource Ecosystem

Workflow Manager

and Orchestrator

ONAP

ARIA

OSM

Design and

Deployment

CI/CD

Pipelines

•Jenkins

•Fabric8

(Jenkins,

Kubernetes

and Docker)

•Other options

– goCD,

concourse.ci

Automation

•Ansible/Puppet

/Chef

Scheduler

(w/ docker/cri-o)

Kubernetes

Mesos

Swarm

Service Mesh

ISTIO

Linkerd

Monitoring

Prometheus

Other

alternatives(https://prometheus.io/docs/i

ntroduction/comparison/)

Tracing

Zipkin

Squash

Logging

ELK

Kubernetes

Master components

API Server – Frontend for K8S control plane

Scheduler

Control-manager

Node Controller

Replication Controller

Endpoints Controller

Service account and token controllers

Etcd – All clustered data is stored

Worker Node

Kubelet - primary node agent and watches pod that has been assigned to node

Kube-proxy – enables service abstraction by maintaining network rules on the host

Kubernetes components

Abstraction levels

POD – Encompasses the containers the pods that are related/microservice

Replication Controller – Defines pod count that corresponds to Service

Service – defines logical set of Pods

Ingress – connecting to external world

NFV Service – Deployment 1

PodAPodA

PodA

PodBPodB

NFV Service – Deployment 2

PodBPodAPodBPodA

Kubernetes Networking - Overview

CNI uses CNI Network plugin to setup container networking

(If container runtime is docker, CNM will not be used)

Plugin responsible for creating network interface to container

Plugin calls IPAM to setup IP address

Plugin needs to implement API for network creation and deletion

Kubernetes

CNI

Network Plugin IPAM

VNF Deployment scenario

Neutron Plugin or Gluon/Proton

Linux Linux Linux Linux

Kubernetes

Datacenter Hardware

Linux

OpenStack

OVS

App

Orchestration Systems

OpenStack APIs

OpenDaylight

Kubernetes APIs

CNI Plugin

KVM KVM ContainerRT

Container RT

App App App

VM Application Container Application

AppApp App

ODL COE Architecture

Host OS

Open vSwitch

Host OS

Open vSwitch

VM

Container App

Container App

Container App

VLAN’s

Orchestrator

OpenDaylight

Kubernetes / Docker

Kuryr

Neutron / Gluon

Container App

Container App

Container App

Container Mgt

(docker, kube-pxy)

Iptables / NAT/FW

Native

Openstack Kuryr

Kuryr is an Openstack project aimed at providing network and storage support for hybrid environments

Bridge between container Networking and Openstack Neutron

Two implementation

Kuryr CNI for Kubernetes

Kuryr libnetwork for docker

Kuryr-Kubernetes Architecture

Kuryr components

Kuryr Controller

Watches Kube API resource with a service account

Secure connection with Neutron API server

Kuryr CNI

Communicates with Kube API

Perform local binding of Neutron port

Watches Pod resources for controller-driven vif

Kuryr Kubernetes modes

Baremetal/side by side

VM and Pods are in the same

Nested

Pods within VM

Uses trunk ports to provide neutron port to containers

Uses VLAN segmentation so POD communication goes to vswitch

Neutron – K8S construct mapping

Kubernetes NeutronNamespace Network

Cluster Subnet Subnet Pool

Service Cluster IP Subnet

External Subnet Floating IPExternal NetworkRouter

POD Port

Service Load Balancer

Openstack Kuryr 101

• Controller• Watches K8S API endpoints to

make sure that the corresponding model is maintained in Neutron

• Updates K8S resource endpoints annotations to keep neutron details required by CNI driver

• Watcher• Used by both Controller

and CNI Driver• Connects to K8S API• Observe Registered

Endpoints and invoke call back handlers

• CNI Driver

Kuryr-K8S integration

ODL COE Baremetal

Installing Kuryr AIO

Pull code

git clone https://github.com/openstack/kuryr-kubernetes

Install Vagrant – sudo apt-get install vagrant

Install Virtual Box – sudo apt-get install virtualbox

Edit local.conf.sample

cd $KURYR_HOME/devstack/

cp local.conf.odl.sample to local.conf.sample

By default only Openstack Neutron is enabled and if you want to install Openstack components

Comment out ENABLED_SERVICE=“” statement in local.conf.sample or

ENABLED_SERVICE=<list needed Openstack components>

cd $KURYR_HOME/conrib/vagrant

vagrant up

vagrant ssh