Application deployment on Openstack using Containers and

DockerBOSTON OPENSTACK MEETUP:

August 14, 2013

Contents

• Introduction to Docker (10 min)

• Docker and OpenStack (5 min)

• Demo of Docker (10 minutes)

• Demo of Docker + OpenStack (10 minutes)

• Q&A

In the 4.5 months since we launched

• >50,000 pulls

• >4,500 github stars

• >100 significant contributors

• >150 projects built on top of docker• UIs, mini-PaaS, Remote Desktop….

• 1000’s of Dockerized applications• Memcached, Redis, Node.js…and Hadoop

• Integration in Jenkins, Travis, Chef, Puppet, Vagrant and OpenStack

• Meetups arranged around the world…with organizations like Ebay, Cloudflare, and Rackspace presenting on their use of Docker

Why all the excitement?

Static website

Web frontend

User DB

Queue Analytics DB

Background workers

API endpoint

nginx 1.5 + modsecurity + openssl + bootstrap 2

postgresql + pgv8 + v8

hadoop + hive + thrift + OpenJDK

Ruby + Rails + sass + Unicorn

Redis + redis-sentinel

Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs +

phantomjs

Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client

Development VM

QA server

Public Cloud

Disaster recovery

Contributor’s laptop

Production Servers

The ChallengeM

ult

iplic

ity

of

Stac

ksM

ult

iplic

ity

of

har

dw

are

e

nvi

ron

men

ts

Production Cluster

Customer Data Center

Do

service

s and

app

s in

teract

app

rop

riately?

Can

I migrate

sm

oo

thly an

d

qu

ickly?

Results in matrix from hell

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development

VMQA Server

Single Prod

Server

Onsite

ClusterPublic Cloud

Contributor’s

laptop

Customer

Servers

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Mu

ltip

licit

y o

f G

oo

ds

Mu

ltip

ilici

tyo

f m

eth

od

s fo

r tr

ansp

ort

ing/

sto

rin

g

Do

I wo

rry abo

ut

ho

w go

od

s interact

(e.g. co

ffee

be

ans

next to

spice

s)

Can

I transp

ort q

uickly

and

smo

oth

ly(e

.g. from

bo

at to train

to

truck)

Cargo Transport Pre-1960

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

Also a matrix from hell

Mu

ltip

licit

y o

f G

oo

ds

Mu

ltip

licit

y o

f m

eth

od

s fo

r tr

ansp

ort

ing

/sto

rin

g

Do

I wo

rry abo

ut

ho

w go

od

s interact

(e.g. coffee

be

ans

next to

spices)

Can

I transp

ort

qu

ickly and

smo

oth

ly(e.g. fro

m b

oat to

train

to tru

ck)

Solution: Intermodal Shipping Container

…in between, can be loaded and

unloaded, stacked, transported

efficiently over long distances,

and transferred from one mode

of transport to another

A standard container that is

loaded with virtually any

goods, and stays sealed until

it reaches final delivery.

Static website Web frontend User DB Queue Analytics DB

Development

VMQA server Public Cloud Contributor’s

laptop

Docker is a shipping container system for code

Mu

ltip

licit

y o

f St

acks

Mu

ltip

licit

y o

f h

ard

war

e en

viro

nm

en

ts

Production

ClusterCustomer Data

Center

Do

services an

d ap

ps

interact

app

rop

riately?

Can

I migrate

smo

oth

ly and

qu

ickly

…that can be manipulated using

standard operations and run

consistently on virtually any

hardware platform

An engine that enables any

payload to be encapsulated

as a lightweight, portable,

self-sufficient container…

Static website

Web frontend

Background workers

User DB

Analytics DB

Queue

Development

VMQA Server

Single Prod

Server

Onsite

ClusterPublic Cloud

Contributor’s

laptop

Customer

Servers

Docker eliminates the matrix from Hell

Why Developers Care

• Build once…(finally) run anywhere*• A clean, safe, hygienic and portable runtime environment for your app.

• No worries about missing dependencies, packages and other pain points during subsequent deployments.

• Run each app in its own isolated container, so you can run various versions of libraries and other dependencies for each app without worrying

• Automate testing, integration, packaging…anything you can script

• Reduce/eliminate concerns about compatibility on different platforms, either your own or your customers.

• Cheap, zero-penalty containers to deploy services? A VM without the overhead of a VM? Instant replay and reset of image snapshots? That’s the power of Docker

• * Today—we require a modern Linux kernel and AUFS. These requirements will be relaxed significantly with the 0.8 release of Docker. See http://blog.docker.io/2013/08/getting-to-docker-1-0/

Why Devops Cares?

• Configure once…run anything• Make the entire lifecycle more efficient, consistent, and repeatable

• Increase the quality of code produced by developers.

• Eliminate inconsistencies between development, test, production, and customer environments

• Support segregation of duties

• Significantly improves the speed and reliability of continuous deployment and continuous integration systems

• Because the containers are so lightweight, address significant performance, costs, deployment, and portability issues normally associated with VMs

AppA

Containers vs. VMs

Hypervisor (Type 2)

Host OS

Server

GuestOS

Bins/Libs

AppA’

GuestOS

Bins/Libs

AppB

GuestOS

Bins/Libs

Ap

p A

’

Do

cker

Host OS

Server

Bins/Libs

Ap

p A

Bins/Libs

Ap

p B

Ap

p B

’

Ap

p B

’

Ap

p B

’VM

Container

Containers are isolated,but share OS and, whereappropriate, bins/libraries

GuestOS

GuestOS

…result is significantly faster deployment, much less overhead, easier migration, faster restart

Why are Docker containers lightweight?

Bins/Libs

AppA

Original App(No OS to takeup space, resources,or require restart)

Ap

p Δ

Bin

s/

AppA

Bins/Libs

AppA’

GuestOS

Bins/Libs

Modified App

Union file system allowsus to only save the diffsBetween container A and containerA’

VMsEvery app, every copy of anapp, and every slight modificationof the app requires a new virtual server

AppA

GuestOS

Bins/Libs

Copy ofApp

No OS. CanShare bins/libs

AppA

GuestOS

GuestOS

VMs Containers

What are the basics of the Docker system?

Source Code

Repository

DockerfileFor A

Docker Engine

DockerContainer

Image Registry

Build

Do

cker

Host 2 OS (Linux)

Co

ntain

er A

Co

ntain

er B

Co

ntain

er C

Co

ntain

er A

Push

SearchPull

Run

Host 1 OS (Linux)

Changes and Updates

Docker Engine

DockerContainer

Image Registry

Docker Engine

Push

Update

Bins/Libs

AppA

Ap

p Δ

Bin

s/

Base Container

Image

Host is now running A’’

Container Mod A’’

Ap

p Δ

Bin

s/

Bins/Libs

AppA

Bin

s/

Bins/Libs

AppA’’

Host running A wants to upgrade to A’’. Requests update. Gets only diffs

Container Mod A’

Docker Futures

• Stabilizing: Docker 0.8 in Early October

• Shrink and stabilize Core• Provide stable, pluggable API

• Expanding runtime environments:

• OpenStack: announced• Docker as various flavors of VM

(AMI, vdf, etc.)-This week• Increase compatible kernel set-

October• Increase compatible Union FS

set-October• IaaS compatibility-Ongoing

• Dev/devops tools integrations: Ongoing

• Chef, Puppet, Vagrant, GitHub, Travis, Buildbot,

• Various flavors of PaaS & IDE

OpenStack / DockerNew hypervisor to enable Nova to deploy Linux containers

Why Docker + OpenStack

• Alternative to VMs within OpenStack

• Easier deployment of OpenStack itself

• Cross cloud application deployment

• At OpenStack Summit we hope to show:• Building and testing an application from source

• Running on a laptop

• Running it, without modification or noticeable downtime, on a public cloud

• Running it, without modification or noticeable downtime, on an openstack cluster

• Doing all of the above using Nova and Horizon

Why a new hypervisor?

• Nova a computing controller for OpenStack

• Nova support for containers is minimal (via LibVirt)

• Enables control of Docker through OpenStack projects (ex: deploy containers via Horizon Web UI)

Current state of the project

• Nova blueprint accepted

• Code review in progress

• Testing in Devstack

• Code available:• https://github.com/dotcloud/openstack-docker

Deploying container with Horizon

Access Docker Registry through Glance

Under the hood

• Implements Nova’s driver API (start, stop, deploy, destroy, etc…)

• Using Docker HTTP Rest API to control containers

Want to learn more:

• www.docker.io:• Documentation• Getting started: interactive tutorial, installation instructions, getting

started guide, • About: Introductory whitepaper:http://www.docker.io/the-whole-

story/

• Github: dotcloud/docker

• IRC: freenode/docker/

• Google groups: groups.google.com/forum/#!forum/docker-user

• Twitter: follow @docker

• OpenStack Code: https://github.com/dotcloud/openstack-docker

• Meetups: Scheduled for Boston, San Francisco, Austin, London, Paris, Boulder…and Nairobi. Go to website for details

www.docker.io