S A I N T L O U I S J A VA U S E R G R O U P M A Y 2 0 1 4

steve@borrelli.org

S T E V E N B O R R E L L I

@stevendborrelli

A B O U T M E

F O U N D E R , A S T E R I S ( J A N 2 0 1 4 ) @

O R G A N I Z E R O F S T L M A C H I N E L E A R N I N G A N D D O C K E R S T L

S Y S T E M S E N G I N E E R I N G M A N A G E M E N T

F I R S T C O M P U T E R :

aster.is

W H Y D O C K E R ?

Docker makes it easy to:

Package

Deploy

Share

Server Applications

Think:

java -jar

vs.

./configure; make install !

D O C K E R FA C T S

Written by Docker, Inc. (Formerly Dotcloud)

Automates the management and control of Linux containers

Rewrite of their proprietary PAAS container engine (written in Python)

Written in Go / Apache 2 License

11,700+ Github stars

D O C K E R T I M E L I N E

M A R C H 2 0 1 3 : I N I T I A L G I T H U B R E L E A S E

M AY 7 , 2 0 1 4 : 0 . 1 1 R E L E A S E

M O N T H LY R E L E A S E C A D E N C E

J A N U A R Y 2 0 1 3 : P R O J E C T S TA R T

M AY 8 , 2 0 1 4 : 0 . 1 1 . 1 R E L E A S E

(LINUX) KERNEL

LAYER FSCGROUPS NAMESPACES

LIBCONTAINER

DOCKER

LXC

D O C K E R A R C H I T E C T U R E

N A M E S PA C E S V S . C G R O U P S Namespaces provide isolation:

• pid (processes)

• net (network interfaces, routing...)

• ipc (System V IPC)

• mnt (mount points, filesystems)

• uts (hostname)

• user (UIDs)

Control groups control resources:

• cpu (CPU shares)

• cpusets (limit processes to a CPU)

• memory (swap, dirty pages,

• blockio (throttle reads/writes)

• devices

• net_cls, net_prio: control packet class and priority

What’s the difference between containers and virtual machines (VMs)?

!

V I RT U A L I Z AT I O N

1966-1972 IBM CP/CMS

1989 Insignia SoftPC

1997 Connectix VirtualPC

1999 VMWare Workstation

2001 IBM AIX LPAR

2002 Xen

2006 Amazon EC2

2007 Sun Logical Domains

2007 Linux KVM

2007 InnoTek VirtualBox

2008 Microsoft Hyper-V

H A R D WA R E V I RT U A L I Z AT I O N

C O N TA I N E R S

P R O C E S S V I RT U A L I Z AT I O N

1979-1982 UNIX Chroot

1998 FreeBSD Jail

2001 Parallels Virtuozzo

2001 Linux-VServer

2005 Solaris Zones

2005 OpenVZ

2008 Linux LXC

2007+PAAS:

Heroku, Joyent, CloudFoundry

2013 Docker

Differences between containers and virtual machines

!• Weaker isolation in containers

• Containers run near-native speed CPU/IO

• Containers launch in around 0.1 second (libcontainer)

• Less memory overhead

N O TA B L E C H A N G E S

0 . 9 : L I B C O N TA I N E R

N O TA B L E C H A N G E S

0 . 1 0 :

• TLS support on docker API

• Systemd integration via API instead of /proc

• Lots of cleanups

N O TA B L E C H A N G E S

0 . 1 1 :

• Release Candidate for 1.0

• Multiple registries

• Direct host network access

• SELinux support

EXAMPLES

R U N N I N G A C O N TA I N E R

Start a container:

Mount host filesystems:

Example: run Zookeeper + Exhibitor

Host ports will be dynamically allocated by docker

Port 2181 on host will be mapped to 2181 on container

M A P P I N G P O RT S

New in 0.11, allows a container to access host adapters:

D I R E C T H O S T N E T W O R K

Port 8080 on the container is 8080 on the host:

I M M U TA B L E S E R V E R S

Physical server lifetime is measured in years.

A container's lifetime can be as short as a few seconds.

Treat containers like a build artifact. 

If you need to make changes, build a new container.

J AVA D O C K E R F I L E

T O M C AT D O C K E R F I L E

L AY E R E D F S I S A G R A P H

R U N N I N G J AVA I N D O C K E R

• Pass in environment variables to Java vars (i.e. in start.sh): java -Dkeystore.password=${KEY_PASS}

• When you run the container, set the vars:

docker run -e SSL_PASS=password tomcat

Problem: keep configuration out of containers

R U N N I N G J AVA I N D O C K E R

• Link from a volume containerdocker run -v /opt/properties -v /opt/ssl \ -name TOMCAT-CFG busybox true docker run -t -i -rm -volumes-from TOMCAT-CFG -name appsrv1 tomcat

• Mount host filesystem:

docker run -v/opt/ssl:/opt/ssl tomcat

Problem: keep configuration out of containers

R U N N I N G J AVA I N D O C K E R

Running stateless containers

• Use a redis/DB to store session data

• Use a shared FS (hdfs/nfs, etc.) or Object storage (Swift, S3) for data

• Send logs to a centralized location

• Docker future: storage plugins

W H AT ’ S N E X T ?

Stripped-down operating systems

W H AT ’ S N E X T ?

New operational models:

Continuous Delivery

Automated routing

Distributed consensus (Paxos, Raft)

Service Discovery (Zookeeper, etcd, serf, skydns, consul)

Distributed scheduling (Fleet, Mesos, YARN)

S U M M A RY

Easy to build, run & share containers

Rapidly expanding ecosystem

Better performance vs. VMs

Layered filesystem gives us git-like control of images.

Reduces complexity of system builds

Q & A