An Open Platform to Build, Ship, and Run Distributed Applications
Who am I?
• Nick Stinemates | @nickstinemates | bitbucket.org/keeb • Work: Head of Business Development / Tech Alliances @
Docker
A long history of work with Atlassian Products
• JIRA critical to Issue and Agile management • Over 400 Bamboo Jobs running / day • Workflow built with SCM integratgion
• … and a special thanks for introducing me to Docker and for the amazing early support of the Docker project
Agenda
• The Matrix from Hell and the Need for Containers • Docker & Containers: how they work • Docker Overview and Docker Ecosystem • Current Usage and Adoption • Creating an Open Environment • Docker Roadmap & Business Model
~2000 2014
Long lived Development is itera2ve and constant
Monolithic and built on a single stack Built from loosely coupled components
Deployed to a single server Deployed to a mul2tude of servers
Apps Have Fundamentally Changed
Static website
Web frontend
User DB
Queue Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8
hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Development VM
QA server
Public Cloud
Disaster recovery
Contributor’s laptop
Production Servers
The Problem in 2014 Mul*p
licity
of
Stacks
Mul*p
licity
of
hardware
environm
ents Production Cluster
Customer Data Center
Do services and apps interact appropriately?
Can I migrate
smoothly and quickly?
The Matrix From Hell
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM QA Server Single Prod
Server Onsite Cluster
Public Cloud
Contributor’s laptop
Customer Servers
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Mul*p
licity
of
Goo
ds
Mul*p
ilicity of
metho
ds fo
r tran
spor*n
g/storing
Do I worry about
how goods interact
(e.g. coffee beans next to spices)
Can I transport quickly and sm
oothly (e.g. from
boat to train to truck)
An Inspiration…and some really ancient history: Cargo Transport Pre-1960
Mul2p
licity
of
Good
s
Mul2p
licity
of
metho
ds fo
r transpor2n
g/storing
Do I worry about
how goods interact
(e.g. coffee beans next to spices)
Can I transport quickly and sm
oothly (e.g. from
boat to train to truck)
Solution: Intermodal Shipping Container
…in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another
A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.
This spawned an Intermodal Shipping Container Ecosystem
• 90% of all cargo now shipped in a standard container • Order of magnitude reduc2on in cost and 2me to load and unload
ships • Massive reduc2on in losses due to theK or damage • Huge reduc2on in freight cost as percent of final goods (from >25% to
<3%) à massive globaliza2on • 5000 ships deliver 200M containers per year
Static website Web frontend User DB Queue Analytics DB
Development VM QA server Public Cloud Contributor’s laptop
Let’s create a shipping container system for applications Mul2p
licity
of
Stacks
Mul2p
licity
of
hardware
environm
ents
Production Cluster Customer Data Center
Do services and apps interact appropriately?
Can I migrate
smoothly and quickly
…that can be manipulated using standard operations and run consistently on virtually any hardware platform
An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development VM QA Server Single Prod
Server Onsite Cluster
Public Cloud
Contributor’s laptop
Customer Servers
And eliminate the matrix from Hell
App A
Step One: Create a lightweight container (vs. VMs)
Hypervisor (Type 2)
Host OS
Server
Guest OS
Bins/ Libs
App A’
Guest OS
Bins/ Libs
App B
Guest OS
Bins/ Libs
App A’
Host OS
Server
Bins/Libs
App A
Bins/Libs
App B
App B’
App B’
App B’ VM
Container
Containers are isolated, but share OS kernel and, where appropriate, bins/libraries
Guest OS
Guest OS
…result is significantly faster deployment, much less overhead, easier migra2on, faster restart
Step 2: Make the containers easy to use, standardized, interoperable, automatable
• With Docker, low level containers get the
following: -‐ Ease of use, tooling -‐ Re-‐usable components -‐ Ability to run on any Linux server today: physical, virtual,
VM, cloud, OpenStack, +++ -‐ (Stay tuned for other O/S’s)
-‐ Ability to move between any of the above in a maber of seconds-‐ no modifica2on or delay
-‐ Ability to share containerized components -‐ Interoperability with all exis2ng devops tools -‐ Self contained environment—no dependency hell -‐ Tools for how containers work together: linking, nes2ng,
discovery, orchestra2on, ++ -‐ “Containeriza2on” is really “Dockeriza2on”
Step 3: Make containers super lightweight
Bins/ Libs
App A
Original App (No OS to take
up space, resources, or require restart)
App Δ
Bins/
App A
Bins/ Libs
App A’
Guest OS
Bins/ Libs
Modified App Copy on write allows us to only save the diffs Between container A and container A’
VMs Every app, every copy of an
app, and every slight modifica2on of the app requires a new virtual server
App A
Guest OS
Bins/ Libs
Copy of App
No OS. Can Share bins/libs
App A
Guest OS
Guest OS
VMs Containers
A set of slides from IBM: Comparing Docker to VMs
9/22/14 16
Step 4: Build an open platform
An Open Pla*orm to Build, Ship, and Run Distributed Applica;ons
An Open Platform…
API
Engine Hub open source soKware at the heart
of the Docker plakorm cloud-‐based plakorm services for distributed
applica2ons
API
An Open Platform…
Any App
Any infrastructure • Physical • Virtual cloud
+ 25K apps + 10K projects
API
Engine Hub cloud-‐based plakorm services for distributed
applica2ons
API
open source soKware at the heart of the Docker plakorm
…to Build, Ship, and Run
Build Ship
Run
Dev
QA
Source
Staging
Physical
Virtual
Cloud
Infrastructure Management
Infrastructure Management
DockerFile
Source Code Repository
TEST TEST
TEST TEST
TEST
GCE RAX IBM
Mac/Win Dev Machine
Boot2Docker
Docker
Analytics DB
Prod Machine
Linux OS
Docker
Docker
++
Users Collab
Provenance Policy
Docker Hub
Registries
Public Curated Private
Docker Hub API
Third Party Tools
Prod Machine
Linux OS
Docker
Docker
Prod Machine
Linux OS
Docker
Docker
VM
Docker
Docker
VM
Docker
Docker
VM
Docker
Docker
QA Machine
Linux OS
Docker
Docker
…Distributed Applications
Data
User DB
Queue
Analytics DB
API Endpoint
Web Frontend
Data
Background Workers
Development VM
QA Server
Public Cloud
Disaster Recovery
Contributor’s Laptop Production Servers
Production Cluster
Customer Data Center
Step 5: Build an Ecosystem
Official Repos & 14K+ Dockerized Apps
Community 540+ Contributors 250+ Meetups on Docker 7M Downloads 10K Projects on GitHub
Support
Enterprise Support Robust Documentation Implementation, Integration, Training Network of Partners
The Docker Platform Docker Engine
Docker Hub
Build, Ship, and Run
Partners
Content
Users
Official Repos & 25 K Dockerized Apps
Partner Ecosystem
Who is using Docker?
And hundreds of others….these are just a sample of those who have published use cases
Since DockerCon (as of 7/25)
• Major inflection points in almost all metrics • Projects: From 6.5K to >10K • Contributors: From 475à545 • Downloads: From 2.75M to ~8M • Boot2Docker Downloads: 400Kà600K • Repositories: 14Kà25K
• Significant activity in enterprise pipeline (200+ F500 opportunities, including major financials, energy, bioinformatics, government)
• Joint announcements, developments with virtually every major IT company
Conclusion
• Multiple forces are driving a reconsideration of how applications should be created, built, deployed, scaled, and managed
• We believe that the right approach is to decouple application management from infrastructure management
• Container based approach (vs. VM approach) provides right level of abstraction
• Enables infrastructure to be managed consistently and stably • Enables applications to be built flexibly and deployed flexibly • Provides greater degree of visibility, control, and management of
what runs where and what components are allowed • Massive cost, speed, efficiency savings • Docker is becoming the standard for containerization
Learn More
• All Videos and Talks from DockerCon: http://bit.ly/1AnvHFz • Demo of DockerHub: see Ben Golub keynote from DockerCon, http://bit.ly/1xHqUL8
demo starts at 29:00 • Sign up for a DockerHub account: https://hub.docker.com/ • Security & Isolation: see above, also blog.docker.io • Use Cases: https://docker.com/resources/usecases/ • Docker project: www.docker.com/ • Follow Docker on Twitter: twitter.com/docker • Take the Docker interactive tutorial: https://docker.com/tryit/ • Join Docker on IRC: botbot.me/freenode/docker/ • Go to the Docker repository on GitHub: github.com/docker/docker/ • Go to a meetup: https://www.docker.com/community/meetups/