Post on 15-Jul-2015
transcript
Bom dia!I'm Jérôme.
First time in Brazil!
Unfortunately, I don't speak Portuguese ☹
Therefore, I will continue in English. Thank you!
5 / 110
Who am I?French software engineer living in California
I have built and scaled the dotCloud PaaS
I know a few things about running containers (in production)
I ♥ Open Source
I think Docker will help us to build better apps!
6 / 110
OutlineEvolution vs Revolution
Containers: evolution or revolution?
How Docker can help us today
How Docker will help us tomorrow
7 / 110
First transatlantic flightVickers Vimy modified bomber from World War I
Altitude: 12,000 ft
Speed: 115 mph
Cross the Atlantic in hours instead of days
10 / 110
RevolutionPaved the way for air travel
Air travel was initially very expensive
Ultimately became the main method for long-distance passenger travel
12 / 110
First transatlantic supersonic jetConcorde
Altitude: 60,000 ft
Speed: 1,350 mph
Cross the Atlantic even faster
14 / 110
EvolutionAwesome technological achievement (Paris/New York in 3.5 hours!)
We haven't done better since 50 years ago
But:
did not replace subsonic jets
did not turn the impossible into possible
16 / 110
Apollo missionsSaturn V
Altitude: the Moon (~1,300,000,000 ft)
Speed: 24,500 mph
Sends people to the Moon, and back
18 / 110
RevolutionExcept for those people who don't believe that we reallywent to the moon, of course...
20 / 110
Space programAltitude and speed: irrelevant
Benefits:
put satellites into orbit
land rovers and probes on Mars, Titan...
examine planets, comets, and more
launch probes outside of the solar system
Picture: Delta Rocket launching a GPS satellite
23 / 110
RevolutionNumerous scientific breakthroughs
And consumer applications:
GPS
satellite TV
internet access in planes (!)
etc.
25 / 110
Virtual machines with an APIOrder time: seconds
Delivery time: seconds
Install time: minutes
This is typically called "IaaS" or "Cloud" ☺
32 / 110
ProblemTraffic patterns have daily peaks
Low traffic during the night (and sometimes, low traffic during week-ends)
Must provision servers according to the peaks
Note:if you are dealing with worldwide traffic, its distribution changes during the day.
35 / 110
Solution: autoscalingAdd servers automatically, when load increases
Remove them automatically, when load decreases
Requires on-demand servers:
available in minutes (not hours or days)
elastic pricing model ("pay for what you use")
automation (API)
This was not possible before.
38 / 110
ProblemOld model: developers write code, ops put it in production and run it
Applications are increasingly complex
It's increasingly hard for ops to fix problems
It's increasingly hard for devs to anticipate them
40 / 110
Solution: DevOpsDesign capacity and scaling upstream
Devs and Ops in the same team
Everybody is on call
No more manual operations (racking machines, deploying OS...)
Everything can be scripted, automated
This was not possible before.
43 / 110
ProblemDeploying new code is risky
Consequence: we are afraid of deploying
Consequence: we deploy less often
Consequence: when we deploy, we deploy bigger changes
Consequence: deploying new code is even riskier!
45 / 110
Solution: blue/green deployment"Blue" servers are in production
Deploy new release to "green" servers
Progressively move traffic from blue to green
If anything goes wrong: go back to blue
Requires:
2x the number of servers (for a short period of time)
... or servers on demand (Cloud!)
This was not possible/realistic before.
48 / 110
KitematicOS X GUI to start containers
Order time: seconds
Delivery time: seconds
Install time: seconds
52 / 110
Docker CLI and APIOrder time: milliseconds
Delivery time: milliseconds
Install time: milliseconds
55 / 110
Containers seen by hosting industryContainers = lightweight virtual machines
Higher density (3x-10x better than VMs)
Faster boot times
Possibility of bare metal performance
$$$
57 / 110
Testing, continuous integrationTesting environments must be:
short-lived (tests can run in seconds)
disposable (destroyed after the tests)
quick to boot (so that tests can start rapidly)
Containers are the perfect fit!
63 / 110
When a test depends on a databasePlan A: use a common "test database"
undefined state
needs to be reloaded before each test
slow (especially on big test sets)
still vulnerable to concurrent access
64 / 110
When a test depends on a databasePlan B: spin multiple copies of the DB in VMs
expensive (spin up 100 VMs if you have 100 tests)
slow (spin up 1 VM, test, repeat)
complex to implement (even with the help of IAAS or Vagrant)
65 / 110
When a test depends on a databasePlan C: spin multiple databases in containers
cheap, low resource usage (even with big databases)
fast boot times (even with big databases)
easy to set up (thanks to CLI and API)
66 / 110
Testing is critical to release quality,
but it does not affect uptime.
(At least, not directly.)
69 / 110
Software deployment before DockerInstall packages A, B, C
Install libraries X, Y, Z
It doesn't work: wrong versions
Try again
Wasted time, frustration
74 / 110
Software deployment with Dockerdocker run joaodubas/orientdb
It works!
Avoid "dependency hell"
75 / 110
Software dependencies before DockerThis software depends on:
library A version X
library B version Y
library C version Z
other hidden, unspecified, unknown dependencies
76 / 110
Software dependencies with DockerThis container image depends on:
Intel 64 bits CPU instructions
Linux kernel ABI
Those things are very, very stable.
77 / 110
Software packaging before DockerPackaging is distribution-specific (different for Red Hat, Debian, OS X...)
Packaging can also be language-specific (Node.js npm, Ruby gems, Python eggs...)
Packaging is a useful, but uncommon skill (who here knows how to make a package?)
Packaging is often done by separate people (ops, maintainers...)
78 / 110
Software packaging with DockerDevelopers can build container images easily
Images are described by a Dockerfile
If you know how to write a shell script, then you know how to write a Dockerfile!
79 / 110
Sample DockerfileFROM debian:jessieMAINTAINER Jessica Frazelle <jess@docker.com>
RUN apt-get update && apt-get install -y \ build-essential \ ca-certificates \ git \ qmlscene \ qt5-qmake \ qt5-default \ qtdeclarative5-dev \ qml-module-qtquick-controls \ qml-module-qtgraphicaleffects \ qml-module-qtquick-dialogs \ qml-module-qtquick-localstorage \ qml-module-qtquick-window2 \ --no-install-recommends
RUN git clone --recursive https://github.com/Swordfish90/cool-retro-term.git /srcWORKDIR /srcRUN qmake \ && make
ENTRYPOINT [ "/src/cool-retro-term" ]
80 / 110
Tech support before Docker"Works For Me"
"Can't reproduce"
"I would appreciate if you could test between 3 and 4am"
"So to trigger the bug you have to install X and Y thenconfigure A, B, and C, then download the extra file, put it inthis directory (which doesn't exist?!?) and then if yourestart three times in approximatively 5 minutes butsometimes it takes longer you will see that the images areshifted by a few pixels but if it doesn't work try to upgradeY to version Z and try all over again..."
84 / 110
Tech support with DockerGet a well-defined, reproducible environment
Define this environment in a Dockerfile
Build this Dockerfile into a container image
Run this container image anywhere
Get exactly the same behavior
85 / 110
Blue/green deployment before DockerMust create server images (slow)
Server images are often specific to IAAS (cannot run them locally for testing)
Servers must be stateless (data files are wiped out at each deploy)
87 / 110
Blue/green deployment with DockerContainer image creation is fast
Container images can be deployed anywhere (local dev env, physical servers, VMs...)
Blue/green can be on the same machine, so data can be preserved across deploys
88 / 110
Onboarding with DockerHire developer*
Give them a computer
Install Docker
git clone ...
docker-compose up ...
Your stack is up and running
*Actually the most difficult part.
91 / 110
Docker Composeweb: build: src/front links: - redis - postgres:db - api - zookeeper:zk
redis: image: redis
postgres: image: postgres
api: build: src/backend links: - redis - postgres:db - cassandra - zookeeper:zk
zookeeper: image: jplock/zookeeper
cassandra: image: spotify/cassandra
92 / 110
Cold, hard dataHow long does it take for a developer to join a new project?
Before Docker: 2 days
After Docker: 2 hours
(Source: Worldline)
93 / 110
Microservices before DockerDifferent languages, frameworks, build systems
Service discovery challenge
Differences between dev and prod environments:
dev = many processes on one VM
prod = many processes on many VMs
different network layout
network layout is visible to the processes
95 / 110
Microservices with DockerEverybody ships containers
docker-compose does service discovery in dev, and offers reliable setup of the whole stack
Differences between dev and prod environments:
dev = many containers on one VM
prod = many containers on many VMs
different network layout
network layout is invisible to the processes
96 / 110
Docker MachineCreate Docker hosts with a single tool
docker-machine create -d virtualbox my-dev
docker-machine create -d digitalocean prod-001
docker-machine create -d amazonec2 prod-002
Work in progress!
100 / 110
Docker SwarmControl a cluster of Docker hosts, as if it were a single host
Talk to Swarm using the Docker API
Use the Docker CLI or any existing Docker client
Swarm talks to Docker hosts with the same API
No change to existing clients and hosts!
102 / 110
Scaling Swarm with MesosOn large-scale clusters (100s, 1000s of nodes), scheduling is a hard problem
Mesos solves the scheduling problem elegantly
Swarm and Mesos will integrate:
you talk to Swarm with the Docker API,
Swarm offloads the hard job to Mesos
103 / 110
Network extensionsBuild overlay networks to connect containers across multiple hosts and multiple clouds
Integrate Docker containers with existing networks (Open vSwitch, VLANs, VXLAN, etc.)
Some extensions:
flannel
pipework
socketplane
weave
104 / 110
Storage extensionsStateful containers (databases) are challenging
Moving a stateless container is easy (deploy image; start container; done)
Moving a stateful container is hard
See: Flocker by ClusterHQ
105 / 110
Docker on the DesktopWhat for?
better dependency management for proprietarybinaries (e.g. Chrome, Skype, Spotify...)
better control over resources (both quantitative and qualitative)
Check Jessica Frazelle's blog: https://blog.jessfraz.com/posts/docker-containers-on-the-desktop.html
106 / 110
CRIU (Checkpoint/Restore In Userspace)Save process state (not just filesystem state!)
Obvious use-case: live migration of containers
Less obvious, but easier use-cases:
slow start processes
JIT compilers
107 / 110
RecapBuild with Dockerfiles.
Test rapidly.
Assemble complex stacks with docker-compose.
Implement powerful software patterns easily: immutable infrastructure, microservice architectures.
Manage machines with docker-machine.
Orchestrate clusters with Swarm.
Leverage the numerous tools in the ecosystem.
Last but not least: all of this is Open Source.
109 / 110
Obrigado! Perguntas?
@jpetazzo @docker
110 / 110