A little bit about me ….

Scott Coulton• 10 + years hosting solutions experience• Extensive experience in architecture across AWS, Puppet, Linux,

Docker and all things open source

https://github.com/scotty-chttps://registry.hub.docker.com/repos/scottyc/

https://forge.puppetlabs.com/scottyc

@scottcoulton

Agenda

Looking at how to use Puppet to deploy docker in a production fault tolerant environment with service discovery using docker compose, docker swarm

and consul

Lets look at the components

What is Docker ???• Docker containers wrap up a piece of software in a complete filesystem that contains everything it needs to run: code, runtime, system tools, system libraries – anything you can install on a server. This guarantees that it will always run the same, regardless of the environment it is running in.

What is Docker Swarm ???

•Docker Swarm is native clustering for Docker. It turns a pool of Docker hosts into a single, virtual host.

•Swarm serves the standard Docker API, so any tool which already communicates with a Docker daemon can use Swarm to transparently scale to multiple hosts: Dokku, Compose, Krane, Flynn, Deis, DockerUI, Shipyard, Drone, Jenkins... and, of course, the Docker client itself.

What is Docker Compose???•Compose is a tool for defining and running multi-container applications with Docker. With Compose, you define a multi-container application in a single file, then spin your application up in a single command which does everything that needs to be done to get it running.

What is Weave ???

•Weave Net connects all your containers into a transparent, dynamic and resilient mesh. This is one of the easiest ways to set up clustered applications that run anywhere..

What is Consul ???•Consul makes it simple for services to register themselves and to discover other services via a DNS or HTTP interface. Register external services such as SaaS providers as well.•Flexible key/value store for dynamic configuration, feature flagging, coordination, leader election and more. Long poll for near-instant notification of configuration changes.

What is Puppet ??

• Puppet is a configuration management solution that allows you to define the state of your IT infrastructure, and then automatically enforces the desired state. Puppet automates every step of the software delivery process, from provisioning of physical and virtual machines to orchestration and reporting; from early-stage code development through testing, production release and updates.

Lets look at a real world example of Docker in action …

So what do we have …

• Jenkins Master (Single ec2 in an autoscaling group of 1)

• Slaves servers are in an auto scaling group

Whats wrong with this design !!!

• Auto scaling is to slow• Fined tune metrics of auto scaling does not support the needs of the app• We can only be as quick as AWS EC2 provisioning

Enter Docker …

Now lets add Docker Swarm …

Lets make it repeatable …

Internal Docker Registry !!!

EC2

• Docker

• Docker swarm

• Docker compose

• Weave

• Docker and Docker compose https://github.com/scotty-c/garethr-docker

• Docker swarm https://forge.puppetlabs.com/scottyc/docker_swarm

• Weave

https://forge.puppetlabs.com/tayzlor/weave

Lets look at some code !!! • Docker

• Docker compose Part 1

• Docker compose Part 2

• Docker swarm

What about security ???

The truth ….. • We can not only apply normal infrastructure controls, we can now add application controls to our infrastructure

• We have hardening guides https://benchmarks.cisecurity.org/downloads/show-single/?file=docker16.100

We are going to build

4 nodes ( 3 in a swarm cluster and 1 manager)Running the following • Docker

• Docker Swarm • Weave• Consul

• Docker Compose

All built with Puppet !!!!

https://github.com/scotty-c/puppet-meetup

Code from tonight