Fabricio - Docker deploy automation

transcript

Docker deploy automation with Fabric

1MOSCOW, October 2016

RINAT KHABIBIEV

https://github.com/renskiy https://www.facebook.com/rinat.khabibiev

Since 2008

http://www.redmadrobot.com

https://www.docker.com/company

SOME DOCKER STATISTICS

DOCKER, CAVEATS

Open source

Network overhead

https://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf

DOCKER, BENEFITS

Simple requirements

CI/CD integration

Doesn’t require Internet connection to update App

The 12-factor App (12factor.net)

REDMADROBOT: TOTAL DOCKERIZATION

Python (Django)

PostgreSQL

RabbitMQ

Elasticsearch

AVAILABLE DEPLOY TOOLS

Vagrant

Fabric

Ansible

Capistrano

Docker compose

Kubernetes

DEPLOY TOOL REQUIREMENTS

Arbitrary environment support

Easy adaptation (copy/paste)

Easy customization

No special education or experience requirements

Docker support

DB migrations apply and rollback

Support of private registry and complex networks

FABRICIO

https://github.com/renskiy/fabricio

DOCKER DEPLOY AUTOMATION TOOL

FABRICIO DEPLOY CONFIG EXAMPLE

# fabfile.py

from fabricio import docker, tasks

class NginxContainer(docker.Container):

image = docker.Image('nginx:1.9')

ports = '80:80'

nginx = tasks.DockerTasks(

container=NginxContainer(name='web'),

hosts=['user@example.com'],

FABRICIO DEPLOY PROCESS

$ fab --list

Available commands:

nginx backup -> pull -> migrate -> update

nginx.deploy backup -> pull -> migrate -> update

nginx.pull pull Docker image from registry

nginx.rollback rollback Docker container to previous version

nginx.update start new Docker container if necessary

$ fab nginx

[user@example.com] Executing task ‘pull’

[user@example.com] run: docker pull nginx:stable

[user@example.com] out: 1.9: Pulling from library/nginx

$ docker ps

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES

ec44b023adf0 nginx:1.9 "nginx -g" 2 minutes ago Up 2 minutes 0.0.0.0:80->80/tcp web

FABRICIO FEATURES

Build Docker images

Create containers from images with provided tags

Unlimited infrastructures

Parallel execution (Fabric feature)

Rollback containers to previous version

Work with public and private Docker registries

DB migrations and rollback, backup and restore

FABRICIO COMMAND PARAMS

# show detailed information about command

$ fab --display nginx

Displaying detailed information for task 'nginx':

backup -> pull -> migrate -> update

Arguments: self, tag=None, force=False, migrate=True, backup=False

# deploy container using image with provided tag

$ fab nginx:1.10

# force new container

$ fab nginx:force=yes

# combo!

$ fab nginx:1.10,force=yes

# rollback container to previous version

$ fab nginx.rollback

FABRICIO DOCKER CONTAINER DEFINITION

class Container(object):

image = None # type: Image

cmd = None

stop_timeout = 10

user = None

ports = None

env = None

volumes = None

links = None

hosts = None

network = None

restart_policy = None

stop_signal = None

def __init__(self, name, **options):

self.name = name

FABRICIO AND PRIVATE DOCKER REGISTRY

# start local Docker registry

$ docker run -d -p 5000:5000 --name registry -v /data/registry:/var/lib/registry registry:2

nginx = tasks.PullDockerTasks(

container=NginxContainer('web'),

$ fab --list

Available commands:

nginx prepare -> push -> backup -> pull -> migrate -> update

nginx.deploy prepare -> push -> backup -> pull -> migrate -> update

nginx.prepare prepare Docker image

nginx.pull pull Docker image from registry

nginx.push push Docker image to registry

nginx.rollback rollback Docker container to previous version

nginx.update start new Docker container if necessary

BUILD DOCKER IMAGES WITH FABRICIO

class MyContainer(docker.Container):

image = docker.Image('my_image')

app = tasks.BuildDockerTasks(

container=MyContainer('my_service'),

build_path='.',

$ fab --list

Available commands:

app prepare -> push -> backup -> pull -> migrate -> update

app.deploy prepare -> push -> backup -> pull -> migrate -> update

app.prepare prepare Docker image

app.pull pull Docker image from registry

app.push push Docker image to registry

app.rollback rollback Docker container to previous version

app.update start new Docker container if necessary

FABRICIO ROLES AND INFRASTRUCTURES

from fabric import api as fab

@tasks.infrastructure

def production():

fab.env.update(roledefs={'front': ['user@example.com']})

nginx = tasks.DockerTasks(

container=NginxContainer(name='web'),

roles=['front'],

$ fab --list

Available commands:

production select production infrastructure to run task(s) on

production.confirm automatically confirm production infrastructure selection

$ fab production nginx

Are you sure you want to select production infrastructure to run task(s) on? [y/N]

FABRICIO: DEPLOYING DJANGO PROJECTS

from fabric import api as fab

from fabricio import docker, tasks

from fabricio.apps.python.django import DjangoContainer

class BaseDjangoContainer(DjangoContainer):

image = docker.Image('my_django')

@property

def env(self):

return 'DJANGO_SETTINGS_MODULE=settings.{}'.format(

fab.env.infrastructure,

django = tasks.BuildDockerTasks(

container=BaseDjangoContainer('api'),

migrate_commands=True,

FABRICIO: DEPLOYING DJANGO PROJECTS

$ fab --list

Available commands:

django prepare -> push -> backup -> pull -> migrate -> update

django.deploy prepare -> push -> backup -> pull -> migrate -> update

django.migrate apply migrations

django.migrate_back remove previously applied migrations if any

django.prepare prepare Docker image

django.pull pull Docker image from registry

django.push push Docker image to registry

django.rollback rollback Docker container to previous version

django.update start new Docker container if necessary

FABRICIO: DATA BACKUP AND RESTORE

from fabricio.apps.db.postgres import PostgresqlBackupMixin

class BackupDjangoContainer(BaseDjangoContainer,

PostgresqlBackupMixin):

volumes = '/data/backup/postgres:/backup'

db_backup_dir = '/backup'

django = tasks.BuildDockerTasks(

container=BackupDjangoContainer('api'),

backup_commands=True,

$ fab --list

Available commands:

django.backup backup data

django.restore restore data

FABRICIO REQUIREMENTS AND INSTALL

Python 2.6 or 2.7

Docker CLI (Linux/Mac/Windows)

Docker 1.9 or greater recommended (remote side)

# virtualenv install

$ pip install --upgrade fabricio

# macOS system-wide install

$ sudo pip install --upgrade fabricio six==1.4.1

FABRICIO ROADMAP

Master-Slave configurations for PostgreSQL

Docker Swarm support

docker-py integration

MySQL?

Non-Django frameworks?

USEFUL LINKS & QUESTIONS

Fabricio: https://github.com/renskiy/fabricio

Author of Fabricio: https://www.facebook.com/rinat.khabibiev

Хабра-блог Redmadrobot: https://habrahabr.ru/company/redmadrobot

Redmadrobot on Facebook: https://www.facebook.com/redmadrobot

The 12-factor App (SaaS dev patterns): https://12factor.net

Docker image with cron: https://hub.docker.com/r/renskiy/cron

Questions

Fabricio - Docker deploy automation

Engineering