Adopt Devops philosophy on yourSymfony projects

An introduction to Devops

by Fabrice Bernhard

MeFabrice Bernhard

@theodo

fabriceb@theodo.fr

Co-founder / CTO of Theodo and Allomatch.com

Theodo creates web-based applications

with open-source web technologies, agile methodologies and the higheststandards of quality

to guarantee rapid application development, risk-free deployment and easymaintenance for every client.

@skoop: Now, let's look for a bar in Paris where I could watch the FC Utrecht match:)

Allomatch.com is the website for watching sports in bars in France and Spain(sportandbar.es)

Allomatch is used by 500 barmen (clients) and visited by more than 200,000unique visitors per month

Peaks on the biggest days can go up to 20,000 people in the 2 hours precedingthe game

Allomatch.com is hosted on a cluster of 6 servers

What is DevOps?Introduction

How many here consider themselves SysAdmins?

How many here have never deployed an application on aserver?

WikipediaDefinition of DevOps

DevOps is a set of processes, methods and systems for communication,collaboration and integration between departments for Development(Applications/Software Engineering), Technology Operations and Quality Assurance(QA).

It relates to the emerging understanding of the interdependence of developmentand operations in meeting a business' goal to producing timely software productsand services

The fundamental DevOps contradictionDevs VS Ops

Developers are asked to deliver new value, often and fast

Operations people are asked to protect the current value

Pro-Change VS Pro-Stability

Silos

Break the silos

DevOps do RADD

DevOps create the infrastructure that empower devs fromthe first line of code to the delivery

How to be DevOps?

Configuration management for rapid, repeatable server setup

Deployment scripts to abstract sysadmin tasks and empower developers

Development VMs with prod configuration to ensure consistency and avoidunexpected system-related bugs

Continuous deployment to make it fast!

DevOps is spreading agility to the whole IT project lifecycle

Rapid and repeatable server setupConfiguration management with Puppet

What is configuration management?

Writing the system configuration of your servers in files

Applying these files automatically

That's it!

Why do configuration management?

To do fast cluster deployment: who wants to manually setup 50 EC2 servers???

To do fast crash-recovery: configuration management is the best documentationfor a server's setup

To have consistent environments for development and production

Puppet or ChefConfiguration management tools

Two popular recent tools for configuration management: Puppet and Chef

A master server contains different "recipes" describing system configurations

Client servers connect to the master server, read their recipe, and apply theconfiguration

Puppet

Puppet references

Let us create a Symfony-ready server with Puppet

Introduction to Puppet manifests

class lighttpd{ package { "apache2.2-bin": ensure => absent, } package { "lighttpd": ensure => present, } service { "lighttpd": ensure => running, require => Package["lighttpd", "apache2.2-bin"], } }

class lighttpd-phpmysql-fastcgi inherits lighttpd{ package { "php5-cgi": ensure => present, } package { "mysql-server": ensure => present, } exec { "lighttpd-enable-mod fastcgi": path => "/usr/bin:/usr/sbin:/bin", creates => "/etc/lighttpd/conf-enabled/10-fastcgi.conf", require => Package["php5-cgi", "lighttpd"], }}

class symfony-server inherits lighttpd-phpmysql-fastcgi{ package { ["php5-cli", "php5-sqlite"]: ensure => present, notify => Service["lighttpd"], } package { "git-core": ensure => present, } exec { "git clone git://github.com/symfony/symfony1.git": path => "/usr/bin:/usr/sbin:/bin", cwd => "/var/www", creates => "/var/www/symfony1", require => Package["lighttpd", "git-core"], }}

class symfony-live-server inherits symfony-server{ file { "/etc/lighttpd/conf-available/99-hosts.conf": source => "/vagrant/files/conf/hosts.conf", notify => Service["lighttpd"], } exec { "lighttpd-enable-mod hosts": path => "/usr/bin:/usr/sbin:/bin", creates => "/etc/lighttpd/conf-enabled/99-hosts.conf", require => File["/etc/lighttpd/conf-available/99-hosts.conf"], notify => Service["lighttpd"], }}include symfony-live-servernotice("Symfony server is going live!")

Why not use shell scripts?

Shell scripts are for administrators. Is all your team composed of admin experts?

Even for admin experts, Puppet and Chef recipes are more readable

Puppet and Chef make inheritance and modules easy

Puppet and Chef are idempotent: running them twice in a row will not breakyour system

Develop and test on the sameenvironment as in production!

VM provisioning with Vagrant

Develop on local Virtual MachinesVagrant

Vagrant is a tool to create local VirtualBox VMs, configured automatically by yourChef recipe or Puppet manifest

It ensures you test on the same environment as your production server

It is VERY easy

All you need is:Vagrant

A Puppet manifest

A few system config files

A Vagrant conf file

DemonstrationVagrant

$ git clone git://github.com/fabriceb/sflive2011vm.git .$ git clone git://github.com/fabriceb/sflive2011.git$ vagrant up

http://127.0.0.1:2011/

Give developers the power to deploythemselves

Scripted deployment

Deployment

Deployment is a very critical task usually done by admins

Remember Murphy's law: "If anything can go wrong, it will"

When things go wrong, most of the time developers have the solution

So give the developers the responsibility to deploy, rollback, correct and deployagain!

Scripting deployment can be VERY easySimple Fabric script example

# fabfile.pyfrom fabric.api import *env.hosts = ['theodo@myserver.com']def deploy(): with cd('/theodo/sflive2011'): run('git pull') run('./symfony doc:build --all --no-confirmation') run('./symfony cc')

$ fab deploy

A good practise: scripting a rollbackAnother Fabric example

# fabfile.pydef deploy(): tag = "prod/%s" % strftime("%Y/%m-%d-%H-%M-%S") local('git tag -a %s -m "Prod"' % tag) local('git push --tags') with cd(path): run('git fetch') tag = run('git tag -l prod/* | sort | tail -n1') run('git checkout ' + tag)def rollback(num_revs=1): with cd(path): run('git fetch') tag = run('git tag -l prod/* | sort | tail -n' + \ str(1 + int(num_revs)) + ' | head -n1') run('git checkout ' + tag)

And why not let Jenkins deployhimself?

Continuous deployment

The Holy Grail of Rapid App Development & Deployment:

Automate everything low value-added

and relax

Isn't it dangerous to trust a machine?Errare humanum est

Of course you need continuous integration with MANY tests

Of course you need some serious monitoring on the production server

Of course you need some good rollback scripts

But aren't that good things to do anyway ?

Good continuous integration is more reliable than a human!

You need to separate dev, pre-prod and prod...Continuous deployment howto

For example with git:

features/* branches for small projects

dev branch for merging team development

master branch for production-ready code

prod/* tags for production

And you need a deployment script + JenkinsContinuous deployment howto

Deployment script using Fabric (for example)

Jenkins (formerly known as Hudson) to test and deploy

Create a new Jenkins project testing only branch master

Specify "Build other projects" in the post-build actions

Don't forget to activate Chuck Norris

Create a second Jenkins project to execute the deploy script

That's it!

Next step

Links

docs.puppetlabs.com

fabfile.org

vagrantup.com

github.com/fabriceb/sflive2011vm

DevOps meetups

groups.google.com/group/paris-devops

and many more devops meetups around the world

Many thanks to Samuel @smaftoul Maftoul, organiser of theParis DevOps meetup, who bootstrapped me on DevOps!

Questions?

@theodo

fabriceb@theodo.fr