Continuous Integration and Automated Deployments for Symfony-based projects

Symfony 2 under control

PHP Minsk User Group

2

INTRODUCTION

3

Max Romanovsky

maxromanovsky.com

max.romanovsky@gmail.com

Maxim_Romanovsky@epam.com

Twitter, LinkedIn, SensioLabs Connect, MoiKrug: maxromanovsky

Who am I

4

Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day.

http://en.wikipedia.org/wiki/Continuous_integration

http://www.martinfowler.com/articles/continuousIntegration.html

What is Continuous Integration?

5

• Improve quality of application

• Understand current situation on project

• Automate your checklists & verifications

• Notify different people on different events

– Build broken? – Developers

– New build deployed to QA? – QA engineers

– New build deployed to UAT/Production? – Customers

How could it help you?

6

Dependencies• Composer• Platform packages• Phar• Pear• Pecl

Build• Phing• phpqatools.org

Continuous Integration• Jenkins CI• Additional PHP-

related & general plugins

• Git

Deployment• Package with

vendors• Capifony• Doctrine Migrations

Our proposal for Symfony-based projects

7

DEPENDENCIES

8

Manages 3rd party dependencies

required at runtime*

• Bundles• PHP libraries• JS/CSS libraries• *Phpunit for hphpa

Composer

getcomposer.org

9

Checklist

• stability• lib-*, ext-*• install vs update• require• version masks• Keep composer.phar

under VCS• Satis for private packages

Composer

github.com/composer/satis

lib-*, ext-*: goo.gl/Z8dFal

10

Phar• Only for CLI libraries used for development and build• Composer, php-cs-fixer

Pear• Build tools• Phing, phpunit*, phpqatools

Pecl• For PHP extensions missing in platform packages

Platform packages (apt, yum, ports)• Use it for all purposes if you can (PHP, extensions etc.)

Other tools

11

What to update?• Composer dependencies• PHARs• PEARs• PECLs• Platform packages

Keep in mind• Update at the beginning of the

iteration• Read changelogs• Subscribe to news feeds• Use * as Composer dependency

version to receive all updates• Not for Symfony itself

Dev

QA

Staging• Performance testing• Check logs

Production

Don’t forget about updates!

12

BUILD

13

Ant

• From Java world

• XML declarative config

• Can run tasks in parallel

• No PHP-specific tasks

• Could be extended (Java)

• IDE support

Phing

• Written in PHP• Ant config

syntax• All tasks by

default are running in the single PHP process

• Could be extended (PHP)

• IDE support

Pake

• Written in PHP• Not so popular• PHP-based

config• github.com/inde

yets/pake/wiki

Build tools

14

build-symfony-ci.xml• Default workflow

build.xml• Overwrites for current

project

build.properties.dist• Default configuration

build.properties• Project configuration

Phing

phing.info

15

Target

• A set of tasks• Could depend

on other targets

• phing <target_name>

Task

• Action executed as a part of target

• Custom tasks can be implemented

Types

• Reference• FileSet• Property• Other

• FileList• Filters

Phing Terminology

16

build

• for dev environment

• outputs to the console

build-ci

• for CI environment

• outputs to XML files

package

• generates artifacts for deployment

• tar.gz archive

• SQL schema dump

deploy

• deploys tar.gz archive to the environment

Main Phing Targets

17

Special clean

prepare

build-tools-self-update

Composer-specific

composer-validate

dependencies-install

dependencies-list-updates update –dry-run

symfony-security-checker security.sensiolabs.org

Phing Targets (1/6)

18

General phplintphp -l

phploc

pdependcharts

phpmd+ custom config

phpcs/phpcs-js+ custom config (PSR-1&2, no var_dump(). console.log() etc)

phpcpd

Phing Targets (2/6)

19

General apidocPHPDoc

projectdocrst2html

phpunitvia exec

phpcbPHP_CodeBrowser UI

php-cs-fixer

hphpaHipHop static analysis

No target for KPHP yet

Phing Targets (3/6)

20

Symfony-specific

copy-configs-ci parameters.yml for CI

cache:warmup app/cache is removed during build

twig:lint for Bundles and app/Resources

assets:install

assetic:dump YUICompressor, Google Closure Compressor, lessphp

Phing Targets (4/6)

21

Symfony-specific

doctrine:database:drop

doctrine:database:create

doctrine:migrations:migrate

doctrine:fixtures:load

doctrine:schema:validate

doctrine:schema:dump –dump-sql

for DDL schema artifact

Phing Targets (5/6)

22

Package package-version version.txt

package-db-schema schema.sql

package-archive

Deployment deploy-maintenance- enable/disable

deploy-backup-db

deploy-clear-doctrine-cache Only if not in APC

deploy-cleanup-old-releases

deploy-artifact artifact name provided via CLI argument

Phing Targets (6/6)

23

CONTINUOUS INTEGRATION

24

CruiseControl• +PHP Under

Control

TeamCity• Commercial

Travis CI• SaaS

Scrutinizer CI• SaaS• PHP-oriented

Jenkins CI• ex-Hudson CI

CI servers

25

• Open-source

• Enterprise

• Lot of plugins

• PHP Template from Sebastian Bergmann

• IDE Integration

• RESTful API

Jenkins CI overview

jenkins-ci.org

26

• No comments git

• green & red signs for buildsgreenballs

• parameterized buildsparameterized-trigger

• git tag selector for builds• disable internal tagging!git-parameter

• copy artifact from one job to anothercopyartifact

• sends emails on successful builds• useful for deployment jobsemail-ext

Jenkins Plugins (common)

goo.gl/IZ2gKE

27

phing

•phpcs•hphpacheckstyle

•phpcpddry

•php_dependjdepend

•phploc using CSV filesplot

•phpmdpmd

•aggregates info from phpcs, hphpa, phpmd, phpcpdviolations

•phpunitxunit

•HTML artifactshtmlpublisher

Jenkins Plugins (PHP-specific)

jenkins-php.org

28

Master

• Bleeding edge• For completed features

Feature

branch

• Merge to master on completion• After code review• Developer executes phing build at least before code review

Tag

• Stable version from master

Bugfix & develop?

• Sprint branch• Master branch• Merge feature branches to sprint branch• Jenkins job for sprint branch

Git workflow

29

• Builds specific branch (master)• Features are verified after merge

build-<branchname>

• Parameterized• Produces .tar.gz artifact from specific

Git tag

build-package-tag

Jenkins Jobs for build

30

• Produces .tar.gz artifact from latest revision of branch• Deploys to QA• Not recommended to have more than one such job

(issues with migrations)• Workaround: multiple QA or complete DB purge on deploy

deploy-qa-<branchname>

• Deploys specific package from build-package-tag to specified environment

• Ability to enable/disable maintenance mode and error message

• Email after deployment

deploy-package-tag

Jenkins Jobs for deployment

31

Deployed .tar.gz archive

• <tag>.<build#>• v0.0.3.26

• <branchname>-<build#>-<revision>-SNAPSHOT• master-12-7c041a90366ce9d958760094c763d809ffbc629e-

SNAPSHOT

Previous deployed version

DB dump before deployment

Doctrine Migrations info

DB schema

Jenkins Jobs for deployment. Artifacts

32

DEPLOYMENT

33

Capistrano• from Ruby world

Capifony• Based on

Capistrano

Shell script• VCS update• Rsync

Phar• Don’t use WebPhar• Silex does not use

it anymore

Zend Server package• No support for

vanilla PHP

PaaS• AWS Elastic

Beanstalk• PagodaBox

Deployment tools

34

Based on Capistrano• Supports Capistrano plugins

Symfony 1&2 deployment

Multistage• qa, staging, uat, production

Maintenance mode

Clustered environments

Capifony

capifony.org

35

Directories• releases/• current/ (symlink)• shared/

Shared• app/config/parameters.yml• app/cache• app/logs• web/uploads

Shared files & dirs are symlink’ed

Shared files & dirs are not shared across

nodes

Directory structure for Capistrano projects

36

`-- /var/www/my-app.com

|-- current → /var/www/my-app.com/releases/20100512131539

|-- releases

| `-- 20100512131539

| `-- 20100509150741

| `-- 20100509145325

`-- shared

|-- web

| `-- uploads

|-- log

`-- config

`-- databases.yml

Directory structure example

37

All you need is package• capistrano-deploy-strategy-archive

Github not available?• Goodbye, vendors!

Decreases deployment time• Lot of vendors• Slow internet connection

Reproducible• You know what you deploy• You can deploy exactly the same package to multiple environments

Deployment via SCM vs package

38

.tar.gz Optimized autoloader

Contains only necessary dirs & files

assets:install assetic:dumpversion.txt &

db-schema.sql

Packaging with Phing

39

DBDeploy Liquibase

Doctrine Migrations

Doctrine Fixtures

Database Migrations

40

User

• Triggers job on Jenkins CI• Choose options (env, maintenance mode)

Jenkins

• Executes single phing target with additional command-line arguments

Phing

• Executes several Capifony commands according to command-line arguments

Capifony

• Performs actions on remote machines

Capifony & Jenkins CI

41

Enables Maintenance mode• maintenance.html in webroot

Backups DB

Clears Doctrine cache• Not for APC cache

Downloads previous deploy metadata• package version• Doctrine Migrations info

Deploys tar.gz archive

Capifony & Phing (1/2)

42

Restarts services• php-fpm or Apache• Doctrine APC cache is cleared

Symfony-specific tasks• copy app/config/parameters.yml• doctrine:migrations:migrate• cache:warmup

Disables Maintenance mode

Performs cleanup for old releases

Capifony & Phing (2/2)

43

Semi-automatical

• from deployment server

• usually CI server

Enable Maintenance

• manually via phing

Manually backup for analysis

• DB• current version• Doctrine

Migrations status

• codebase

Rollback (1/2)

44

Switch to previous release

• Manually via capifony CLI

• Recover DB• Rollback

Doctrine Migrations

• Restore DB dump

• Custom

Additional tasks

• Restore web server config

• Restart service• cache:warmup

Disable Maintenance

• Manually via phing

Rollback (2/2)

45

Write rollback guide• Step-by-step• Verify it periodically

Enable monitoring on prod• Zabbix• Munin• New Relic• Pinba• Graylog2

Rollback tips

46

SUMMARY

47

• Currently not available online

• Planning to release in 1-2 months on GitHub

Where to download?

48

Deployment•Cronjob / CLI script handling•Web server (Apache, nginx) config manipulations

•Ext- and lib- dependency verification via Composer

What is not implemented yet

Questions?