Rock-solid Magento Development and Deployment Workflows

2 October 26, 2015

San Francisco, CA

Janine

Fiona

that’s me

Leo

Continuous Development

Continuous Delivery

Continuous Integration

Continuous Deployment

Continuous Monitoring

Continuous

Jenkins

Everything sounds better with “continuous”

plan

code build

test

deploy operate

http://kirbiecravings.com/2014/08/blueberry-mug-cake-2.html

code

CMS

Magento

Solr

ERP

Project

Applications

Magento Magento

117

Magento

118

Magento

119

Builds

Release

Staging

Magento

72

Magento

103

Magento

119

Magento

72

Magento

103

Magento

119

Magento

72

Magento

103

Magento

119

Environment

Servers

Instances

Staging Integration Devbox Production Deploy

transient, potentially broken

Staging Integration Devbox Production Deploy

VirtualBox + Vagrant

Jenkins Server

identical setup

One size doesn’t fit all!

What is it? A simple shell script. That’s it! No Ant, no Phing, no fancy PHP tool/framework

Where is it used?

In the build job of a Jenkins server

What does it need to know? Where to get all the source code from

What does it do? add metadata, composer, minify, packaging, upload to S3

What is it? A couple of chef cookbooks. Run by AWS OpsWorks or Vagrant,…

Where is it used?

On the target servers

What does it need to know? The role of the server (web server, devbox, Redis,…)

What does it do? Installing and configuring web server, PHP, MySQL, Redis,…

What is it? A simple shell script. Or the “deploy” chef cookbook Or EasyDeployWorkflows

Where is it used?

On the target servers

What does it need to know? How to get the build and where to deploy it to

What does it do? (Resetting the environment), getting the build, extracting it, managing release folders and symlinks, triggering installer, cleanup old releases

What is it? A simple shell script. Sometimes triggered inside “deploy” before_migrate.rb

Where is it used?

On the target servers, called from deployment step

What does it need to know? What environment this is. Injected settings

What does it do? Run modman, apply settings, trigger update scripts, customize cache prefix, indexing,…

doesn’t know anything about the app, except where to get it

from and where to put it

knows how to get the app up and

running

no creating CMS blocks

no creating product attributes

no updating of any configuration settings!

no clearing the cache

no purging CDN or Reverse Proxy

content!

media directory (sometimes minified)

Database dump

(Vanilla) Magento core

Modules

Scripts

Settings

Tools

Structure Project

project web root root

This is the complete project

This is where your webserver points to.

project web root root

htdocs

<Name>

<theme>

layout

template

code

design

etc

locale

<area>

core

community

local

<area>

<language>

modules

<package>

<Namespace>

<package>

*.xml

*.csv

<theme>

app

skin

lib

htdocs

<Name>

<theme>

layout

template

code

design

etc

locale

<area>

core

community

local

<area>

<language>

modules

<package>

<Namespace>

<package>

*.xml

*.csv

<theme>

app

skin

lib

Module2

default

layout

template

code

design

etc

locale

frontend

local

frontend

de_DE

modules

base

My

base

My_Module2.xml

My_Module2.csv

default

app

skin

etc, Block, …

js, css, img, …

my_module2

my_module2

Magento My_Module My_Module2

Module

default

layout

template

code

design

etc

locale

frontend

local

frontend

de_DE

modules

base

My

base

My_Module.xml

My_Module.csv

default

app

skin

etc, Block, …

js, css, img, …

my_module

my_module

htdocs

Module

default

layout

template

code

design

etc

locale

local

frontend

de_DE

modules

My

base

*.xml

app

My_Module2.xml

My_Module2.csv

my_module2

My_Module.xml

My_Module.csv

etc, Block, …

my_module

Module2

etc, Block, …

my_module2

my_module

by @colinmollenhour

htdocs

code

core

community

local

app

app/code/…/My/Module

.modman

My_Module

app/design/…

skin/frontend/…

app/etc/modules/My_Module.xml

modman


My_Module2

app/design/…

skin/frontend/…


modman

<theme>

layout

template

design

etc

locale

<area>

<area>

<language>

modules

<package>

<package>

*.xml

*.csv

<theme>

skin

modman doesn’t even need to be in the web root!

Vanilla Magento

absolutely no core hacks!

…and core bugs you’ve fixed (and submitted to

Magento, of course…)

(except official patches)

modman “connects” existing modules to the Magento core, but how do we manage them?

htdocs

code

core

community

local

app


.modman

My_Module

app/design/…

skin/frontend/…


modman


My_Module2

app/design/…

skin/frontend/…


modman

<theme>

layout

template

design

etc

locale

<area>

<area>

<language>

modules

<package>

<package>

*.xml

*.csv

<theme>

skin

tools

n98-magerun.phar

phpunit.phar

modman

EnvSettingsTool

composer.phar

htdocs

.modman

Configuration

Installer

(Metadata, Doc,…)

My_Module

My_Module2

app,…

and usually there’s even more stuff…

These should come from different origins

“inspired” by composer/installers

handles package types

<?php namespace Aoepeople\ComposerInstallers; class MagentoInstaller extends BaseInstaller { protected $locations = array( 'module' => '.modman/{$name}/', 'source' => 'htdocs/' ); }

• Fully compatible with magento-hackathon/magento-composer-installer

• Adds support for type “magento-source”

• Only puts packages into place

• Native modman script is used to deploy modules

• Wrapped in a simple installer script

{ "minimum-stability":"dev", "require":{ "aoepeople/composer-installers": "*", "fbrnc/aoe_scheduler": "*", "fbrnc/aoe_cachecleaner": "*", "aoepeople/magento_enterprise": "1.13.1.0", "aoepeople/envsettingstool": "*" }, "config": { "bin-dir": "bin" } }

replaces Hackathon Installer

some modules

Vanilla Magento Source

Tools,…

Webroot: /var/www/<projectName>/<environmentName>/current/htdocs

var

var

<projectName>

releases

<environmentName>

shared

www

media

build_117

build_118

build_119

.modman

htdocs

Configuration

tools

install.sh

app,…

var

media

previous

latest

current

next

My_Module

My_Module2

https://github.com/AOEpeople/Magento_Boilerplate

discover use

code

review

add

modman

add

composer git integrate test

deploy

download

good luck with that!

pay $xx

to author provide FTP

access

seriously?!

one-click

install

upload

“Step 1”

upload

“Step 2”

clear

caches

Code Quality Modules

Not enterprise ready

Not production ready

Not cloud ready Bad performance

Security

Urls

Database Settings

Payment Provider Configuration Feature

Flags…

Everything that’s different between two

environments

Zettr

Every file is a potential security risk

Don’t deploy anything you don’t need to

operate the website

.git*

./htdocs/includes

./htdocs/downloader

./htdocs/pkginfo

./htdocs/LICENSE*

./htdocs/RELEASE_NOTES.txt

./htdocs/phpunit.xml*

./htdocs/*.sample

./htdocs/var

./htdocs/media

./.modman/Aoe_TemplateHints

./.modman/Aoe_Profiler

./.modman/EcomDev_PHPUnit

…

production, staging,…

devbox and integration environment

Vagrant Boxes

Awesome stuff

Every single line potentially

can break production!

Bratwurst (Germans)

Donut (Americans)

Pizza (Italians)

master

Feature B

Feature A

Hotfix (if master is not modified)

Hotfix (if master is modified)

Feature C

build

install on integration

tests

build


tests

install on deploy

build


tests

deploy to stage

deploy to prod

build


tests

deploy to stage

deploy to prod

build


tests

deploy to stage

deploy to prod

single mainline

no commits, only --no-ff

merges

Version Control

GIT, SVN,…

PHP lint

Version Control Code Reviews

Crucible,…

Version Control

Code Reviews

Build

Files

DB

Installer

Settings

Version Control Code Reviews Build

Files

DB

Installer

Settings

Static Code Analysis

CodeSniffer

php-pmd

php-cpd

php-depend


Files

DB

Installer

Settings


Install on “Integration”

First system where code from all

developers/teams meets each other

unstable system, might be broken,

will be rebuilt without warning


Files

DB

Installer

Settings



Unit Tests

PHPUnit


Files

DB

Installer

Settings



Unit Tests Integration Tests

AOE PMD (for Magento)

broken links, HTTP headers, W3C validity,…

Test features that rely on other parts (e.g. framework, external services,…)

External components (SSL, Redis, Varnish, ERP, Data Import,…)

Basically this is also PHPUnit, but addressing

other things


Files

DB

Installer

Settings



Unit Tests Integration Tests Acceptance Tests

Selenium

Firefox

Chrome

browser

size

640

980

1280

1900

US languages

UK DE FR IT

language

size

browser

touchstone job


Files

DB

Installer

Settings




Install on “Staging”

Behat, PDIFF,…

Same setup as production, but different machines

Client QA / approval


Files

DB

Installer

Settings





Stress

Tests

JMeter

Spin up EC2 instances to

produce traffic

Again: multiple servers or autoscaling setup


Files

DB

Installer

Settings





Stress

Tests

Install on “Production”

A/B Deployments on different

scales

Plan B: rollback

Zero Downtime


Files

DB

Installer

Settings





Stress

Tests


Deployment != Feature Release


Files

DB

Installer

Settings





Stress

Tests


Enable Features

Version Control

Code Reviews Build Static Code Analysis


Unit Tests Integration Tests

Acceptance Tests



Enable Features

Pipeline Visualization

#284

#283

#282

#281

#280

Build Static Code

Analysis Unit Tests


Integration Tests

Acceptance Tests

#285

for team dashboards

type “YESIKNOWWHATIMDOING” + click

automatically

http://joind.in/talk/view/15588

Follow me on twitter!

My blog

Date post:	08-Jan-2017
Category:	Technology
Upload:	aoe
View:	6,523 times
Download:	2 times

Rock-solid Magento Development and Deployment Workflows

Technology