Scale out your Magento application withElasticBeanstalk

About usCofounder & CTO -

on Github on Twitter

Bitbull S.r.l. @Bitbull_itbitbull-teamkesonno

Cofounder - Cofounder -

on Github on Twitter

Corley S.r.l. @CorleyCloudUpCloo LTD @UpCloo

wdalmutwalterdalmut

- Internet Of Things! @ Turin [CFP] - 15 Nov - CloudComputing @ Turin [CFP ASAP]

internetof.itwww.cloudconf.it

ElasticBeanstalkScale your app workers and web apps

Queue DaemonsLoad balanced web applications

Mainly for Service Oriented Architecture

Autoscaling applied to web applications

Autoscaling in practice

Dynamically added and removed!

Machine to Machine

Closed-loop control system

Scale apps is not simpleHow to handle dev/testing/production? (dynamic env)How to install/update softwares? (dynamic env)How to handle user sessions? (more than one node)How to handle/tail logs? (dynamic env)How to monitor all instances (dynamic env)And more... (all things are moving!)

Run different environment per app

Typically you run: Production, Testing, Development

How to deploy the app?ElasticBeanstalk updates app using Git on pushElasticBeanstalk updates app using a pre-packaged Zip file

ElasticBeankstalk offers also API

Git Push -> Jenkins Build ZIP package -> ElasticBeankstalk Deploy

Different app versions per environment

Easy distributed app deploy/rollback and testing/production application management

Different configuration per environment

Upgrade your environment and switch your production without downtime

Production Env Swap

My production environment

Add a new environment

Swap production environment

ElasticBeanstalk swap env URLs in order to simplify the upgrade

Destroy your old environment

COST-SAVING!it is easier create and destroy environments than upgrade them

Environment Management

Control your ApplicationConfigurations

Application variablesAll environment variables are ported to your application in $_SERVER

You can pass everything like: Memcached and Mysql configurations etc.

Scaling made easy

We need more customizations!Create a folder in your project root with name .ebextensions and

append your configuration files with extension .config

ElasticBeanstalk will use them during the application provisioning

Cronjobs runs on the leader instance only.ebextensions/05_cron_jobs.config

container_commands:    01_magento_cron_job:    command: "cat .ebextensions/magento_cron_job.txt > /etc/cron.d/magento_cron_job && chmod 644 /etc/cron.d/magento_cron_job"    leader_only: true

All configuration files are just simple YAML files

Monitor your environment

You can monitor many metrics with CloudWatchUDP/IP CloudWatch agent on local machine: https://github.com/wdalmut/cloudwatch-agent

What we are missing?

Logs!

Grab all active instances logs

But my application logs?commands:    21_application_logs:        command: echo "/var/app/current/var/log/*.log" > myapp.conf        cwd: /opt/elasticbeanstalk/tasks/bundlelogs.d    22_application_logs:        command: echo "/var/app/current/var/log/*.log" > myapp.conf        cwd: /opt/elasticbeanstalk/tasks/systemtaillogs.d    23_application_logs:        command: echo "/var/app/current/var/log/*.log" > myapp.conf        cwd: /opt/elasticbeanstalk/tasks/taillogs.d    24_application_logs:        command: echo "/var/app/current/var/log/*.log" > myapp.conf        cwd: /opt/elasticbeanstalk/tasks/publishlogs.d

.ebextensions/06_prepare_logs.config

The Magento side

Use Composer for dependenciesElasticBeanstalk uses Composer in order to prepare your application

So, we can use composer hooks in order to connect all env variables toour Magento configuration

http://github.com/magento-hackathon/magento-composer-installer

Composer hooks{    /** other composer configs **/    "scripts": {        "post‐update‐cmd": [            "Corley\\Deploy\\Magento::localConf"        ],        "post‐install‐cmd": [            "Corley\\Deploy\\Magento::localConf"        ]    },}

<?phpnamespace Corley\Deploy;

use Composer\Script\Event;

class Magento{    public static function localConf(Event $event)    {        // Generate a local.xml based on environment variables    }}

The golden rule:

The filesystem is not reliable

Keep media in a shared locationok but, what could we use?

NFS?is a single point of failureis not at web scale

Amazon S3?does not work out-of-the-boxmaybe in future...

Database?

Why not?work out-of-the-box (and is dead simple to configure)

read access to db only once for each instance......may be reduced to only one using a CDN (CloudFront)

Cache and sessionsAll cache backend and session handlers except "File" are fineBest options are Redis and Memcached (both available via AmazonElastiCache)

TO BE DONEsitemap.xmlExchange data for third-party integration (ERP, CRM etc.)Integration with services (eg. bank gateway) that require IPs whitelist

Use a proxyIP reservation

Store media on Amazon S3Use DynamoDB session handler

Thanks for listening

Any question?