Jan Löffler, CTO Plesk

AutoScaling WordPress with Docker & AWS

code.talks 2016

Sebastian Vettel

Who uses WordPress?

Usain Bolt Katy Perry Snoop Dog LL Cool J

Kim Kardashian The Rolling Stones Sylvester Stallone Mötley Crüe

Beyoncé Justin Bieber Justin Timberlake Alanis Morissette

Usain Bolt Katy Perry Snoop Dog LL Cool J

Kim Kardashian Mötley Crüe

Beyoncé Alanis MorissetteJustin Bieber

TechCrunch Metro The New Yorker BBC America

MTV News The Official Star Wars Blog Sony Music Facebook Newsroom

Time.com CNN UPS Mozilla Blog

http://w3techs.com/technologies/history_overview/content_management/all/y

# WEBSITES WORLDWIDE

1.085.358.537http://www.internetlivestats.com/total-number-of-websites/

# WEBSITES WORLDWIDE

1.085.358.692http://www.internetlivestats.com/total-number-of-websites/

38% of E-Shops

26,6%

2,7%

2,2%

#SPEED

High Traffic Sites???

NEED

FOR

SPEED

my-wordpress-site.com

my-wordpress-site.com

MYSQL ServerLoad-Balancer

my-wordpress-site.com

MYSQL ServerLoad-Balancer

File Storage

my-wordpress-site.com

MYSQL Server

Load-Balancer

File Storage

CDN

MYSQL Server

Easy-peasy, isn’t it?

DEPLOYMENT

doubled to 30%http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2016-state-cloud-survey

my-wordpress-site

Modern Web-Development

2016

IMMUTABLE INFRASTRUCTURE

IMMUTABLE STACKS

ELB myapp-v1

jans-wordpress.com

EC2+ Docker

EC2+ Docker

EC2+ Docker

100%

IMMUTABLE STACKS

ELB myapp-v1

EC2+ Docker

EC2+ Docker

EC2+ Docker

ELB myapp-v2

EC2+ Docker

EC2+ Docker

jans-wordpress.com

90% 10%

IMMUTABLE STACKS

ELB myapp-v1

EC2+ Docker

EC2+ Docker

EC2+ Docker

ELB myapp-v2

EC2+ Docker

EC2+ Docker

jans-wordpress.com

0% 100%

APP INSTANCE

VM (EC2)

Basis Image (AMI)

Docker Container

Application

✓ VPC✓ Security

Groups✓ AutoScaling✓ CloudWatch✓ CloudFront✓ Route53

✓ Docker Runtime✓ Logging✓ Monitoring✓ Security config

✓ Debian✓ Nginx✓ php7-fpm✓ NewRelic

AWS

API

Docker Registry

docker pull

docker push

AMI

THE WISH

THE REALITY

$ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface

$ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$

$ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$

$ describe-reserved-instances$ $ describe-reserved-instances$ describe-reserved-instancesmodifications$ $ describe-reserved-instances$ describe-route-tables$ $ describe-scheduled-$ describe-scheduled-$ describe-security-group$ describe-security-groups$ $ describe-snapshot-attribute$ $ describe-snapshots$ $ describe-spot-datafeed$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-instance$ describe-spot-price$ describe-stale-security$ describe-subnets$ $ describe-tags$ $ describe-volume-attribute$ $ describe-volume-status$ $ describe-volumes$ $ describe-vpc-attribute$ $ describe-vpc-classic$ describe-vpc-classic$ describe-vpc-endpoint$ describe-vpc-endpoints$ $ describe-vpc-peering$ describe-vpcs$ $ describe-vpn-connections$ $ describe-vpn-gateways$ $ detach-classic-link$ detach-internet-gateway$ $ detach-network-interface$ $ detach-volume$ $ detach-vpn-gateway$

210 calls for EC2 only

$ accept-vpc-peering-connection$ $ allocate-address$ $ allocate-hosts$ $ assign-private-ip-addresses$ $ associate-address$ $ associate-dhcp-options$ $ associate-route-table$ $ attach-classic-link-vpc$ $ attach-internet-gateway$ $ attach-network-interface$ $ attach-volume$ $ attach-vpn-gateway$ $ authorize-security-group-egress$ $ authorize-security-group-ingress$ $ bundle-instance$ $ cancel-bundle-task$ $ cancel-conversion-task$ $ cancel-export-task$ $ cancel-import-task$ $ cancel-reserved-instances-listing$ $ cancel-spot-fleet-requests$ $ cancel-spot-instance-requests$ $ confirm-product-instance$ $ copy-image$ $ copy-snapshot$ $ create-customer-gateway$ $ create-dhcp-options$ $ create-flow-logs$ $ create-image$ $ create-instance-export-task$ $ create-internet-gateway$ $ create-key-pair$ $ create-nat-gateway$ $ create-network-acl$ $ create-network-acl-entry$ $ create-network-interface

$ create-reserved-instances-listing$ $ create-route$ $ create-route-table$ $ create-security-group$ $ create-snapshot$ $ create-spot-datafeed-subscription$ $ create-subnet$ $ create-tags$ $ create-volume$ $ create-vpc$ $ create-vpc-endpoint$ $ create-vpc-peering-connection$ $ create-vpn-connection$ $ create-vpn-connection-route$ $ create-vpn-gateway$ $ delete-customer-gateway$ $ delete-dhcp-options$ $ delete-flow-logs$ $ delete-internet-gateway$ $ delete-key-pair$ $ delete-nat-gateway$ $ delete-network-acl$ $ delete-network-acl-entry$ $ delete-network-interface$ $ delete-placement-group$ $ delete-route$ $ delete-route-table$ $ delete-security-group$ $ delete-snapshot$ $ delete-spot-datafeed-subscription$ $ delete-subnet$ $ delete-tags$ $ delete-volume$ $ delete-vpc$ $ delete-vpc-endpoints$ $ delete-vpc-peering-connection$

$ delete-vpn-connection$ $ delete-vpn-connection-route$ $ delete-vpn-gateway$ $ deregister-image$ $ describe-account-attributes$ $ describe-addresses$ $ describe-availability-zones$ $ describe-bundle-tasks$ $ describe-classic-link-instances$ $ describe-conversion-tasks$ $ describe-customer-gateways$ $ describe-dhcp-options$ $ describe-export-tasks$ $ describe-flow-logs$ $ describe-host-reservation-offerings$ $ describe-host-reservations$ $ describe-hosts$ $ describe-id-format$ $ describe-identity-id-format$ $ describe-image-attribute$ $ describe-images$ $ describe-import-image-tasks$ $ describe-import-snapshot-tasks$ $ describe-instance-attribute$ $ describe-instance-status$ $ describe-instances$ $ describe-internet-gateways$ $ describe-key-pairs$ $ describe-moving-addresses$ $ describe-nat-gateways$ $ describe-network-acls$ $ describe-network-interface-attribute$ $ describe-network-interfaces$ $ describe-placement-groups$ $ describe-prefix-lists$ $ describe-regions$

$ describe-reserved-instances$ $ describe-reserved-instances$ describe-reserved-instancesmodifications$ $ describe-reserved-instances$ describe-route-tables$ $ describe-scheduled-$ describe-scheduled-$ describe-security-group$ describe-security-groups$ $ describe-snapshot-attribute$ $ describe-snapshots$ $ describe-spot-datafeed$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-fleet$ describe-spot-instance$ describe-spot-price$ describe-stale-security$ describe-subnets$ $ describe-tags$ $ describe-volume-attribute$ $ describe-volume-status$ $ describe-volumes$ $ describe-vpc-attribute$ $ describe-vpc-classic$ describe-vpc-classic$ describe-vpc-endpoint$ describe-vpc-endpoints$ $ describe-vpc-peering$ describe-vpcs$ $ describe-vpn-connections$ $ describe-vpn-gateways$ $ detach-classic-link$ detach-internet-gateway$ $ detach-network-interface$ $ detach-volume$ $ detach-vpn-gateway$

210 calls for EC2 only

S3ELB

RDS

CloudFrontCloudWatch

AutoScaling

Route53

Events

IAM

S3api

SNS

PLESK WORDPRESS AWS SCALER

https://github.com/plesk/wordpress-aws-scaler

AWS

manage-wordpress.sh

Docker Registry

docker pull

docker push

AMI

PLESK WORDPRESS AWS SCALER

manage-wordpress.sh create

Too quick? How does it work internally?Install AWS CLI (https://docs.aws.amazon.com/cli/latest/userguide/installing.html)

$ sudo pip install awscli$ aws configure$ aws --version

Clone WordPress AWS Scaler git repo (https://github.com/plesk/wordpress-aws-scaler)

$ git clone https://github.com/plesk/wordpress-aws-scaler.git$ cd wordpress-aws-scaler

Build Docker Image and Upload to Docker Hub (https://hub.docker.com/explore/)

$ docker build -t janloeffler/wordpress-aws-scaler:latest .$ docker push janloeffler/wordpress-aws-scaler:latest

List existing AWS resources

List existing AWS Resources

$ aws ec2 describe-vps $ aws ec2 describe-security-groups $ aws ec2 describe-instances$ aws rds describe-db-instances[...]

Do this for all AWS components

$ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events

Create new resources

Create new AWS Resources

$ aws ec2 create-vpc$ aws ec2 create-security-group[...]

Do this for all AWS components

$ aws autoscaling $ aws ec2 $ aws cloudfront $ aws cloudwatch $ aws elb $ aws rds $ aws s3 $ aws route53 $ aws events

Create new Launch ConfigurationGenerate EC2-User-Data Script to run directly after booting EC2 VM

$ cat >ec2-user-data.sh <<EOL#!/bin/bashdocker pull janloeffler/wordpress-aws-scaler:latestdocker run -d -p 80:80 -p 443:443 -e WORDPRESS_DB_HOST='${DB}' -e WORDPRESS_DB_USER='${DB_USERNAME}' -e WORDPRESS_DB_PASSWORD='${DB_PASSWORD}' -e WORDPRESS_DB_NAME='${DB_NAME}' -e WORDPRESS_DB_PREFIX='${WORDPRESS_DB_PREFIX}' -e WORDPRESS_URL='http://${ELB}' -e WORDPRESS_TITLE='${WORDPRESS_TITLE}' -e WORDPRESS_USER_EMAIL='${WORDPRESS_USER_EMAIL}' -e NEWRELIC_KEY='${NEWRELIC_KEY}' -e NEWRELIC_NAME='${NEWRELIC_NAME}' -e S3_KEY='${IAM_USER_KEY}' -e S3_SECRET='${IAM_USER_SECRET}' -e S3_BUCKET='${S3_BUCKET_NAME}' -e S3_BUCKET_URL='${S3_URL}' -it janloeffler/wordpress-aws-scaler:latestEOL

Create new Launch Configuration for AutoScaling

$ aws autoscaling create-launch-configuration --launch-configuration-name $LC_NAME --image-id $AMI --instance-type $INSTANCE_TYPE --key-name $KEYNAME --security-groups $SEC_GROUP_ID --user-data file://ec2-user-data.sh

You must love JSON !!! ;-){

"Reservations": [{

"OwnerId": "699328319947","ReservationId": "r-041ac13209e0d4eef",

"Groups": [],

"RequesterId": "226008221399","Instances": [

{"Monitoring": {

"State": "enabled"

},"PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com",

"State": {"Code": 16,

"Name": "running"

},"EbsOptimized": false,

"LaunchTime": "2016-09-05T12:19:54.000Z","PublicIpAddress": "52.210.200.157",

"PrivateIpAddress": "172.31.25.83",

"ProductCodes": [{

"ProductCodeId": "4jvb72q6a56js2x7jzd24jar5","ProductCodeType": "marketplace"

}

],"VpcId": "vpc-fffbe19a",

"StateTransitionReason": "","InstanceId": "i-0ea3384a24a3c6aba",

"ImageId": "ami-64385917",

"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal","KeyName": "Marko Heijnen",

"SecurityGroups": [{

"GroupName": "WordPress-Docker-Test",

"GroupId": "sg-1b8dde7c"}

],

"ClientToken": "70aa81e1-10a3-4b83-acc7-62702015d06e_subnet-cff90cb9_1",

"SubnetId": "subnet-cff90cb9","InstanceType": "m3.medium",

"NetworkInterfaces": [{

"Status": "in-use",

"MacAddress": "06:c0:22:65:b4:ed","SourceDestCheck": true,

"VpcId": "vpc-fffbe19a","Description": "",

"Association": {

"PublicIp": "52.210.200.157","PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com",

"IpOwnerId": "amazon"},

"NetworkInterfaceId": "eni-2f4efa63",

"PrivateIpAddresses": [{

"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal","Association": {

"PublicIp": "52.210.200.157",

"PublicDnsName": "ec2-52-210-200-157.eu-west-1.compute.amazonaws.com","IpOwnerId": "amazon"

},"Primary": true,

"PrivateIpAddress": "172.31.25.83"

}],

"PrivateDnsName": "ip-172-31-25-83.eu-west-1.compute.internal","Attachment": {

"Status": "attached",

"DeviceIndex": 0,"DeleteOnTermination": true,

"AttachmentId": "eni-attach-7f9031a0","AttachTime": "2016-09-05T12:19:54.000Z"

},

[…]

WP CUP – RACE DAY 1

single-server

avg 3.86 sec for 947 hits

auto-scaling

avg 1.45 sec for 9.866 hits

Fastest Lap 216 ms180 RPS < 3sec

Fastest Lap 99 ms1.000 RPS < sec

No Caching!!!

Plesk Mission

“We simplify the lives of web professionals.”

Docker

WordPress CDN

ELB & AutoScaling

DB

Route53

code.talks 2016, Hamburg

“High traffic sites with WordPress, Docker & AWS”

How to auto-scale WordPress on AWS and make it fly!

Passion for Web Development

Jan Löffler

CTO Plesk

● @jlsoft2

● jan@plesk.com

Marko Heijnen

WordPress Contributor

● @markoheijnen

● mheijnen@plesk.com

https://github.com/plesk/wordpress-aws-scaler

https://www.slideshare.net/plesk

“When you walk through your website architecture and you wonder who could be so stupid to test it with 10.000 parallel requests per second. But if you want to be successful and play in the first league, it is important to think about it before your traffic increases”

Jan Löffler @jlsoft2

https://github.com/plesk/wordpress-aws-scaler

BACKUP

Web Development Key Tools – what web developers need and use

59

26,6% of all websites worldwide

2,7% 2,2%(high traffic sites)

70% of developers use git as primary source management solutionIncreasing usage by web agencies

38% of all online stores worldwide

runs on top of WordPress as plugindeveloped by automattic

used by 30% of IT companiesincreased from 13% to 30% in 12 monthsfurther 35% plan introduction

Sources: w3techs.com, Rightscale, StackOverflow Survey 2015, 2016, Netcraft

311% growth

17.2% usage26.8% growth

17.9% usage14.9% growth

16% of all webservers22% of all active sites

continuously growing, while Apache is shrinking

WordPress Management

Free SSL everywhere

http2 50+ extensions

CLI

Automatic Updates

Site Migration Free support

30+ languages nginxServer Security DNS

✔

Plesk Onyx: Continuous Delivery Deployment Pipeline (sample)

61

Jenkins runs as Docker container managed by Plesk

Plesk Multi Server Extension installs three environments

Apps managed by Plesk via Docker