Home > Technology > Cloud Computing & WordPress - Scalability and High Availability - wpcampbo13

Cloud Computing & WordPress - Scalability and High Availability - wpcampbo13

Date post: 27-Jan-2015
Category:
Author: corley-srl
View: 107 times
Download: 1 times
Share this document with a friend
Description:
Cloud Computing & WordPress - Scalability and High Availability @ WordPress WordCamp Bologna 2013 by Gabriele Mittica and Walter Dal Mut - www.corley.it - www.upcloo.com
Embed Size (px)
Popular Tags:
of 52 /52
WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 WordPress & Cloud Computing: Scalability and High Availability Gabriele Mittica & Walter Dal Mut
Transcript
  • 1. WordPress & Cloud Computing:Scalability and High AvailabilityGabriele Mittica & Walter Dal MutWORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13

2. CLOUD COMPUTINGCloud computing refers to the delivery of computing and storagecapacity as a service to a heterogeneous community of end-recipients.Cloud computing entrusts services with a users data, softwareand computation over a network.It has considerable overlap with software as a service (SaaS). WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 3. CLOUD COMPUTINGIs the cloud just a fad?1.No.Its a rational evolution of IT architecture towards a more efficient wayof managing resources and designing Web apps efficiently.Is the cloud cheap?2.No.The cloud allows you to pay the right for each service involved. Is the cloud just a scalable vps?3. No. The cloud is a set of services designed to meet specific computing needs. WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 4. CLOUD COMPUTING - access to unlimited resources - scalable architecture - no hardware dependency - pay as you go - geographical redundancy - high availability - increased competition for start-up WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 5. AMAZON WEB SERVICESWORDCAMP BOLOGNA - 9 FEBBRAIO 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 6. AMAZON WEB SERVICES - SIGNIN WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 7. AMAZON WEB SERVICES - CONSOLEWORDCAMP BOLOGNA - 9 FEBBRAIO 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 8. AMAZON WEB SERVICES - LINKS - Home page: http://aws.amazon.com/ - About AWS: https://aws.amazon.com/what-is-aws/ - All products: http://aws.amazon.com/products/ - Dev area: http://aws.amazon.com/resources/ - Documentation: http://aws.amazon.com/documentation/ - SDK: http://aws.amazon.com/code/ - Community: https://forums.aws.amazon.com/index.jspa - AWS Blog: http://aws.typepad.com/ - Events: https://aws.amazon.com/about-aws/events/ - Services Health Dashboard: http://status.aws.amazon.com/ - Pricing Calculator: http://calculator.s3.amazonaws.com/calc5.html WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 9. WORDPRESS ONAMAZON WEB SERVICESWORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 10. WORDPRESS ON AWS WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 11. WORDPRESS ON AWS WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 12. WORDPRESS ON AWS WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 13. BOOSTING WORDPRESS WITH: S3, CF, CLOUDSEARCH, SESWORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 14. PLUGINS FOR AWS S3 & CLOUDFRONT WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 15. PLUGINS FOR AWS S3 & CLOUDFRONT Download credentials: WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 16. PLUGINS FOR AWS S3 & CLOUDFRONT Grant full access to S3, CloudSearch, SES: WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 17. PLUGINS FOR AWS S3 & CLOUDFRONTTipical use of WordPress: html and media files are provided by apache(http request): WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 18. PLUGINS FOR AWS S3 & CLOUDFRONT Uploads your wordpress Services involved: attachements to S3 with an Simple Storage Service (S3): option CloudFront distribution.S3 provides a simple web servicesinterface that can be used to store and This WordPress plugin allows you retrieve any amount of data, at any to use Amazons Simple Storage time, from anywhere on the web. Service to host your media for It provides 99.999999999% durability. your WordPress powered blog with an optional CloudFrontCloudFront: distribution.CF can be used to deliver your entirewebsite, including dynamic, static andstreaming content using a global Plugin homepage:network of edge locations. http://wordpress.org/extend/plug Over 30 edge locations. ins/tantan-s3-cloudfront/ WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 19. PLUGINS FOR AWS S3 & CLOUDFRONTCreate a new bucket in S3 console: WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 20. PLUGINS FOR AWSEdit the distribution setting the bucket as origin! WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 21. PLUGINS FOR AWS S3 & CLOUDFRONTDownload, activate and customize the plugin: WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 22. PLUGINS FOR AWS S3 & CLOUDFRONTStatic files hosted on S3 and served by CloudFront: WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 23. PLUGINS FOR AWS S3 & CLOUDFRONT Thanks to S3 and CloudFront, you can easily send all your media filesthrough the content Delivery Network WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 24. PLUGINS FOR AWS - CLOUDSEARCH Create a scalable search engineServices involved: for your content published on WordPress. This WordPress plugin allows you CloudSearch: to use Amazons CloudSearch to Amazon CloudSearch is a fully-managed provide a smart search engine to search service in the cloud that allows your final users.customers to easily integrate fast andhighly scalable search functionality intotheir applications Plugin homepage:It supports over 8 millions docs. http://wordpress.org/extend/plug ins/lift-search/ WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 25. PLUGINS FOR AWS - CLOUDSEARCH Starting a new engine server with CloudSearch: WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 26. PLUGINS FOR AWS - CLOUDSEARCH Documentation at http://getliftsearch.com/documentation/ WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 27. PLUGINS FOR AWS SIMPLE EMAIL SERVICE WP-SES is a plugin that redirects Services involved: all outgoing WordPress emails through Amazon Simple Email Service (SES) for maximum email Simple Email Service (SES): delivrability.SES is a highly scalable and cost- effective bulk and transactional email- You can download this plugin on sending service for businesses and official website http://wp- developers ses.com/Only $0.10 per thousand. WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 28. PLUGINS FOR AWS SIMPLE EMAIL SERVICE WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 29. PLUGINS FOR AWS SIMPLE EMAIL SERVICE First, install like any other plugin: - Upload and activate the plugin - The setting are in settings / WP SES Then, proceed to the settings: - Fill the email address and name to use as the sender for all emails - Fill in Amazon API credentials - Save changes (Important !) - Ask to add the email as a confirmed sender - Click on the link you got by email from Amazon SES - Refresh the plugin, send a test email - If ok, ask Amazon to go out of sandbox into production mode - Once in production mode, you can use the top button to activate the plugin. WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 30. PLUGINS FOR AWS SIMPLE EMAIL SERVICE All emails sent by Simple Email Service: WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 31. PLUGINS FOR AWS BEFORE: WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 32. PLUGINS FOR AWS AFTER: WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 33. SCALABLE WORDPRESS HOW TO MAKE WORDPRESS SCALABLE ON THE CLOUDWORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 34. A TYPICAL SCALABLE INFRASTRUCTURE WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 35. RDBMS START FROM THE ENDComputation LB listens for incoming connections and route requests to web applications Web applications use RDBMS to get stored information Sessions and performance improvements are handled byMemcached instances.Static resources distribution CDN Content Distribution Network A globally distributed endpoints Serve static files (also dynamic if needed) Native connections with S3 (Simple Storage Service) It handles all static resources, in this way our web servers have to handle only dynamic calls WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 36. RDBMS START FROM THE END MySQL RDBMS Relational Database Management System How it scales? Read Replica Pros (In terms of scalability) Simple to do Simple management Cons You can scale only read operations The master instance has to handle all write operations (bottleneck on writes) WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 37. READ REPLICA ON AWS From RDS service tab on the AWS console right click on a running instance and create a Read Replica DB Instance Configure the read-replica and create it through the graphical console. WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 38. IN ORDER TO PROMOTE A SLAVE TO MASTER?Similar to master creation Select a read-replica Right-click and promote Read ReplicaDiscover more on RDS: http://aws.typepad.com/aws/amazon-rds/ WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 39. NOW HAVE A LOOK ON WEB INSTANCES All web instances scales out instead scales up Scale out? What it means? Instead increase VM performances (more RAM, more CPU, more IO etc. etc.) open new VM and serve requests from these instances Load balancer route incoming connections to VMs using common algorithms Round robin techniques Based on VMs average load WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 40. PROBLEMS WE NEVER TALK ABOUTSession management If we open and close servers runtime we have to maintain PHP sessions in order to handle user logins and other features related to sessionsDatabase connections All MySQL connectors handle just one connection No x RDB connections a the same timeSoftware and Plugins maintenance How can we have the same version of WordPress and WP Plugins if VMs starts and stops continuously? How can we handle software updates?What about logs? How can we centralize the log management? WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 41. DELEGATE SESSION MANAGEMENT TO MEMCACHE Memcache(d) servers are not only useful distributed in RAM caching servers but also they can manage PHP session for us. Memcache infrastructure is simple to create andmaintain Elasticache Service of AWS No software modification We have just to configure the PHP interpreter(compile with memcache/memcached support) session.save_handler = memcache session.save_path = "tcp://1.cache.group.domain.tld:11211" WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 42. DELEGATE CONNECTIONS TO MYSQL NATIVE DRIVER MySQL native driver?Available from PHP >=5.3Compile PHP with mysqlnd support --with-mysqli=mysqlnd --with-pdo=mysqlnd --with- mysql=mysqlndWARN mysql extension is deprecated as of PHP 5.5.0 Delegate to mysqlnd_ms the master/slavemanagementhttp://www.php.net/manual/en/book.mysqlnd- ms.php WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 43. DELEGATE CONNECTIONS TO MYSQL NATIVE DRIVER The simple JSON configuration is divided in two main section { "myapp": { "master": { Master "master_0": { Slaves "host": "localhost", "port": "3306" } myapp is the hostname that we use }, "slave": { instead the real mysql host address. "slave_0": { "host": "192.168.2.27", Eg. "port": "3306" } mysql_connect(myapp, user, } passwd); } new Mysqli(myapp, user, } passwd); new PDO(mysql:dbname=testdb;host=my app); WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 44. START TALKING ABOUT ELASTIC COMPUTE CLOUD ELB Elastic Load Balancer Distributed load balancer on AWS regions (eu-west-1, 2, 3 youhave to select in how many region you are available) Watch EC2 status thanks to a ping strategyPage check every x minutes/seconds Turn on/off EC2 instances automatically thanks to alarms(CloudWatch raise alarms)Receive Alarms from CloudWatch and engage scale operationsYou can raise CPU alarms, Network Alarms, VM status alarms and many others in order to increase or decrease the actual number of EC2 Scale strategy is not simple and you have to understand how yourapplication worksCPU is the simplest way but remember that the bandwidth is limited by network interfaces and bottlenecks can obfuscate the CPU alarm and your application stucks in weird and strange situations. WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 45. AUTOSCALING WITH ELB + EC2 + CLOUDWATCH If servers start and stops continuously, we have tofind solutions to stay fresh and updated also onsoftware When a server starts, it has to create a validenvironment in order to provides web pages.Strategies?Compile and bundle all softwares in one instance image It is very simple but all software becomes old very quickly andwhen you have to release an update you have to compile a newimage and update all load balancers configurations. It is a long andcomplex operationUse EC2_USER_DATA feature provided by AWS You can run a shell script when your instances bootstraps. It is moreflexible because you can create a skeleton (PHP + libraries) anddownload all software runtime during the boot operation WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 46. THE PROBLEM WITH SOFTWARE MANAGEMENT Use SVN (Subversion) to download the latest version of WordPress Probably is not a good idea use the trunk but you can use tags in order to stay aligned in all VMs svn checkout http://core.svn.wordpress.org/tags/3.5.1/ mywebsite http://codex.wordpress.org/Installing/Updating_WordPress_with_Su bversion Use SVN externals to download your plugins cd mywebsite/wp-content/plugins/ svn propset svn:externals akismet http://plugins.svn.wordpress.org/akismet/tags/2.5.7/ svn up Create/Download your WordPress configuration file during VM bootstrap WORDCAMP BOLOGNA - 9 FEB 2013 @WORDCAMPBOLOGNA # WPCAMPBO13 47. HOW WE CAN DOWNLOAD WP AND PLUGINS? If you ran 10 servers execute commands could be hard. You can use tools to run command on a server list Capistrano (Ruby) https://github.com/capistrano/capistrano Fabric (Python) https://github.com/fabric/fabric Use CLOTH for AWS EC2 instances https://github.com/garethr/cloth WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 48. HOW TO UPDATE CONFIGURATIONS RUNTIME?#! /usr/bin/env pythonfrom __future__ import with_statementfrom fabric.api import *from fabric.contrib.console import confirm EC2 instances are dynamic with dont know address, for that reason we can use taggingfrom cloth.tasks import *system to execute commands on a group of instancesenv.user = "root"env.directory = /mnt/wordpress fab nodes:"^production.*" tailenv.key_filename = [/home/walter/Amazon/wp-cms.pem]Execute the tail command on all instances with a name that starts with [email protected] reload():Eg."Reload Apache configuration" production.web-1run(/etc/init.d/apache2 reload) production.log [email protected] tail():"Tail Apache logs"run(tail /var/log/syslog) WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 49. EXAMPLE OF FABRIC USAGE WITH CLOTH We create and destroy instances thanks to alarms but when we close an instance we lose immediately all apache logs (or equivalent) How we can manage logs? The simplest way is to use Rsyslog clusters Rsyslog is an opensource software that forwarding log messages in an IP network Rsyslog implement the basic syslog protol That means that we can configure apache logs to sysloginstead using normal text files. In this way we can collect all logs in one group of VM andwork on these files later thanks to other technologies. WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 50. ALSO LOG MANAGEMENT IS NOT SIMPLE Collecting logs is not the latest operation because you have to analyse and reduce information Move logs to S3 bucket Time based Analyze logs with Hadoop Map Reduce on the cloud with Elastic Map Reduce service(EMR) Use script languages on top of Hadoop in order tosimply the log analysis HIVE Data Warehouse infrastructure (datasummarization) Pig High level platform for creating MapReduce program WORDCAMP BOLOGNA - 9 FEB [email protected] # WPCAMPBO13 51. The end? Really you want the Red pill?find out just how deep the rabbit hole goes 52. RELATORIGabriele MitticaWeb: www.gabrielemittica.comTwitter: @gabrielemitticaWalter Dal MutWeb: www.walterdalmut.comTwitter: @walterdalmutWORDCAMP BOLOGNA - 9 FEBBRAIO 2013 @WORDCAMPBOLOGNA # WPCAMPBO13


Recommended