Achieve Cloud-Scale by implementing Elastic Load Balancing, Asset Deployment,

and Synchronization

Chris Haddad Vice President Technology Evangelism

Follow me on Twitter @cobiacomm

December 2012

Achieve Cloud-Scale

Agenda

• Scale Objectives

• Architecting for Scale

• Elastic Load Balancing

• Asset Deployment Synchronization and Optimization

• Getting started

Resource Tier

Share Business Capabilities at Scale

Functional Role

Client Tier

Integration Services

Integration Services Resource Services

Functional Role

Presentation and Mashups

Functional Role

Functional code

Presentation Role

Presentation and Mashups

Presentation and Mashups

Resource Services

Private Applications

Public Cloud Services

Business Proces

Business Process

Business Process Business Process and

Business Rules

Scale across Clouds

Mobile Application

Application Portfolio

Hybrid Platform Services

Presentation Services

Resources Services

Functional Services

Business Process and

Rules Services

Integration Services

Composite Application

Business Service Portfolio

Integration Services

Cloud API Endpoint

Infrastructure as a Service (IaaS)

SaaS Application

Web application Endpoint

Maintain Availability by Scaling across Regions and Zones Multiple IaaS (hybrid) Deployment

Avail

ab

ilit

y

Co

st

LOWEST

HIGHEST

5

Private cloud (data center)

Zone 1

Zone 2

Amazon EC2

Zone 1

Zone 2

Rackspace Cloud

Zone 1

Zone 2

Scale Challenges (a subset)

• Tenant and service partitioning

– Traffic Routing

• Dynamically registering service instances

– Right-sizing service instances to match demand

• Code deployment into multiple Cloud instances

– Minimizing Cloud instance footprint

Architecting for Scale

Cloud Scale

On-demand

Self-service

Create tenant environment

Deploy Service

Configure Service

Elastic Scalability

Match capacity with demand

Load balancing

Dynamic Clustering

Resource Pooling

Asset deployment

Asset Synchronization

Consumption based Funding

Meter usage

Bill per scale unit

Mission-critical Cloud Scale requires new architecture components

Availability Scalability Management

State replication Tenant partitioning Private jet mode

Management nodes

Session replication Ghost deployment Logging infrastructure

Multiple load balancers with keepalived or DNS RR

BAM 2.0 architecture Deployment synchronization

Auto-scaling

Elastic Load Balancer

Cloud-aware Reference Architecture

Cloud Platform

Cloud Management

PaaS Manager

Cloud Governance

Identity Management

Asset Repositories and Registries(tenant code, service

endpoints, meta-data, configuration, policies)

Infrastructure as a Service (AWS, Eucalyptus, OpenStack,

CloudStack)

Platform as a Service Run-time Framework (i.e. WSO2 Stratos, Cloud Foundry, RedHat OpenShift)

Service-aware, tenant-aware

Elastic Load Balancer

Stratos Controller

Asset/Code Deployer

Asset/Code Synchronizer

Metering and Billing

Cloud Native Container(s)

Application Platform Services (web server, database, ESB)

Tenant 1 Tenant (n)

Platform as a Service Reference Architecture

Carbon ESB

Cartridge

Carbon

App Server

Cartridge

Stratos Controller

Message Service

Logging Service

Security Service

Registry Service

Relational Data

Service

Elastic Load

Balancer

Cloud Controller/ Auto-scaler

Artifact Distribution Controller

Deployment Synchronizer

Management Console

Column Storage Service

File Storage Service

Task Mgmt Service

Billing Service

Other Carbon

Cartridges

PHP

Cartridge

Any

Pluggable

Cartridge

Infrastructure Cloud (EC2, vmWare, Rackspace, OpenStack, Eucalyptus, etc)

Load Monitor

IaaS

Pa

aS

Fo

un

da

tio

n

aP

aa

S

iPa

aS

Dynamic Instance Clustering

Hybrid group

N WK2

WK1

WK3 WK4

Static members

M6

M5

M7

Dynamic members

Join (IP, Port) Notify

Tenant-awareness impacts Cloud Scaling

Elastic Load Balancer Config

Private Service Clouds

Deployment Synchronization

• Allows you to synchronize deployment artifacts across nodes in a cluster

• Also includes meta data synchronization

Deployment Synchronization • Allows you to synchronize deployment artifacts across

nodes in a cluster

• Also includes meta data synchronization

Management Node Config – carbon.xml

• <DeploymentSynchronizer>

• <Enabled>true</Enabled>

• <AutoCommit>true</AutoCommit>

• <AutoCheckout>true</AutoCheckout>

• <RepositoryType>svn</RepositoryType>

• <SvnUrl>http://10.100.3.115/svn/repos/wsas</SvnUrl>

• <SvnUser>wso2</SvnUser>

• <SvnPassword>wso2123</SvnPassword>

• <SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>

• </DeploymentSynchronizer>

AutoCommit/Checkout = RW

Deployment Optimization

• Four variants

• Lazy initialization

• Virtual proxy

• Value holder

• Ghost

• Two aspect of lazy loading in Carbon

• Lazy loading global configuration

• Lazy loading artifacts

Ghost deployment for services carbon.xml - <EnableGhostDeployer>true</EnableGhostDeployer>

Lazy loading with Ghost Deployment

What is your scale value baseline?

Scaling a Cloud Business

• Measuring Revenue versus Cost

Source: http://giffconstable.com/2009/11/freemium-business-model-template/

Tenant Scaling Method Impacts Value Proposition

• Single Application Tenant per

• Dedicated Application Server Container

Multiple Application Tenants per Shared Application Platform Service

Container

Cloud Topology and Value

Tenant Context

Container

Tenant Partition

Application Solution Tenant Traffic

Tenant-1

AS-1

Tenant-1

ESB-1

Tenant-1

Tenant-2

AS-2

Tenant-2

ESB-2

Tenant-2

Tenant-3

ESB-3

Tenant-3

Five (5) Container Instances, Three Tenants

Cloud Topology and Value

Tenant Context

Container

Service Partition

Application Solution Tenant Traffic

ESB

ESB-1

Tenant-1 Tenant-3

ESB-2

Tenant-2

Web Application

AS-1

Tenant-1 Tenant-2

Three (3) Container Instances, Three Tenants

Cloud Scale = High Availability at a cost

26

Sin

gle

No

de

Pri

mar

y-Se

con

dar

y, s

ingl

e LB

Mu

lti-

no

de

acti

ve

clu

ster

-

Sin

gle

zon

e

Mu

lti-

regi

on

Pri

mar

y-Se

con

dar

y,

wit

h m

ult

iple

LB

s Mu

lti-

zon

e

M

ult

i-Ia

aS

Key Metrics

• Foundation • Time to create new application environment

• Time to redeploy application

• Optimize • Minimum and maximum scale

• Scale frequency (i.e. time to scale up/down)

• Transformation • Time and effort required integrating business process, event

processor – creating a complex app.

• Time and effort required to apply policy across tenant(s)

• Cost to operate application per user or transaction

Quick Start Use Cases

Service level management and elastic scale • Use cases

• Ensure application satisfies consumer demand while maximizing resource utilization • Scale workload processing and increase performance while

minimizing infrastructure spend

• Load test application service • Demonstrate multi-tenant web application

• Key Metrics • Minimum and maximum scale • Scale frequency (i.e. time to scale up/down)

Quick Start Use Cases

Consumption based pricing and billing

• Use cases

• View service logs

• View bill by business value

• Key Metric

• Cost to operate application per user or transaction

Resources

• Try Stratos right now:

– https://stratoslive.wso2.com/

• Read about Stratos:

– http://wso2.com/cloud/stratos/

– Source Download available

• Contact us:

– bizdev@wso2.com

Resources

• Information on tenant-aware load balancing • http://sanjeewamalalgoda.blogspot.com/2012/03/tenant-aware-load-balancer-is-upcoming.html

• http://sanjeewamalalgoda.blogspot.com/2012/05/tenant-aware-load-balancer.html

• Information on long running performance • http://kishanthan.wordpress.com/2012/06/27/lazy-loading-pattern-in-wso2-stratos

• Lazy loading deployment artifacts • http://blog.afkham.org/2011/11/lazy-loading-deployment-artifacts-in.html

• Scaling Stratos • http://srinathsview.blogspot.com/2012/06/scaling-wso2-stratos.html

• http://blog.afkham.org/2011/09/how-to-setup-wso2-elastic-load-balancer.html

• http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html

Resources

• Auto-scaler service deployment • http://nirmalfdo.blogspot.com/2012/07/autoscaler-service-deployment.html

• Auto-scaler service • http://nirmalfdo.blogspot.com/2012/07/wso2-autoscaler-service-part-i.html

• Pass-through transport – performance comparison

• http://wso2.org/library/articles/2012/03/wso2-esb-message-transfer-mechanisms-comparative-benchmarks/

• Automatic failover for WSO2 ELB • http://gonesimple.org/2012/09/24/automatic-fail-over-for-wso2-elb/