Scalability, Availability & Management of WSO2 Carbon

Afkham Azeez

Director of Architecture

WSO2 Inc

1

About Me

• PMC member Apache Axis, Committer Synapse & Web Services

• Member, Apache Software Foundation

• Co-author, Axis2 Web Services

• Director of Architecture, WSO2 Inc

• Blog: http://blog.afkham.org

• Twitter: afkham_azeez

2

Agenda

• A brief look at the WSO2 platform

• Carbon clustering for scalability & availability

• WSO2 Elastic Load Balancer

• Worker node & management node separation

• Deployment synchronization

• Logging

• Lazy loading

3

WSO2 Offerings • WSO2 Carbon

• Full platform of servers for deployment on-premise, in private or public cloud

• Products share a consistent architecture and core platform services (e.g. logging, management, security, identity, caching) through OSGi and the “Carbon Core”

• Includes ESB, AppServer, Data Services, Governance, Identity, Business Process, and more

• WSO2 Stratos

• Platform-as-a-Service (PaaS) Foundation

• Supports running servers as elastic, metered, billed, multi-tenant with self-service

• Including all Carbon Servers, PHP, Jetty, and a growing list through a standard Cartridge model

• WSO2 StratosLive

• http://stratoslive.wso2.com

• WSO2’s Public PaaS

• An instance of Stratos running in the cloud with all Carbon Servers available

4

Consistent Architecture

• Carbon: A consistent set of class-leading enterprise servers

• The same products run either on-premise or in the cloud, single-tenant or multi-tenant

• Utilize the same Carbon core runtime for a seamless experience

• Stratos: A cloud platform for enterprise, hybrid and public deployment

• Extends the deployment to support full self-service, elastic scaling, metering and billing

• Supports Carbon and native server runtimes

• Including Java and non-Java servers such as Jetty and PHP

• Re-uses the same core Carbon architecture to offer core PaaS services including:

• Identity, Logging, File, Relational Storage, Column Storage, Code Deployment, etc

• Both projects share a common set of OSGi modules and a core runtime architecture

5

WSO2 SOA Platform

6

WSO2 Carbon

7

Carbon Clustering

• Membership types

• Static

• Dynamic

• Hybrid

• Membership modes

• Multicast

• Well-known address

8

Static membership

• Predefined members

• Other (non-predefined) nodes cannot join

M2 M1

M3 M4

Static group

N

9

Dynamic membership

• No predefined members

• Nodes can join & leave

M2 M1

M3 M4

Dynamic group

N Join

10

Hybrid membership

• Some predefined (well-known) members, and some dynamic members

• Nodes can join & leave

• Membership revolves around the static members

Hybrid group

N

M2 M1

M3 M4

Static members

M6 M5

M7

Dynamic members

Join (IP,

Port)

11

Multicast based membership management

N

M2

M1

M3

M4

Join (IP,

Port)

12

Well-known Address (WKA) based membership management

Hybrid group

N WK2

WK1

WK3 WK4

Static members

M6

M5

M7

Dynamic members

Join (IP,

Port)

Notify

13

Multicast vs. WKA

Multicast WKA

All nodes should be in the same subnet Nodes can be in different networks

All nodes should be in the same multicast

domain No multicasting requirement

Multicasting should not be blocked

No fixed IP addresses or hosts required At least one well-known IP address or host

required

Failure of any member does not affect

membership discovery

New members can join with some WKA

nodes down, but not if all WKA nodes are

down

Does not work on IaaSs such as Amazon

EC2

IaaS-friendly

Requires keepalived, elastic IPs or some

other mechanism for remapping IP

addresses of WK members in cases of

failure 14

Multicast vs. WKA – how to decide?

• Multicast

• Cluster is going to be setup in a network where multicasting is allowed

• WKA

• Cloud based deployment

• Members are distributed across datacenters & regions

• Multicasting blocked

15

Configuring Clustering

<parameter name="membershipScheme">multicast | wka</parameter>

Parameters related to multicast based membership discovery

<parameter name="mcastAddress">228.0.0.4</parameter>

<parameter name="mcastPort">45564</parameter>

<parameter name="mcastFrequency">500</parameter>

Parameters that describe the local member

<parameter name="domain">wso2.carbon.domain</parameter>

<parameter name="localMemberHost">127.0.0.1</parameter>

<parameter name="localMemberPort">4000</parameter>

<parameter name="properties">…

16

Configuring Clustering Static/well-known members <members> <member> <hostName>127.0.0.1</hostName> <port>4000</port> </member> <member> <hostName>127.0.0.1</hostName> <port>4001</port> </member> </members>

State replication

<stateManager class="org.apache.axis2.clustering.state.DefaultStateManager"

enable=”true">

17

HTTP Session Replication

• catalina-server.xml • <Cluster className="org.wso2.carbon.core.session.CarbonTomcatSimpleTcpCluster"/>

• <Valve className="org.wso2.carbon.tomcat.ext.valves.CarbonTomcatSessionReplicationValve"/>

• web.xml • <distributable/>

18

Elastic Load Balancer 2.0

• New sysadmin-friendly configuration language

• High performance PassThrough transport

• Tenant-aware load balancing

• Ability to dedicate clusters for tenants (private jet mode)

• Improved auto-scaler

• Separate IaaS-aware Cloud controller takes care of spawning new instances on different IaaSs

19

ELB Config

20

Passthru Transport Latency Additional time taken via router/ESB/ELB vs direct

-1.00

1.00

3.00

5.00

7.00

9.00

11.00

13.00

15.00

10 25 50 100 250 500

La

ten

cy (

ms)

Concurrent clients

Axiom

Binary

Relay

Passthru

21

Tenant-aware LB

22

Private Jet mode

• Analogy

• Economy class

• no SLA management, only elasticity

• Business class

• elasticity plus SLA guarantees

• Private Jet

• Guaranteed isolated VMs or machines for a specific tenant

• Still elastically scaled

Private Jet Mode

24

Auto-scaling

25

Management & Worker Node Separation

26

Management & Worker Node Separation

• Proper separation of concerns - management nodes specialize in management of the setup while worker nodes specialize in serving requests to deployment artifacts

• Only management nodes are authorized to add new artifacts into the system or make configuration changes

• Worker nodes can only deploy artifacts & read configuration

• Lower memory foot in the worker nodes because the management console related OSGi bundles are not loaded

• Improved security - management nodes can be behind the internal firewall & be exposed to clients running within the organization only, while worker nodes can be exposed to external clients.

27

Deployment Synchronization

• DepSync allows you to synchronize deployment artifacts across nodes in a cluster

• Also includes meta data synchronization

28

Deployment Synchronization

29

Management Node Config – carbon.xml

<DeploymentSynchronizer>

<Enabled>true</Enabled>

<AutoCommit>true</AutoCommit>

<AutoCheckout>true</AutoCheckout>

<RepositoryType>svn</RepositoryType>

<SvnUrl>http://10.100.3.115/svn/repos/wsas</SvnUrl>

<SvnUser>wso2</SvnUser>

<SvnPassword>wso2123</SvnPassword>

<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>

</DeploymentSynchronizer>

AutoCommit/Checkout = RW

30

Management Node Config – catalina-server.xml

<Connector protocol="org.apache.coyote.http11.Http11NioProtocol” port="9763” proxyPort="8300” …

<Connector protocol="org.apache.coyote.http11.Http11NioProtocol” port="9443" proxyPort="8263” …

31

Worker Node Config - Create worker node

ant createWorker

Buildfile: /Users/azeez/software/wso2carbon-core-4.0.2/bin/build.xml

createWorker:

[input] You are about to delete all the front-end components from the server runtime. Do you really want to proceed? (y, [n])

32

Worker Node Config - Start worker node

./wso2server.sh -DworkerNode

33

WorkerNode Config – carbon.xml

• <!--

• Host name or IP address of the machine hosting this server

• e.g. www.wso2.org, 192.168.1.10

• This is will become part of the End Point Reference of the

• services deployed on this server instance.

• -->

• <HostName>appserver.wso2.com</HostName>

34

Worker Node Config – carbon.xml

<DeploymentSynchronizer>

<Enabled>true</Enabled>

<AutoCommit>false</AutoCommit>

<AutoCheckout>true</AutoCheckout>

<RepositoryType>svn</RepositoryType>

<SvnUrl>http://10.100.3.115/svn/repos/wsas</SvnUrl>

<SvnUser>wso2</SvnUser>

<SvnPassword>wso2123</SvnPassword>

<SvnUrlAppendTenantId>true</SvnUrlAppendTenantId>

</DeploymentSynchronizer>

AutoCheckout = RO

35

Logging

36

Lazy Loading

• Four variants

• Lazy initialization

• Virtual proxy

• Value holder

• Ghost

• Two aspect of lazy loading in Carbon

• Lazy loading global configuration

• Lazy loading artifacts

37

Ghost deployment for services

carbon.xml - <EnableGhostDeployer>true</EnableGhostDeployer>

38

Lazy loading with Ghost Deployment

39

BAM 2.0

TODO

40

BAM 2.0

41

Summary

Availability Scalability Management

State replication Tenant partitioning

Private jet mode

Management nodes

Session replication Ghost deployment Logging infrastructure

Multiple load balancers with

keepalived or DNS RR

BAM 2.0 architecture Deployment synchronization

Autoscaling

Elastic Load Balancer

42

References

Information on tenant-aware load balancing http://sanjeewamalalgoda.blogspot.com/2012/03/tenant-aware-load-balancer-is-upcoming.html

http://sanjeewamalalgoda.blogspot.com/2012/05/tenant-aware-load-balancer.html

Information on long running performance http://kishanthan.wordpress.com/2012/06/27/lazy-loading-pattern-in-wso2-stratos

Lazy loading deployment artifacts http://blog.afkham.org/2011/11/lazy-loading-deployment-artifacts-in.html

Scaling Stratos http://srinathsview.blogspot.com/2012/06/scaling-wso2-stratos.html

http://blog.afkham.org/2011/09/how-to-setup-wso2-elastic-load-balancer.html

http://blog.afkham.org/2011/09/wso2-load-balancer-how-it-works.html

43

Auto-scaler service deployment http://nirmalfdo.blogspot.com/2012/07/autoscaler-service-deployment.html

Auto-scaler service http://nirmalfdo.blogspot.com/2012/07/wso2-autoscaler-service-part-i.html

Pass-through transport – performance comparison http://wso2.org/library/articles/2012/03/wso2-esb-message-transfer-mechanisms-comparative-

benchmarks/

Automatic failover for WSO2 ELB http://gonesimple.org/2012/09/24/automatic-fail-over-for-wso2-elb/

44

Questions?

45

http://www.flickr.com/photos/oberazzi/