Load Balancing, Failover and Scalability with ColdFusion

transcript

Load Balancing, Failover and Scalabilitywith ColdFusion

PresentersMike Collins, Sr. ConsultantPriyank Shrivastava Sr. Product Consultant

Today’s Agenda

Modeling for ScalabilityColdFusion and Web Server

ConfigurationsColdFusion ClusteringLoad Balancing with ConnectorUsing the Tomcat Connector AdminSession Management StrategiesDemo Load Balancing and Failover

Application Scaling Techniques – Physical and Virtual

Physical Server Scaling Vertical Horizontal Distributed

VM Server Scaling Managed Experience Vertical Horizontal Distributed

Popular Environments VMWare ESX Microsoft HyperV Citrix XenServer

Business Critical Apps need to be designed to provide the end user uninterrupted service. We will be looking look at building an environment to support Business Critical Apps

Modeling for Scalability

Physical Servers Techniques

Many instances on each peer

Greatest utilization of hardware

Most cost-effective Need to consider

hardware failures

Vertical Horizontal

Peers are on other physical servers

Protects against hardware failures

Effective management of workloads

higher costs

Distributed

Peers are located across physical servers

Each server has multiple CF Instances

Protects against hardware failures

More utilization of available CPU

Effective management of workloads

Virtual - VM Scaling Techniques

Most Popular Scaling Model Managed environments perfect for

creating new servers Easily create reusable VM images VMs become the building blocks

All in One - Web server and Application Server

Web server VMs - Application Pool VMs Popular Environments

VMWare, HyperV

VM Building Blocks

Need to establish your building block VMs OS CF Version and settings Web server JVM Settings

Establish Vertical scalability Scale Up Test each VM

Establish how many CF Instances can you run on each VM Establish how large a VM would be in terms of vCPU and memory, JVM settings

Scale Out Test How many VMs do you need to meet your Response Time SLAs

without reaching 70%-80% saturation of CPU? Create a testing environments where you can get accurate

numbers

VM Building Blocks – Know the limitations

Test a single building block find it’s limits so you know critical metrics to determine how many vMs you need to serve expected load

Change the building blocks JVM Heap, vCPUs, and retest limits tp find a sweet spot giving you the best throughput and performance

Sample VM Topologies

Let’s take a look at some sample topologies

Department Based Topology with Shared Data API

Finance

Load Balancer Loan Processing \PDF Generation

REST Application Tier

REST DATA Tier- DSNs

HTML\JSON\JDBC

Database

HTML\JSON\JDBC

Company Wide Topology

Account Management

Load Balancer

HTML\JSON\JDBC

Database

eCommerce SiteEnd Users

Back End

CRM Apps

Utilities

Event Gateways

Frontend eCommerce with separate Web Server Pool

Load Balancer

Database

Web Servers

Shoppers CRM Apps

eCommerce

Event Gateway

Orders

Corp Data Access

JDBC REST

Database

Client App – API Based Topology

Single Page Apps REST API base applications JavaScript Apps Mobile API Access

JSON\HTML

Load Balancer(s)

Application Failover Factors

Application Failover Three servers is the ideal number

for a business critical app You can take one out of the pool

and still have 2 two remaining live.

Your tolerance will vary Continuous Monitoring and

Alerts Monitoring is part of failover Often will stop or shorten an

outage The CF Unresponsive thread alert

with stack traces is a tool

Syncing your Code and Resource Files

Peers will need access to uniform set of source code and resource files Common ways

Each peer can synch from a common network drive Build script can move files to all the servers

Always possibility of something getting out of sync Need to consider if code could be sensitive to one server being updated prior to

another Usually not an issue but scenarios could exist

LB, Scaling and Failover by Category

Category OneCost $ One server \ VM 2-3 Instances of

CF Load Balanced

with Tomcat Connector

Shared code base

Session might be lost on failover

Email Alerts

Category ThreeCost $$$ Three servers \

VMs 2-3 Instances per

server Load Balance

with Hardware LB Code is synched

or build script pushes to servers

Seamless session state on failover

Cached Content Full monitoring

Category FourCost $$$$ Six or more servers \

VMs 2-3 Instances per

application Load Balance with

Hardware LB Multi location VM replication for

backup Code is synched Seamless session

state on failover Dist Cached

Content Full Monitoring

Category TwoCost $$ Two servers \ VMs 2-3 Instances per

server Load Balanced with

Tomcat Connector or Hardware Load Balancer

Code is synched or build updates both servers

Smaller chance of lost sessions on failover

Email Alerts

Load Balancing Scalability and Failover with ColdFusionPriyank Shrivastava| Sr. Product ConsultantLoad Balancing, Scalability and Failover with ColdFusionPriyank Shrivastava | Sr. Product Consultant

Distributed Environment

Reliability

Availability

Scalability

Advantages of Distributed Environment

How to setup Distributed Environment?

Distributed Environment Single Server Setup

Copy these folders from ColdFusion directory to Webserver machine

How to setup Distributed Environment

Required directories copied

Install Microsoft Visual C++ Redistributable (2008 and 2012 32 bit/64 bit)

Command to create the connector

Required directories copied

Install Microsoft Visual C++ Redistributable (2008 and 2012 32 bit/64 bit)

How to setup Distributed Environment with ColdFusion Cluster?

Distributed Environment with Cluster

Distributed Environment with Cluster ColdFusion Cluster setup

Distributed Environment with Cluster

Copy these additional XMLs from ColdFusion config folder to Webserver machine

How to setup Distributed Environment in a cluster

Command to create the connectorHow to setup Distributed Environment in a cluster

Workers Properties file

Connector Properties

Connector port

Check the ColdFusion Handlers

Check the ISAPI Filter

Check the Connector folder and files

Check if the ISAPI_REDIRECT.dll is correct

Check List

Microsoft Visual C++ Redistributable (2008 and 2012 32 bit/64 bit)

Port should be open

Required files needs to be copied

Things to remember

Reference links

http://blogs.coldfusion.com/post.cfm/setting-up-coldfusion-in-distributed-envionment

https://helpx.adobe.com/coldfusion/configuring-administering/using-multiple-server-instances.html

Thank you

cfsup@adobe.com

cf.install@adobe.com

Priyank Shrivastava.

priyank_adobe

Clustering and Load Balancing Decision Points

Do you have a Load Balancer routing traffic, monitoring connectivity?

Do we need to share state across peers?

Do you have any tolerance for throwing errors on failover?

Can the server go down for 15 minutes from time to time?

Each application will have different requirements

Clustering and Load BalancingColdFusion

Applications

ColdFusion Clustering Basics

Clustering Basics Allows for 2 or more peer servers to work together Peers can be local or remote Uses Multicast to auto recognize when peers are up or down Can be defined in the ColdFusion Administrator When the Web Server Connector is setup it builds the load balancing rules based on

the cluster you have defined. Additional Properties and settings are located in these files

Cluster.xml Server.xml

Getting Started with Clustering – Creating Instances

Uses the cfusion instance to seed the new instance

Do no use cfusion inside a defined cluster Use cfusion as your baseline instance

Setup any default datasources and settings in cfusion

Settings are brought over Make sure JVM arguments do not

conflict such as port numbers on any custom settings

Review ports in {instancename}/runtime/config/server.xml

JVMRoute Property should be unique in server.xml

Requires CF Enterprise

ColdFusion Cluster Setup

What does it do when I setup a cluster?

When you create a cluster you are building the Network support to allow the peers to auto discover each other

Needed for in memory session replication and failover

Settings are added to the server.xml file

It does not setup Load Balancing

Load Balancing is setup when the connector is defined

Cluster Setup – Remote Instances are supported

Default 8985/8987 (in jetty.xml)

Must be unique for session affinityWeighted round robin

ColdFusion Clustering and Load Balancing

Clustering Auto discovery of peers required to

support in memory session replication

Edits the Server.xml with multicast settings so peers can have a backchannel to communicate

Clustering setup in the CF Admin does nothing for Load Balancing

Essentially the Cluster Setup in the CF Admin gives you the support for in memory session replication

Load balancing is done when you define the web server connector

But wait Adobe does not recommend in memory session replication

Do I even need to setup a cluster if I just want load balancing No you don’t However you might want to use it to

initially setup your connector files to get started with load balancing

Once your connector settings are setup to load balance you can delete your cluster in the CF Admin

Load balancing does not require CF Enterprise just make sure each server has it’s own Standard license.

ColdFusion Connector Setup for a Cluster

Run wsconfog.exe or remote command

Once completed you will get a numbered directory with configuration

Your load balancing is now defined in the workers.properties file

Connector Setup for Cluster – workers.properties

You will get a worker.properties that looks something this

Hosts Can be local or remote The CF Server needs a Connector defined in

the server.xml CF Servers can define more then one

connector as well Status Worker

Allows you to monitor and edit peers Mount to a protected url path Restarting website in IIS will reload connector

settings CF 11 Updater 4 and below- Make sure to

save any settings because this file is replaced if you update rebuild the Connector. U5 now backs up properties files if you update the connector.

worker.list=mycluster,statusworker.mycluster.type=lbworker.mycluster.balance_workers=peer1,peer2,peer3worker.mycluster.sticky_session=falseworker.balancer.method=Bworker.peer1.type=ajp13worker.peer1.host=localhostworker.peer1.port=8013worker.peer1.max_reuse_connections=250worker.peer1.lbfactor=1worker.peer1.route=peer1worker.peer2.type=ajp13worker.peer2.host=localhostworker.peer2.port=8016worker.peer2.max_reuse_connections=250worker.peer2.lbfactor=1worker.peer2.route=peer2worker.status.type=statusworker.status.mount=/admin/status/jk

Changing the Workers.Properties

You can ecit worker properties anytime

Add Peers Disable Peers Chance Load Balancer settings Simply Restart web site reload

the connector which will bring in new settings.

Using the Connector Admin

Built into the Tomcat Connector

Options Around Session Management

Sticky Session – Server Affinity Must turn on J2EE Sessions Easy setup - Works great Must have some tolerance for errors on a failover Needs in memory session replication to get seamless failover, however session failover has limitations and is not

recommended by Adobe CF team Client Scope

Setup client scope to be stored in database Great solution for seamless failover Great solution for round robin load balancing Need to probably write your own purge routine to best control timing of the purges and to control locking

Distributed Caching – EHCache Great solution similar to Client Scope just different mechanism Can take the work off your database Needs some expertise to setup and maintain Check out the EHCache Session from CFSummit 2015 about some great new features

Custom Solution Nothing to stop you from writing your own solution into the database, shared file system etc

Hardware LB vs Software LB

Hardware LB is the most common on scalable

architectures Network Probe determines outage Also supports HTTP probe which can call a CF

page to determine database health etc

Software LB offers low cost option to seamlessly take out peers from the pool Tomcat Connector Load Balancing HAProxy

Demo Application Failover

What we will see in the demo Create 3 peers that are load balanced with worker.properies

settings Create Load on Web server(s) Application is setup to use Client scope for user information Client scope is defined to go to the database Gracefully take server out of pool using Tomcat Connector

admin Restart IIS website to bring in new connector settings Watch peers pick up traffic Look to see of server threw any errors Place server back into pool Restart IIS website to bring in pool changes made with

Tomcat connector admin Peers pick up traffic

Application Server Failover

Questions

Load Balancing, Failover and Scalability with ColdFusion

Technology