Date post: | 15-Feb-2017 |
Category: |
Software |
Upload: | mcollinscf |
View: | 550 times |
Download: | 3 times |
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Load Balancing, Failover and Scalabilitywith ColdFusion
PresentersMike Collins, Sr. ConsultantPriyank Shrivastava Sr. Product Consultant
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Today’s Agenda
Modeling for ScalabilityConfiguring Web ServersColdFusion ClusteringLoad Balancing with ConnectorUsing the Tomcat Connector AdminSession Management StrategiesDemo Load Balancing and Failover
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Application Scaling Techniques – Physical and Virtual
Physical Server Scaling Vertical Horizontal Distributed
VM Server Scaling Managed Experience Vertical Horizontal Distributed
We will be looking look at building an environment to support Business Critical Apps
Business Critical Apps need to be designed to provide the end user uninterrupted service.
Same concepts exist for both Physical and Virtual Scaling
Modeling for Scalability
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Physical Servers Techniques
Many instances on each peer
Greatest utilization of hardware
Most cost-effective Need to consider
hardware failures
Vertical Horizontal
Peers are on other physical servers
Protects against hardware failures
Effective management of workloads
higher costs
Distributed
Peers are located across physical servers
Each server has multiple CF Instances
Protects against hardware failures
More utilization of available CPU
Effective management of workloads
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 5
Virtual - VM Scaling Techniques
Most Popular Scaling Model Managed environments perfect for
creating new servers Easily create reusable VM images VMs become the building blocks
All in One - Web server and Application Server
Web server VMs - Application Pool VMs Popular Environments
VMWare, HyperV
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 6
VM Building Blocks
Need to establish your building block VMs OS CF Version and settings Web server JVM Settings
Scale Up Test each VM Establish how many CF Instances can you run on each VM Establish how large a VM would be in terms of vCPU and
memory, JVM settings
Scale Out Test How many VMs do you need to meet your Response Time SLAs
without reaching 70%-80% saturation of CPU? Create a testing environments where you can get accurate
numbers
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 7
VM Building Blocks – Know the limitations
Test a single building block find it’s limits so you know critical metrics to determine how many vMs you need to serve expected load
Change the building blocks JVM Heap, vCPUs, and retest limits and find a sweet spot giving you the best throughput and performance
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 8
Sample VM Topologies
Let’s take a look at some sample topologies
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 9
Department Based Topology with Shared Data API
Finance
Load Balancer Loan Processing \PDF Generation
REST Application Tier
REST DATA Tier- DSNs
HTML\JSON\JDBC
Database
Sales
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 10
HTML\JSON\JDBC
Company Wide Topology
Account Management
Load Balancer
REST Application Tier
REST DATA Tier- DSNs
HTML\JSON\JDBC
Database
eCommerce SiteEnd Users
Back End
CRM Apps
Utilities
Event Gateways
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 11
Frontend eCommerce with separate Web Server Pool
Load Balancer
Database
Web Servers
Shoppers CRM Apps
eCommerce
Event Gateway
Orders
Corp Data Access
JDBC REST
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 12
Database
Client App – API Based Topology
Single Page Apps REST API base applications JavaScript Apps Mobile API Access
JSON\HTML
REST Application Tier
REST DATA Tier- DSNs
Load Balancer(s)
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 13
Application Failover Factors
Application Failover Three servers is the ideal number
for a business critical app You can take one out of the pool and
still have 2 two remaining live. Your tolerance will vary
Great tool for Maintenance Gracefully add and remove peers
Continuous Monitoring and Alerts Monitoring is part of failover Often will stop or shorten an outage The CF Unresponsive thread alert
with stack traces is a tool
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Syncing your Code and Resource Files
Peers will need access to uniform set of source code and resource files Common ways
Each peer can synch from a common network drive Build script can move files to all the servers
Always possibility of something getting out of sync Need to consider if new code could be sensitive to one server being updated prior to
another Usually not an issue but scenarios could exist
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 15
LB, Scaling and Failover by Category
Category OneCost $ One server \ VM 2-3 Instances of
CF Load Balanced
with Tomcat Connector
Shared code base
Session might be lost on failover
Email Alerts
Category ThreeCost $$$ Three servers \
VMs 2-3 Instances per
server Load Balance
with Hardware LB Code is synched
or build script pushes to servers
Seamless session state on failover
Cached Content Full monitoring
Category FourCost $$$$ Six or more servers \
VMs 2-3 Instances per
application Load Balance with
Hardware LB Multi location VM replication for
backup Code is synched Seamless session
state on failover Dist Cached
Content Full Monitoring
Category TwoCost $$ Two servers \ VMs 2-3 Instances per
server Load Balanced with
Tomcat Connector or Hardware Load Balancer
Code is synched or build updates both servers
Smaller chance of lost sessions on failover
Email Alerts
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Load Balancing Scalability and Failover with ColdFusionPriyank Shrivastava| Sr. Product ConsultantLoad Balancing, Scalability and Failover with ColdFusionPriyank Shrivastava | Sr. Product Consultant
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Reliability
Availability
Scalability
Advantages of Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
How to setup Distributed Environment?
Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Distributed Environment Single Server Setup
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Copy these folders from ColdFusion directory to Webserver machine
How to setup Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Copy these folders from ColdFusion directory to Webserver machine
How to setup Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Copy these folders from ColdFusion directory to Webserver machine
How to setup Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Required directories copied
Install Microsoft Visual C++ Redistributable (2008 and 2012 32 bit/64 bit)
Command to create the connector
How to setup Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Command to create the connector
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Required directories copied
Install Microsoft Visual C++ Redistributable (2008 and 2012 32 bit/64 bit)
Command to create the connector
How to setup Distributed Environment
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
How to setup Distributed Environment with ColdFusion Cluster?
Distributed Environment with Cluster
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Distributed Environment with Cluster ColdFusion Cluster setup
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Distributed Environment with Cluster
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Copy these additional XMLs from ColdFusion config folder to Webserver machine
How to setup Distributed Environment in a cluster
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Command to create the connectorHow to setup Distributed Environment in a cluster
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Workers Properties file
Connector Properties
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Connector port
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Check the ColdFusion Handlers
Check the ISAPI Filter
Check the Connector folder and files
Check if the ISAPI_REDIRECT.dll is correct
Check List
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Microsoft Visual C++ Redistributable (2008 and 2012 32 bit/64 bit)
Port should be open
Required files needs to be copied
Things to remember
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Reference links
http://blogs.coldfusion.com/post.cfm/setting-up-coldfusion-in-distributed-envionment
https://helpx.adobe.com/coldfusion/configuring-administering/using-multiple-server-instances.html
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Thank you
Priyank Shrivastava.
priyank_adobe
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 44
Clustering and Load Balancing Decision Points
Do you have a Load Balancer routing traffic, monitoring connectivity?
Do we need to share state across peers?
Do you have any tolerance for throwing errors on failover?
Can the server go down for 15 minutes from time to time?
Each application will have different requirements
Clustering and Load Balancing
ColdFusion Apps
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
ColdFusion Clustering Basics
Clustering Basics Allows for 2 or more peer servers to work together Peers can be local or remote Uses Multicast to auto recognize when peers are up or down Can be defined in the ColdFusion Administrator When the Web Server Connector is setup it builds the load balancing rules based
on the cluster you have defined. Additional Properties and settings are located in these files
Cluster.xml Server.xml
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 46
Getting Started with Clustering – Creating Instances
Uses the cfusion instance to seed the new instance
Do no use cfusion inside a defined cluster
Use cfusion as your baseline instance Setup any default datasources and
settings in cfusion Settings are brought over Make sure JVM arguments do not
conflict such as port numbers on any custom settings
Review ports in {instancename}/runtime/config/server.xml
JVMRoute Property should be unique in server.xml
Requires CF Enterprise
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 47
ColdFusion Cluster Setup
What does it do when I setup a cluster?
When you create a cluster you are building the Network support to allow the peers to auto discover each other
Needed for in memory session replication and failover
Settings are added to the server.xml file
It does not setup Load Balancing
Load Balancing is setup when the connector is defined
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Cluster Setup – Remote Instances are supported
Default 8985/8987 (in jetty.xml)
Must be unique for session affinityWeighted round robin
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 49
ColdFusion Clustering and Load Balancing
Clustering Auto discovery of peers required to
support in memory session replication
Edits the Server.xml with multicast settings so peers can have a backchannel to communicate
Clustering setup in the CF Admin does nothing for Load Balancing
Essentially the Cluster Setup in the CF Admin gives you the support for in memory session replication
Load balancing is done when you define the web server connector
But wait Adobe does not recommend in memory session replication
Do I even need to setup a cluster if I just want load balancing No you don’t However you might want to use it to
initially setup your connector files to get started with load balancing
Once your connector settings are setup to load balance you can delete your cluster in the CF Admin
Load balancing does not require CF Enterprise just make sure each server has it’s own Standard license.
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 50
ColdFusion Connector Setup for a Cluster
Run wsconfog.exe or remote command
Once completed you will get a numbered directory with configuration
Your load balancing is now defined in the workers.properties file
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 51
Connector Setup for Cluster – workers.properties
You will get a worker.properties that looks something this
Hosts Can be local or remote The CF Server needs a Connector defined in
the server.xml CF Servers can define more then one
connector as well Status Worker
Allows you to monitor and edit peers Mount to a protected url path Restarting website in IIS will reload connector
settings CF 11 Updater 4 and below- Make sure to
save any settings because this file is replaced if you update rebuild the Connector. U5 now backs up properties files if you update the connector.
worker.list=mycluster,statusworker.mycluster.type=lbworker.mycluster.balance_workers=peer1,peer2,peer3worker.mycluster.sticky_session=falseworker.mycluster.method=Bworker.mycluster.retries=10worker.mycluster.retry_interval=200
worker.peer1.type=ajp13worker.peer1.host=localhostworker.peer1.port=8013worker.peer1.max_reuse_connections=250worker.peer1.lbfactor=1worker.peer1.route=peer1
<more peers>
worker.status.type=statusworker.status.mount=/admin/status/jk
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 52
Changing the Workers.Properties
You can edit worker properties anytime Add or Remove Peers Change Load Balancer settings Restarting the web site will reload the
connector which will bring in the new settings.
Test this scenario in your environment Could throw 404s for a second
depending on setup
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 53
Using the Connector Admin
Built into the Tomcat Connector
Changes go live immediately
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Options Around Session Management
Sticky Session – Server Affinity Must turn on J2EE Sessions Easy setup - Works great Must have some tolerance for errors on a failover Needs in memory session replication to get seamless failover, however session failover has limitations and is not
recommended by Adobe CF team Client Scope
Setup client scope to be stored in database Great solution for seamless failover Great solution for round robin load balancing Need to probably write your own purge routine to best control timing of the purges and to control locking
Distributed Caching – EHCache Great solution similar to Client Scope just different mechanism Can take the work off your database Needs some expertise to setup and maintain Check out the EHCache Session from CFSummit 2015 about some great new features
Custom Solution Nothing to stop you from writing your own solution into the database, shared file system etc
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential. 55
Hardware LB vs Software LB
Hardware LB is the most common on scalable
architectures Network Probe determines outage Also supports HTTP probe which can call a CF
page to determine database health etc
Software LB offers low cost option to seamlessly take out peers from the pool Tomcat Connector Load Balancing HAProxy
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Demo Application Failover
What we will see in the demo Create 3 peers that are load balanced with worker.properies
settings Create Load on Web server(s) Application is setup to use Client scope for user information Client scope is defined to go to the database Gracefully take server out of pool using Tomcat Connector
admin Restart IIS website to bring in new connector settings Watch peers pick up traffic Look to see of server threw any errors Place server back into pool Restart IIS website to bring in pool changes made with
Tomcat connector admin Peers pick up traffic
Application Server Failover
© 2015 Adobe Systems Incorporated. All Rights Reserved. Adobe Confidential.
Questions