ConfidentialConfidential

ApplicationPerformance Testing

and Tuning on theCloud

Beyond Cloud 101Seminar Series 2010

ConfidentialConfidential

Welcome to Summer Webinar Series

CSS Corp and Amazon Web Servicespresent Beyond Cloud 101 Seminar Series:

• June 29, 2010: Achieving Success with Clouds• August 16, 2010: Seamless Design and

Deployment of Hybrid Clouds• September 28, 2010: Application Performance

Tuning on the Cloud• October 26, 2010: Integrated Cloud Provisioning,

Operations and Management

ConfidentialConfidential

Today’s Installment

Application Performance Testing and Tuning onthe Cloud

This webinar will focus on utilizing the cloud as aperformance testing platform. Topics covered willinclude:

1. Performance Testing on the Cloud2. Cloud Performance Tuning Lessons3. Cloud Based Testing as a Service

ConfidentialConfidential

Presenters

Ahmar Abbas – SVP, Global InfrastructureManagement at CSS Corp

Anil Parambath – Vice President,Performance Testing at CSS Corp

Jay Bannerjee – Lead Global Alliances atAmazon Web Services

Matt Tavis – Principal Solutions Architectat Amazon Web Services

ConfidentialConfidential

Cloud Market: Is It Really This Big?

Focus Area % of RespondentsHybrid Clouds 26%Private Clouds -IaaS - Want tobuild my own 16%Public Clouds -IaaS / PaaS -Planning tomove to avendor 8%Still Evaluating 51%

100%

Time Horizon % of Respondents3 Months or Less 27%

4 to 6 Months 13%

More than 6 Months 14%Unknown 46%

100%

Source: CSS survey of registrants 6/10-9/10

The global cloud computing market isexpected to grow at a compoundedannual rate of 28 percent from $47billion in 2008 to $126 billion by 2012

Source: IBM – June 2010

Worldwide cloud services will accountfor $68.3 billion in revenues in 2010,up 16.6 percent from $58.6 billion lastyear. The research firm expects thatnumber to climb to $148.8 billion in2014.

Source: Gartner – June 2010

ConfidentialConfidential

CSS Cloud Enablement Tools**

CSS HybridFox for Private/ Public Cloud Management

CSS CloudBuddy Personal – A tool to manageAmazon S3 cloud storage

CSS CloudBuddy Analytics for analyzing Cloudservices logs

CSS CloudSmart Tool forprovisioning

** Available under GNU license at code.google.com

ConfidentialConfidential

CSS Cloud Platforms

7

CSS CloudValut – An email archiving Platform

CSS CloudBuddy Enterprise for extending theCloud storage to the Enterprise Users

© 2010 CSS Corporation | Confidential

CSS Instance and Application MonitoringPlatform

CSS CloudTestGo – Load Generation Platform

ConfidentialConfidential

Cloud Services and Solution

Cloud Monitoring and

Management Services

Cloud Monitoring and

Management Services

Cloud Design, Orchestration,

Automation and Testing

Services

Cloud Design, Orchestration,

Automation and Testing

Services

Cloud Support ServicesCloud Support Services

© 2009 CSS Corporation

AMI Development

Provisioning Systems

Deployment Automations

Backup / Restore

Automation

Application Optimization

Application &

Infrastructure Testing

Peak Load Management

Instance management

Application specific monitoring

and management

Auto-scaling

Auto deployment of agents

Application specific agent

configuration

Agent-less monitoring

24 x 7 NOC and Tier 1, 2, 3

delivered from US & India

SLA Management

24 x 7 Customer Support services

24 x 7 Technical Assistance

services

Multi-lingual delivered from US,

Poland, Philippines and India

Multi-channel including voice,

web, chat and email support

Knowledge Engineering

SLA Management

ConfidentialConfidential

Cloud BasedPerformance Testing

Anil Parambath

ConfidentialConfidential

Opportunities & Challenges for Development andTesting

On-demand build/sandbox hardware

Collaboration/Shared Test labs

Test Automation

Unlimited hardware/Global locations for Loadgeneration

Instant scalability for cloud apps

ConfidentialConfidential

Performance Testing Challenges - Traditional

Environment• Dedicated performance testing environment

• Requirement to test on multiple environments

• Setup & maintenance of a production-like or scaled-down environment

• Hardware for load generation infrastructure especially at high loads

Simulating Internet cloud• Most users come over the internet, from all over the globe

• Testing internet facing network infrastructure

Tools & Licensing• High costs involved in procurement & support

• License costs increase as virtual user count increases

• Protocol bundles & plug-ins cost over-and-above the standard tool cost

ConfidentialConfidential

Performance Testing Cloud Applications

Testing instant scalability and downscaling

Varying hardware characteristics

Managing large load tests, tracking data

Test Data management

Testing Web 2.0 applications

ConfidentialConfidential

Environment

• Dedicated performance testing environment

• Test on multiple environments

• Setup of a production-like environment

• Production-like data volumes

• Additional hardware for load generation

infrastructure

• Simulating Global Internet

Cloud based Performance Testing

Tools• High procurement & support costs

• License costs skyrocket with user count

• Exorbitant over-&-above costs

• Managing large load tests

• Analyzing large data sets

© 2009 CSS Corporation

The Environment - Cloud • On-demand access to environments

• Linux / Unix / Windows

• Environment setup

• Pre-built image – customize & store

• Cost – Pay-As-You-Go• Hourly usage of environment

• Global coverage• Simulate load from EC,WC, Europe & Asia

The Tool – CSS JLTT • Concept & Implementation

• CSS Load-Testing tool bundled with cloudmonitoring tools

• Similar feature-set & metrics to marketstandard tools.

• Cost - Pay-As-You-Go

• Data Analysis tool & Tuningrecommendations

ConfidentialConfidential

CloudTestGo - Performance Testing as a Service

CSS•CSS Load-Testing tool bundled withmonitoring tools

› Feature-set similar to Load-Runner,WebLOAD, & other standard tools

› HTTP/HTTPS, WAP, ISO-8583, & CORBA

• Performance Engineers & Analysts

•Cost – Pay as you go

CSS•CSS Load-Testing tool bundled withmonitoring tools

› Feature-set similar to Load-Runner,WebLOAD, & other standard tools

› HTTP/HTTPS, WAP, ISO-8583, & CORBA

• Performance Engineers & Analysts

•Cost – Pay as you go

Cloud• On-demand access to environments

› Linux / Unix / Windows

•Environment setup in minutes

› Pre-built image – customize & store

› Fully setup image – archive & retrieve

•Cost – Pay-As-You-Go

› Hourly usage of environment

Cloud• On-demand access to environments

› Linux / Unix / Windows

•Environment setup in minutes

› Pre-built image – customize & store

› Fully setup image – archive & retrieve

•Cost – Pay-As-You-Go

› Hourly usage of environment

CloudTestGo• Avoid investment in testing hardware

› No hardware procurement/setup delays

› No hardware maintenance costs

•Avoid investment in tool licenses

› No support costs

› No user based costs

• Analysis & recommendations in 2 to 4 days

•Generate very high user loads

CloudTestGo• Avoid investment in testing hardware

› No hardware procurement/setup delays

› No hardware maintenance costs

•Avoid investment in tool licenses

› No support costs

› No user based costs

• Analysis & recommendations in 2 to 4 days

•Generate very high user loads

© 2010 CSS Corporation | Confidential

ConfidentialConfidential

Testing models

LoadGeneration

over theInternet

Testing Cloudbased

Environments

ConfidentialConfidential

CloudTestGo Components

Recorder & Scripting Engine

Graphing & Analysis

Run/Scenario Designer

Run Dashboard

ConfidentialConfidential

Service LevelsBasic

TestingAdvanced

Performance TestingFull Service

PerformanceEngineering

Scripting Level Recorded & Playback

Advanced scripting,behavior simulation,

error handling,integrated with data

loader

Advanced scripting,behavior simulation,

error handling,integrated with data

loader

PerformanceStatistics in report

Client side statisticsClient & Server side

statistics. Element levelperformance data.

Detailed report withClient & Server side

statistics with root causeanalysis and solution

recommendation.

Recommended forload testing network

capabilities

Recommended fortesting transaction

websites.

Performanceimprovementguaranteed*

Notes:1. Target environment and test bed will be setup by the client2. For server side statistics, monitors will need to be deployed at the server side.

* After implementation of the fixes by the client engineering team.

ConfidentialConfidential

Benefits Service Benefits

• Manage large user, multi-location loads

• Analyze client-side & server-side statistics & detail report with tuning

recommendations

• Instantly convert scripts of commercial tools into JLTT scripts

Cost• Pay-As-You-Go - Pricing based on real time testing demands

• Save - On staffing, infrastructure setup & maintenance

• Licensing - Generate 1000s of v-users at a fraction of the cost of other tools

Time

• Reduced dependence on IT for hardware and setup

• On-demand availability - instantly test service uptimes & QoS

Value

• Simulate user experience – mode of access, number of users

• Application Tuning – flexibility to experiment

― Top honors overall 2009― Top vendor for Innovation,Customer Satisfaction 2009

CSS Corp.# 1 ITO technology

support vendor

ahmar.abbas@csscorp.comanil.mp@csscorp.com