Load Test Cloud PoC v 9.1

CLOUD LOAD TEST PoC

Highly Confidential Document

PERFORMANCE ENGINEERING LAB WIPRO TECHNOLOGIES

Proof of Concept Load Testing in the Linux Cloud using Open Source Tools

PROPRIETARY NOTICE This document contains confidential information. In consideration of receipt of this document, the recipient agrees to maintain such information in confidence and to not disclose this information to any person outside of Wipro Technologies without the written permission of the author. This Document contains total number of < > pages inclusive cover page.

Page 1 of 39

HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC


TABLE OF CONTENTS1. EXECUTIVE SUMMARY..............................................................................................3 2. POC DETAILS...........................................................................................................5 2.1 Scope and Objective of the POC...........................................................................5 2.2 About TSung, Grinder and Jmeter in Load Testing Perspective............................5 .....................................................................................................................7 1.1 2.3 Platform Details................................................................................................8 3. TEST DETAILS..........................................................................................................16 3.1 Test Approach & Procedure................................................................................16 Installing Apache JMeter............................................................................................22 3.2 CHALLENGES AND RESOLUTION STEPS...........................................................24 3.3 OBSERVATIONS AND ACHIEVEMENTS...............................................................30 4. CONCLUSION...........................................................................................................37 5. GLOSSARY...............................................................................................................38 6. POC TEAM................................................................................................................39

Page 2 of 39


CLOUD LOAD TEST PoC


1. EXECUTIVE SUMMARYThis Proof of Concept (PoC) is an initiative taken by Wipro Cloud Testing Practice and Performance Engineering Practice in-order to evaluate the capability of the existing open-source load testing tools that can be used to inject or simulate heavy load (as much as 1 million simultaneous users) on web-based applications. This PoC had to be carried out in Linux Based Amazons Cloud Environment. The tools that were selected were TSung, Grinder and Jmeter. As this PoC was first of its kind, it is appropriate to mention that team assigned had to proceed on Hit-and-Try mode at every step of this PoC execution as documentation available was poor. There were humungous challenges that the team had come across in carrying out this successfully. Most of the information is captured in this document so that it can be referred and used in future assignments. The outcome of the PoC is that TSung is able to inject up to 50,000 simultaneous users per agent machine, whereas Grinder and Jmeter is able to simulate up to ____ and 1000 request/sec (throughput) respectively. TSung majorly showed positive results in achieving the objective of this PoC. Below is the TSung Graph that was showcases the simulation of 0.9 million Simultaneous Users with 1 Controller and 11 Agent machines in the Amazons Ubuntu Cloud.

As a result of this PoC, Wipro Testing Services declares that using TSung tool, Wipro customers will be able to avail On-demand Performance Verification services, pay-as-you-go service using a flexible, affordable, hourly / per test usage model.Page 3 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC


Page 4 of 39


CLOUD LOAD TEST PoC


2.

POC DETAILS

The purpose of this PoC is to present the Observations and Conclusions arrived at the Proof of Concept using various Open Source Tools: Grinder, TSung and Jmeter. The intention is to validate each Tools capability to ramp up 1m simultaneous users or simultaneous transactions on a Linuxbased Cloud Environment.

2.1 Scope and Objective of the POCFor each of the Tools such as TSung, Grinder and Jmeter, this POC objective is to:

Benchmark the Scalability per server Benchmark the Horizontal Scalability Arrive at Peak Elasticity and Benchmark the same

2.2 About TSung, Grinder and Jmeter in Load Testing PerspectiveTSUNGTsung is a heavy duty, load-testing tool that is used by organizations to benchmark the performance of their web applications and other large applications. The tool offers several key advantages over traditional injection tools, including: Advantag Description e High Tsung can simulate up to 50,000 simultaneous users per performan physical computer, whereas traditional injection tools can ce usually only simulate up to 1,000 users Ease of Based on XML configuration, Tsung removes the need for use developers to write complex scripts. The tool handles tasks such as managing cookies automatically, so developers can focus on designing the scenarios not managing lines of code Protocol Tsung is protocol independent and can be used to test Jabber, independe XMPP, HTTP, SOAP, LDAP and PostgreSQL servers. The solution nce can also be easily extended to provide support for new protocols Support The testing load can be distributed across a cluster of client for cluster machines, to allow benchmarking to be carried out in largeenvironme scale server environments. In addition, several IP addresses can nts be used on a single machine Greater Developers can use Tsung to simulate different types of users flexibility during the same benchmark and even define the proportion of the various behaviors in the test. This enables organizations to carry out benchmark tests that reflect realistic businessPage 5 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC

Detailed graphical reports

scenarios Tsung enables developers to monitor the target server. The tool produces detailed statistics on performance, errors and target cluster behavior, which can be viewed in graph format. All statistics are produced in real time, so developers and systems administrators can view the results as the test progresses


GRINDERThe Grinder is a load testing framework that makes it easy to run a distributed test across a number of load injector machines. Test scripts are written in Jython, and HTTP scripts can be recorded easily from a browser session. This is a GUI based performance testing tool. The framework is comprised of three types of process or program (that can run test scripts across a number of machines): worker processes, agent processes, and the console. The responsibilities of each of the process types are: Worker processes o Interpret Jython test scripts and perform tests using a number of worker threads Agent processes o Manage worker processes The console o Coordinate the other processes o Collate and display statistics o Script editing and distribution As The Grinder is written in Java, each of these processes is a Java Virtual Machine (JVM).

Page 6 of 39


CLOUD LOAD TEST PoC

Number of simultaneous users = number of (worker processes x agent processes x console)


For heavy duty testing, agent process has to be started on each of several load injector machines. The worker processes they launch can be controlled and monitored using the console. Note that it is recommended to run one agent on each load injector for optimum results. JMETER Apache JMeter could be used to test functional and performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). It is primarily used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. Also, it can be used to perform a functional test on websites, databases, LDAPs, webservices etc.

Page 7 of 39


CLOUD LOAD TEST PoC


1.1

2.3 Platform Details

2.3.1 Pre-requisites: Software and ToolsThe following are the various pre-requisites for effective functioning of TSung, Grinder, Jmeter respectively. Below are the common tools that were required to connect to the cloud machines and perform load tests: Putty to access the controller machine by providing Amazons IP and access key VNC viewer for Remote Desktop Connection The below are the tool specific pre-requisites for its effective functioning:

TSUNG Ubuntu Operating System 9.1 JDK 1.6 Erlang R13B01 TSung 1.3.2 Apache Tomcat Server 6.0 Sample Application (web page of 25KB, 50KB, 100 KB data) Amazon Cloud Watch and HyperHQ - for monitoring Gnuplot and perl5 - for HTML reports Python and mathplotlib - for graphical output with tsung-plotter SSH access to remote machines without password (use a RSA/DSA key without pass-phrase or ssh-agent) - for distributed tests Bash WinSCP to transfer local files to cloud machine and vice versa Note: Tsung is protocol independent and can be used to test Jabber, XMPP, HTTP, SOAP, LDAP and PostgreSQL servers. The solution can also be easily extended to provide support for new protocols.

GRINDER Java5 Install the packages: sudo apt-get -y install openjdk-6-jre zip apache2 Test the Java installation: java -version Test Apache: curl http Jython

Page 8 of 39


CLOUD LOAD TEST PoC


JMETER

Java Environment A fully compliant Java 1.2 environment is required for Apache JMeter to execute.(Note: Hopefully this will be changed back to 1.1 in the final release. Since Apache JMeter uses only standard Java APIs (java.*), please be informed that the JRE might fail to execute because of a class implementation could be missing, or not fully compliant. Java JFC (Swing components)

Swing GUI extensions are required for Apache JMeter.Note: Due to incompatibility in the package names, the precompiled version works only with Swing placed under the package javax.swing which is its final destination found in the Java 2 (tm) platform. JSSE [optional]

JSSE is used to provide an SSL implementation to enable testing of https protocol. Java Compiler [optional]A Java compiler is not needed since the distribution includes a precompiled java binary archive. Note: A compiler is required if you plan to recompile the source code or if you plan to build your own plugins.

2.3.2 PoC LandscapeThe following is landscape that was made available for the PoC through the Amazon Cloud. The below numbers represent the peak number of machines for each tool ecosystem.

Page 9 of 39


CLOUD LOAD TEST PoC


TSUNG Controller VM Configuration #Instances Ag ent AUT 8 ECU, 4 Core, 15 GB 4 AUT 8 ECU, 4 Core, 15 GB 4 AUT 8 ECU, 4 Core, 15 GB 4 Load Balancer Am azon Load Balancer 1 Load Balancer Am azon Load Balancer 1 Load Balancer Am azon Load Balancer 1

26 ECU, 8 Core, 8 ECU, 4 Core, 15 68 GB GB 1 Controller 12 GRINDER Ag ent 8 ECU, 4 Core, 15 GB 4 JMETER Controller Ag ent 20 ECU, 8 Core, 7 GB 3

VM Configuration #Instances

8 ECU, 4 Core, 15 GB 1

VM Configuration #Instances

4 ECU, 2 Core, 7.5 GB 1

2.3.3 Environment details and Connection Steps 2.3.3.1 TSUNGFor this PoC, 20 machines were procured from AWS (Amazon Web Services) Service Provider. Each machine is a high configuration machine which is one of the 4 configurations: 8ECU, 4 Core, 16 GB 26 ECU, 8 Core, 68 GB 20 ECU, 8 Core, 7 GB 4 ECU, 2 Core, 7.5 GBNote: ECU = 1.6 GHz processor.

The following are the machine details that were used for each tools PoC. TSUNG - MACHINE DETAILS: Details VM Configuration Controller 26 ECU, 8 Core, 68 GB Agent 8 ECU, 4 Core, 15 GB AUT 8 ECU, 4 Core, 15 GB

Page 10 of 39


CLOUD LOAD TEST PoC

# of Instances

1

12


4

GRINDER - MACHINE DETAILS: Details VM Configuration # of Instances Controller 8 ECU, 4 Core, 15 GB 1 Agent 8 ECU, 4 Core, 15 GB 4 AUT 8 ECU, 4 Core, 15 GB 4

JMETER - MACHINE DETAILS: Details VM Configuration # of Instances Controller 4 ECU, 2 Core, 7.5 GB 1 Agent 20 ECU, 8 Core, 7 GB 3 AUT 8 ECU, 4 Core, 15 GB 4

Page 11 of 39


CLOUD LOAD TEST PoC


As per these details, each tool was set up as per the figure below and was connected to the cloud machines through the connection steps explained further.

Figure: Cloud Setup and Wipro Backbone access

Connection Steps to Amazon Cloud from Wipro Backbone:Double click on putty.exe file and follow the below steps to do SSH. (SSH is a network protocol that allows data to be exchanged using a secure channel between two networked devices. Used primarily on Linux and Unix based systems to access shell accounts, SSH was designed as a replacement for Telnet and other insecure remote shells, which send information, notably passwords, in plaintext, rendering them susceptible to packet analysis. The encryption used by SSH is intended to provide confidentiality and integrity of data over an unsecured network, such as the Internet.)

Page 12 of 39


CLOUD LOAD TEST PoC


Go to Auth and locate the .ppk file (through the browse button) as shown below.

Under the session tab, provide the IP and the port and click save and then open

Page 13 of 39


CLOUD LOAD TEST PoC


Enter login as root and the Cloud machine is ready to start the tests. (Incase,if it prompts for the password type in Wipro@123)

Page 14 of 39


CLOUD LOAD TEST PoC


Page 15 of 39


CLOUD LOAD TEST PoC


3. TEST DETAILS3.1 Test Approach & Procedure 3.1.1 Tsung: Test Approach

Inorder to carry out a successful load scenario execution, the following are the approach steps that were required to be executed.Set-up the pre-requisites Record the Business Scenarios Update main configuration file with recorded session Ex ecute the TSung script

Monitor the load ex ecution

i n c d l y v r o f s t a p e R

Generate report and Analyze

Conclude

Figure: Test Approach As per the above process, it is required to start the client and the server side machines listed in the Tsung Configuration file. To increase the number of users, it is possible to add more agent machines to the client tag of the Tsung configuration file.

Tsung: End-to-End Test Procedure Once the pre-requisite is set-up, follow the installation instructions for the first time and proceed towards configuring, recording, executing and generating reports for analysis. Installation of Tsung: Download the installation file: TSung-1.3.2.tar.gz Untar the TSung-1.3.2.tar.gz by typing the command tar -zxvfTSung-1.3.2.tar.gz

Page 16 of 39


CLOUD LOAD TEST PoC


Go to TSung-1.3.2 folder and run ./Configure make make install

Recording with https protocol: Set Proxy setting to locahost:8090 in browser Start recording by typing command as:

> TSung-recorder start Open a browser and type http://www.abcabc.com (while login it opens https site) when we click on Signin It displays the address as http://-www.abcabc.com/login.asp

Steps to record the Tsung Script: Set the proxy settings to localhost:8090 Run the command from terminal > tsung-recorder p

start. It takes http as default protocol. (Note: Various protocols that tsung supports are webdav, mysql, postgresql etc) Open a browser and access the URL, do transactions once you are done with the transactions logout Run the command from terminal> tsung-recorder stop To check the status of the recording Run the command from terminal> tsung-recorder status Once you stop the recording it will create a .xml file in /root/.tsung/xxx.xml floder

Below is the format of tags that is used in the main configuration file of tsung: Page 17 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) Gecko/20050513 Galeon/1.3.21 Mozilla/5.0 (Windows; U; Windows NT 5.2; frFR; rv:1.7.8) Gecko/20050511 Firefox/1.0.4


For this PoC scenario, please refer the TSung Main Configuration file (in xml format) below:200request6Agents. xm l.xm l

Running Tsung Script: After recording, goto /home/Ubuntu/.tsung folder Type this command to run the tsung Script: tsung -f xxx.xml start Report Generation: Now, Goto /root/root/.tsung/log/log_timestamp folder Run the script by typing this command Note: It takes time to run the script (as per the time specified in the script)

/usr/lib/tsung/bin/tsung_stats.pl It will generate the graph.html, reports.html and images folder Copy these folder and .html files to your system using WinSCP Open the graph.html and reports.html to analyze the reports To get a comparison report, run the following command tsplot -d

Example: tsplot "scenario1" 20100818-05:00/tsung.log "scenario2" 2010081805:13/tsung.log -d /home/ubuntu/.tsung/report Note: tsung.log files will be generated for each scenario in the log folder located at /root/root/.tsung/log/log_timestamp folder as shown in below figure.

Page 18 of 39


CLOUD LOAD TEST PoC


Figure: Comparison report Note: In some cases it might take a long time to run the script and to check the processes typeps -aef |grep erl (Controller M/c) kill -9 ps -aef|grep beam(Agent Machine) kill -9

3.1.2 Grinder: End-to-End Test ProcedureInstallation of Grinder: First ensure java is installed on the machine where you will be running the scripts. If not, please Install Java Download and unzip the grinder Installer Download the jython2.1 class file Add the grinder.jar to your CLASSPATH variable - Set CLASSPATH = /usr/local/bin/grinder-3.4/lib/grinder.jar: $CLASSPATH Install the jython2.1 class file - Java jython2-1 Setting Classpath: export CLASSPATH=/usr/local/bin/grinder-3.4/lib/grinder.jar: $CLASSPATH java net.grinder.Grinder To Start Grinder Console: Use putty.exe connect to Grinder controller system, if you already logged into different machine ssh -i /home/ravi.pem root@controller IP Start httpd service with this command: service httpd start Start vncserver with this command: vncserver Now log into controller machine with VNCViewer from the terminal type this command: java net.grinder.ConsolePage 19 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC


Note: The Grinder console is the user interface which provides the below features for the grinder. The Grinder console provides an easy way to control multiple testclient machines, display test results, and control test runs. Using the grinder console one can start and restart the test The console is used to coordinate the actions of the worker processes by sending them start, reset, and stop commands. Used to visualize test results in real-time.

To Start Grinder Agent: Use putty.exe grinder agent system, if you already logged into different machine then do ssh -I /home/ravi.pem root@agent IP Goto root folder of Grinder tool for ex: cd /usr/local/bin/grinder3.4/examples Update the console IP in Grinder agents "grinder.properties" file Now start the Agent by typing this command: java net.grinder.Grinder Eg.Page 20 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC

The settings in the grinder.properties file are: grinder.processes=5 grinder.threads=3 grinder.runs=4 grinder.logDirectory=log grinder.numberOfOldLogs=2 #grinder.initialSleepTime=500 #grinder.sleepTimeFactor=0.01 #grinder.sleepTimeVariation=0.005


Figure: Settings in the grinder.properties file To Record in Grinder: Set the proxy settings to localhost: 8001 in the browser in the controller machine only. (access control m/c through VNC) Run the below mentioned command in the controller machine (access controller m/c through VNC) Type the below command from Controller Machine in its terminal. Java net.grinder.TCPProxy console http > Filename.py and hit enter. Open the browser from the Controller machine and start recording; for eg. start browsing through the website it will capture the script. Once you have done with the recording click on Stop button from the window. Now, copy or move the file to >mv Groc_Login.py/home/grinder3.4/examples/Groc_Login.py (where the grinder is installed to view this file in Grinder Console window). Set Grinder.properties file according to the requirements Executing the test scriptsPage 21 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC


To run the grinder test scripts; In the same directory where grinder.properties file is there, you will find your recorded scripts files, if not copy those script files to this directory which you would like to run. For example: httpscript.py- run java net.grinder.Grinder from that directory From the Console Click File > Start Collection and then start processes.

Below is the report screen on the Grinder Console that is shown as an output after execution of the test(s).

3.1.3 Jmeter: End-to-End Test Procedure Installing Apache JMeter If your system meets the above requirements, thanks to the beauty of Java you don't have to do anything else: under the directory /bin you will find thePage 22 of 39 HIGHLY CONF IDENTIAL

CLOUD LOAD TEST PoC

binary application and the system wrappers you need to run Apache JMeter. You may want to go the running instructions to continue. URL: http://www.jajakarta.org/jmeter/1.5.1/en/installing.html


Page 23 of 39


CLOUD LOAD TEST PoC


3.2 CHALLENGES

AND

RESOLUTION STEPS

The following table depicts the high level issues and challenges faced by the PoC team during this assignment.

Tsung: Challenges and Resolution StepsS.n o TSung Challenges Understanding the tool Tsung as very less or ambiguous 1 documentation was available Downloading the Tool on the Linux machine with 2 dependencies Installing Erlang on the local RedHat Linux machine 3 Steps Taken to Resolve Surfed through the internet to collect all sorts of information and tried to proceed with the available steps or came out with self steps As these were pre-requisites, surfed the internet to get the latest version of the software Tsung, Erlang, Gnuplot and Perl5 downloaded and installed Tried the installation version R12B first. Team faced with some error while installing then downloaded the latest version R14A. This gave us some additional error. Finally when we tried to install the version R13B. The Erlang installation went successfully. After installation of the Erlang, while minor issues popped-up still installation of the Tsung tool went successfully with the help of documents

Installing Tsung Tool on the linux machine - local Linux 4 RedHat machine 5 Installing Erlang on the linux First installation of Erlang software, the version used was machine Amazon Cloud 64bit "otp_src_R14A.tar.gz") Linux Fedora machine *we encountered errors for the command: "./configure" configure: error: No curses library functions found configure: error: /bin/sh '/root/TsungFiles/otp_src_R14A/erts/configure' failed for erts Second we tried to install Erlang s/w again ( the version used "otp_src_R13B.tar.gz") *we encountered errors for the command: "./configure" configure: error: No curses library functions found

Page 24 of 39


CLOUD LOAD TEST PoC


configure: error: /bin/sh '/root/TsungFiles/otp_src_R13B/erts/configure' failed for erts Finally we got the Fedora version of the Erlang s/w "otp_src_R12B...". We could successfully install Erlang.

Installing Tsung on the linux Tried to install Tsung 1.3.2 initially. Installation was not happening at all. Tried machine - cloud 64bit to find a 64bit compatible version of Tsung. But, the team couldnt find one. Hence we moved to the 32bit machine in cloud. 6 Linux Fedora machine Installing Erlang and Tsung on Team was successfully able to install Erlang (version 12B.) on the 32bit Fedora the linux machine - Cloud machine. But again the team was not able to install or find Fedora 32bit Linux Fedora machine compatible version of TSung. Hence it was moved to an Ubuntu machine on 7 the cloud. Installing of Erlang & TSung on We were successfully able to install the dependencies on the Ubuntu cloud Ubuntu machine in the cloud machine followed by Erlang & TSung tool 8 environment Exploring the recording With the help of the user manual & self exploration of the tool we were able to options in TSung identify the way in which the recording can happen successfully. 9 Recording the TSung script on The Ubuntu machine on the cloud does not have a GUI/Console to start a the cloud Ubuntu machine TSung recording. Hence we planned to record the TSung script on the local Linux machine copy the same to the cloud machine to run. 10 Recording the TSung script on TSung Tool listens to all the communication through a listening port (8090). the local Linux machine for an After this was set in the Browser connection settings. Session xml files got 11 intranet site. generated successfully. Replaying the TSung script User manual, sample xmls & other documentation on the internet helped to get after enhancements on the the TSung replay xml file in working condition. The replay script should be in a local Linux machine particular format of xml file. 12

Page 25 of 39


CLOUD LOAD TEST PoC


Replaying the new TSung script for sample page after enhancements on Ubuntu cloud machine/Local Linux 13 machine Recording TSung script for a sample page deployed outside the Wipro intranet on the local Linux machine 14 Replaying the new TSung script for sample page after enhancements on Ubuntu cloud machine/Local Linux 15 machine. 16 Enhancing the TSung script to accommodate multiple agents configuration, multiple option setting and other settings. 17 https issue while recording 18 https issue while execution 19 Inability to scale-up the number of simultaneous users 20 TS plot errors: a) While generating a comparison report, faced errors as matplotlib is not installed. b) DISPLAY environment

We are facing the error on trying to replay the script: - Permission denied (public key). (Solution : Need the SSH to be set as passwordless for client/server) We know that the local machine needs proxy setting in the browser to access the external pages, outside the firewall. Hence we faced the limitation of the local machine to record only the scripts for the sites within the firewall. We had to make modification in the TSung configuration file to get it work for the external sites as well. We were facing the error on trying to replay the script as - Kernel pid terminated (application controller).and more such errors These issues was resolved when we reinstalled TSung again on both the machines. This a very challenging task of enhancements because of the complexity in understanding the TSung xml file and adding enhancements to the file. Its also very challenging to make sure the enhancements are working as expected. Involves lot of trial & error method. In config file, parent_proxy attribute is set to TRUE to proceed with recording and instead of https:// we gave http://- Due to above step, it wasnt allowing to execute from backbone and so had to reset the parent_proxy attribute to FALSE and recorded without proxy in the cloud machine directly Load Balancer was used to distribute load to the additional servers and handle the peak requests & get maximum responses a) Installed matplotlib b) resolved by editing the tsplot file located at /usr/bin folder with the following text: import matplotlib matplotlib.use(Agg)

Page 26 of 39


CLOUD LOAD TEST PoC


variable is not set

As we are using Remote Connection to access the machine, need to make the drawing done virtually on the file instead of server.

Grinder: Challenges and Resolution Steps S.n o Grinder Challenges 1 2 Steps Taken to Resolve

Page 27 of 39


CLOUD LOAD TEST PoC


Jmeter: Challenges and Resolution StepsS.n o Jmeter Challenges First to Install & configure the JMeter software in Linux 1 cloud environment Launching JMeter tool through command line arguments 2 in Linux cloud environment Running JMeter users/threads in JMeter Controller 3 machine Using remote agents as load generator machines and 4 connecting to Controller machine RMI communication happening through server ports 5 from controller m/c to agents Getting OutofMemory & GC limit exceeded error in all 3 6 agent machines Generated throughput of 2500 req/sec after using 3 7 agents also Running the Jmeter in each agent machine to act as load 8 generator Every time in cloud environment the agents ip address 9 will be changing Running test in Jmeter GUI mode may affect the 10 throughput value Steps Taken to Resolve Unzip the Jmeter software file and made config settings in properties file Collected the information from jakarta jmeter help file to use the tool properly Created a test plan for execution of users in specific duration Add all the agent machine IP's from cloud environment to remote hosts parameter field in JMeter Properties file which is available in the JMeter software bin folder Default port set as 1099 Increased the JVM heap size in Application server properties file Added the thread stack size in all the agents machine and got the throughput of 3370 req/secs Running the agent with Jmeter server mode Need to update in controller Jmeter software bin folder / jmeter.properties file. In this file under remote_hosts parameter just need to add the agents new ip address every time. Also tried running with non-gui mode and save the Jmeter test log file and verified the throughput value. It's giving the same value as we got when running in GUI mode.

Page 28 of 39


CLOUD LOAD TEST PoC


Recording the script using badboy software and then 11 convert to Jmeter script file Exploring the recording options in both badboy & Jmeter 12 tool Replaying the Jmeter script after enhancements on the 13 local Linux machine 14 Tuning the JVM stack in agents machine

Installed the software in client machine and created the script for jmeter test execution With the help of the user manual & self exploration of the tool we were able to identify the way in with a recording can happen successfully. User manual, sample xmls & other documentation on the internet helped to get the Jmeter script in working condition. Added the thread stack size argument in the Jmeter file which is available in the Jmeter software bin folder

Page 29 of 39


CLOUD LOAD TEST PoC


3.3 OBSERVATIONS

AND

ACHIEVEMENTS

3.3.1 Tsung: Observations and AchievementsAs per the End-to-end Test Procedure and the following configuration data changes in the TSung main configuration file, the results generated are shown in the graph below. S.N o 1 2 3 4 5 XML /Attribute Load Duration Arrival Rate Server Client Number of requests in session Data 10 minutes 50000 Amazon Load Balancer Agents(1-12 Machines) 200

More Details of the scenario is provided below: Main Statistics:

Page 30 of 39


CLOUD LOAD TEST PoC


Response Time:

Network Throughput:

Page 31 of 39


CLOUD LOAD TEST PoC


Simultaneous Users:

Observations: One Agent Server is able to scale upto 45,000 to 60,000 simultaneous users

Page 32 of 39


CLOUD LOAD TEST PoC


Amazons load balancer enabled in increasing the number of users to hit the server by balancing the load across the servers Number of simultaneous users has increased consistently with respect to the increase in the number of agents (from 1 agent to 11 agents) When the agents are increased beyond 11, the number of simultaneous users start to degrade in numbers

Achievements: Successfully utilized the Amazon Cloud Services to install and run tsung scripts even with poor documentation and forum support By using an open-source tool and the Ubuntu environment, huge cost savings is achieved by the organization as this POC requires high-end machines to generate such heavy loads Significantly, TSung was able to reach the number of simultaneous users to 0.9 million simultaneous users

Page 33 of 39


CLOUD LOAD TEST PoC


3.3.2 Grinder: Observations and Achievements These are observations with iperf command and Grinder is utilizing network bandwidth with 1KB file and 25KB File. For 1KB file we are getting 3650 TPS and when we use 25KB file 1750 TPS are observed in Grinder tool. Server to Client with iperf (300sec) 631 Mbits/Sec Client to Server with iperf (300sec) 616 Mbits/Sec Grinder with 1KB File (AWS Monitor) 40Mbits/Sec Grinder with 25KB file. (AWS Monitor) 39Mbits/sec

Grinder Readings: Work Proces ses 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 3 3 3 Respon se Time(m s) 2.5 2.8 3.06 3.33 3.97 2.54 2.65 2.89 2.82 3.15 3.12 3.63 4.03 4.39 4.65 6.68 7.64 10.4 52.04

Ap pl. 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2

Agen ts 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 3 3

Threa ds 1 2 3 4 5 6 7 8 9 10 11 12 13 15 10 10 10 10 10

TPS (mean) 375 692 955 1090 1210 1990 2290 2460 2830 2850 3150 3020 2980 3150 3760 4160 7300 8220 1710

TPS (Peak) 468 1130 1850 2660 2810 3820 5270 6330 4800 5220 6780 4790 8030 5250 5920 10200 11000 12100 2210

Errors 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

Page 34 of 39


CLOUD LOAD TEST PoC


3.3.3 Jmeter: Observations and Achievements Jmeter: Controller Throughp 90 Error ut % % (per sec) KBPS 46 4 0 165.3 4118.3 18 6 0 172.5 4297.3 60 0 298.9 7447.2 10 8 0 347.8 8666.1 10076. 58 0 404.4 3 11596. 70 0 465.4 8 11 12518. 3 0 502.4 5 14329. 93 0 575.1 8 14233. 75 0 571.3 5 13405. 65 0 538 8 Jmeter: Agent Throughp 90 Error ut % % (per sec) 22 11 8 28 26 24 4 40 6 0 0 0 0 0 0 0 0 502.5 658.3 877.7 1078.6 37.5 13.8 786.4 27.4

Sampl es 100 1000 3000 4500 5500 7500 9000 12000 13000 15000

Sampl es 100 1000 3000 4500 5500 7500 9000 12000

KBPS 12520. 6 16402. 9 21868. 9 26874. 9 933.9 343.9 19594. 9 683.6

Page 35 of 39


CLOUD LOAD TEST PoC


Page 36 of 39


CLOUD LOAD TEST PoC


4. CONCLUSIONTSung is successfully able to scale up to 0.9 m concurrent users on websites. It is yet to be proved to be working on web-applications. This will be taken up as PoC Phase II. .. . The below table gives more information on various tools: TSUNG GRINDER JMETER Per Server Upto 50,000 Only upto 1,000 Scalability Simultaneous Requests Users generated/sec

Using TSung tool, Wipro will be able to provide On-demand Performance Verification services, pay-as-you-go service using a flexible, affordable, hourly / per test usage model.

Page 37 of 39


CLOUD LOAD TEST PoC


5. GLOSSARY Amazon Web Services (AWS): The Amazon Web Services are a collection of remote computing services (also called web services) that together make up a cloud computing platform, offered over the Internet by Amazon.com. The most central and well-known of these services are Amazon EC2 and Amazon S3. Scalability Testing: This test is to measure an applications capability to scale up or scale out in terms of any of its non-functional capability - be it the user load supported, the number of transactions, etc. TSung Tool: A robust benchmarking tool Tsung is an open-source distributed load testing tool. It is protocolindependent and can be used to stress and benchmark http and jabber servers. It simulates user behaviour using an xml description file, reports many measurements in real time (statistics can be customized with transactions, and graphics generated using gnuplot). For http, it supports 1.0 and 1.1, has a proxy mode to record sessions, supports get and post methods, cookies, and basic www-authentication. It also has support for ssl.

Page 38 of 39


CLOUD LOAD TEST PoC


6. POC TEAMInitiative Drivers:Sharad Kumar Engineering Practice Aditya Kittur Laksminarayana Prasad Praveen P Practice Krishna K Mariyanna Practice : : : : : Principal Consultant - Performance Technical Manager Cloud Testing Services Technical Manager Cloud Testing Services Sr. Consultant Performance Engineering Delivery Head - Performance Engineering at Wipro Testing Services

Program Lead by:Harsha R P Engineering Practice : Project Lead at Wipro Performance

Contributors:Jayasree T Nazeer A Pavana N S Dharmendra Soni Ramanathan G A Ravi Krishna N Kasturi Narra : : : : : : : Performance Technical Consultant Sr. Performance Technical Consultant Sr. Project Engineer Sr. Project Engineer Sr. Technical Consultant Project Engineer Project Engineer at Wipro Technologies

Authors:Harsha R P, Praveen P, Jayasree T, Nazeer A at Wipro Technologies

Page 39 of 39


Date post:	14-Jul-2015
Category:	Documents
Upload:	venkatesh-elangovan
View:	694 times
Download:	1 times

Load Test Cloud PoC v 9.1

Documents