11 © ALP International CorporationMarch 2001 MD-CFUG
Automating your Website Testing
ByBy
Andrew L. Pollner and Lei WangAndrew L. Pollner and Lei Wang
ALP International CorporationALP International Corporation
22 © ALP International CorporationMarch 2001 MD-CFUG
Why Load Test?
To Simulate Multiple Users on a SystemTo Simulate Multiple Users on a System To Measure end-to-end performanceTo Measure end-to-end performance To Measure Server Performance:To Measure Server Performance:
– Database (Oracle)Database (Oracle)– Application (CF)Application (CF)– Middleware (Web Logic)Middleware (Web Logic)– Web (Netscape)Web (Netscape)
33 © ALP International CorporationMarch 2001 MD-CFUG
When Should Loadtesting Occur?
When sizing a serverWhen sizing a server After Application Has StabilizedAfter Application Has Stabilized When other users are not on the systemWhen other users are not on the system After notifying Sys Admin.s, DBA’s, Test After notifying Sys Admin.s, DBA’s, Test
ManagersManagers
44 © ALP International CorporationMarch 2001 MD-CFUG
What’s Needed for Loadtesting?
Knowledge of:Knowledge of: ApplicationApplication DatabaseDatabase EnvironmentEnvironment LoadRunner!LoadRunner!
55 © ALP International CorporationMarch 2001 MD-CFUG
Loadtesting Concepts - Part I
TransactionTransaction
RendezvousRendezvous
Measured event (i.e. Time it takes to Measured event (i.e. Time it takes to return data to a screen after a return data to a screen after a QUERY button is pressed)QUERY button is pressed)
Synchronized point where Synchronized point where transaction is to occur transaction is to occur simultaneouslysimultaneously
66 © ALP International CorporationMarch 2001 MD-CFUG
Loadtesting Concepts - Part II
Vuser Vuser
SgroupSgroup
ScenarioScenario
A defined user type which A defined user type which
executes a specific testexecutes a specific test
A group of Vusers which with a A group of Vusers which with a
characteristic profilecharacteristic profile
A combination of sgroupsA combination of sgroups
77 © ALP International CorporationMarch 2001 MD-CFUG
Types of Vusers
WebWeb C/S GUIC/S GUI Terminal EmulationTerminal Emulation SQLSQL
88 © ALP International CorporationMarch 2001 MD-CFUG
Using LoadRunner - Web
Record HTTP ScriptRecord HTTP Script Generalize traceGeneralize trace Add Transaction points:Add Transaction points:
– start_transaction(“ClaimWriteback”)start_transaction(“ClaimWriteback”)– end_transaction(“ClaimWriteback”, PASS)end_transaction(“ClaimWriteback”, PASS)
Add Rendezvous:Add Rendezvous:– rendezvous(“Login”) rendezvous(“Login”)
rendezvous(“ClaimWriteback”)rendezvous(“ClaimWriteback”)
99 © ALP International CorporationMarch 2001 MD-CFUG
Login
Enter Data
Save Data
Login
Enter Data
Save Data
Login
Enter Data
Save Data
Rendevouz 1 ==>
Rendevouz 2 ==>
Rendevouz 3 ==>
Start Synchronization
1010 © ALP International CorporationMarch 2001 MD-CFUG
Login
Enter Data
Save Data
Login
Enter Data
Save Data
Login
Enter Data
Save Data
Rendevouz 1 ==>
Rendevouz 2 ==>
Rendevouz 3 ==>
Step Synchronization
1111 © ALP International CorporationMarch 2001 MD-CFUG
Using LoadRunner - DB
Get SQL traceGet SQL trace Generalize SQL traceGeneralize SQL trace Add Transaction points:Add Transaction points:
– start_transaction(“ClaimWriteback”)start_transaction(“ClaimWriteback”)– end_transaction(“ClaimWriteback”, PASS)end_transaction(“ClaimWriteback”, PASS)
Add Rendezvous:Add Rendezvous:– rendezvous(“Login”) rendezvous(“Login”)
rendezvous(“ClaimWriteback”)rendezvous(“ClaimWriteback”)
1212 © ALP International CorporationMarch 2001 MD-CFUG
Product Categories
GUI/Web vusersGUI/Web vusers– Netscape BrowserNetscape Browser– IE BrowserIE Browser– Client ServerClient Server
RTE vusersRTE vusers– Terminal EmulationTerminal Emulation
SQL (dB) vusersSQL (dB) vusers– Client ServerClient Server
1313 © ALP International CorporationMarch 2001 MD-CFUG
LoadRunner Family of Products
Both Client Both Client ANDAND Server Load Testing Server Load Testing Wide range of platformsWide range of platforms
– Win 2000Win 2000
– NTNT
– Win95/98Win95/98
– UNIXUNIX
ReportingReporting– Automatically for each userAutomatically for each user
– By transactionBy transaction
1414 © ALP International CorporationMarch 2001 MD-CFUG
Single point of control: ControllerSingle point of control: Controller
Client AND Server Load TestingTesting the Site
WAN/Internet
SYBASE SERVER
NTSQLSERVER
1515 © ALP International CorporationMarch 2001 MD-CFUG
Single point of control: ControllerSingle point of control: Controller Drive multiple Drive multiple realreal NT GUI clients :End-to-End NT GUI clients :End-to-End
Client AND Server Load TestingTesting the Site
SYBASE SERVER
NT
SQLSERVER
WAN/Internet
1616 © ALP International CorporationMarch 2001 MD-CFUG
Single point of control: ControllerSingle point of control: Controller Drive multiple Drive multiple realreal NT GUI clients : End-to-End NT GUI clients : End-to-End Stress SQL Server with multiple Stress SQL Server with multiple virtualvirtual NT clients NT clients
Client AND Server Load TestingTesting the Site
SYBASE SERVER
NT
NT
SQLSERVER
WAN/Internet
1717 © ALP International CorporationMarch 2001 MD-CFUG
Single point of controlSingle point of control Drive multiple Drive multiple realreal clients clients Stress SQL Server Stress SQL Server virtualvirtual clients NT clients NT Stress Sybase: Stress Sybase: virtualvirtual clients NT/UNIX clients NT/UNIX
Client AND Server Load TestingTesting the Enterprise
SYBASE SERVER
NT/UNIX
NT WAN/Internet
1818 © ALP International CorporationMarch 2001 MD-CFUG
Advanced Loadtest Measurement
Mean vs. MedianMean vs. Median 90% Percentile90% Percentile Performance under loadPerformance under load System Performance MeasurementSystem Performance Measurement
– CPUCPU– SWAPSWAP– DISKDISK
1919 © ALP International CorporationMarch 2001 MD-CFUG
Analyzing Loadtest Results - ICLIENT LAN WAN
GUI
ApplicationProtocol SQL Server
Total System Performance: Client, LAN, SQL Server, WAN, Sybase
Sybase
SERVERSERVER
Client
Load
2020 © ALP International CorporationMarch 2001 MD-CFUG
Transaction Performance: LAN, SQL Server, WAN & Sybase
Analyzing Loadtest Results - IICLIENT LAN WAN
GUI
ApplicationProtocol SQL Server Sybase
SERVERSERVER
Client
Load
Server
Load
2121 © ALP International CorporationMarch 2001 MD-CFUG
Analyzing Loadtest Results - IIICLIENT LAN WAN
GUI
ApplicationProtocol SQL Server
Transaction Performance: WAN, Sybase
Sybase
SERVERSERVER
Client
Load
Server
Load
2222 © ALP International CorporationMarch 2001 MD-CFUG
Analyzing Loadtest Results - IVCLIENT LAN WAN
GUI
ApplicationProtocol SQL Server Sybase
SERVERSERVER
Transaction Performance: SYBASE
Client
Load
Server
Load
2323 © ALP International CorporationMarch 2001 MD-CFUG
Analyzing Loadtest Results - VCLIENT LAN WAN
GUI
ApplicationProtocol SQL Server Sybase
SERVERSERVER
Client
Load
Server
Load
Transaction Response Time: Client
Transaction Response Time: LAN, SQL Server
2424 © ALP International CorporationMarch 2001 MD-CFUG
Conclusions
Loadtesting:Loadtesting: Is a necessary component in testing an Is a necessary component in testing an
application and its environmentapplication and its environment Has multiple purposesHas multiple purposes Comes in many flavorsComes in many flavors Is an extension to Functional testingIs an extension to Functional testing Is complex!Is complex!
2525 © ALP International CorporationMarch 2001 MD-CFUG
DEMO!
2626 © ALP International CorporationMarch 2001 MD-CFUG
Virtual User Generator
2727 © ALP International CorporationMarch 2001 MD-CFUG
LoadRunner Controller
2828 © ALP International CorporationMarch 2001 MD-CFUG
Controller Vusers
2929 © ALP International CorporationMarch 2001 MD-CFUG
Vuser Protocol
3030 © ALP International CorporationMarch 2001 MD-CFUG
Transaction Report
3131 © ALP International CorporationMarch 2001 MD-CFUG
Page Faults Graph
3232 © ALP International CorporationMarch 2001 MD-CFUG
On-line Monitors
3333 © ALP International CorporationMarch 2001 MD-CFUG
Q&A