Essence of Performance and Load Testing VanQ 30 April 2009

7/31/2019 Essence of Performance and Load Testing VanQ 30 April 2009

http://slidepdf.com/reader/full/essence-of-performance-and-load-testing-vanq-30-april-2009 1/42

Essence of Performance and Load Testing

April 30th , 2009Gihan B Madawala

Argus Technologies Ltd

CREO Products Inc

Chancery Software Ltd

Meridian Project Systems Ltd

Modular Mining Systems Canada Ltd.



Topics

Why do we need to test performance Myths surrounding performance testing

Performance / Scalability testing

Single User Performance testing Worst-case scenario

Performance bottlenecks

Performance and Scalability Targets Performance Testing Lab

Performance Team



Topics Contd..

Microsoft Testing Tools Commercial Performance Testing Tools

Process of analyzing performance and

Scalability issues Tool selection strategy

Problems with manual perf testing

Tool Evaluation Process

Some Performance relatedissues/solutions

Books & Reference materials



Bad comments/Questions/Answers

It’s the first release of this feature so of course it is going to be slow !

It only takes couple of seconds so what is thebig dealQ: Why is this page too slow ?

A: It is because we have TWO users in thesystem

Q: How many users that we can support withthis system?

A: 20 users per server



Good comments/questions/Answers

“ The technology of the product is basedupon current solid architecture and easy towork with”

“The system is flexible to build upon and can

be managed easily by our personnel.” Product has the architecture required to

function in large projects. It not onlyperforms well but it scales

Q: How many users can we support with oursystem?

A: 400 users per server



Why do we need to test

performance

Studies show performance was one of the

most important qualities of a application

Experience shows Performance issues are themost likely to “bite with surprise” released

into live operation

Lack of knowledge on Performance &Scalability testing



Myths surrounding Performance

testing

We do not need to measure System Performance

Our system is incredibly impressive with all thefeatures, users will not mind if it is little slow

We do not really need precise measurement goals

Performance testing has to be done late in theproject.

Anyone can measure performance; it does not

require any skills or expertise Any test lab/tool is about the same as any other one

and you can mimic real life usage of the app



Performance testing should start early in the

project and should be a on going process

Performance andtesting practices

OperationalPerformance was

Acceptable (48%)

OperationalPerformance was NOT

Acceptable (52%)

Reviewed or stimulated

performance during

requirements anddesign phases

21% 0%

Tested early in

development

35% 6%

Tested late in

development

38% 26%

Did not do

performance or load

testing

6% 60%



Performance testing / Scalability

Testing

The purpose of performance testing is to measuresystem’s performance for agreed requirements

Scalability testing (Load Testing, Stress testing) is

measurement of performance under heavy load: Peakor worst-case conditions :-

Performance usually is measured as weighted mix of response time, throughput (Network Bandwidth) andavailability

Response time – Measure of how long the systemtakes to complete a task or group of tasks



Single user performance testing

How important is to measure single user

performance

Gives an early indication of system designfeasibility

Can be measured with existing set of tools

available Cost of single user testing is low and can be

done prior to multiple user testing



Worst-case scenario

If the test pass and scale well with worst-case we know for sure that the application

will perform better for a lesser load. Full data population is necessary

Project Ex: (worst-case)

-Larger data packet size

-More frequency data packets

-More client threads



Performance and Scalability Targets

Achieve the specified Response time for oneuser

Maintain the response time within a specifiedrange when the system is under load

Maximize the number of active users peravailable bandwidth

Minimize network bandwidth usage

CPU usage of machines should be < 80%under heavy load



Performance Bottlenecks

Client CPU

Database Server CPU

Client, Server Memory

Network Bandwidth Disk I/O

Transaction processing speed

Serialization and de-serialization speed Network latency

Limitations of the 2nd /3rd party servers/add-ins



Performance Testing Lab

Simulation of realistic scenario

Having adequate hardware

Having feature rich testing tools Test Automation and Unit tests

Performance testing expertise

Creating real test data



Performance Team

Performance Test Engineer

Senior Developer

Database Specialist Hardware Engineer

Network Specialist



Microsoft Testing Tools

PERFMON

SQL Profiler

ACT (Application Centre Test) WAST (Web Application Stress Tool)

TEAM TEST (Visual Studio Team System)

windebug /ddk



Some Industry Leading Performance

and Scalability Tools

Load Runner (Mercury now HP) *

Silk Performer (Segue now Boland) *

QA Load ANTS

Open STA (Open Source)

E-Load (Emprix) SHUNRA

SPIRENT Test Centre



Process of Analyzing Performance and

Scalability Issues

Non Invasive action (Low hanging fruits)1 Analyzing of Performance counters

2 Profiling

3 Load testing tool analysis. Two steps

- Run concurrent users- Run specific number of users continuously

4 Simulate different network conditions

5 Tracing and debugging .NET components

6 Use of windebug /ddk Invasive Action

1.Re-factoring 2.Architectural change



Tool selection Strategy

# Areas to Test GUI Automation

(Feature Testing)

Single Userperformanc

e Multiple User

Performance Stress/Stability Testing

1 Web Clients TestPartner/TestComplete

Manual/Benchmark

Tools/Visual

Studio LoadRunner/SilkPerformer/

TestComplete LoadRunner/SilkPerformer

/TestComplete

2 Office client TestPartner/TestComplete

Manual/Benchmark

Tools/VisualStudio LoadRunner/SilkPerformer LoadRunner/SilkPerformer

3

Central server – DB server TestPartner/TestComplete

Manual/Benchmark

Tools/Visual

Studio LoadRunner/SilkPerformer/

TestComplete LoadRunner/SilkPerformer

/TestComplete

4

NetworkInfrastructure SUNRA/SPIRENT

Manual/Benchmark

Tools/Visual

Studio SUNRA/SPIRENT SUNRA/SPIRENT

5 HardwareInterfaces Internal Simulators

Manual/Benchmark

Tools/Visual

Studio Internal Simulators Internal Simulators

6

Mobile Client – Windows CE TestComplete

Manual/Benchmark

Tools/Visual

Studio Internal Simulators Internal Simulators



Problems with Manual Performance

testing in the Lab

Difficult to mimic concurrency operations

Cannot be repeated easily

Difficult to setup and organize

Lab could be expensive

Very time consuming

Cannot afford to have large scale performance

labs in every development locationsFull NFRs cannot be tested



Commercial tools Vs Internal tools

Consider the scope of features needed

Estimate the ROI

How soon you need the tool Does the test team has performance

testing expertise

Customer requirements for benchmarksand proper test results



Tool Evaluation Process

Phase 1 – Fact finding of requirements,

suitable vendors and preparing aproposal for project stakeholders –

5 weeks

Phase 2 - Tool demonstrations in lab –

6 weeks (2 weeks for each vendor)

Phase 3 – Complete Evaluation Process –

1 week

Phase 4 – Negotiation and Purchase of a tool – 4 weeks

Phase 5 – Tool deployment – 1 week



Issue # 1 – Testing Non Functional

Requirements

Unless it is tested we cannot guarantee thatthe application going to work for the stated

NFRs. That is if we say that our application can

support 500 mobile clients we need to test

these scenarios before clamming the limits of

our system. We cannot test 50 clients and say

that the system should work with 500 clients

calculating performance parameters byinterpolating them.



Solution #1: Always test/simulate full

NFRs

Shouldn’t interpolate single user data for

multiple users

Try to test with worst-case scenario

Different bottlenecks restrain achievingbetter performance/Scalability.

Proper testing Lab Use tools to simulate and test full NFRs



Issue # 2 – Cost of fixing Performance

Defects

We need to start testing Performanceand scalability early in the development

life cycle. More you do this early it is

easy to find solutions and less costly tothe project.



Solution #2: Reducing Project Costs



Issue # 3 – Unable to gauge performance

Performance optimizations shouldn’t beintegrated haphazardly. We need to have

mechanism and tools to monitor

performance and scalability regularly. If it is not with weekly builds we should at

least try to monitor this at least once a

month with set of test scenarios covering

good part of operations of the

application. It would be ideal if we can

automate this.



Solution #3: Regular Performance

Measurements – Heartbeat of the project



Issue # 4- Unrealistic/Untested NFRs

We need to better scrutinize the NFRrequirements coming from Marketing to

really understand that we can support

them with the technology and theplatform that we select for the product.





Issue # 5 – Agile Performance Testing

If Agile method has been adopted forthe development process try to do

performance and scalability testing

in each sprint tied down to userstories. This will give developers

immediate feedback of the status of

performance of the particular area.



Solution # 5 – Traditional Performance

Testing



Solution # 5 – Agile Performance Testing



Issue # 6 – Testing with unrealistic data

We need to test systems with morerealistic data as possible. We should

create different sizes/profiles of

databases and benchmark results foreach category. We can develop these

data sets together with the

development team as this is

important for them as well.



Solution # 6 – Data Generation

We need to programmaticallygenerate application data keeping

the data integrity.

Do this for different profiles

(standard, heavy and ceiling)

Update and maintain this dataThis is a responsibility of both test

and dev teams

I # 7 l k f t f db k till



Issue # 7 – lack of customer feedback till

end of the project

Many projects we find major issues

because our process does not has

room to get customer feedback aearly in the project.



Solution # 7 – More input from Support

Dept and Customers

Provide Implementation/Supportdepartment with working

software/system before the application

transitioned to collect feedbackGet customer feedback if possible before

the release

Expose identified dev and test teammembers for real customer environments

if possible.



Issue # 8: Growing customer demands

Once a customer buys your product withtime their requirements overpass theoriginal application requirements.

Mostly this is to do with handlingincrease amount of data. Somecustomers may even require to keepeven 10 years worth of data in the

running system. This could be a majorchallenge if not planned properly.



Solution # 8: Capacity Planning

Collecting data from Customers/field

Consolidate this data for a 6 months/1

year targets

Focus more on most frequent & mostimportant features

Get the Performance Team to start on

these targets early

Do Capacity planning for both software

and hardware/network



Microsoft Performance Testing Labs

Can use for benchmarking

Can use testing tools with 1000’s of user

licenses

Can use Microsoft domain experts on

Performance

Worth investment towards improving

system performance



Books & Reference materials

MSDN articles

Pattern & Practices (Performance

tuning MS .NET Applications) – By MS

ACE Team)

Improving .NET Application

Performance and Scalability – by Rico

Mariani, Scott Barber)



Discussion

Q&A

http://images.google.com/imgres?imgurl=http://xerto.free.fr/toad1.jpg&imgrefurl=http://xerto.free.fr/index.php%3Fpage%3Dt10mignons&h=407&w=267&sz=13&hl=en&start=6&um=1&tbnid=oVvJ5o0MJXLBwM:&tbnh=125&tbnw=82&prev=/images%3Fq%3DToad%2BMario%26svnum%3D10%26um%3D1%26hl%3Den%26sa%3DX

http://images.google.com/imgres?imgurl=http://www.pick-up-lines.info/mario_pick_up_lines.jpg&imgrefurl=http://www.pick-up-lines.info/mario_pick_up_lines.htm&h=614&w=703&sz=92&hl=en&start=152&um=1&tbnid=qx8S1GlmIrLzHM:&tbnh=122&tbnw=140&prev=/images%3Fq%3DToad%2BMario%26start%3D144%26ndsp%3D18%26svnum%3D10%26um%3D1%26hl%3Den%26sa%3DN

Date post:	05-Apr-2018
Category:	Documents
Upload:	khushboo-tambi
View:	213 times
Download:	0 times

Essence of Performance and Load Testing VanQ 30 April 2009

Documents