Test Expo 2009 Site Confidence & Seriti Consulting Load Test Case Study

Commercial in Confidence - © Copyright 2008 NCC Group plc - all rights reserved

SERITI CONSULTING

Load Testing Case Study

Some hints and tips on load-testing your website...

1


Introductions

Before you start

Designing the test

On the Day…

Analysing the results and taking action

The vendor bit…

Questions?

2

Agenda


SERITI CONSULTING

Before you start

Working out what to test and how much to test it

5


New website still under development?

Existing website about to head into a business peak period?

New marketing campaign about to launch?

The Met Office is predicting snow...

6

Why do you want to load test?


“Load test” vs. “Performance Test”

Performance vs Load Test?

This affects:How you will do itWhat you will do during the testWho needs to be involvedWhat information you record!

7


Load vs. Performance?

Load Test = "Black Box"I only care if the whole system can handle the

loadIf it can't I probably don't have a lot of clues as

to why not...

Performance Test = "White Box"Looking inside the App and Infrastructure

architecture to identify the key elementsDesigning tests to stress each elementMonitoring those key elements during the test

8


Test In-house or use 3rd party?

Project triangle decisionTimeMoneyScope

Resources required...Testing software e.g. LoadRunner, Selenium, Site ConfidencePeople to script itHardware & network to generate the loadAccessibility of test environments (or use Prod)Reporting interface to get the right stats & graphsPeople to interpret the results...

9


SERITI CONSULTING

Designing the test

Working out what to test and how much to test it

10


Decide on your critical user journeys

Home page

Search pages

Details pages

Upload/download

Checkout/e-commerce

SEO Crawl – Google can hammer sites!

12


Set your targets

How many transactions/sec?

How many concurrent users?

What is the mix of the user journeys?Don't underestimate the impact of concurrence...

What's your "safety margin"We used 40%

13


TIP - Web Analytics is your BFF

Leverage your site metrics!Google Analytics, WebTrends or Omniture etc

Use the information to identify your peak loads across different user journeys

It is very common for people to test their sites at very unrealistic rates... Or focus on the wrong parts…• Don’t forget deep linking!

14


TIP – Management Speak..

The Management tends to ask “how many users will the site support”…

How many users do what, how often?Users on playing online poker are different to users reading

“The Register”…

I find it best to talk about “page impressions” since that is a metrics they are probably familiar with by now…

15


Concurrent users & “think time”

Take the number of page impression per hour

Divide by 3600

That gives you the page impressions ("transactions") per second target

Then take the number of "concurrent users" and divide by the "think time" (dwell time) to give you transactions a second1200 users with a 60s think time = 20 transactions /sec

600 users with 30s think time = 20 transactions /sec

300 users with 15s think time = 20 transactions /sec

100 users with a 5s think time = 20 transactions /sec

16


Scripts Profiles Scenarios

Design the test…

Scripts - the actual user journey (“clicks”)

Profiles - how many, how fast, how long etc

Scenarios - combining profiles to match the traffic patterns on your site…

The “mix” of user journeys at any one time

17


What does a Profile look like?

19


3 Quick Tips…

Baseline your network firstDo a “GET” for a 100K JPEG and see how many you get before you

saturate your network

Do “single profile” scenarios firstWork from simple to complexTest each element before you try combinations

Think about “keyhole” tests to isolate specific application functionsRather than simulate the “click by click” of the user just “POST” (form

submit) directly to the siteThis enables you to focus on the app performance rather than the UI

elements (you need to do both though!)

21


One more tip…

DON’T FORGET YOUR BATCH PROCESSESHow does the system perform during…

• Backups• Re-indexes• ETL activities• Any other batch process that stresses the system…

Many a site dies when these processes run

And if you are truly 24x7 online – when is your window?

22


On the day – running the test!

Who does what, where & when?

23


Plan out the timetable!

Make the most of your testing window…

Plan carefully but BE REALISTIC!

Allow a lot of time for review between each scenario run...

You WILL change your plans on the day...Be flexible!

Get the right people – Operations, QA, Network, Database etc

24


On the day...

DON’T - Suddenly remember that you forgot to tell your hosting provider...And watch their IDS system think it was a DDOS attack and drop all

your packets...

DO – keep an eye on the time (and ensure that time is sync’s across the systems)You will want to correlate events!

DO – use a meaningful name for each run and add commentsSo you know how many users are added, over what ramp up period,

and run for how long at a glance

DO – think about cleaning up after you! If you have been running against Prod you might have created a lot of

dummy transactions (and your web logs will be HUGE!)

25


SERITI CONSULTING

Analysing the results and taking action

Data -> Information -> Knowledge -> Action

26


Did you hit your targets?

Where there any odd peaks/troughs?

Was “mixed scenario” performance different?

Was performance consistent across test runs?Did your releases make things better (or worse)?

What was the Max/Mean/Min valuesAnd what was the variance?

Lots of questions…

27


Before and After a Network upgrade…

28


Lower & longer is better

29


Faster response, with more users…

30


Use the performance stat (if you gathered any) to isolate the weak linksUpgrade/re-factor & re-test

Use the (bad) stats to get more budget to fix the problem…

Use the (good) stats to get more a bonus because you did such a good job…

Taking action…

31


SERITI CONSULTING

The vendor bit…

Over to James…

32


Site Confidence Clients…

34


Talk to Site Confidence on their stand at the TestExpo after the presentation or if you would like a copy of the presentation

Email James – [email protected]

Visit their website - www.siteconfidence.com

Contact Details – Site Confidence

35

mailto:[email protected]

http://www.siteconfidence.com/


If you have any questions around web hosting (IIS, Cloud)Performance tools (Appmetrics, Avicode)Operations & ITIL (service-now.com)Configuration Management (Configuresoft ECM)Near-shore development, functional test (Endava)

Email me – [email protected]

Connect with me on LinkedInhttp://www.linkedin.com/in/stephenthair

Contact Details – Seriti Consulting

36

mailto:[email protected]


Any Questions?

37

Date post:	14-May-2015
Category:	Technology
Upload:	stephen-thair
View:	1,405 times
Download:	1 times

Test Expo 2009 Site Confidence & Seriti Consulting Load Test Case Study

Technology