8/6/2019 Delivering Quality Software Through Formal Testing
1/75
Leonard Aye
MICT Park
Yangon
22 January 2010
8/6/2019 Delivering Quality Software Through Formal Testing
2/75
Leonard Aye
lenaye
2
8/6/2019 Delivering Quality Software Through Formal Testing
3/75
Do we need to test software?
Definitions of Good quality software
Categories of Software
Definition of Testing Development models & testing processes Case studies: Waterfall
V-Model
Agile (SCRUM)
Building a test team
Q&A
3
8/6/2019 Delivering Quality Software Through Formal Testing
4/75
Date: June 4th 1996
Place: European Space Agency Launch Site
Location: French Guiana, South America
4
8/6/2019 Delivering Quality Software Through Formal Testing
5/75
5
Rocket: Ariane 5
8/6/2019 Delivering Quality Software Through Formal Testing
6/75
Why did it blow up? There was an error in the software that calculates the
rockets height and direction.
Error occurred when a 64-bit Floating Point Numberwas converted to a Signed 16-bit Integer Number,
causing the resulting Integer number to overflow
There was no error handling for this type of
Overflow error because this condition was neverdiscovered during testing.
6
8/6/2019 Delivering Quality Software Through Formal Testing
7/75
Whats the cost of this failure? Development time: 11 years
Total cost: US $11 Billion
Whats the Conclusion? There should have been more testing!
7
8/6/2019 Delivering Quality Software Through Formal Testing
8/75
Whatis testing?
What typesof testing are there?
Whatdo we test exactly?
How much testing is enough?
How much time do we need for testing?
How manytesters do you need?
What makes a good tester?
8
8/6/2019 Delivering Quality Software Through Formal Testing
9/75
What is BAD quality software? Its a bad piece of software
if it is full of bugs (errors)!
Its a bad piece of software even if its 100% bugs free but
it doesnt do what the user want
9
8/6/2019 Delivering Quality Software Through Formal Testing
10/75
What is BAD quality software? Its a bad piece of software
if it is full of bugs (errors)!
Its a bad piece of software even if its 100% bugs free, but
it doesnt do what the user want
Its a bad piece of software
even if its 100% bugs free and
it does exactly what the user want, but
not in the way the user wants it
10
8/6/2019 Delivering Quality Software Through Formal Testing
11/75
What is BAD quality software? Its a bad piece of software
even if its 100% bugs free and
it does exactly what the user want, and it works in the way the user wants it, but
it works sloooowly
11
8/6/2019 Delivering Quality Software Through Formal Testing
12/75
It works without serious errors
Easy to install, configure and use
Has features the users want Works in the way the users expect it to work
Responsive and high-performance
Good aesthetics, e.g. clean and uncluttered
interface and ease of navigation Has good and extensive documentation
Good technical support
12
8/6/2019 Delivering Quality Software Through Formal Testing
13/75
3 main types Off-the-shelf /Packaged software
Bespoke software
Customised software
13
8/6/2019 Delivering Quality Software Through Formal Testing
14/75
Off-the-Shelf/Packaged software Ready-made software which you can pick up off the
shelf in a supermarket. Some Examples
Desktop Applications:
Windows Vista, Windows 7, Mac OS X
MS Office
Photoshop
Chrome browser
Online sites & serives: Google
Youtube
14
8/6/2019 Delivering Quality Software Through Formal Testing
15/75
Bespoke software Software specifically written for one customer or
one organisation only
2 types of bespoke software New computerised system for a business
Replace existing computerised system with new
(faster/better) system
15
8/6/2019 Delivering Quality Software Through Formal Testing
16/75
Customised software A generic application which is configured for
each customer/client
Some commercial off-the-shelf examples:
Accounting packages, e.g. Sage, MS Money
Payroll systems, e.g. Zpay, Intuit
Ecommerce packages, e.g. E-Volved, Actinic, EROL
16
8/6/2019 Delivering Quality Software Through Formal Testing
17/75
17
Off-the-Shelf Bespoke/Customised
User Requirements From within your own
organisation
From customer
Project duration No limits, except for external
pressure
Defined by customer
Deployment via websites or shops Defined with customer
Testing process Defined by development
process
Defined by development
process
Customer feedback via email or website Direct from customer
Customer interaction Little or none High
8/6/2019 Delivering Quality Software Through Formal Testing
18/75
A definition Testing is the process of comparing the invisible
to the ambiguous, so as to avoid the unthinkable
happening to the anonymous. James Bach
18
8/6/2019 Delivering Quality Software Through Formal Testing
19/75
A few well known definitions Testing is a process used to identify the correctness and
completeness of computer software.
Testing is verifying software features against the system
requirements Testing is a process of verifying that a program functions properly.
Preferred definition Testing is a process of finding bugs in the software
Kaner, Falk & Nguyen, Testing Computer Software, 1999
Testing is a process of trying to break the software duringDevelopment so that it can be fixed, thereby reducing companystime and money after the product is released.
19
8/6/2019 Delivering Quality Software Through Formal Testing
20/75
Integration Testing
System Testing Functional Testing
Non-Functional Testing
User Acceptance Testing
End-to-End Testing
Regression TestingExploratory Testing
20
8/6/2019 Delivering Quality Software Through Formal Testing
21/75
Part 1 Do we need to test software?
Definitions of Good quality software
Categories of Software
Definition of Testing
Development models & test processes Case studies:
Waterfall
V-Model Agile (SCRUM)
Building a test team
Q&A
21
8/6/2019 Delivering Quality Software Through Formal Testing
22/75
Software development model defines a set of ruleson how a software is to be built.
There are many and varied development models: Agile
DSDM (Dynamic Systems Development Method)
FDD (Feature Driven Development)
Iterative
RAD (Rapid Application Development)
RUP (Rational Unified Process)
Scrum
TDD (Test Driven Development) V-Model
Waterfall
XP (eXtreme Programming)
22
8/6/2019 Delivering Quality Software Through Formal Testing
23/75
Traditional Iterative/Incremental
V-Model Agile
Waterfall FDD (Feature Driven Development)
RAD (Rapid Application Development)
DSDM (Dynamic Systems Development Method)
ScrumRUP (Rational Unified Process)
XP (eXtreme Programming)
TDD (Test Driven Development)
23
8/6/2019 Delivering Quality Software Through Formal Testing
24/75
Waterfall model is a software
development process in which
progress is seen as flowing steadily
downwards (like a waterfall) through
the phases of:- Requirements
Design
Construction
Testing, and
Maintenance.
24
8/6/2019 Delivering Quality Software Through Formal Testing
25/75
25
Requirements
Analysis
Design
Construction
Testing
Maintenance
8/6/2019 Delivering Quality Software Through Formal Testing
26/75
Small to medium scale projects, i.e. < 10 people
and/or timescale ~6-9 months
Requirements are clearly defined and unlikely to
change throughout the life of the project
26
8/6/2019 Delivering Quality Software Through Formal Testing
27/75
28
msn shopping
Price comparison site
Online Merchants send in their
product catalogue, daily/weekly
Products are categorised and
published on their site Business model
Online Merchants get greater
exposure to their products from
users on msn
User clicks are recorded and
Charge per click-throughs
8/6/2019 Delivering Quality Software Through Formal Testing
28/75
29
8/6/2019 Delivering Quality Software Through Formal Testing
29/75
Environment
30
Pr 6
SQL
ServerSQL
Server
Pr 1 Pr 2
SQL
Server
Staging
SQL
Server
Dev
SQL
Server
MerchantMerchant Merchant Merchant
Load Balancer
Test
SQL
Server
8/6/2019 Delivering Quality Software Through Formal Testing
30/75
Requirements Website is non-business critical
Overnight switchover (big bang)
Website must display consistently on all existing web
browsers
Website must have high availability (24x7)
Website must be able to support high user traffic
Merchant products upload
8/6/2019 Delivering Quality Software Through Formal Testing
31/75
How do we test this system? System Testing
Functional Testing
Non-Functional Testing
End-to-End testing
What do we need to test this? Test Servers (Application and DB server) with all necessary
apps installed, i.e. IIS & SQL Server User PCs (Win 98, Win ME, Win 2K, Win NT) with IE 3, IE4,
IE4.01 SP1 ,IE 4.02 SP2, etc.
Test Data (manually created dummy data)
32
8/6/2019 Delivering Quality Software Through Formal Testing
32/75
System Testing Ensuring the system meets the user/design
requirements
Functional Testing Aim is to find errors (bugs)
Does it work?
Does it do what its supposed to do?
e.g. functional testing a car would be:-
Check that engine starts if you turn on the ignition key
Check that the break lights come on if you press on the breakpedals
Check that radio comes on if you turn on the radio
33
8/6/2019 Delivering Quality Software Through Formal Testing
33/75
System Testing UI Test cases
Layout of products
Product description too long
Display of special characters
Missing images (at site and from merchant)
Missing information, e.g. Detailed description
Missing mandatory information, i.e. price
Special offer (normal price, sale price, and sale indicator) Incorrect or dead-links back to merchant site
Functional Test cases
Correct categorisation of products
Product filtering by type, by brand, by store, by price range(pre-defined or manual)
Product sorting
Product searching via Search box
Product listing per page
Non-Functional Testing
Web server breakdown
SQL server breakdown
Load/Stress testing
Average page load times: < 10ms
Min concurrent user: 400
34
8/6/2019 Delivering Quality Software Through Formal Testing
34/75
Non-Functional Testing
Aim is NOT to find faults but tomeasure performance againstthe required targets
How does it work? Fast, slow, etc.
Is it easy to use? (Usability testing)
Can users of all different versions of OSs and browsers access thesite? (Compatibility testing)
Will the system be available in case of server failures (Reliabilityand Availability testing)
Can the system cope with the growth of data volume or uservolume in the future (Load testing)
e.g. non-functional testing of a car would be:-
Check that it accelerates from 060 mph in under 10 secs
Check that the electric door opens/closes 100,000 times withoutfailure
35
8/6/2019 Delivering Quality Software Through Formal Testing
35/75
Usability testing
Invited people from outside the office and asked them to
use the website as normal users, i.e.
Search for products
Find a particular product Filter products by price
Filter products by brand, etc.
This is followed by questionnaires for their feedback
Amending the layout and design of the site based on users
feedback
36
8/6/2019 Delivering Quality Software Through Formal Testing
36/75
Compatibility testing
Testing that the application/website is compatible to use on
as many supported browers and OSs as possible
Set up a test laboratory with several PCs running different
OSs and web browsers Check how the site displayed on each type of browser
Use VMWare to set up Virtual Machines
37
8/6/2019 Delivering Quality Software Through Formal Testing
37/75
Reliability testing
Testing that the system is still available in the event of failure
in any part of the system
Manually turned off each web servers and check that the user
transactions are re-routed correctly Manually turned off each DB server and check the user data
request is served by a different DB server
38
WS 6
SQL
ServerSQL
Server
WS 1 WS 2
SQL
Server
Load Balancer
8/6/2019 Delivering Quality Software Through Formal Testing
38/75
Availability testing
Testing that the system is still available in the event of total
system and site failure
Set up a duplicate/back up site at a different location
Manually turned off main site and check that the usertransactions are re-routed to backup site
Check for data synchronisation between the main DBs and
backup DBs
39
W
S
6
SQL
Ser
ver
SQL
Ser
ver
W
S
1
W
S
2
SQL
Ser
ver
Load Balancer
W
S
6
SQL
Ser
ver
SQL
Ser
ver
W
S
1
W
S
2
SQL
Ser
ver
Load Balancer
Sync
London Derby
8/6/2019 Delivering Quality Software Through Formal Testing
39/75
40
SQL
Server
SQL
Server
SQL
Server
SQL
Server
SQL
Server
SQL
Server
Load Balancer
Load Injectors Load Controller
0
10
20
30
40
50
60
70
Users 50 100 150 200 250 300 350 400 450
Load testing analysisTime (ms)
Concurrent Users
Load Testing Example
8/6/2019 Delivering Quality Software Through Formal Testing
40/75
Summary: Duration: 10 months from concept to production
Written entirely ASP and SQL
Team:
1 PM
2 Merchant Liaisons
4 Dev
2 DBAs
4 Testers (London, Dublin, Outsourced company)
Big-Bang deployment
Old system turned off Friday night and new system brought
up on Monday morning
41
8/6/2019 Delivering Quality Software Through Formal Testing
41/75
8/6/2019 Delivering Quality Software Through Formal Testing
42/75
Part 1 Do we need to test software?
Definitions of Good quality software
Categories of Software
Definition of TestingDevelopment models & testprocesses Case studies:
Waterfall (examples of non-functional testing)
V-Model Agile (SCRUM)
Building a test team
Q&A
43
8/6/2019 Delivering Quality Software Through Formal Testing
43/75
44
User
Requirements
System Design
Module Design
Construction
Integration
Testing
System
Testing
User Acceptance
Testing
Int. Test
Design
System Test
Design
User Acceptance
Test Design
8/6/2019 Delivering Quality Software Through Formal Testing
44/75
Accepts that testing takes roughly 50% of project time
Activities like test planning and test case designs take
place at the beginning of the project well before code
is written, i.e. User Acceptance tests are written once Requirements are approved
System Acceptance tests are written once System Designs are
completed
Integration tests are written once Module Designs are completed
45
8/6/2019 Delivering Quality Software Through Formal Testing
45/75
LCH.Clearnet Clearing house for London Stock Exchange
Also acts as counter-party for trade for bonds, swaps, futures, options.
Volume of trades cleared per year: 1.7 billions
Value of transactions per year: 600 trillion (2008) Project: GCS (Generic Clearing System)
To combine separate individual clearing systems into a single generic clearing
system using new technology and tools, i.e.
HP/Unix servers with Oracle 9
BEA WebLogic for business logic and processing
IBM MQ Series for messaging
J2EE, XML for application development
Case tools: Rational SuiteRose for Requirements & UML models, Requisite Pro for
designing use-cases, TestManager for test case management, ClearQuest for defect
management.
46
8/6/2019 Delivering Quality Software Through Formal Testing
46/75
Development Process System consists of 4 main system components
Development teams structured to match the system components
Using V-Model, requirements for each component were captured, system
models were designed, interface specification were created and codingstarted.
Total Dev team is around 120 people:
~50 Devs, 15 Testers, 4 Test automation Devs
Plus BAs, System Analysts, DBAs, various managers.
47
System
Interface
Trade
Validation
TradeProcessing
Engine
Risk
Management
Settlement
& Clearing
SystemInterface
8/6/2019 Delivering Quality Software Through Formal Testing
47/75
Implication for testing Each system component is to be built with all features fully implemented.
Each component requires its own component testing.
Without having interface to other components, testing each component requires specially
created Test Harnesses
Once each component is completed and tested, they will then be integratedone component at a time.
Integration testing is required between each component.
Once all the components are fully integrated, the entire system will be tested.
System testing is required for the complete system.
Additionally, End-to-End plus performance and stress testing are required.
48
System
Interface
Trade
Validation
TradeProcessing
Engine
Risk
Management
Settlement
& Clearing
SystemInterface
8/6/2019 Delivering Quality Software Through Formal Testing
48/75
Test Harnesses
Q: How to test A without B, and vice-versa?
A:Driversto simulate Up-stream interface
Stubsto simulate Down-stream interface
49
Component
A
Component
B
8/6/2019 Delivering Quality Software Through Formal Testing
49/75
Testing Component A in isolation 1 Driver: Usually a piece code, or an application, which can
send all possible input data range and data type
2 Stubs: To intercept output data from Module A and, if required,
send confirmation back to Module A.
50
Component
ADriver
Stub
Stub
8/6/2019 Delivering Quality Software Through Formal Testing
50/75
Integration Testing To check that when multiple system components are integrated
together, the combined component worked as a single unit.
Understanding the interface specifications between
components and generating test data required to testintegration
Entering data using specially written Drivers and verifying
results via Stubs or other means, e.g. direct SQL calls to DB
tables.
51
8/6/2019 Delivering Quality Software Through Formal Testing
51/75
User Acceptance Testing To verify with the users that the system meets their
requirements
Test cases usually prepared by Testers with theinvolvement of End Users
Tests are run to show positive system behaviour as
well as negative system behaviour, e.g. handling of
error conditions
Tests are run by Users with Testers present
52
8/6/2019 Delivering Quality Software Through Formal Testing
52/75
Outcome of GCS project Project took too long to implement
After 2.5 years we were still at Integration Testing stage
Requirements constantly changed due togovernment and industry regulations
Changing requirements meant the each system has to be
redesigned (data models, interface specs, functional specs),
recoded and retested
After 4 years, company decided to cancel the projectafter spending US $75 million
53
8/6/2019 Delivering Quality Software Through Formal Testing
53/75
8/6/2019 Delivering Quality Software Through Formal Testing
54/75
Traditional Iterative/Incremental
V-Model Agile
Waterfall FDD (Feature Driven Development)
RAD (Rapid Application Development)
DSDM (Dynamic Systems Development Method)
Scrum
RUP (Rational Unified Process)
XP (eXtreme Programming)
TDD (Test Driven Development)
55
8/6/2019 Delivering Quality Software Through Formal Testing
55/75
Part 1 Do we need to test software?
Definitions of Good quality software
Categories of Software
Definition of TestingDevelopment models & testprocesses Case studies:
Waterfall (functional & non-functional testing)
V-Model (integration and UAT)
Agile/Iterative (SCRUM)
Building a test team
Q&A
56
8/6/2019 Delivering Quality Software Through Formal Testing
56/75
Marshall Wace Asset Management Hedge funds investment company
Buys/Sells stocks, currencies and futures
Total assets managed: ~ US $15 Billion Regularly gives a return of > 10% interest
57
8/6/2019 Delivering Quality Software Through Formal Testing
57/75
Project summary To build a new trading system to replace
existing one using new technologies C#, .Net 3, WCF, WGF, XOML
New system must be flexible to react tochanging trading laws and regulations
System is critical to the business It must guarantee 100% reliability
Minimum disruption to the business when newsystem is deployed New system must be introduced slowly and co-exist with the
current system.
58
8/6/2019 Delivering Quality Software Through Formal Testing
58/75
59
Trade idea ValidationOrder
Execution
RestrictionsSettlementTrade Book
Existing system
8/6/2019 Delivering Quality Software Through Formal Testing
59/75
SCRUM
60
8/6/2019 Delivering Quality Software Through Formal Testing
60/75
SCRUM Summary
Each iteration is called a Sprint lasting 2-4 weeks.
Requirements are added to Product Backlog
At the start of the sprint, requirements are picked from the
Backlog to be developed. Usually 5-6 but no more than 10 requirements per sprint.
The sprint team is no bigger than 5-6 people, consisting of
developers and 1 or 2 testers.
Daily meetings for 15 mins. What youve done yesterday
What youre going to do today
Any problems thats stopping you from doing your job.
61
8/6/2019 Delivering Quality Software Through Formal Testing
61/75
Key principles Deliver working software incrementally
Fast ROI (Return on Investment)
Managing changing requirement is part of theprocess
Each increment is time-boxed
Within each increment, all steps of software
lifecycle takes place, i.e. analysis, design, dev &test (mini-waterfall), severaltimes
62
8/6/2019 Delivering Quality Software Through Formal Testing
62/75
Testing in SCRUM As features/functions are added incrementally
over many Sprints, testing has to ensure that:-1. All new features work correctly
2. All existing features do not break.
How? Functional tests: check new features for bugs
Regression tests: check existing features continue to work
Exploratory tests: uncovers unexpected behaviour
63
Sprint 1 Sprint 2 Sprint 3
8/6/2019 Delivering Quality Software Through Formal Testing
63/75
How does SCRUM cope with change? New requirements are added
Existing requirements removed or reprioritised
Each new Sprint only deals with the most up todate requirements
Urgent requirements are brought into current
Sprint by removing planned requirement(s)
64
8/6/2019 Delivering Quality Software Through Formal Testing
64/75
Testing in SCRUM Within the Sprint
Specifications are usually not written down
Communication with Devs and Users is key Functional testing of features as soon as implemented
User Acceptance testing before delivery to user
65
8/6/2019 Delivering Quality Software Through Formal Testing
65/75
Testing in SCRUM Across several Sprints
Regression tests are run at the beginning of new Sprint
In simplest terms, regression means re-running of existing
tests from previous Sprints, to make sure all existing
features continue to work after new code has been added.
But
Sprint 1 = Functional tests 1 (Ft1)
Regression in Sprint 2 = Ft1 Regression in Sprint 3 = Ft2 + Ft1
No of test cases grow very quickly and can become
unmanageable
66
8/6/2019 Delivering Quality Software Through Formal Testing
66/75
Regression testing
67
0
20
40
60
80
100
120
140
160
180
200
1 2 3 4 5
Tests
Cumulative
8/6/2019 Delivering Quality Software Through Formal Testing
67/75
Managing regression tests Selectively re-run only those tests which are
relevant, i.e. if modules A, D and E are affected in
current sprint, rerun tests from modules A, D andE from previous sprints
Automated testing
Custom tool
Commercial tool (Robot, QTP) Open source tools (Selenium, Ruby, Watir)
68
8/6/2019 Delivering Quality Software Through Formal Testing
68/75
Selected low-volume, low-risk trades
Trade is replicated in both new and old system and outputs
compared
69
Trade ideaTrade
ValidationOrder
Execution
RestrictionsSettlementTrade book
Trade validation Order Execution
RestrictionsSettlementTrade book
8/6/2019 Delivering Quality Software Through Formal Testing
69/75
Project Summary Project timescale is 2 years but users were able
to start using the system within 3 months
Deployment was seamless to users Able to change the system when Credit Crunch
happened, i.e.
Disabling short-selling of stocks
Removing Bear Sterns related processes Implementing new changes in law in Hedge Funds
trading
70
8/6/2019 Delivering Quality Software Through Formal Testing
70/75
Part 1 Do we need to test software?
Definitions of Good quality software
Categories of Software
Definition of Testing
Development models & test processes Case studies:
Waterfall (functional & non-functional testing)
V-Model (integration and UAT) Agile/Iterative (SCRUM)
Building a test team
Q&A
71
8/6/2019 Delivering Quality Software Through Formal Testing
71/75
How many testers do you need?
Difference between Developer & Tester
72
8/6/2019 Delivering Quality Software Through Formal Testing
72/75
How many testers do you need? Dev/Test Ratio
Technology consultancies (Accenture, IBM, Logica) =
6-7:1 Global software company, e.g. Microsoft = 1:1
NASA Space shuttle project = 1:3
NASA Space shuttle life support systems = 1:7
Most IT firms = 4:1
73
8/6/2019 Delivering Quality Software Through Formal Testing
73/75
74
This code hasn't yet been tested.
I dont know if it has any bugs.
I only changed one line of code.
Anything is possible, given enough
time.
Let me fix this one final bug, and we
can ship tomorrow.
Its an "undocumented feature"
I like to build things.
This code hasn't yet been tested.
I dont know if its going to work.
The whole system has to be retested.
Everything has flaws, and given enough
time I can prove it.
If you fix this bug, youll add 2 new bugs.
Its a bug.
I like to break things.
Developer speaks The way Tester thinks
8/6/2019 Delivering Quality Software Through Formal Testing
74/75
75
Is it half full or half empty?
Developer (an optimist):
The glass is half full.
Tester (a pessimist):The glass is half empty.
Really good Tester (a realist):
The glass is twice as big as it needs
to be.
8/6/2019 Delivering Quality Software Through Formal Testing
75/75
Books worth reading Testing Computer SoftwareKaner, Falk & Nguyen, 1999, ISBN: 0471358460
Managing the Testing ProcessBlack, 2009, ISBN: 0470404159
How to Break Web Software: Functional and Security Testing of Web Applications and Web Services Andrews & Whitaker,2006, ISBN: 0321369440
Professional Associations BCS SIGIST (British Computer SocietySpecial Interest Group in Software Testing):
http://www.bcs.org/server.php?show=nav.9262
Software Testing Institute: http://www.softwaretestinginstitute.com/
International Institute for Software Testing (IIST): http://www.testinginstitute.com/
Obtaining Professional qualifications BCS offers ISEB 3 types of Certification examinations in Software Testing: Foundation, Intermediate and Practitioner.
http://www.bcs.org/server.php?show=nav.10920
IIST offers 2 types Certifications on software testing: Certified Software Test Professional and Certified Test Manager.http://www.testinginstitute.com/certification.php
Online resources Agile manifesto : http://agilemanifesto.org/
Scrum Alliance: http://www.scrumalliance.org/
James Bach (proponent of Agile testing, Exploratory testing) : http://www.satisfice.com
Open source test tools: http://www.opensourcetesting.org
Most popular test management tool is TestLink: http://www.teamst.org/
Most popular online bug reporting tool is Bugzilla: https://bugzilla.mozilla.org/
Selenium (automated test tool for testing websites on Firefox): http://seleniumhq.org/
Free Tester magazine (available every 3 months) from SIGIST: http://www.bcs.org/server.php?show=nav.9265
http://www.bcs.org/server.php?show=nav.9262http://www.bcs.org/server.php?show=nav.9262http://www.bcs.org/server.php?show=nav.9262http://www.bcs.org/server.php?show=nav.9262http://www.softwaretestinginstitute.com/http://www.testinginstitute.com/http://www.bcs.org/server.php?show=nav.10920http://www.testinginstitute.com/certification.phphttp://www.testinginstitute.com/certification.phphttp://agilemanifesto.org/http://www.scrumalliance.org/http://www.satisfice.com/http://www.opensourcetesting.org/http://www.teamst.org/https://bugzilla.mozilla.org/http://seleniumhq.org/http://seleniumhq.org/https://bugzilla.mozilla.org/http://www.teamst.org/http://www.opensourcetesting.org/http://www.satisfice.com/http://www.scrumalliance.org/http://agilemanifesto.org/http://www.testinginstitute.com/certification.phphttp://www.bcs.org/server.php?show=nav.10920http://www.testinginstitute.com/http://www.softwaretestinginstitute.com/http://www.bcs.org/server.php?show=nav.9262