Back to Basics – Testing Process and Methodology Fundamentals

Software Testing (back to the basics)

Critical activities to make your testing

efforts more successful!

June 3, 2014

DOES YOUR TEST STRATEGY FAIL MISERABLY?

3©Alliance Global Services 2014

a story from the past

At the conclusion of 3 years of development and QA efforts

Development built an application

QA tested an application

QA tested and signed-off project completion

Application deployed out to the field

End-users could not conduct their business

End-users rejected the applicationo Lost (Time, Money, Reputation)

Application was pulled from production

QA was blamed for the failure

An assessment of the overall project determined that many issues contributed to the failure

When asked what was the cause: Your QA Strategy sucks!

“QA has the responsibility of maintaining the quality level of a company and its products.”


responsibility of QA

“QA provides process and a system to ensure that company

developed products work and that they meet or exceed

customer requirements and expectations!”


testing basics

Description of what is going to be built

o Test acceptance criteria

Test cases

o Traceability

Functional test coverage

o Ad-hoc Testing

Positive and negative testing

Non-functional testing

o Performance / Security / Usability

Regression testing

o Risk based testing and test case priority

Defect management

Test metrics

User Acceptance Testing

UNDERSTANDING WHAT YOU ARE BUILDING


what are you building


requirements

user stories

goalsroles

reason

epic

themes

requirements


written requirements / documentation

Tree Swing

FR: 001


waterfall and agile

Waterfall SDLC CyclesPlanning Phase Design Phase Construction Phase QA Phase

Planning Design Construction Testing

Agile SDLC CyclesFeature 1 Feature 2 Feature 3 Feature 3

Planning

Design

Construction

Testing

Planning

Design

Construction

Testing

Planning

Design

Construction

Testing

Planning

Design

Construction

Testing

Time


test acceptance criteria

pass

fail

=

<>


summary

Requirements

Written Documentation

Location to store and maintain and search the documentation

Process to review requirements

Requirements that are testable


TEST CASES


test cases

What is a test case?

How to determine the level of effort required for your test cases…


no formal test case

Complete faith in testers

QA in-depth knowledge of system

Access to business

Access to development

Testers remain on the project for

the duration

Example #1:

Testing CompleteX


Minimal test case

Complete faith in testers

QA in-depth knowledge of system

Access to business

Access to development

Testers remain on the project for

the duration

Example #2:

Test the functionality of the login screen to be sure that the user can login.


formal test case

Inexperienced test team

Have experienced lack of quality

in the past

Anyone can execute the test case

Required for automated testing

Example #3:

Test the functionality of the login screen:1: Navigate to the login

screen2: Enter a valid user name3: Enter a valid password4: Click on the login button5: Run again pressing enter

on the login button

Expected results: should be left on the home screen


risk based testing and test case priority

RBT helps project managers to balance Risk

with project constraints like Schedule,

Resource and Cost. This is achieved by

prioritizing the system to identify the areas

that carry a higher business risk. Hence

Projects can perform intelligent testing over

complete testing

*About 80% of the Defects come from 20% of the Modules

Identification of critical defect at an early stage of testing

Considerable Decrease in QA cost

Faster Time-to-Market

Increase involvement between Business and Testing

Collaborative working model between SME’s and testers

Business can make informed decision on the Risk v/s Testing Level forrelease acceptability

Over Testing the application

Large amount of effort is spent of areas of the applicationwhich carry very low risk

Minimal involvement of business in Quality Control

High Cost

Conventional Testing Risk Based Testing

Conventional Testing V/S RBT


simple risk and priority calculator

Scenario Probability of Failure

Complexity of Module

Size of Change

Frequencyof Use

Impact of Failure

Cost of Failure

Priority

Test Case 2 1 1 1 1 1 7

Test Case 1 2 3 4 4 2 16

Test Case 4 4 1 1 2 2 14

Test Case 4 3 3 3 4 4 21

1=High, 2=Medium, 3=Low, 4=None1=Big, 2=Medium, 3-Small, 4=None


traceability

Original Documentation of what you are

Building

Dev

elo

pm

en

tQuality

Assurance and Testing Efforts

FinishedProduct


FUNCTIONAL TEST COVERAGE


test coverage reporting


positive and negative testing

Positive Testing – System works as expected

Negative Testing – system can handle unexpected input

Expected result when certain data is used.

What happens when unexpected data is used?


non-functional testing

Performance Testing – meet SLAs

Security Testing – protected data and property

Usability Testing – end user can successfully use product


ad-hoc testing

Let the testers and SME’s play with the system!

Chase down that rabbit hole

The Grandma test

Test engineers are unbound by formal practices


regression testing

Verifies that previously tested features are not impacted by

new development or defect fixes

Generally a mixture of positive and negative test cases

Can be designed for complete coverage or partial coverage

through risk-based test planning


user acceptance testing

User Acceptance Testing (UAT) is a final test of the application

by the users of the application before release.

o Last chance to get a new perspective outside of Engineering

o Testing can be formal scripts or ad-hoc “play with it” testing

It is not based on detailed requirements and specs

o High level contractual agreements and

o General look and feel, and usability from the end-users point of view

not the engineers.

SDLC Differences

o Waterfall: this is last step before release

o Agile: this is done within the iteration by story during the Product

Owner Acceptance and often as waterfall type release UAT testing

of system as a whole


defect and metrics management

Defect management

o Best done with a tool

o Memory

o Excel

Test Metrics

o Track the progress and output of testing

o Defects found per cycle

o Cases created / executed

o Pass / fail ratios

o Overall test coverage


Thank you

Date post:	01-Jun-2015
Category:	Technology
Upload:	alliance-global-services
View:	237 times
Download:	0 times

Back to Basics – Testing Process and Methodology Fundamentals

Technology