Learn Software Testing & Prepare for ISTQB Foundation Certification Exam >>>>>>>>>>>>>>>>>>>>>> www.softwaretestinggenius.com <<<<<<<<<<<<<<<< <<<<<<
Transcript
Slide 1
Learn Software Testing & Prepare for ISTQB Foundation
Certification Exam
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Contents Part - 1: Quality Part - 2: Software Testing &
Development Models Part - 3: Different Stages of SDLC with STLC
Part - 4: Testing Techniques Part - 5: Types of Testing Part - 6:
Testing Life Cycle Part - 7: Defects Management Part - 8: Test
Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Quality has two elements QA - Quality Assurance QC - Quality
Control Part 1: Quality
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Quality from the Customers Viewpoint Fitness for use, or other
customer needs Producers Viewpoint Meeting requirements What is
Quality ?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Software quality includes activities related to both Process,
and the Product Quality Assurance is about the work process Quality
Control is about the product Quality Function
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Quality assurance activities are work process oriented. They
measure the process, identify deficiencies, and suggest
improvements. The direct results of these activities are changes to
the process. These changes can range from better compliance with
the process to entirely new processes. The output of quality
control activities is often the input to quality assurance
activities. Audits are an example of a QA activity which looks at
whether and how the process is being followed. The end result may
be suggested improvements or better compliance with the process.
What is Quality Assurance?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Quality control activities are work product oriented. They
measure the product, identify deficiencies, and suggest
improvements. The direct results of these activities are changes to
the product. These can range from single-line code changes to
completely reworking a product from design. They evaluate the
product, identify weaknesses and suggest improvements. Testing and
reviews are examples of QC activities since they usually result in
changes to the product, not the process. QC activities are often
the starting point for quality assurance (QA) activities. What is
Quality Control?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Prevention is better than cure...... but not everything can be
prevented! Cure Detection Prevention Prevention and Detection
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
QA Assurance Process Preventive Quality Audit QC Control
Product Detective Testing QA and QC Broad Difference
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Does this software product work as advertised? Functionality,
Performance, System & User Acceptance... Testing Will the users
be able to do their jobs using this product? Installability,
Compatibility, Load/Stress... Testing Can they bet their business
on this software product? Reliability, Security, Scalability...
testing Quality... its all about the End-User
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Real World Software Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Testing What is testing Objectives of Testing Part 2: Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Definition: Testing is process of trying to discover every
conceivable fault or weakness in a work product. Testing is a
process of executing a program with the intent of finding an error.
A good test is one that has a high probability of finding an as yet
undiscovered error. A successful test is one that uncovers an as
yet undiscovered error What is Testing?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Testing is a process used to identify the correctness,
completeness and quality of developed computer software. Testing,
apart from finding errors, is also used to test performance,
safety, fault-tolerance or security. Software testing is a broad
term that covers a variety of processes designed to ensure that
software applications function as intended, are able to handle the
volume required, and integrate correctly with other software
applications. What is Testing?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
What are the right things to test? Are you testing an entire
product or some components? Which functions are critical for a
product? Which functions are highly visible when they fail? Which
functions have been highly error-prone? Which functions are most
often used? Which functions are changed most often / recently?
Which functions suffered turnover of employees? Where is high
geographic distribution of work? Which functions require new skills
/ technology? Looks Impossible to do it all!
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Objective of testing is to find all possible bugs (defects) in
a work product Testing should intentionally attempt to make things
go wrong to determine if things happen when they shouldn't or
things don't happen when they should. What is objective of Testing?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Setting up criteria for testing Criteria can be at phase level
Entry Criteria Parallelism Vs Ready for use Exit Criteria
Completeness Vs Risk of release Suspension Criteria Show stopper
bugs Crossing a threshold number of bugs Developers producing a new
version making the old one redundant Resumption Criteria Above
hurdles being cleared
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
General Testing Principles Testing shows presence of Defect
Exhaustive Testing is Impossible Early Testing Defect Clustering
Pesticide Paradox Testing is context dependent Absence-of-error
fallacy
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Identify defects when the software doesnt work Verify that it
satisfies specified requirements verify that the software works
Summary
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
A mature view of software testing is to see it as a process of
reducing the risk of software failure in the field to an acceptable
level [Bezier 90]. Mature view of software testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The Goal of a software tester is to find defects And find them
as early as possible. What exactly Does a Software Tester Do?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Testers hunt errors Detected errors are celebrated - for the
good of the work product Testers are destructive - but creatively
so Testing is a positive and creative effort of destruction Testers
pursue errors, not people Errors are in the work product, not in
the person who made the mistake Testers add value by discovering
errors as early as possible What does testing mean to testers?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
By examining the users requirements, internal structure and
design, functional user interface etc By executing the code,
application software executable etc How testers do it?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Waterfall Model for Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Simple and easy to use. Easy to manage due to the rigidity of
the model each phase has specific deliverables and a review
process. Phases are processed and completed one at a time. Works
well for smaller projects where requirements are very well
understood. Advantages of Waterfall Model
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Adjusting scope during the life cycle can kill a project No
working software is produced until late during the life cycle. High
amounts of risk and uncertainty. Poor model for complex and
object-oriented projects. Poor model where requirements are at a
moderate to high risk of changing. Disadvantages of Waterfall Model
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
It is a process of establishing requirements, designing,
building and testing a system, done as a series of smaller
development Increment produced by Iteration is tested at several
levels as part of its development Regression testing is important
after all iterations Verification and verification can be carried
outs on each iteration Disadvantages of Waterfall Model
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Simple and easy to use. Each phase has specific deliverables.
Higher chance of success over the waterfall model due to the
development of test plans early on during the life cycle. Works
well for small projects where requirements are easily understood.
Advantages of W - Model
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Very rigid, like the waterfall model. Little flexibility and
adjusting scope is difficult and expensive. Software is developed
during the implementation phase, so no early prototypes of the
software are produced. Model doesnt provide a clear path for
problems found during testing phases. Disadvantages of W - Model
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stages of SDLC with STLC Part 3: Different stages of SDLC with
STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Different stages of SDLC with STLC Stage-1 Requirement
Gathering DA: Defining requirements to establish specifications is
the first step in the development of software. TA: Reviewing the
requirements e.g the requirements should not have ambiguous words
like (may or may not). It should be clear and concise.
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stage-2 Functional Specification DA: It describes the products
behaviors as seen by an external observer, and contains the
technical information and data needed for the design. The
functional specification defines what the functionality will be TA:
In order to make the functional specifications accurate we have
review our functional specifications. Different stages of SDLC with
STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stage-3 Design DA: The software specifications are transformed
in to design models that describe the details of the data
structures, system architecture, interface and the components. TA:
Each design product is reviewed for quality before moving the next
phase of the software development. Different stages of SDLC with
STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stage-4 Code DA: In this phase the designs are translated into
code. TA: Code review is a process of verifying the source code.
Code review is done the find and fix the defects that are
overlooked in the initial development phase, to improve overall
quality of code. Different stages of SDLC with STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stage-5 Building Software DA: In this phase we build different
software units and integrate them one by one to build single
software. TA: Unit testing & Integration testing Different
stages of SDLC with STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stage-6 Building system DA: After the software has been build
we have the whole system considering all the non-functional
requirements like installation procedure, configuration etc. TA:
System testing & Acceptance testing Different stages of SDLC
with STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Stage-7 Release for use After the whole product has been
developed and the required level of quality has been achieved and
the software is release for the actual use of the customers.
Different stages of SDLC with STLC
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Testing Techniques Verification and Validation Part 4: Testing
Techniques
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Find all the missing information Who What Where When Why How
Verification What to Look For?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Simply giving a document to a colleague and asking them to look
at it closely which will identify defects we might never find on
our own. Peer Review
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Informal meetings, where participants come to the meeting;and
the author gives the presentation. Objective: To detect defects and
become familiar with the material Elements: A planned meeting where
only the presenter must prepare A team of 2-7 people, led by the
author Author usually the presenter. Inputs: Element under
examination, objectives for the walkthroughs applicable standards.
Output: Defect report Walkthrough
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Formal meeting, characterized by individual preparation by all
participants prior to the meeting. Objectives: To obtain defects
and collect data. To communicate important work product
information. Elements: A planned, structured meeting requiring
individual preparation by all participants. A team of people, led
by an impartial moderator who assure that rules are being followed
and review is effective. Presenter is reader other than the author.
Other participants are inspectors who review, Recorder to record
defects identified in work product Inspection
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
An important tool specially in formal meetings like inspections
They provide maximum leverage on on verification There are generic
checklists that can be applied at a high level and maintained for
each type of inspection There are checklists for
requirements,functional design specifications, internal design
specifications, for code Checklists : the verification tool
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
There are two main strategies for validating software White Box
testing Black Box testing Validation Strategies
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
White Box Testing Deals with the internal logic and structure
of the code The tests are written based on the white box testing
strategy incorporate coverage of the code written, branches, paths,
statements and internal logic of the code etc. Normally done the
developers Validation Strategies
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
White Box Testing can be done by: Data Coverage Code Coverage
White Box testing Methods
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Date Coverage Data flow is monitored or examined through out
the program. E.g. watch window we use to monitor the values of the
variables and expressions. White Box testing Methods
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Code Coverage Its a process of finding areas of a program not
exercised by a set of test cases, Creating additional test cases to
increase coverage Code coverage can be implemented using basic
measure like, statement coverage, decision coverage, condition
coverage and path coverage White Box testing Methods
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Black Box Testing Does not need any knowledge of internal
design or code Its totally based on the testing for the
requirements and functionality of the work product/software
application. Tester is needed to be thorough with the requirement
specifications of the system and as a user, should know how the
system should behave in response to the particular action.
Validation Strategies
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The following are commonly used Black Box methods : Equivalence
partitioning Boundary-value analysis Error guessing Black Box
testing Methods
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
An equivalence class is a subset of data that is representative
of a larger class. Equivalence partitioning is a technique for
testing equivalence classes rather than undertaking exhaustive
testing of each value of the larger class. Equivalence Partitioning
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
If we expect the same result from two tests, you consider them
equivalent. A group of tests from an equivalence class if, They all
test the same thing If one test catches a bug, the others probably
will too If one test doesnt catch a bug, the others probably wont
either Equivalence Partitioning
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
For example, a program which edits credit limits within a given
range ($10,000-$15,000) would have three equivalence classes: Less
than $10,000 (invalid) Between $10,000 and $15,000 (valid) Greater
than $15,000 (invalid) Equivalence Partitioning
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Partitioning system inputs and outputs into equivalence sets If
input is a 5-digit integer between 10,000 and 99,999 equivalence
partitions are 99,999 The aim is to minimize the number of test
cases required to cover these input conditions Equivalence
Partitioning
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Equivalence classes may be defined according to the following
guidelines: If an input condition specifies a range, one valid and
two invalid equivalence classes are defined. If an input condition
requires a specific value, then one valid and two invalid
equivalence classes are defined. If an input condition is Boolean,
then one valid and one invalid equivalence class are defined.
Equivalence Partitioning
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Divide the input domain into classes of data for which test
cases can be generated. Attempting to uncover classes of errors.
Based on equivalence classes for input conditions. An equivalence
class represents a set of valid or invalid states An input
condition is either a specific numeric value, range of values, a
set of related values, or a Boolean condition. Equivalence classes
can be defined by: If an input condition specifies a range or a
specific value, one valid and two invalid equivalence classes
defined. If an input condition specifies a Boolean or a member of a
set, one valid and one invalid equivalence classes defined. Test
cases for each input domain data item developed and executed.
Equivalence Partitioning Summary
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Bugs lurk in corners and congregate at boundaries Boris Beizer
Boundary value analysis
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
A technique that consists of developing test cases and data
that focus on the input and output boundaries of a given function.
In same credit limit example, boundary analysis would test: Low
boundary plus or minus one ($9,999 and $10,001) On the boundary
($10,000 and $15,000) Upper boundary plus or minus one ($14,999 and
$15,001) Boundary value analysis
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Large number of errors tend to occur at boundaries of the input
domain BVA leads to selection of test cases that exercise boundary
values BVA complements equivalence partitioning. Rather than select
any element in an equivalence class, select those at the ''edge' of
the class Examples: For a range of values bounded by a and b, test
(a-1), a, (a+1), (b-1), b, (b+1) If input conditions specify a
number of values n, test with (n-1), n and (n+1) input values Apply
1 and 2 to output conditions (e.g., generate table of minimum and
maximum size) Boundary value analysis
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Account number 567 invalid valid invalid Number of digits:
First character: invalid: zero valid: non-zero
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Based on the theory that test cases can be developed based upon
the intuition and experience of the Test Engineer For example, in
an example where one of the inputs is the date, a test engineer
might try February 29,2000 or 9/9/99 Error Guessing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Error guessing is an ad hoc approach, based on intuition and
experience, to identify tests that are considered likely to expose
errors The basic idea is to make a list of possible errors or error
prone situations and then develop tests based on the list. E.g.
Where one of the input is the date, a test engineer may try
February 29, 2000 A=B/C What if C is 0? Error Guessing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Part 5: Various Types of Testing Validation is done at two
levels Low Level Unit testing Integration Testing High Level
Function Testing System Testing Acceptance Testing Validation
Activities
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Searches for defect and verifies the functionality of software,
depending upon the context of the development It includes testing
of functional and non-functional characteristics It occurs with
access to code being tested and with the support of development
environment Defects are fixed as soon as they are found with out
formally recording incident If test cases are prepared and
automated before coding, it is termed as test-first approach or
test-driven development. Unit Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Integration Testing Integration testing tests interface between
components, interaction to different parts of system. Greater the
scope of Integration, more it becomes to isolate failures to
specific component or system, which may leads to increased risk.
Integration testing should normally be integral rather than big
bang, in order to reduce the risk of late defect discovery Non
functional characteristics (e.g. performance) may be included in
Integration Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Function Testing It is used to detect discrepancies between a
programs functional specification and the actual behavior of an
application. The goal of function testing is to verify whether your
product meets the intended functional specifications laid out the
development documentation. When a discrepancy is detected, either
the program or the specification is incorrect. All the black box
methods are applicable to function based testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
It is concerned with the behavior of whole system as defined by
the scope of development project It includes both functional and
non-functional requirement of system System testing falls within
the scope of black box testing. On building the entire system, it
needs to be tested against the system specification. An Independent
testing team may carry out System Testing System Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Usability Testing The typical aim of usability testing is to
cause the application to fail to meet its usability requirements so
that the underlying defects can be identified, analyzed, fixed, and
prevented in the future. Performance testing is testing to ensure
that the application response in the limit set by the user.
Performance Testing Subject the system to extreme pressure in a
short span. E.g Simultaneous log-on of 500 users Saturation load of
transactions Stress Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Configuration Testing Configuration testing is the process of
checking the operation of the software you are testing with all
these various types of hardware. Compatibility Testing The purpose
of compatibility testing is to evaluate how well software performs
in a particular hardware, software, operating system, browser or
network environment.
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Acceptance Testing Acceptance testing may assess the system
readiness for deployment and use The goal is to establish
confidence in the system, parts of system or non-functional
characteristics of the system Following are types of Acceptance
Testing: User Acceptance Testing Operational Testing Contract and
Regulation Acceptance Testing Alpha and Beta Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
In development Testing, main objective is to cause as many
failures as possible. In Acceptance Testing, main objective is to
confirm that system work as expected. In Maintenance Testing, main
objective is to make sure that no new errors have been introduced.
In Operational testing, main objective may be to access system
characteristics such as reliability and availability. Objectives of
Different Types of Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Other Testing Types Other than validation activities like unit,
integration, system and acceptance we have the following other
types of testing Mutation testing Progressive testing Regression
testing Retesting Localization testing Internationalization testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Mutation testing is a process of adding known faults
intentionally, in a computer program to monitor the rate of
detection and removal, and estimating the umber of faults remaining
in the program. It is also called Be-bugging or fault injection.
Mutation testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Most test cases, unless they are truly throw-away, begin as
progressive test cases and eventually become regression test cases
for the life of the product. Progressive Testing and Regressive
Testing Regression testing is not another testing activity It is a
re-execution of some or all of the tests developed for a specific
testing activity for each build of the application Verify that
changes or fixes have not introduced new problems It may be
performed for each activity (e.g. unit test, function test, system
test etc) Regression Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Testing software that has been tested before. Why? Peripheral
testing for bug fixes Retesting the old version functionality with
the new version Regression Test Why retest? Because any software
product that is actively used and supported must be changed from
time to time, and every new version of a product should be retested
Retesting
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The process of adapting software to a specific locale, taking
into account, its language, dialect, local conventions and culture
is called localization. Localization Testing The process of
designing an application so that it can be adapted to various
languages and regions without engineering changes.
Internationalization Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Test Types : The Target of Testing Testing of functions
(functional testing) It is the testing of what the system does
Functional testing considers external behavior of the system
Functional testing may be performed at all test levels Testing of
software product characteristics (non-functional testing) It is the
testing of How the system works Nonfunctional testing describes the
test required to measure characteristics of systems and s/w that
can be quantified on varying scale Non-functional testing may be
performed at all levels
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Testing of software structure/architecture (structural testing)
Structural testing is used in order to help measure the
thoroughness of testing through assessment of coverage of a type of
structure Structural testing may be performed at all levels.
Testing related to changes (confirmation and regression testing)
When a defect is detected and fixed then the software should be
retested to confirm that the original defects has been successfully
removed. This is called Confirmation testing Regression Testing is
the repeated testing of an already tested program, after
modification, to discover any defects as a result of changes.
Regression Testing may be performed at all levels. Test Types : The
Target of Testing
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Software Development and Software Testing Life Cycle Part-6 :
Software Development & STLC It is the process of defining a
testing project such that it can be properly measured and
controlled It includes test plan, test strategy, test requirements
and testing resources Test Planning
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Parts of Test Planning Commn Mgmt Risk Mgmt Test Script And
Scheduling Identifying Test Deliverables Identifying Env needs
Identifying Skill sets / Trng Setting Entry / Exit Criteria
Deciding Test Strategy Scope Mgmt Preparing A Test Plan Test
Planning Start Here
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
It is the process of defining test procedures and test cases
that verify that the test requirements are met Specify the test
procedures and test cases that are easy to implement, easy to
maintain and effectively verify the test requirements. Test Design
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Test Design Techniques During Test Design, the test basis
documentation is analyzed in order to determine what to test i.e
test condition A test condition is defined as an item that could be
verified by one or more test cases Test case consist of a set of
input values, execution preconditions, expected results and
expected post conditions; are developed and described by using test
design techniques Expected results should ideally be defined prior
to test execution Test cases are put in an executable order, this
is test procedure specification If tests are run using test
execution tool, the sequence of actions is specified in a test
script The test procedures and automated test scripts are
subsequently formed into a test execution schedule
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Specification (White-box) Based techniques Models, either
formal or informal; are used for the specification of problem to be
solved From these models test cases can be derived systematically
Structure (Black-box) Based techniques Information about how the
software is constructed is used to derive the test cases The extent
of coverage of the software can be measured for existing test cases
and further test cases can be derived systematically Experienced
Based techniques The knowledge and experience of people are used to
derive test cases Knowledge about likely defects and their
distribution Categories of Test Design Techniques
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Specification based/White Box techniques Equivalence
Partitioning Boundary value analysis Decision Table (Cause and
Effect) Testing Use case Testing Structure/Black Box based
techniques Code Coverage Branch Coverage Statement Coverage 100%
decision coverage guarantees 100% statement coverage, but not
vice-versa Categories of Test Design Techniques
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Test Planning is a continuous activity and is performed in all
the life cycle processes and activities Test Planning activities
includes: Defining the overall approach Integrating and
coordinating the testing activities into software life cycle
activities Assigning resources for different tasks defined Defining
the amount, level of detail, structure and templates for test
documentation Selecting metrics for monitoring and controlling test
preparation Making decisions about what to test, what roles will
perform the test activities, when and how test activities should be
done, how the test results will be evaluated and when to stop the
testing Test Management
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Exit Criteria Defines when to stop testing Exit criteria may
consist of Thoroughness measures, such as coverage of code,
functionality or risk Estimates of defect density or reliability
measures Cost Residual risk Schedules such as those based on time
to market Test Management
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Risk is defined as combination of likelihood and its impact it
would have on the user. Risk and Testing The level of risk will be
determined by the likelihood of an adverse event happening and the
impact Risk associated with testing can be classified in two
categories: Project Risk: The risks that surround the projects
capability to deliver its objective Product Risk: Potential
failures in the software or system are known as product risk, as
they are a risk to the quality of the product
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Suppliers Issues Failure of a third party Contractual Issues
Organizational Factors Skill and staff shortage Personal and
training issues Potential issues, such as problem with testers
communication, failure to follow up the information found in
Testing Improper attitude towards testing Technical Issues Problem
in defining the right requirement The extent that requirements can
be met given existing constraints Quality of design, code and tests
Project Risk Objectives
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Product Risks Objective Error prone software delivered
Potential that the software/hardware could cause harm to
company/individual Poor software characteristics Software that does
not perform its intended functions A risk based approach to testing
provides proactive opportunities to reduce the levels of product
risks, starting in the initial stages of project Product Risk
Objectives
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
It is the process of creating test procedures and test cases
that verify the test requirements Automated Testing using Tools
Manual Testing Test Development
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The basis of test is the source material (of the product under
test) that provide the stimulus for the test. In other words, it is
the area targeted as the potential source of an error:
Requirements-based tests are based on the requirements document
Function-based tests are based on the functional design
specification Internal-based tests are based on the internal design
specification or code. Function-based and internal-based tests will
fail to detect situations where requirements are not met.
Internal-based tests will fail to detect errors in functionality
Test Basis
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
An activity in which a system or component is executed under
specified conditions, the result are observed or recorded, and an
evaluation is made of some aspect of the system or component. A set
of one or more test cases. Test
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Definition of result A necessary part of a test case is a
definition of the expected output or result. Repeatability Two
basic requirements for all validation tests
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Following matrix can be used to "trace" requirement statements
to this use case. Requirement IDUse Case SRS0006Upload Photos Use
Case The above trace-ability matrix tells that the use case "Upload
Photos" satisfies the requirement SR0006 (Easy to upload photos,
create galleries and enter info about the photo).
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
A set of test inputs, execution conditions, and expected
results developed for a particular objective The smallest entity
that is always executed as unit, from beginning to end A test case
is a document that describes an input, action, or event and an
expected response, to determine if a feature of an application is
working correctly A test case should contain particulars such as
test case identifier, test case name, objective, test
conditions/setup, input data requirements, steps, and expected
results Test case may also include prerequisites Test Case
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Procedure: This is the process a tester needs to perform to
invoke the condition and observe the results. A test case procedure
should be limited to the steps needed to perform a single test
case. Expected Result: This is the observable result from invoking
a test condition. If you cant observe a result, you cant determine
if a test passes or fails. In the previous example of an e-commerce
shipping option, the expected results would be specifically defined
according to the type of shipping the user selects. Pass/Fail: This
is where the tester indicates the outcome of the test case. For the
purpose of space, I typically use the same column to indicate both
"pass" (P) and "fail" (F). In some situations, such as the
regulated environment, simply indicating pass or fail is not enough
information about the outcome of a test case to provide adequate
documentation. For this reason, some people choose to also add a
column for "Observed Results." Test Case Components
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com > www.softwaretestinggenius.com
> www.softwaretestinggenius.com www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
>>>>>>>>>>>>>>>>>>>>>
www.softw">
Report a defect The point of writing Problem Reports is to get
bugs fixed. What Do You Do When You Find a defect?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Summary Date reported Detailed description Assigned to Severity
Detected in Version Priority System Info Status Reproducible
Detected by Screen prints, logs, etc. Some typical defect report
fields
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Project Manager Executives Development Customer Support
Marketing Quality Assurance Any member of the Project Team Who
reads the defect reports?
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Part 8: Test Automation Automate tests that absolutely cannot
be done manually (e.g., stress test) Automate those tests that are
part of the regression bed / smoke test Automate tests that are run
repeatedly (e.g., tests that are run at least 20 times over the
next quarter) Automate tests with no manual intervention Automate
tests for standards (e.g., JDBC compliance) Automate tests that are
covering areas static but important areas Principles of Test
Automation 1: Choose carefully what to automate
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Automate tests for highly visible areas Minimize automating
change-prone areas Between GUI and non-GUI portion automation, go
for automating non- GUI portions first Automate tests for
dependencies to catch ripple effects early Automate areas where
multiple combos are possible (pros and cons) Automate areas that
are re-usable Automate easy areas to show low hanging fruits Part
8: Test Automation Principles of Test Automation # 1: Choose
carefully what to automate
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Principles of Test Automation # 2: Ensure Automation Covers
Full Circle Automatic Analysis Fish Bone Diagrams Problem
Identification Test Capture Test Execution Results Comparison Test
Planning Automation Planning Corrective Action Implementation
Automatic Rollover to next runs Incorporation into Regression
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Compatibility to Platform Portability across platforms
Integration with TCDB, DR and SCM 2-way mapping to source code (may
not be possible in services) Scripting Language Compatible to
Multiple Programming Environments Configurability Test Case
Reusability Selective Execution Smart Comparison Reliable Support
Current documentation Principles of Test Automation # 3: Choose
Proper Automation Tool
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Resources for Installation Resources for ongoing execution
People Resources Principles of Test Automation # 4: Plan for
Infrastructure
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Training Development Testing the Tests Sync-ing with product
version changes Principles of Test Automation # 5: Account for
Gestation Period
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Start small Dont try to automate everything at the same time
Allow time for evolving standards Principles of Test Automation #
6: Run a Trial & Calibrate the Tool
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Config FileScenarios Tools Results Reports/ Metrics Test
Framework Test cases Report Generator ExecuteRead Execute Modify
Write Read Write Test Automation Components
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Planning Reqmts Design Test Design Test planning Test reqmts
Coding The V - Model of Software Development Test Execution Process
of Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The W - Model of Test Automation Process of Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The work on automation should go in parallel with product
development The requirements for automation spans over to multiple
phases for multiple releases like the product requirements The
automation can be carried out in parallel with test design
Sometimes the test cases need to be modified to suit the automation
requirements; where automation exist the test cases need not be
elaborate Where a dedicated team exist for automation, the schedule
for automation can be independent of product releases, with some
(tested) deliverables marked for each product release Process of
Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
There are plenty of tools available and rarely does one tool
meet all the requirements The test tools are expensive (both in
upfront costs and running costs) Test tools also require good
amount of training and only few vendors available for training
Training may not always keep pace with new versions of the tools
Test tools expect the users to learn new language/scripts and may
not use standard languages/scripts Deploying a test tool requires
equal amount of effort as deploying a new product in a company
never underestimate the effort and pain involved! Common
Experiences in Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Migrating from one test tool to another may be difficult and
requires good amount of effort Test tools are one generation behind
and may not provide backward / forward compatibility (eg. JAVA SDK
support) Good number of test tools requires their libraries linked
with product binaries Causes portions of the testing to be repeated
after those libraries are removed (eg. Performance) Test tools are
not 100% cross platform They are supported only on some platforms
and the sources generated from these tools may not be compatible on
other Developing sharewares/public domain test tools may not get
same amount of participation/involvement/support as of
standards/products (eg. As against Linux) Common Experiences in
Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The experiences Test tools may not go through same amount of
evaluation for new requirements (eg Year 2000, 508) The test tools
increases the system requirements and requires the H/W and S/W to
be upgraded at compile/run-time The test tools are capable of
testing only the product, not the impact because of the
product/test tool to the system or network Good number of test
tools cant differentiate between a product failure and the test
suite failure Causing increased analysis time and manual testing
Common Experiences in Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
The experiences The test tools may not provide good degree of
trouble shooting / debug/error messages to help in analysis
Resulting in increased printf/log messages in the test suite The
test tools determine the results based on messages and screen co-
ordinates at run-time Intelligence needed to proactively find out
the changes Common Experiences in Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Automation shouldnt be considered as stop-gap arrangement to
engage test engineers (when no test execution, do automation!).
Test Automation, like any other project, should start with the end
in mind A separate team in the organization looking at automation
requirements, tool evaluation and developing generic test suites
would add more value (may not always apply to testing services
organization) Automation doesnt stop with automating the test cases
alone. The test suite needs to be linked with other tools for
increased effectiveness (e.g., Test case database, Defect filing,
auto mails, preparing automatic reports, etc) Automation doesnt
stop with recording & playing back the user commands; Automated
tool should be intelligent enough to say what was expected, why a
test case failed and give manual steps to reproduce the problem
Common Pitfalls in Test Automation
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com
Thank You
>>>>>>>>>>>>>>>>>>>>>>
www.softwaretestinggenius.com