7/29/2019 Pressman7Ch17 Modfied by Arvind
1/57
UNIT NO 3Software Testing Strategies and Method
Marks 20
Software Engineering: A PractitionersApproach, 7/eby Roger S. Pressman
Modified by
Mr. Arvind S. Sardar (Lecturer)
MIT Polytechnic Rotegaon
. 1
7/29/2019 Pressman7Ch17 Modfied by Arvind
2/57
Software Testing
2
Testing is the process of exercising a programwith the specific intent of finding errors priorto delivery to the end user.
Testing is a set of activity that can be plannedin advance and conducted systematically.
7/29/2019 Pressman7Ch17 Modfied by Arvind
3/57
What Testing Shows
3
errors
requirements conformance
performance
an indicationof quality
7/29/2019 Pressman7Ch17 Modfied by Arvind
4/57
Testing Objectives Testing is a process of executing a program with the intent of finding
an error.
A good test case is one that has a high probability of findingan as-yet-undiscovered error.
A successful test is one that uncovers an as-yet-undiscovered error.
4
7/29/2019 Pressman7Ch17 Modfied by Arvind
5/57
Testing Principles All tests should be traceable to customer requirements.
Tests should be planned long before testing begins.
The Pareto principle applies to software testing.
Testing should begin in the small and progress toward testing in the
large.
Exhaustive testing is not possible.
To be most effective, testing should be conducted by an independent
third party. 5
7/29/2019 Pressman7Ch17 Modfied by Arvind
6/57
Testability Operability It operates cleanly
Observability The results of each test case are readily observed
Controllability The degree to which testing can be automated and
optimized
DecomposabilityTesting can be targeted
Simplicity Reduce complex architecture and logic to simplify
tests
Stability Few changes are requested during testing
Understandability Of the design 6
7/29/2019 Pressman7Ch17 Modfied by Arvind
7/57
What is a Good Test? A good test has a high probability of finding an error
A good test is not redundant.
A good test should be best of breed
A good test should be neither too simple nor too complex
7
7/29/2019 Pressman7Ch17 Modfied by Arvind
8/57
Test Plan, Test Case and test
Data
Test Plan:-
Atest plan is a document detailing a systematic approach
to testing a system such as a machine or software.
A test plan documents the strategythat will be used toverifyand ensure that a product or system meets its
design specifications and other requirements.
A test plan can be defined as a document describing the
scope, approach, resources, and schedule of
intended testing activities.
It identifies test items, the features to be tested, the testing
tasks, who will do each task, and any risks requiring
contingency planning.
8
http://en.wikipedia.org/wiki/Machinehttp://en.wikipedia.org/wiki/Softwarehttp://en.wikipedia.org/wiki/Softwarehttp://en.wikipedia.org/wiki/Machine7/29/2019 Pressman7Ch17 Modfied by Arvind
9/57
Test Plan, Test Case and test
Data
Test Data:-
Test Data is data which has been specifically identified for
use in tests, typically of a computer program.
Example:- Confirmatory way :- To verify that a given set of input to a
given function produces some expected result.
To challenge the ability of the program to respond to
unusual, extreme, exceptional, or unexpected input.
9
7/29/2019 Pressman7Ch17 Modfied by Arvind
10/57
Test Plan, Test Case and test
Data
Test cases:-
A set of test inputs, execution conditions, and expected
results developed for a particular objective, such as to
exercise a particular program path or to verify compliancewith a specific requirement.
Formal test cases
A formal written test-case is characterized by a known
input and by an expected output, which is worked out
before the test is executed. The known input should test a
precondition and the expected output should test a
postcondition.
10
7/29/2019 Pressman7Ch17 Modfied by Arvind
11/57
Test Plan, Test Case and test
Data
Informal test cases
Test cases can be written based on the accepted normal
operation of programs of a similar class.
Test cases are not written at all but the activities andresults are reported after the tests have been run.
11
7/29/2019 Pressman7Ch17 Modfied by Arvind
12/57
Test Plan, Test Case and test
Data
Test case format To test the correct behavior / functionality, features of an
application.Includeds:
Test case ID - Test case description Test step or order of execution number related
requirement(s) Depth - Test category Author Check boxes for whether the test can be or has been automated Pass/Fail - Remarks
12
7/29/2019 Pressman7Ch17 Modfied by Arvind
13/57
Strategic Approach Strategy providesA rod map- That describe the steps to be
conducted for testing, When these steps are planed and
taken, How much effort, Time and Recourses are required.
So Strategy incorporates
1. Test Planning
2. Test Case design
3. Test Execution and
4. Resultant data collection and evaluation.
Flexible enough
Rigid enough
13
7/29/2019 Pressman7Ch17 Modfied by Arvind
14/57
Strategic Approach Characteristics
To perform effective testing, you should conduct
effective technical reviews. By doing this, many errors
will be eliminated before testing commences. Testing begins at the component level and works
"outward" toward the integration of the entire computer-
based system. Different testing techniques are appropriate for different
software engineering approaches and at different pointsin time.
Testing is conducted by the developer of the softwareand (for large projects) an independent test group.
Testing and debugging are different activities, but
debugging must be accommodated in any testingstrategy.
14
7/29/2019 Pressman7Ch17 Modfied by Arvind
15/57
V & V Verificationrefers to the set of tasks that ensure that
software correctly implements a specific function. Does the product meet its specifications? Validation refers to a different set of tasks that ensure
that the software that has been built is traceable tocustomer requirements. Boehm [Boe81] states thisanother way:
Does the product perform as desired? Verification: "Are we building the product right?" Validation: "Are we building the right product?"
15
7/29/2019 Pressman7Ch17 Modfied by Arvind
16/57
Misconceptions The developer of software should do no testing at all
That the software should be "tossed over the wall" to strangers who will
test it mercilessly
Testers get involved with the project only when the testing steps are
about to begin.
16
7/29/2019 Pressman7Ch17 Modfied by Arvind
17/57
Who Tests the Software?
17
developer independent tester
Understands the system
but, will test "gently"
and, is driven by "delivery"
Must learn about the system,
but, will attempt to break it
and, is driven by quality
7/29/2019 Pressman7Ch17 Modfied by Arvind
18/57
Testing Strategy
18
System engineering
Analysis modeling
Design modeling
Code generationUnit test
Integrat ion test
Validat ion test
System test
7/29/2019 Pressman7Ch17 Modfied by Arvind
19/57
Testing StrategyWe begin bytesting-in-the-small and move toward
testing-in-the-large
For conventional software
The module (component) is our initial focus
Integration of modules follows
For OO software
our focus when testing in the small changes from anindividual module (the conventional view) to an OOclass that encompasses attributes and operations andimplies communication and collaboration
19
7/29/2019 Pressman7Ch17 Modfied by Arvind
20/57
Strategic Issues Specify product requirements in a quantifiable manner
long before testing commences. State testing objectives explicitly. Understand the users of the software and develop a
profile for each user category. Develop a testing plan that emphasizes rapid cycle
testing. Build robust software that is designed to test itself Use effective technical reviews as a filter prior to testing
Conduct technical reviews to assess the test strategyand test cases themselves.
Develop a continuous improvement approach for thetesting process.
20
7/29/2019 Pressman7Ch17 Modfied by Arvind
21/57
Unit Testing
21
moduleto be
tested
test cases
results
software
engineer
Take Verification effort on the
smallest unit of software
7/29/2019 Pressman7Ch17 Modfied by Arvind
22/57
Unit Testing
22
interface
local data structures
boundary conditions
independent paths
error handling paths
moduleto be
tested
test cases
7/29/2019 Pressman7Ch17 Modfied by Arvind
23/57
Unit Test Environment
23
Module
stub stub
driver
RESULTS
interface
local data structures
boundary conditions
independent paths
error handling paths
test cases
7/29/2019 Pressman7Ch17 Modfied by Arvind
24/57
Unit Test Environment
24
Driver:- is noting more than a Main Program that
accept test date, Passes such data to the
component and prints relevant results.
Stub:- Stubs serve modules that are subordinate the
component to be tested
7/29/2019 Pressman7Ch17 Modfied by Arvind
25/57
Integration Testing Strategies
25
Is a systematic technique for constructing the software
architecture and conducting test to uncover error
associated with Interfacing
Build program structure that has been dictated bydesign
Options: The big bang approach
All component are combined in advance.
An incremental construction strategyThe program is constructed and tested in smallincrements, where errors are easier to isolate.
7/29/2019 Pressman7Ch17 Modfied by Arvind
26/57
Top Down Integration
26
Top-down integration testing is an incremental approach to
construction of program structure.
Modules are integrated by moving downward through thecontrol hierarchy, beginning with the main control module
(main program)
.
7/29/2019 Pressman7Ch17 Modfied by Arvind
27/57
Top Down IntegrationThe integration process is performed in a series offive steps: The main control module is used as a test driver and stubs
are substituted for all components directly subordinate to
the main control module. Depending on the integration approach selected (i.e.,
depth or breadth first), subordinate stubs are replaced one
at a time with actual components.
Tests are conducted as each component is integrated.
On completion of each set of tests, another stub is
replaced with the real component.
Regression testing may be conducted to ensure that new
errors have not been introduced
27
7/29/2019 Pressman7Ch17 Modfied by Arvind
28/57
Top Down Integration
28
top module is tested withstubs
stubs are replaced one ata time, "depth first"
as new modules are integrated,some subset of tests is re-run
A
B
C
D E
F G
7/29/2019 Pressman7Ch17 Modfied by Arvind
29/57
Bottom-Up Integration
29
Begins construction and testing with atomic modules
A bottom-up integration strategy may be implemented with the
following steps:
1. Low-level components are combined into clusters (sometimescalled builds) that perform a specific software sub function.
2. A driver (a control program for testing) is written tocoordinate test case input and output.
3. The cluster is tested.4. Drivers are removed and clusters are combined moving
upward in the program structure.
7/29/2019 Pressman7Ch17 Modfied by Arvind
30/57
Bottom-Up Integration
30
drivers are replaced one at atime, "depth first"
worker modules are grouped intobuilds and integrated
A
B
C
D E
F G
cluster
7/29/2019 Pressman7Ch17 Modfied by Arvind
31/57
Sandwich Testing
31
Top modules aretested with stubs
Worker modules are grouped intobuilds and integrated
A
B
C
D E
F G
cluster
7/29/2019 Pressman7Ch17 Modfied by Arvind
32/57
Regression Testing Regression testingis the re-execution of some subset of tests that
have already been conducted to ensure that changes have notpropagated unintended side effects
Whenever software is corrected, some aspect of the software
configuration (the program, its documentation, or the data thatsupport it) is changed.
Regression testing helps to ensure that changes (due to testing or forother reasons) do not introduce unintended behavior or additionalerrors.
Regression testing may be conducted manually, by re-executing asubset of all test cases or using automated capture/playback tools.
32
7/29/2019 Pressman7Ch17 Modfied by Arvind
33/57
Regression TestingThe regression test suite (the subset of tests to be executed) containsthree different classes of test cases:
A representative sample of tests that will exercise all software
functions.
Additional tests that focus on software functions that are likely to be
affected by the change.
Tests that focus on the software components that have been changed.
33
7/29/2019 Pressman7Ch17 Modfied by Arvind
34/57
Smoke Testing A common approach for creating daily builds for product software
Smoke testing steps: Software components that have been translated into code are
integrated into a build.
A build includes all data files, libraries, reusable modules, andengineered components that are required to implement one ormore product functions.
A series of tests is designed to expose errors that will keep the buildfrom properly performing its function.
The intent should be to uncover show stopper errors that have
the highest likelihood of throwing the software project behindschedule.
The build is integrated with other builds and the entire product (inits current form) is smoke tested daily.
The integration approach may be top down or bottom up .
34
7/29/2019 Pressman7Ch17 Modfied by Arvind
35/57
Smoke Testing
Benefits when it is applied on complex, time critical software
engineering projects:
Integration risk is minimized- Smoke tests are conducted daily.
The quality of the end-product is improved -The approach is
construction (integration) oriented, smoke testing is likely to
uncover both functional errors and architectural and
component-level design defects.
Error diagnosis and correction are simplified- Errors uncovered
during smoke testing are likely to be associated with newsoftware increments
Progress is easier to assess-With each passing day, more of the
software has been integrated and more has been demonstrated
to work.
35
7/29/2019 Pressman7Ch17 Modfied by Arvind
36/57
Alpha and Beta Testing
1. The alpha test is conducted at the developer's site by a customer.
The software is used in a natural setting with the developer
"looking over the shoulder" of the user and recording errors and
usage problems.Alpha tests are conducted in a controlled environment.
2. The beta test is conducted at one or more customer sites by the
end-user of the software.
Unlike alpha testing, the developer is generally not present.
Therefore, the beta test is a "live" application of the software inan environment that cannot be controlled by the developer.
The customer records all problems (real or imagined) that areencountered during beta testing and reports these to thedeveloper at regular intervals.
36
7/29/2019 Pressman7Ch17 Modfied by Arvind
37/57
System Testing
Software is incorporated with other system elements (e.g.,
hardware, people, information), and a series of system
integration and validation tests are conducted.
System testing is actually a series of different tests whose primarypurpose is to fully exercise the computer-based system.
37
7/29/2019 Pressman7Ch17 Modfied by Arvind
38/57
Recovery Testing
Recovery testing is a system test that forces the software to fail in
a variety of ways and verifies that recovery is properly performed.
If recovery is automatic (performed by the system itself),reinitialization, check pointing mechanisms, data recovery, and
restart are evaluated for correctness.
If recovery requires human intervention, the mean-time-to-
repair (MTTR) is evaluated to determine whether it is within
acceptable limits.
38
7/29/2019 Pressman7Ch17 Modfied by Arvind
39/57
Security Testing
Security testing attempts to verify that protection mechanisms
built into a systemwill, in fact, protect it from improper
penetration.
During security testing, the tester plays the role(s) of theindividual who desires to penetrate the system.
39
7/29/2019 Pressman7Ch17 Modfied by Arvind
40/57
Stress Testing
Stress testing executes a system in a manner that demands
resources in abnormalquantity, frequency, or volume.
Example:-
Special tests may be designed that generate ten interrupts persecond, when one or two is the average rate,
Input data rates may be increased by an order of magnitude to
determine how input functions will respond,
Test cases that require maximum memory or other resources
are executed,
Test cases that may cause thrashing in a virtual operating system
are designed,
Test cases that may cause excessive hunting for disk-resident
data are created. 40
7/29/2019 Pressman7Ch17 Modfied by Arvind
41/57
Performance Testing Performance testing is designed to test the run-time performance of
software within the context of an integrated system.
Performance testing occurs throughout all steps in the testing
process. Even at the unit level, the performance of an individualmodule may be assessed as white-box tests are conducted.
Performance tests are often coupled with stress testing and usually
require both hardware and software instrumentation.
41
7/29/2019 Pressman7Ch17 Modfied by Arvind
42/57
White Box Testing
Is a test case design method that uses the control structure of the
procedural design to derive test cases.
The "status of the program" may be examined at various points
to determine if the expected or asserted status correspondsto the actual status.
The software engineer can derive test cases
Guarantee that all independent paths within a module have been
exercised at least once,
Exercise all logical decisions on their true and false sides,
Execute all loops at their boundaries and within their operational
bounds, and
Exercise internal data structures to ensure their validity.
42
7/29/2019 Pressman7Ch17 Modfied by Arvind
43/57
Black Box Testing
Black-box testing alludes to tests that are conducted at the
software interface.
Designed to uncover errors,
To demonstrate that software functions are operational, That input is properly accepted and output is correctly produced,
The integrity of external information (e.g., a database) is
maintained.
Examines some fundamental aspect of a system with little regard
for the internal logical structure of the software.
43
7/29/2019 Pressman7Ch17 Modfied by Arvind
44/57
Debugging: A Diagnostic Process
44
7/29/2019 Pressman7Ch17 Modfied by Arvind
45/57
Debugging Software testing is a process that can be systematically plannedand specified. Test case design can be conducted, a strategy can
be defined, and results can be evaluated against prescribed
expectations. Debugging occurs as a consequence of successful testing.
That is, when a test case uncovers an error, debugging is the
process that results in the removal of the error.
Although debugging can and should be an orderly process, it is
still very much an art.
45
7/29/2019 Pressman7Ch17 Modfied by Arvind
46/57
7/29/2019 Pressman7Ch17 Modfied by Arvind
47/57
The Debugging Process
47
7/29/2019 Pressman7Ch17 Modfied by Arvind
48/57
Debugging Effort
48
time requiredto diagnose thesymptom and
determine thecause
time requiredto correct the errorand conductregression tests
7/29/2019 Pressman7Ch17 Modfied by Arvind
49/57
7/29/2019 Pressman7Ch17 Modfied by Arvind
50/57
Consequences of Bugs
50
damage
mildannoying
disturbing
serious
extreme
catastrophic
infectious
Bug Type
Bug Categor ies: function-related bugs,system-related bugs, data bugs, coding bugs,design bugs, documentation bugs, standardsviolations, etc.
7/29/2019 Pressman7Ch17 Modfied by Arvind
51/57
Debugging Techniques
51
brute force / testing
backtracking
induction
deduction
7/29/2019 Pressman7Ch17 Modfied by Arvind
52/57
Brute Force The most common and least efficient method for isolating the
cause of a software error.
Apply when all else fails.
Using a "let the computer find the error" philosophy
52
7/29/2019 Pressman7Ch17 Modfied by Arvind
53/57
Backtracking Used successfully in small programs.
Beginning at the site where a symptom has been uncovered,
the source code is traced backward (manually) until the site ofthe cause is found.
Unfortunately, as the number of source lines increases, the
number of potential backward paths may become
unmanageably large.
53
7/29/2019 Pressman7Ch17 Modfied by Arvind
54/57
Cause elimination Is manifested by induction or deduction and introduces the
concept of binary partitioning.
Data related to the error occurrence are organized to isolatepotential causes.
54
7/29/2019 Pressman7Ch17 Modfied by Arvind
55/57
Correcting the Error Is the cause of the bug reproduced in another part of the program?In
many situations, a program defect is caused by an erroneous
pattern of logic that may be reproduced elsewhere.
What "next bug" might be introduced by the fix I'm about to make?
Before the correction is made, the source code (or, better, the
design) should be evaluated to assess coupling of logic and data
structures.
What could we have done to prevent this bug in the first place? This
question is the first step toward establishing a statistical softwarequality assurance approach. If you correct the process as well as the
product, the bug will be removed from the current program and may
be eliminated from all future programs.
55
7/29/2019 Pressman7Ch17 Modfied by Arvind
56/57
Final Thoughts Think-- before you act to correct
Use tools to gain additional insight
If youre at an impasse, get help from someone else Once you correct the bug, use regression testing to
uncover any side effects
56
7/29/2019 Pressman7Ch17 Modfied by Arvind
57/57