Post on 22-Dec-2015
transcript
Chapter 11: Testing
The dynamic verification of the behavior of a program on a finite set of test cases, suitable selected from the usually infinite execution domain, against the expected behavior
Testing Fundamentals
Testing is the one step in software engineering process that could be viewed as destructive rather than constructive. A successful test is one that breaks the software.
A successful test is one that uncovers an as yet undiscovered defect.
Testing can not show the absence of defects, it can only show that software defects are present.
For most software exhaustive testing is not possible.
Testing Unit testing
White-box Code walkthroughs and inspections
Integration testing Bottom-up Top-down Sandwich Big Bang
Functional testing (black-box) Performance testing Acceptance testing Installation testing
System testing
Functional testing Performance testing Acceptance testing Installation testing
System Testing Process
Functional testing: does the integrated system perform as promised by the requirements specification?
Performance testing: are the non-functional requirements met?
Functional Testing
Test cases derived from requirements specification document Black box testing Independent testers Test both valid and invalid input and the
success of the test is determined by the produced output
Equivalence partitioning Boundary values
Equivalence partitioning
Reduces the number of test cases to a necessary minimum and select the right test cases to cover all possible scenarios
Example: method accepts an int value for month (1..12) as a parameter
Performance Testing
Stress tests Volume tests Recovery tests Security tests Timing tests
Environmental tests Quality tests Maintenance tests Documentation tests Human factors
(usability) tests
Acceptance Tests
Enable the customers and users to determine if the built system meets their needs and expectations
Written, conducted, and evaluated by the customers
Types of Acceptance Tests
Pilot test: install on experimental basis
Alpha test: in-house test Beta test: customer pilot Parallel testing: new system operates
in parallel with old system
Installation test
Does the system run at the customer site(s)?
System Testing
Pop quiz
How does system testing differ from unit and integration testing?
How does unit testing differ from integration testing?
In a method, a variable did not get initialized properly.
Which type of testing would most likely expose this defect?
A. Unit testingB. Integration testingC. Functional testingD. Performance testingE. Acceptance testingF. Installation testing
A gas pump system is supposed to allow the user to choose whether or not a receipt is printed, but the print function has not been implemented.
Which type of testing is most likely to expose this defect?
A. Unit testingB. Integration testingC. Functional testingD. Performance testingE. Acceptance testingF. Installation testing
A configuration file used by the reporting subsystem is not placed in the correct directory in the customer's environment.
Which type of testing is most likely to expose this defect?
A. Unit testingB. Integration testingC. Function testingD. Performance testingE. Acceptance testingF. Installation testing
The customer is unhappy with the number of screens that must be traversed before getting to the parts list screen, a screen accessed frequently when using the system.
Which type of testing is most likely to expose this defect?
A. Unit testingB. Integration testingC. Function testingD. Performance testingE. Acceptance testingF. Installation testing
Short answer question
What is the difference between verification and validation testing?
Test Documentation Test plan: describes system and plan for
testing all functions and characteristics Test case specification: details each test
and defines criteria for evaluating each feature
Test incident report: results of each test Test report summary: lists all failures
from the tests that need to be investigated
Test Plan (part of 480 QA plan)
Define the subsystems to be tested (and not tested)
Describe the process to follow for unit, integration, system, and acceptance testing
List the test cases
Plan the test schedule
Test Documentation
Defect Tracking Form
Regression Testing
Identifies new faults that may have been introduced as current ones are being corrected
Verifies that a new version or release still performs the same functions in the same manner as an older version or release
Quality Assurance concerns
Software reliability: operating without failure under given condition for a given time interval
Software availability: operating successfully according to specification at a given point in time
Software maintainability: for a given condition of use, a maintenance activity can be carried out within stated time interval, procedures and resources
Testing Safety-Critical Systems Recognize that testing cannot remove all faults
or risks
Assume that every mistake users can make will be made Do not assume that low-probability, high-
impact events will not happen
Emphasize requirements definition, testing, code and specification reviews, and configuration control Cleanroom testing
Different Levels of Failure Severity
Catastrophic: causes death or system loss Critical: causes severe injury or major
system damage Marginal: causes minor injury or minor
system damage Minor: causes no injury or system damage
One-minute quiz
What is meant by regression testing?
Is regression testing used for verification or validation?