1

OO testing and testing strategies

• Testing object-oriented software

• Testing strategies

• Higher-order testing

2

Object - oriented testing

• begins by evaluating the correctness and consistency of the OOA and OOD models

• testing strategy changesthe concept of the ‘unit’ broadens due to encapsulation

integration focuses on classes and their execution across a ‘thread’ or in the context of a usage scenario

validation uses conventional black box methods

• test case design draws on conventional methods, but also encompasses special features

3

OOT strategy

• classes are the smallest testable unit

• inheritance defines new context for methods– behavior of inherited methods can be changed because of methods that are

called within

– methods have to be tested per class

• objects have states: testing methods have to take that into account

• class testing is the equivalent of unit testing– operations within the class are tested

– the state behavior of the class is examined

• OO testing concentrates on the states of the objects and their interactions

4

Integration applied three different strategies

• thread-based testing• integrates the set of classes required to respond to one

input or event

• use-case based testing• integrates the set of classes required to respond to one use

class

• starts with independent (ground) classes

• cluster testing• integrates the set of classes required to demonstrate one

collaboration

5

OOT - test case design (1)

Berard [BER93] proposes the following approach:

1. each test case should be uniquely identified and should be explicitly associated with the class to be tested,

2. the purpose of the test should be stated,

3. a list of testing steps should be developed for each test and should contain [BER94]:

6

OOT - test case design (2)

A a list of specified states for the object that is to be tested

B a list of messages and operations that will be exercised as a consequence of the test

C a list of exceptions that may occur as the object is tested

D a list of external conditions (i.e., changes in the environment external to the software that

must exist in order to properly conduct the test)

E supplementary information that will aid in understanding or implementing the test.

7

Problems in testing OO software

• Testing shall report errorsinternal state of object

• Encapsulation hides internal things

• Inheritance: test cases of the superclass may may have to be adapted

8

OOT methods: Scenario-based testing

• Concentrates on what the user does NOT what the system does

• Based on use cases

• Described by– a test– specific user needs

9

OOT methods: random testing

• random testingidentify operations applicable to a class

define constraints on their use

identify a minimum test sequence• an operation sequence that defines the minimum

life history of the class (object)

generate a variety of random (but valid) test sequences

• exercise other (more complex) class instance life histories

10

OOT methods: partition testing (1)

• Partition testingreduces the number of test cases required to

test a class in much the same way as equivalence partitioning for conventional software

state-based partitioning• categorize and test operations based on their

ability to change the state of a class

11

OOT methods: partition testing (2)

• Partition testingattribute-based partitioning

• categorize and test operations based on the attributes that they use

category-based partitioning• categorize and test operations based on the

generic function each performs e.g.– initialization

– state changing

– queries

– termination

12

State diagram based testing

• Use state diagram to determine a sequence of events

• Tests should achieve all state coverage

• All transitions should be tested at least once

• Test invalid events

13

OOT methods: Inter-class testing (1)

• Inter-class testingFor each client class, use the list of class

operators to generate a series of random test sequences. The operators will send messages to other server classes.

For each message that is generated, determine the collaborator class and the corresponding operator in the server object.

14

OOT methods: Inter-class testing (2)

• Inter-class testingFor each operator in the server object ( that

has been invoked by messages sent from the client object), determine the messages that it transmits.

For each of the messages, determine the next level of operators that are invoked and incorporate these into the test sequence

15

OO testing and testing strategies

• Testing object-oriented software

• Testing strategies

• Higher-order testing

16

Verification & validation

• Verification: Are we building the product right

• Validation: Are we building the right product

17

Testing strategy

code

design

requirementshigh-order

tests

integration test

unittest

18

Unit testing

Test cases

Moduleinterface

local data structures

boundary conditions

independent paths

error handling paths

19

Unit testing environment

Test cases

interface

local data structures

boundary conditions

independent paths

error handling paths

driver

Module

stub stub

RESULTS

20

Why integration testing?

• If all units work individually, why doubt that they work together?

• Putting units together - interfacing

21

Integration testing

BigBang!

incrementalconstruction

strategy

incremental“builds”

regressiontesting

22

Top down integration

A

GFB

E

C

D

top module is tested with stubs

stubs are replaced one at a time, “depth first”

as new modules are integrated,some subset of tests is re-run

23

Bottom-up integration

A

GFB

E

C

D

drivers are replaced one at a time, “depth first”

worker modules are grouped intobuilds and integrated

cluster

24

Sandwich testing

A

GFB

E

C

Dworker modules are grouped intobuilds and integrated

cluster

top modules are tested with stubs

25

OO testing and testing strategies

• Testing object-oriented software

• Testing strategies

• Higher-order testing

26

High-order testing

• validation test

• system test

• alpha and beta test

• other specialized testing

27

Alpha & Beta test

software

developer site customer site

customer tests

Alpha test

software

developer site customer site

customertests

Beta test

developerreviews

28

Test specification (1)

1. Scope of testing

2. Test plan

3. Test procedure n (description of test for build n)

4. Actual test results

5. References

6. Appendices

29

Test specification (2)

2. Test planA. test phases and builds

B. schedule

C. overhead software

D. environment and resources

30

Test specification (3)

3. Test procedure n A. order of integration

1. purpose

2. modules to be tested

B. unit tests for modules in build1. description of tests for module n

2. overhead software description

3. expected results

31

Test specification (4)

3. Test procedure n C. test environment

1. special tools or techniques

2. overhead software description

D. test case data

E. expected results for build n

32

Debugging: Symptoms & causes

• Symptom and cause may be geographically separated

• symptom may disappear when another problem is fixed

• cause may be due to a combination of non-errors

• cause may be due to a system or compiler error

• cause may be due to assumptions that everyone believes

• symptom may be intermittentcausesymptom