CSC 480 Software Engineering

Testing - I

Plan project

Integrate & test system

Analyze requirements

Design

Maintain

Test unitsImplement

Software Engineering Roadmap:

Identify corporate practices

Test units (parts) separately - use implementations - apply discipline - gain coverage

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Learning Goals

• Understand meaning of unit testing

• Distinguish black box vs. white box testing

• Attain proper test coverage

• Validation and Verification

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Verification:are we building the thing right?

Validation:are we building the right thing?

Meaning of “V&V” (Boehm)

Graphics reproduced with permission from Corel.

1. Introduction to unit testing

Golden Rules of Testing

Goal of testing: maximize the number and

severity of defects found per dollar spent

– thus: test early

Golden Rules of Testing

Goal of testing: maximize the number and

severity of defects found per dollar spent

– thus: test early

Limits of testing: Testing can only determine the

presence of defects, never their absence

– use proofs of correctness to establish “absence”

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Testing: the Big Picture

Function

Module

Module combination

2. Integration tests

3.

System tests

1.Unittests

Testing: the Big Picture

Methods

Combinations of methods in class

Packages of classes

OO:

Include use-cases

Function

Module

Module combination

2. Integration tests

3.

System tests

1.Unittests

Elaboration

Unified ProcessInception Construction Transition

Requirements

Analysis

Design

Implemen-tation

Test

Jacobson et al: USDP

Prelim.iterations

Iter.#1

Iter.#n

Iter.#n+1

Iter.#m

Iter.#m+1

Iter.#k

…..

Elaboration

Unified ProcessInception Construction Transition

Requirements

Analysis

Design

Implemen-tation

Test

Jacobson et al: USDP

Prelim.iterations

Iter.#1

Iter.#n

Iter.#n+1

Iter.#m

Iter.#m+1

Iter.#k

….. …..

Unit Tests Integration tests ... System tests

RoadMap for Unit Testing

1. Plan for unit testing-- see section SSS

Requirements

Test

results

Detailed design

. . . .

RoadMap for Unit Testing1. Plan for

unit testing-- see section SSS

Requirements

Unit test plan

2. Acquire test set-- see section SSS

Products of

prior testing

3. Execute unit test-- see section SSS

Test set

Test

results

Code

under test

Detailed design

Identify largest

trouble spots

IEEE, 1986

2. Test types

Black-, Gray-, & White-box Testing

Black box… requirements*

Actual outputcompared

with required

* from previous phase

Input determinedby...

Result

Black-, Gray-, & White-box Testing

Black box… requirements

Actual outputcompared

with required output

White box

Gray box… requirements &key design elements

Input determinedby...

Result

…designelements

Confirmationof expected

behavior

As for black- and white box

testing

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Test Input Possibilities

interest rate0%

25%

principal

$100 $100M

inflation

estimate

1%

20%Infinitely many legal values:

choose a finite sample.

Infinitely many illegal values:

choose a finite sample.

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Test Input Partitioning and Boundaries

interest rate0%

25%

principal

$100 $100M

inflation

estimate

Boundaries

1%

20%

Equivalence partitions

An illegal region

Range of valid inputs

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Testing Ranges: Elementary Cases

1. within range

2. at the boundaries of the range

3. outside the range (“illegal”)

range

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Covering Every Statement is Not Sufficient (Myers)

u>1 andv==0

x = x/u

u==2 orx>0

++x

No

Yes

No

Yes

Required program

Covering Every Statement is Not Sufficient (Myers)

u>1 andv==0

x = x/u

u==2 orx>0

++x

No

Yes

Code attempt to implement flowchart

if( (u>1) && (v==0) ) (1)x = x/u; (2)

if( (u==2) || (x>3) ) (3)++x; (4)

u=2, v=0 and x=3 • executes every line (1) - (4) • gives the correct output x= 2.5 However, line (3) is wrong

No

Yes

Required program

Paths to be Checked

Parameter & settings make

sense?

Parameter name too

long?

N

YN

Set _name to “defaultName"

Y

Truncate name

Set _name to parameter

Paths to be Checked

Parameter & settings make

sense?

Parameter name too

long?

N

YN

Decision Coverage

Set _name to “defaultName"

Y

Truncate name

Set _name to parameter

Adapted from Software Engineering: An Object-Oriented Perspective by Eric J. Braude (Wiley 2001), with permission.

Flow Chart vs. Flow Graph

Binary search (Java)

class BinSearch { public static void search ( int key, int [] elemArray, Result r ) { int bottom = 0 ; int top = elemArray.length - 1 ; int mid ; r.found = false ; r.index = -1 ; while ( bottom <= top ) { mid = (top + bottom) / 2 ; if (elemArray [mid] == key) { r.index = mid ; r.found = true ; return ; } // if part else { if (elemArray [mid] < key) bottom = mid + 1 ; else top = mid - 1 ; } } //while loop } // search } //BinSearch

This is an encapsulation of a binary search function that takes an array of ordered objects and a key and returns an object with 2 attributes namely

index - the value of the array index found - a boolean indicating whether or not the key is in the array

An object is returned because it is not possible in Java to pass basic types by reference to a function and so return two valuesthe key is -1 if the element is not found

1

2

3

4

5

67

89

8

Binary search flow graph

1

2

3

4

65

7

while bottom <= top

if (elemArray [mid] == key

(if (elemArray [mid]< key8

9

bottom > top

• 1, 2, 3, 8, 9• 1, 2, 3, 4, 6, 7, 2• 1, 2, 3, 4, 5, 7, 2• 1, 2, 3, 4, 6, 7, 2, 8, 9• Test cases should be derived so that all of these

paths are executed• A dynamic program analyser may be used to

check that paths have been executed

Independent paths

Cyclomatic Number

• In graph-theoretic terms, the cyclomatic number of a graph G is defined as

C = E – N + 2Pwhere – N is the number of nodes– E is the number of edges– P is the number of connected components (usually, 1)

Cyclomatic Complexity

• McCabe suggested that the number C can be used as a source-code metric – that reflects the complexity of a program’s control flow

• C defines the number of linearly independent paths in a program– It can be used as an upper bound for the number of

tests that must be conducted

Alternative Ways to Compute C

• There are other ways that can be used to determine the cyclomatic number C– C = P + 1

where P is the number of predicate nodes contained in the flow graph

– C = Rwhere R is the number of regions into which

the flow graph divides the plane