-- Finishing Design Unit-- Intro to Glass Box Testing

ECEN5543 / CSCI 5548 SW Eng of Standalone Programs

University of Colorado, Boulder

Many Examples

Pure Fabrication – an idea to group together some common behaviorMany OO design patterns are examples

AdapterStrategyCommand...

Traditional cohesion – top two strongestinformationalfunctional

Introduction to Glass Box Testing

TestingSoftware Engineering of Standalone

Programs

What is Glass Box Testing?Sometimes called Structure TestsExercise the internal logic of a program and traverse particular execution paths

(Functional, performance, and stress testing are collectively referred to as black box testing.)

What are major activities?Decide which paths need to be exercised.Derive test data to exercise those paths.Determine the test coverage criterion.Execute the test cases.Measure the test coverage achieved.

Why determine test coverage criterion?

Why not just test it all?What does “all” mean?Program units (classes, clusters of classes) usually contain too many paths to permit exhaustive testing.For example, loops introduce combinatorial numbers of execution paths and make exhaustive testing impossible.

What if you could test all paths?

Would the program be correct then?The program might be missing paths

Code for branch alternative not thereMore likely detect missing paths how?

Might have computational errors not revealed in the test data shown, even on the path where the error occurs

We call that coincidental correctnessFor example, suppose code has x = A + A where it should have A*A but you test with A = 2.

When are you done with unit testing? Decide on criterion

Statement coverageBranch coverageLogical path coverage

1 5

4

3

2

Statement CoverageFind a set of test cases to ensure that every statement is executed at least once

Branch coverageFind a set of test cases to ensure that each branching statement is taken in each direction at least onceHow is that different from statement coverage?

Logical Path CoverageAcknowledges that the order in which branches are taken can be significantEnsure that every path is executed at least onceA path is some feasible combination of statements and branchesNot in your life time ...

Branch Testing -- May not accomplish what you think

Consider the following pseudocode:if (X > 3 and then Y = 2) then

Z := 1endif;if (X = 4 or else Z > 3) then

Z := Z + 1;endif;What happens with test case x=5,

y=2, z=3 and x=4, y=3, z=3?

Common rule of thumb re unit test completion

85 to 90 % of branch coveragethat is, 85 to 90% of all branch alternatives have been traversed by the set of unit test cases.

Strikes a balance between excessive number of test cases and leaving test completion to the intuition of each programmer

According to studies, functional + performance + stress tests based on functional requirements and programmer’s intuition achieve ....

Limitations

May not be possible to reach 100% branch coverage because it may be hard to find input data that will exercise a nested branching statement in a particular way.Beware of trend plots

# errors discovered per unit of time vs. timeExpect buildup to maximum value and then trail offProblems with this?

Control Flow Diagrams

Provide visual representation of flow of control

IFWhileFor“Do until”Case

Control flow examples - 1

Control flow examples - 2

Control flow examples - 3

Control flow examples - 4

Why look at control flow?Easy to spot complexityEasy to develop basis set for statement coverage

Complexity leads to faults

What do you find makes code hard to understand?What does the control flow diagram look like for that?McCabe’s Cyclomatic Complexity

What is being measured? Number of branchesNumber of closed regions

What is NOT being measured?

Beware: you can analyze YOUR error data against your modules and determine what characteristics are fault-prone in your application area.

Basis set for statement coverage testing

Draw the flow graphCount the regions including the “outside”

N is the number of linearly independent paths through the program control structureA linearly independent path includes one node of the control flow graph not contained in any otherCreate a basis set of linearly independent paths

Prepare test cases to force execution of each path in the basis set.Execution of these test cases yields ______ coverage

Code sample

/* Copyright (c) 1995 Sun Microsystems, Inc. All Rights Reserved.* Permission to use, copy, modify, and distribute this software and its documentation for * NON-COMMERCIAL purposes and without fee is hereby granted provided that this * copyright notice appears in all copies. Please refer to the file “copyright.html” for * further important copyright and licensing information.

* SUN MAKES NO REPRESENTATION OR WARRANTIES ABOUT THE

* SUITABILITY OF THE SOFTWARE, EITHER EXPRESS OR IMPLIED,

* INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF

* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-

* INFRINGEMENT. SUN SHALL NOT BE LIABLE FOR ANY DAMAGES

* SUFFERED BY LINCENSEE AS A RESULT OF USING, MODIFYING OR

* DISTRIBUTING THIS SOFTWARE OR ITS DERIVATIVES.

import java.io.*;import java.util.vector; class ListOfNumbers {

private Vector victor;final int size = 10;

 public ListOfNumbers () {

int i;victor = new Vector(size);for (i = 0; i < size; i++)

victor.addElement(new Interger(i));}

public void writeList() {PrintStream pStr = null;

 System.out.println(“Entering try

statement”);int i;pStr = new PrintStream(new BufferedOutputStream(new FileOutputStream(“OutFile.txt”)));

  for (i = 0; i < size; i++)pStr.println(“Value at: “ + i + “ = “ +

victor.elementAt(i));pStr.close()

}}