1
Graph Coverage (3)Graph Coverage (3)
Reading AssignmentReading AssignmentP. Ammann and J. Offutt
“Introduction to Software Testing”◦Section 2.2◦Section 2.3
2
3
OutlineOutlineData Flow Criteria
◦DU Pairs and DU PathsGraph Coverage for Source Code
◦Basic Block◦Control Flow Graph (CFG) – If statements◦CFG – Loops
CFG – Test requirements and Test paths
Data Flow Graph Coverage for Source Code
4
Data Flow CriteriaData Flow Criteria
Definition (def) : A location where a value for a variable is stored into memory
Use : A location where a variable’s value is accessed
Goal: Try to ensure that values are computed and used correctly
5
Data Flow CriteriaData Flow Criteria
0
2
1
63
5
4X = 42
Z = X-8
Z = X*2
Defs: def (0) = {X}
def (4) = {Z}
def (5) = {Z}
Uses: use (4) = {X}
use (5) = {X}
6
DU Pairs and DU PathsDU Pairs and DU PathsDU pair : A pair of locations (li, lj)
such that a variable v is defined at li and used at lj
Def-clear : A path from li to lj is def-clear with respect to variable v if v is not given another value on any of the nodes or edges in the path
7
DU Pairs and DU PathsDU Pairs and DU PathsReach : If there is a def-clear path
from li to lj with respect to v, the def of v at li reaches the use at lj
du-path : A simple subpath that is def-clear with respect to v from a def of v to a use of v
du (ni, nj, v) – the set of du-paths from ni to nj
du (ni, v) – the set of du-paths that start at ni
8
Data Flow Test CriteriaData Flow Test Criteria
All-defs coverage (ADC)All-defs coverage (ADC) : For each set of du-paths : For each set of du-paths SS = = dudu ( (nn, , vv), TR contains at least one path ), TR contains at least one path dd in in SS..
All-uses coverage (AUC)All-uses coverage (AUC) : For each set of du-paths to : For each set of du-paths to uses uses SS = = dudu ( (nnii, n, njj, , vv), TR contains at least one path ), TR contains at least one path dd
in in SS..
• Then we make sure that every def reaches all possible uses
• First, we make sure every def reaches a use
9
Data Flow Test CriteriaData Flow Test Criteria
All-du-paths coverage (ADUPC)All-du-paths coverage (ADUPC) : For each set : For each set SS = = dudu ( (ni, njni, nj, , vv), TR contains every path ), TR contains every path dd in in SS..
• Finally, we cover all the paths between defs and uses
10
Data Flow Testing Data Flow Testing ExampleExample
0
2
1
63
5
4X = 42
Z = X-8
Z = X*2
All-defs for X
[ 0, 1, 3, 4 ]
All-uses for X
[ 0, 1, 3, 4 ]
[ 0, 1, 3, 5 ]
All-du-paths for X
[ 0, 1, 3, 4 ]
[ 0, 2, 3, 4 ]
[ 0, 1, 3, 5 ]
[ 0, 2, 3, 5 ]
11
Graph Coverage Criteria Graph Coverage Criteria Subsumption Subsumption
Node Coverage
NC
Edge Coverage
EC
Edge-Pair Coverage
EPC
Prime Path Coverage
PPC
Complete Path Coverage
CPC
All-DU-Paths Coverage
ADUP
All-uses Coverage
AUC
All-defs Coverage
ADC
12
Control Flow Graphs (CFG) – Control Flow Graphs (CFG) – Source CodeSource CodeA CFG models all executions of a
method by describing control structures
Nodes : Statements or sequences of statements (basic blocks)
Edges : Transfers of control
13
Control Flow Graphs (CFG) – Control Flow Graphs (CFG) – Source CodeSource CodeBasic Block : A sequence of
statements such that if the first statement is executed, all statements will be (no branches)
CFGs are sometimes annotated with extra information◦ branch predicates◦ defs◦ uses
Rules for translating statements into graphs …
14
CFG : The if Statement CFG : The if Statement if (x < y){ y = 0; x = x + 1;}else{ x = y;}
4
1
2 3
x >= yx < y
x = yy = 0
x = x + 1
if (x < y){ y = 0; x = x + 1;}
3
1
2x >= y
x < y
y = 0x = x + 1
15
CFG : The if-Return StatementCFG : The if-Return Statement
if (x < y){ return;}print (x);return;
3
1
2x >= y
x < y
return
print (x)return
No edge from node 2 to 3.The return nodes must be distinct.
16
LoopsLoops
Loops require “extra” nodes to be added
Nodes that do not represent statements or basic blocks
17
CFG : while and for LoopsCFG : while and for Loops
x = 0;while (x < y){ y = f (x, y); x = x + 1;}
1x = 0
43y =f(x,y)x = x + 1
x >= yx < y
for (x = 0; x < y; x++){ y = f (x, y);}
1
x = x + 1
2
3 5
x >= yx < y
y = f (x, y)
4
2
dummy node
x = 0implicitly
initializes loop
implicitly increments loop
18
CFG : The case (switch) CFG : The case (switch) StructureStructure
read ( c) ;switch ( c ){ case ‘N’: y = 25; break; case ‘Y’: y = 50; break; default: y = 0; break;}print (y);
5
1 read ( c );
c == ‘N’
y = 0;break;
2 43
c == ‘Y’ default
y = 50;break;
y = 25;break;
print (y);
19
Example Control Flow – Example Control Flow – computeStatscomputeStatspublic static void computeStats (int [ ] numbers){ int length = numbers.length; double med, var, sd, mean, sum, varsum;
sum = 0; for (int i = 0; i < length; i++) { sum += numbers [ i ]; } med = numbers [ length / 2 ]; mean = sum / (double) length;
varsum = 0; for (int i = 0; i < length; i++) { varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean)); } var = varsum / ( length - 1.0 ); sd = Math.sqrt ( var );
System.out.println ("length: " + length); System.out.println ("mean: " + mean); System.out.println ("median: " + med); System.out.println ("variance: " + var); System.out.println ("standard deviation: " + sd);}
20
Control Flow Graph for Control Flow Graph for computeStatscomputeStatspublic static void computeStats (int [ ] numbers){ int length = numbers.length; double med, var, sd, mean, sum, varsum;
sum = 0; for (int i = 0; i < length; i++) { sum += numbers [ i ]; } med = numbers [ length / 2 ]; mean = sum / (double) length;
varsum = 0; for (int i = 0; i < length; i++) { varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean)); } var = varsum / ( length - 1.0 ); sd = Math.sqrt ( var );
System.out.println ("length: " + length); System.out.println ("mean: " + mean); System.out.println ("median: " + med); System.out.println ("variance: " + var); System.out.println ("standard deviation: " + sd);}
i = 0
i >= length
i < lengthi++
i >= lengthi < length
i = 0
i++
1
2
3
54
6
87
21
Control Flow TRs and Test Control Flow TRs and Test Paths – ECPaths – EC
1
2
3
54
6
87
TRA. [ 1, 2 ]B. [ 2, 3 ]C. [ 3, 4 ]D. [ 3, 5 ]E. [ 4, 3 ]F. [ 5, 6 ]G. [ 6, 7 ]H. [ 6, 8 ]I. [ 7, 6 ]
Test Path[ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ]
Edge Coverage
22
Control Flow TRs and Test Paths Control Flow TRs and Test Paths – EPC– EPC
1
2
3
54
6
87
TRA. [ 1, 2, 3 ]B. [ 2, 3, 4 ]C. [ 2, 3, 5 ]D. [ 3, 4, 3 ]E. [ 3, 5, 6 ]F. [ 4, 3, 5 ]G. [ 5, 6, 7 ]H. [ 5, 6, 8 ]I. [ 6, 7, 6 ]J. [ 7, 6, 8 ]K. [ 4, 3, 4 ]L. [ 7, 6, 7 ]
Test Pathsi. [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ]ii. [ 1, 2, 3, 5, 6, 8 ]iii. [ 1, 2, 3, 4, 3, 4, 3, 5, 6, 7, 6, 7, 6, 8 ]
Edge-Pair Coverage
23
Control Flow TRs and Test Paths Control Flow TRs and Test Paths – PPC– PPC
1
2
3
54
6
87
TRA. [ 3, 4, 3 ]B. [ 4, 3, 4 ]C. [ 7, 6, 7 ]D. [ 7, 6, 8 ]E. [ 6, 7, 6 ]F. [ 1, 2, 3, 4 ]G. [ 4, 3, 5, 6, 7 ]H. [ 4, 3, 5, 6, 8 ]I. [ 1, 2, 3, 5, 6, 7 ]J. [ 1, 2, 3, 5, 6, 8 ]
Test Pathsi. [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ]ii. [ 1, 2, 3, 4, 3, 4, 3, 5, 6, 7, 6, 7, 6, 8 ]iii. [ 1, 2, 3, 4, 3, 5, 6, 8 ]iv. [ 1, 2, 3, 5, 6, 7, 6, 8 ]v. [ 1, 2, 3, 5, 6, 8 ]
Prime Path Coverage
24
Data Flow Graph Coverage Data Flow Graph Coverage for Source Codefor Source Codedef : a location where a value is
stored into memory◦x appears on the left side of an
assignment (x = 44;)◦x is an actual parameter in a call
and the method changes its value◦x is an input to a program
25
Data Flow Graph Coverage for Data Flow Graph Coverage for Source CodeSource Codeuse : a location where variable’s
value is accessed◦x appears on the right side of an
assignment◦x appears in a conditional test◦x is an actual parameter to a method◦x is an output of the program◦x is an output of a method in a
return statement
26
Example Data Flow – Example Data Flow – computeStatscomputeStatspublic static void computeStats (int [ ] numbers)
{ int length = numbers.length; double med, var, sd, mean, sum, varsum;
sum = 0; for (int i = 0; i < length; i++) { sum += numbers [ i ]; } med = numbers [ length / 2 ]; mean = sum / (double) length;
varsum = 0; for (int i = 0; i < length; i++) { varsum = varsum + ((numbers [ I ] - mean) * (numbers [ I ] - mean)); } var = varsum / ( length - 1.0 ); sd = Math.sqrt ( var );
System.out.println ("length: " + length); System.out.println ("mean: " + mean); System.out.println ("median: " + med); System.out.println ("variance: " + var); System.out.println ("standard deviation: " + sd);}
27
8
1
2
4
3
5
6
7
Control Flow Graph for Control Flow Graph for computeStats computeStats
( numbers )sum = 0length = numbers.length
i = 0
i >= length
i < length
sum += numbers [ i ]i++
med = numbers [ length / 2 ]mean = sum / (double) length;varsum = 0i = 0
i >= length
i < length
varsum = …i++
var = varsum / ( length - 1.0 )sd = Math.sqrt ( var )print (length, mean, med, var, sd)
28
8
1
2
4
3
5
6
7
CFG for computeStats – With Defs CFG for computeStats – With Defs & Uses& Uses
def (1) = { numbers, sum, length }
def (2) = { i }
def (5) = { med, mean, varsum, i }use (5) = { numbers, length, sum }
def (8) = { var, sd }use (8) = { varsum, length, mean, med, var, sd }
use (3, 5) = { i, length }
use (3, 4) = { i, length }
def (4) = { sum, i }use (4) = { sum, numbers, i }
use (6, 8) = { i, length }
use (6, 7) = { i, length }
def (7) = { varsum, i }use (7) = { varsum, numbers, i, mean }
29
Defs and Uses Tables for Defs and Uses Tables for computeStatscomputeStats
Node Def Use
1 { numbers, sum, length }
2 { i }
3
4 { sum, i } { numbers, i, sum }
5 { med, mean, varsum, i }
{ numbers, length, sum }
6
7 { varsum, i } { varsum, numbers, i, mean }
8 { var, sd } { varsum, length, var, mean, med, var, sd }
Edge Use
(1, 2)
(2, 3)
(3, 4) { i, length }
(4, 3)
(3, 5) { i, length }
(5, 6)
(6, 7) { i, length }
(7, 6)
(6, 8) { i, length }
30
DU Paths for computeStatsDU Paths for computeStatsvariable DU Pairs DU Paths
numbers (1, 4)(1, 5)(1, 7)
[ 1, 2, 3, 4 ][ 1, 2, 3, 5 ][ 1, 2, 3, 5, 6, 7 ]
length (1, 5)(1, 8)(1, (3,4))(1, (3,5))(1, (6,7))(1, (6,8))
[ 1, 2, 3, 5 ][ 1, 2, 3, 5, 6, 8 ][ 1, 2, 3, 4 ][ 1, 2, 3, 5 ][ 1, 2, 3, 5, 6, 7 ][ 1, 2, 3, 5, 6, 8 ]
med (5, 8) [ 5, 6, 8 ]var (8, 8) No path neededsd (8, 8) No path neededsum (1, 4)
(1, 5)(4, 4)(4, 5)
[ 1, 2, 3, 4 ][ 1, 2, 3, 5 ][ 4, 3, 4 ][ 4, 3, 5 ]
variable DU Pairs DU Pathsmean (5, 7)
(5, 8)[ 5, 6, 7 ][ 5, 6, 8 ]
varsum (5, 7)(5, 8)(7, 7)(7, 8)
[ 5, 6, 7 ][ 5, 6, 8 ][ 7, 6, 7 ][ 7, 6, 8 ]
i (2, 4)(2, (3,4))(2, (3,5))(4, 4)(4, (3,4))(4, (3,5))(5, 7)(5, (6,7))(5, (6,8))(7, 7)(7, (6,7))(7, (6,8))
[ 2, 3, 4 ][ 2, 3, 4 ][ 2, 3, 5 ][ 4, 3, 4 ][ 4, 3, 4 ][ 4, 3, 5 ][ 5, 6, 7 ][ 5, 6, 7 ][ 5, 6, 8 ][ 7, 6, 7 ][ 7, 6, 7 ][ 7, 6, 8 ]
31
Test Cases and Test PathsTest Cases and Test Paths
There are 12 unique DU paths[ 1, 2, 3, 4 ][ 1, 2, 3, 5 ][ 1, 2, 3, 5, 6, 7 ][ 1, 2, 3, 5, 6, 8 ][ 2, 3, 4 ][ 2, 3, 5 ]
[ 4, 3, 4 ][ 4, 3, 5 ][ 5, 6, 7 ][ 5, 6, 8 ][ 7, 6, 7 ][ 7, 6, 8 ]
5 expect a loop not to be “entered”
5 require at least one iteration of a loop
2 require at least two iteration of a loop
32
Test Cases and Test PathsTest Cases and Test Paths
Test Case : numbers = (44) ; length = 1Test Path : [ 1, 2, 3, 4, 3, 5, 6, 7, 6, 8 ]Note: At least one iteration of a loop
Test Case : numbers = (2, 10, 15) ; length = 3Test Path : [ 1, 2, 3, 4, 3, 4, 3, 4, 3, 5, 6, 7, 6, 7, 6, 7, 6, 8 ]Note: At least two iterations of a loop
Other DU paths require arrays with length 0 to skip loopsBut the method fails with divide by zero on the statement … mean = sum / (double) length;
A fault wasfound
33
Key Points (1/2)Key Points (1/2)Data Flow Criteria tries to ensure that
values are computed and used correctly.Graph coverage criteria sub-sumptionControl Flow Graph – Source Code
◦ If statement◦ Loops◦ Case statements
Applying the graph test criteria to control flow graphs is relatively straightforward◦ Most of the developmental research work
was done with CFGs◦ A few subtle decisions must be made to
translate control structures into the graph
Key Points (2/2)Key Points (2/2)DU Pairs
◦ def : a location where a value is stored into memory
◦ use : a location where variable’s value is accessed
◦ Test cases and DU Paths
34