47
From Symptom to Cause: Localizing Errors in Counterexample Traces Mayur Naik, Purdue University Thomas Ball, Microsoft Research Sriram K. Rajamani, Microsoft Research
From Symptom to Cause:
Localizing Errors in Counterexample Traces
Mayur Naik, Purdue University
Thomas Ball, Microsoft Research
Sriram K. Rajamani, Microsoft Research
Model Checking
+ Fully automatic: Does not require the user to provide annotations.
+ Transparent: Produces a source-level error trace (counterexample).
− An error trace represents a symptom of the error as opposed to its cause.
− State-of-the-art model checkers report only one error trace.
The Problem
1. How do we localize the cause of the errorin an error trace?
2. How do we produce multiple error traces having distinct causes?
Note: Problem is relevant to other error-detection
techniques as well.
What is a “Cause”?
We define a “cause” to be those parts of an error trace not contained in any correct trace.
The program fragments containing the cause are rendered unreachable.
The model checker is invoked again to produce additional error traces.
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Example
Error #1: Lock acquired in succession
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Correct Trace Computation
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Error Cause Localization
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Error Recovery
Insert halt Unreachable from entry of main in future runs of the model checker
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Error #2: Lock held on exit
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Correct Trace Computation
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Error Cause Localization
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Error Recovery
Insert halt Unreachable from entry of main in future runs of the model checker
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ...
}
AcquireLock();
if (...)
ReleaseLock();
else {
...
}
return;
}
Final (Error-Free) Program
main()
{
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ...
}
AcquireLock();
if (...)
ReleaseLock();
else {
halt; ...
}
return;
}
A technique that exploits correct traces for error cause localization.
Efficient algorithm for computing correct traces.
Experimental results in the context of the SLAM toolkit.
Our Results
Transitions and Edges
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Transitions and Edges
<(3, L), (5, U)>
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Transitions and Edges
project(<(3, L), (5, U)>) = (3, 5)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
High-Level Algorithm
while true do
switch ModelCheck(G, ve) of // ve is of the form assert(e)
case FAILURE(T):
let C = GetCorrectTransitions(G, ve) and
K = project(T) \ project(C) in
if K = Ø then
break
for each (vi, vj) in K do
insert a halt statement between vi and vj case SUCCESS:
break
Computing Correct Transitions
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Transitions in Error Trace (T)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Reachable state-space computed by model checker
(4,L)
(1,U)
(2,L)
(3,L)
(5,U)
(5,L)
ve ≡ assert(s==U)
Computing Correct Transitions
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Transitions in Error Trace (T) Correct Transitions (C)
(4,L)
(1,U)
(2,L)
(3,L)
(5,U)
(5,L)
ve ≡ assert(s==U)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Computing Correct Transitions
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Transitions in Error Trace (T)
<(3, L), (5, U)>
Correct Transitions (C)
(4,L)
(1,U)
(2,L)
(3,L)
(5,U)
(5,L)
ve ≡ assert(s==U)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Computing Correct Transitions
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Transitions in Error Trace (T)
<(2, L), (3, L)>
<(3, L), (5, U)>
Correct Transitions (C)
ve ≡ assert(s==U)
(4,L)
(1,U)
(2,L)
(3,L)
(5,U)
(5,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Computing Correct Transitions
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
Transitions in Error Trace (T)
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
Correct Transitions (C)
ve ≡ assert(s==U)
(4,L)
(1,U)
(2,L)
(3,L)
(5,U)
(5,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 AcquireLock();
Example 1: An omission error
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Transitions in Error Trace (T)
Error Cause Localization
K = project (T) \ project (C)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Transitions in Error Trace (T)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
Error Cause Localization
K = project (T) \ project (C)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Transitions in Error Trace (T)
Error Cause Localization
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
Transitions in Error Trace (T)
K = project (T) \ project (C)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Error Cause Localization
K = project (T) \ project (C)
= { (2, 4), (4, 5) }
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
Transitions in Error Trace (T)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Experimental ResultsName of driver LOC
mouse packet filter 984
serial mouse port 7441
keyboard packet filter 1067
IEEE 1394 bus driver 5818
keyboard class driver 13161
i8042 port 22168
packet-based DMA 24971
serial port 30905
Experimental ResultsName of driver LOC Number of edges in
error trace error cause
mouse packet filter 984 73 0
110 4
serial mouse port 7441 56 1
keyboard packet filter 1067 73 0
107 4
IEEE 1394 bus driver 5818 45 7
44 7
60 1
81 3
85 0
keyboard class driver 13161 158 0
i8042 port 22168 127 1
124 5
packet-based DMA 24971 75 1
serial port 30905 248 3
Experimental ResultsName of driver LOC Number of edges in Error cause
localized?error trace error cause
mouse packet filter 984 73 0 No
110 4 Yes
serial mouse port 7441 56 1 Yes
keyboard packet filter 1067 73 0 No
107 4 Yes
IEEE 1394 bus driver 5818 45 7 Yes
44 7 Yes
60 1 Yes
81 3 Yes
85 0 No
keyboard class driver 13161 158 0 No
i8042 port 22168 127 1 Yes
124 5 Yes
packet-based DMA 24971 75 1 Yes
serial port 30905 248 3 Yes
main() {
1 int status = S;
2 if (*)
3 status = foo();
else {
4 foo();
5 status = S;
}
6 assert(status==x);
}
Example 2: A variable-value error
Program state is of the form (status, x)
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x;
}
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
3 status = foo();
else {
4 foo();
5 status = S;
}
6 assert(status==x);
}
Example 2: A variable-value error
Program state is of the form (status, x)
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x;
}
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
Error Cause Localization
K = project (T) \ project (C)
= Ø
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
3 status = foo();
else {
4 foo();
5 status = S;
}
6 assert(status==x);
}
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x;
}
High-Level Algorithm
while true do
switch ModelCheck(G, ve) of // ve is of the form assert(e)
case FAILURE(T):
let C = GetCorrectTransitions(G, ve) and
K = project(T) \ project(C) in
if K = Ø then
break
for each (vi, vj) in K do
insert a halt statement between vi and vj case SUCCESS:
break
High-Level Algorithm
while true do
switch ModelCheck(G, ve) of // ve is of the form assert(e)
case FAILURE(T):
let C = GetCorrectTransitions(G, ve) and
K = project(T \ C) in
if K = Ø then
break
for each (vi, vj) in K do
insert a halt statement between vi and vj case SUCCESS:
break
main() {
1 int status = S;
2 if (*)
3 status = foo();
else {
4 foo();
5 status = S;
}
6 assert(status==x);
}
Example 2: A variable-value error
Program state is of the form (status, x)
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x;
}
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
K = project (T \ C)
= { (4, 5), (5, 6) }
Error Cause Localization
enum { S, F } x = S;
int foo() {
if (*)
x = S;
else
x = F;
return x;
}
(2,(S,S))
(4,(S,S))
(5,(S,F))
(1,(S,S))
(6,(S,F))
Error Trace
(6,(S,S))
(3,(S,S))
Correct Traces
(2,(S,S))
(4,(S,S))
(5,(S,S))
(1,(S,S))
(6,(F,F))
main() {
1 int status = S;
2 if (*)
3 status = foo();
else {
4 foo();
5 status = S;
}
6 assert(status==x);
}
Example 1: An omission error
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Transitions in Error Trace (T)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
Error Cause Localization
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Transitions in Error Trace (T)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
K = project (T \ C)
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
Transitions in Error Trace (T)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
K = project (T \ C)
Error Cause Localization
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
Error Cause Localization
Correct Transitions (C)
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
Transitions in Error Trace (T)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
K = project (T \ C)
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
K = project (T \ C)
= { (2, 4), (4, 5), (5, 6) }
1 AcquireLock();
2 if (...)
3 ReleaseLock();
else {
4 ...
}
5 ...
6 AcquireLock();
...
(1,U)
(2,L)
(5,U)
(3,L)
(6,U)
(1,U)
(2,L)
(5,L)
(4,L)
(6,L)
Correct Transitions (C)Transitions in Error Trace (T)
Error Cause Localization
<(1, U), (2, L)>
<(2, L), (4, L)>
<(4, L), (5, L)>
<(5, L), (6, L)>
<(1, U), (2, L)>
<(2, L), (3, L)>
<(3, L), (5, U)>
<(5, U), (6, U)>
Limitations
Control-based approach fails to localize the cause when every edge in the error trace is contained in some correct trace.
Transition-based approach localizes the cause to a suffix of the error trace.
Model Imprecision: Infeasible paths can misguide error cause localization using either approach.
Multiple Counterexamples Verisim [Bhargavan et al., TSE ’02]
Error Cause Localization Explaining counterexamples [Jin et al., TACAS ’02] Explaining type errors [Wand, POPL ’86; Johnson & Walz, POPL ’86;
Beaven & Stansifer, LOPLAS ’93; Duggan & Bent, SCP ’96; Chitil, ICFP ’01; Tip & Dinesh, TOSEM ’01]
Program Slicing [Weiser, TSE ’84] Algorithmic Debugging [Shapiro, Ph.D. thesis ’82] Delta Debugging [Zeller, FSE ’99]
Anomaly Detection Static: Meta-Level Compilation [Hallem et al., PLDI ’02] Dynamic: Daikon [Ernst et al., TSE ’01], DIDUCE [Hangal &
Lam, ICSE ’02]
Related Work
Conclusions
We have presented a technique for localizing the causes of errors in counterexample traces.
A combination of the control-based and transition-based approaches appears promising.
Our technique is quite general and should be applicable to error detection tools based on data-flow analysis as well.
