soma@ist.tugraz€¦ · Software Maintenance Delta Debugging 4 . Printing this page makes Mozilla...

transcript

Software Maintenance Delta Debugging

Debugging

soma@ist.tugraz.at

Outline - Delta Debugging

Motivation

Basic definitions Test result Interference Inconsistency

Algorithms Minimization Differences

Areas of application

Motivation

Initial Situation Large Input that causes a failure

Challenge Find a smaller input that still causes the failure

Solution Use the principle of divide and conquer Minimization by Delta Debugging Developed by Andreas Zeller et al.

(www.st.cs.uni-sb.de/dd/)

Printing this page makes Mozilla crash

Quelle: Zeller, A.; Society, I. C. & Hildebrandt, R.: Simplifying and Isolating Failure-Inducing Input IEEE Transactions on Software Engineering, 2002, 28, 2002

Excerpt:

How do minimize this input?

Web page has over 40,000 characters

Remove single characters 40,000 tests

Better:

First: cut away large parts

Later: increase granularity

Narrowing down hypothesis

Source: A. Zeller, www.st.cs.uni-sb.de/dd/

Possible failure cause Set up first hypothesis Test first hypothesis

Second hypothesis

Third hypothesis

Fourth hypothesis

Minimizing ...

Minimizing ...

< <SELECT NAME=„priority“ MULTIPLE SIZE=7>

Minimizing ...

< <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7> <SELECT NAME=„priority“ MULTIPLE SIZE=7>

. . . <SELECT NAME=„priority“ MULTIPLE SIZE=7>

Problem description

Given:

Testing function test(x) mapping input x to ,,.

An input C with test(C) = .

Search for a smaller c ⊂ C so that test(c‘) = .

Basic definitions Testing function

Takes a program and an input

The test either

succeeds (PASS, ),

produces the failure (FAIL, ) or

produces indeterminate results (UNRES, )

DD Process

Test case x

Program

test(x)

Delta Debugging minimize

Basic definitions

Interference

Combination of several inputs reveals a bug

No interference: test(x) = iff „1“⊆ x test(x) = otherwise

Interference: test(x) = iff „18“⊆ x test(x) = otherwise

Basic definitions

Inconsistency Input a cannot be applied without input b Example C = “12345678” test(x) = ? iff (“2“⊆ x ۸ “7“ ⊄ x) ۷ (“7“⊆ x ۸ “2“ ⊄ x) test(x) = iff “8“⊆ x ⊆ C test(x) = otherwise

Basic definitions

=⊆∀ )(: ctestCc ≠⊇∀→ )'(:' ctestcc

=⊆∀ )(: ctestCc ≠⊆∀→ )'(:' ctestcc

• Monotony

Algorithm – Version 1

|∆1| ≈ |∆2|

Example 1

Input C = “12345678“ Function test test(x) = iff “7“ ⊆ x ⊆ C test(x) = otherwise

Example 2

Input C = “12345678“ Function test test(x) = iff “37“ ⊆ x ⊆ C test(x) = , otherwise

Interference

Cannot deal with interference!

|∆1| ≈ |∆2|

Divide and conquer!

Example 2

Input C = “12345678“ Function test test(x) = iff “37“ ⊆ x ⊆ C test(x) = , otherwise

Example *

Input C = “12345678“ Function test test(x) = if “18“ ⊆ x ⊆ C test(x) = if “35“ ⊆ x ⊆ C test(x) = otherwise

Several faults

Does not work in case of several errors!

Instead use the result of

Example **

Input C = “12345678“ Function test: test(x) = ? iff (“2“ ⊆ x ۸ “7“ ⊄ x)

۷ (“7“ ⊆ x ۸ “2“ ⊄ x) (case 1)

test(x) = iff “8“ ⊆ x ⊆ C ۸ not case 1 test(x) = otherwise

Inconsistency

Example 4

Input C = “12345678“ Function test: test(x) = if “34“ ⊆ x ⊆ C test(x) = if “7“ ⊆ x ⊆ C test(x) = otherwise

Minimal solution?

Minimality criterias

Global minimum An input c ⊆ C is said to be the global minimum iff

/ Requires testing all 2|C| subsets

Local minimum An input c ⊆ C is said to be the local minimum iff

/ Requires 2|c| -2 tests

|||'|)'(:')( ccctestCcctest <∧=⊆∃∧=

=⊂∃∧= )'(:')( ctestccctest

Minimality criterias

The problem of checking local and global minimality is a decision problem which is NP-complete. n-minimality An input c ⊆ C is n-minimal iff

≠⇒≤−⊂∀ )'(|'|||:' ctestncccc

Properties of ddmin

ddmin returns a 1-minimal result.

The complexity of ddmin is O(|c|2).

In the best case the complexity of ddmin is Ω(log2(|c|)).

bug.c / gcc V2.95.2 (Intel-Linux) #define SIZE 20 double mult(double z[], int n) int i,j; i=0; for(j=0;j<n;j++) i = i + j +1; z[i] = z[i]*(z[0]+1.0);

return z[n]; void copy(double to[], from[], int count) int n = (count + 7 ) / 8; switch (count % 8) do case 0: *to++ = *from++; ...

while (--n > 0); return mult(to,2);

int main (...) ... return copy(y,x,SIZE);

WHICH PART CAUSES GCC TO

CRASH?

bug.c / gcc V2.95.2 (Intel-Linux) #define SIZE 20 double mult(double z[], int n) int i,j; i=0; for(j=0;j<n;j++) i = i + j +1; z[i] = z[i]*(z[0]+1.0);

return z[n]; void copy(double to[], from[], int count) int n = (count + 7 ) / 8; switch (count % 8) do case 0: *to++ = *from++; ...

while (--n > 0); return mult(to,2);

int main (...) ... return copy(y,x,SIZE);

Remove half of the program and try gcc again.

If there is a crash, repeat removing

substrings. Otherwise, choose

a different substring.

(BINARY SEARCH)

bug.c / gcc V2.95.2 (Intel-Linux) Result after minimization:

double mult(double z[],int n) int i,j; for(;;) i=i+j+1; z[i]=z[i]*(z[0]+1.0); return z[n];

The resulting substring is substantially smaller!

Isolation of differences

Example 1

Input C = “12345678“ Function test: test(x) = ? iff (“37“⊄ x) (Case 1) test(x) = iff “6“ ⊆ x ⊆ C ۸ not case 1 test(x) = otherwise

Areas of application

Test case reduction Minimizing failure inducing input User input Data input

Code reduction Identifying software changes that lead to failures

Case Study (Zeller 2002)

Mozilla web browser Example Crash after 95 user actions Automatic reduction to 3 relevant user actions Automatic minimization of 44 bug reports

Conclusions

Successfully used to minimize test cases

Does not (always) find the global/local minimum

Not only for inputs of type STRING

Helps to locate bugs in software

Example ***

Input C = “12345678“ Function test: test(x) = ? iff (“26“⊄ x) (Case 1) test(x) = iff “7“ ⊆ x ⊆ C ۸ not case 1 test(x) = otherwise

Example ****

Input C = “12345678“ Function test: test(x) = ? iff (“17“⊄ x) test(x) = if “2“ ⊆ x ⊆ C test(x) = if “8“ ⊆ x ⊆ C test(x) = otherwise

soma@ist.tugraz€¦ · Software Maintenance Delta Debugging 4 . Printing this page makes Mozilla...

Documents