Fault Localization Where is the root cause
Tho T. Quan SAVE research group Faculty of Computer Science and Engineering Hochiminh City University of Technology Hochiminh, Vietnam [email protected]
Outline
Introduction
Fault Localization: An Odyssey
Academic View
Demonstrations
Conclusion
About SAVE
System Analysis and VErification
Founded in 2008, by Dr. Nguyen Hua Phung
Current leader: A/Prof. Quan Thanh Tho
www.cse.hcmut.edu.vn/~save
Research topics
Formal methods to design, verify and secure computer-based systems
Improvement of programming productivity and performance
Education and practical tools to assist teaching and practicing programming and software engineering
Research Directions
Polymorphism Virus
Mobile Security Checking
Petri Nets – PAT – Model Checking
Automatic Composition of Prototype
Software Testing
Fault Localization
Fault Localization: An Odyssey
Concept
Tour
Fault Localization - Concept
Intuition view
Program introduced abnormal behaviors
Test case failed
Where is the bugs
Approaches
Memory dump
Breakpoint setting
Print out intermediate values
Issue: Locate the suspicious code
Fault Localization - Concept
Fault Localization
Locate the code that may cause the bug
Based on testing results
Automatic or Semi-Automatic
Let the tour commence
More insight analysis
SQL Fault localization
Expert-based Localization
SQL inspection
Expert asset
Does it really work? - Testing with Siemens suite
Fault Localization: Under Academic View
Overview
Program
Test suite
Test result: - Passed/failed test cases - Program spectra
Fault localization techniques
The ranked list of elements based on suspicious degrees
The set of suspicious elements
The root cause
spectrum
model
proof
Fault Localization Techniques
Model-based
Proof-based
Spectrum-based
FL techniques: Model-based
Produce a mathematical model of the program
Solve the model to find suspicious elements
FL techniques: Proof-based
Produce an unsatisfied formula for an error
Find the elements in the formula which cause the formula satisfied when removed
FL techniques: Spectrum-based
Based on spectra of testing result
Returned a ranked list of suspicious elements
Some Definitions
S(e): suspicious score of element e
Nef(e): failed executions involving e
Nnf(e): failed executions NOT involving e
Nep(e): passed executions involving e
Nnp(e): passed executions NOT involving e
Taratula, Ochiai and Jaccard
Tarantula Ochiai Jaccard
3 0.5 0.7 0.5
4 1 1 1
6 0 0 0
n = 1 n= -1
Tarantula Ochiai Jaccard
3 0.5 0.58 0.33
4 1 1 1
6 0 0 0
n = 1 n= -1 n = 0
Open Issues
Impact of test-case sets
Missing code problem
Test-case sets
Better strategy of test-case generation yields better fault localization performance
0.0;1.0;2.0 1.0;4.0;3.0 2.0;2.0;3.0
0.0;1.0;2.0 1.0;4.0;3.0 2.0;2.0;3.0 1.0;2.0;1.0
Missing code problem
Fault localization aims at finding suspicious code that causes error
When the logic error caused by missing some code no code
available to be “suspected”
Case Study: The PROVE system
Program Verification system
http://elearning.cse.hcmut.edu.vn/provegroup/index.jsp
Verify student programming works
Produce counter-example
Show execution path
Show suspected code
Conclusion
Fault localization is the next step of testing
Spectrum-based is deemed suitable to be applied in industry
Applied in an online educational tool at HCMUT
Chances for collaboration with industry