Crowd Debugging

Fuxiang CHEN Sunghun KIM

1ESEC/FSE 2015 (BERGAMO, ITALY, Aug 30 – Sep 4)

Debugging is Hard

2

[ICSE ‘13 Nguyen]•Sig. amt. of time•Root cause found, fixing difficult

[ESEC/FSE ’09 Ashok]•Sig. amt. of time•Similar bugs debugged

What’s wrong with my code?

Some Static Analysis tools you can try

- faster/cheaper than manual inspection

3

But….. they are underusedThey are known for.. 1.producing many warnings …. AND2.Many of them are false positives…. AND3.Ways In which warnings are presented (counterintuitive, confusion)

4

[ICSE ‘13 Johnson]

Another way to debug is using Stack Overflow

5

6

Oh my god….. This piece of code looks so similar to mine..

7

So, what does it means by many recurring questions?Many developers are facing many similar issues…

Challenges for developers?

• Needs frequent visit• Rapid growth of crowd knowledge• May not even know you have problem

8

Overview of Approach

9

Code-PairsDatabase

Code Clone Detection Filtering

Observation: Same Code Element in Question & Answer

10

Stack Overflow Question Code Fragment

Stack Overflow Answer Code Fragment

Observation: Same Code Element in Title & Question

11

Stack Overflow Question Title

Stack Overflow Question Code Fragment

Code-PairsDatabase

Question Title

12

Code-PairsDatabase

Code Clone Detection

Filtering

13

Question Code

Database

Target Code

SolrXMLSerializer.java (lucene)

13

Code-PairsDatabase

Code Clone Detection

Filtering

Question Code

Database

14

SearchingSimilar Code

CCFinderX

30 tokens

Code-PairsDatabase

Code Clone Detection

Filtering

Question Code

Database

15

Found!

Code-PairsDatabase

Code Clone Detection

Filtering

Stack Overflow Question Code Fragment

Target Code Fragment

16

Code-PairsDatabase

Code Clone Detection

Filtering

Title & Element Filters

Non-Essential

Code Filters

Similarities Filters

17

Title & Element Filters

Non-Essential Code Filters

Similarities Filters

Stack Overflow Question Title

Target Clone

18

Title & Element Filters

Non-Essential Code Filters

Similarities Filters

Target CloneStack Overflow Answer Code Fragment

• [ICSE ‘11 Kawrykow]• Non-essential code-changes• Can cause inaccurate representation of software development effort

19

Title & Element Filters

Non-Essential Code Filters

Similarities Filters

• SO Clone < 4

• Overridden methods

• Single Statement Block

20

Title & Element Filters

Non-Essential Code Filters

Similarities Filters

21

Similarity Filter #1

•SO Clone/Answer Ratio > 0.8

Similarity Filter #2

•SO Clone/Answer Ratio > 0.37

•Target Clone/Answer Ratio > 0.25

Title & Element Filters

Non-Essential Code Filters

Similarities Filters

Overview of Approach

22

Code-PairsDatabase

Code Clone Detection Filtering

Title & Element Filters

Non-Essential

Code Filters

Similarities Filters

23

Subject Version # Files K LOCcommons-lang 3.4-snapshot 274 63JFreeChart 1.0.19 654 96joda-time 2.4 315 80JStock 1.0.7r 290 50JStudyPlanner 1.0 34 3JStudyPlanner 2 2.0 56 4log4j 2.0.1 905 61lucene 4.9.0 4826 684

RQ1: (Detectability) How many warnings can bedetected using our technique?

24

RQ2: (Confirmed Bugs) How many warnings from our technique are confirmed as bugs by developers?

RQ3: (Comparison) 1) How many confirmed bugs can also be detected in FindBugs, JLint and PMD, & 2) How many bugs are missing which can be found in FindBugs, JLint and PMD??

RQ1: (Detectability) How many warnings can bedetected using our technique?

18925

26

RQ2: (Confirmed Bugs) How many warnings from our technique are confirmed as bugs by developers?

27

Defective Code : AxisEntity.java (Line 139 – 145) Explanation : Using instanceof for comparison is asymmetric

Suggestion : Use getClass() instead of instanceof Explanation : getClass() is asymmetric

Reference URL : http://www.stackoverflow.com/questions/7132649

Email Bug Report

Subject # of warnings # of reported warnings

# of confirmed bugs

# of rejected warnings

commons-lang 3 0 0 0JFreeChart 152 152 152 0joda-time 0 0 0 0JStock 2 1 1 0JStudyPlanner 19 18 18 0JStudyPlanner 2

1 0 0 0

log4j 2 0 0 0lucene 10 10 0 1Total 189 181 171 1

28

29

JFreeChart

JStudyPlanner

Thanks for the feedback. I agree with your suggestion and will fix these cases…- David Gilbert (Project Leader)

Thank you very much for your message. This is very valuable to me… Once again your research is very helpful. I’ll do all the checks in service objects to handle NullPointerExceptions.

- Oleg Lukin (Project Owner)

RQ3: (Comparison) 1) How many bugs can also be detected in FindBugs, JLint and PMD, &

30

RQ3: (Comparison) 2) How many bugs are missing which can be found in FindBugs, JLint and PMD?

31

3

LIMITATIONS AND THREATS TO VALIDITY• Data Type Insensitive• Code Terms outside Code Blocks• Evaluation of warnings from other Static Analysis tools

32

• New way to debug your code

• Find bugs within well maintained open source code

• Locates and explains the bug

• Provide the crowd’s solution with explanation.

33

Key Takeaways