ARIES: Refactoring Support Tool for Code Clone

transcript

1Software Engineering Laboratory, Department of Computer Science, Graduate School of Information Science and Technology, Osaka University

Yoshiki Higo1, Toshihiro Kamiya2, Shinji Kusumoto1, Katsuro Inoue1

1Osaka University2National Institute of Advanced Industrial Science and Technology

{y-higo, kamiya, kusumoto, inoue}@ist.osaka-u.ac.jp

Contents

Background

Proposed Refactoring Support Method

Step1, Step2, Step3

Metrics used in Step3

Refactoring Support Tool Aries

Case Study

Overview

Filtering Conditions

Evaluation Method

Result

Conclusion

Background

What is code clone?

a code fragment that has identical or similar fragments in the same or different files in a system

introduced in the source program because of various reasons such as reusing code by `copy-and-paste’

makes software maintenance more difficult.

copy-and-pastecopy-and-paste

Contents

Background

Step1, Step2, Step3

Case Study

Overview

Evaluation Method

Result

Conclusion

Our Refactoring Support Method for Code Clone

We have proposed a refactoring support method for code clone

We adopt three steps to get refactoring-oriented code clones quickly

First Step: Run CCFinder to get token-based code clonesSecond Step: Extract structural parts from code clones detected by CCFinderFinal Step: Characterize extracted code clones to predict how they can be refactored

We have implemented our method as a tool AriesIn this presentation, we explain about our method and report the result of a case study using Aries

Our Refactoring Support Method: First Step: CCFinder (Outline)

CCFinder directly compares source code on token unit, and detects code clones

Normalization of name space

Replacement of names defined by user

Removal of table initialization

Consideration of module delimiter

CCFinder can analyze the system of millions line scale in practical use time

Source files

Lexical analysis

Transformation

Token sequence

Match detection

Transformed token sequence

Clones on transformed sequence

Formatting

Clone pairs

1. static void foo() throws RESyntaxException { 2. String a[] = new String [] { "123,400", "abc", "orange 100" }; 3. org.apache.regexp.RE pat = new org.apache.regexp.RE("[0-9,]+"); 4. int sum = 0; 5. for (int i = 0; i < a.length; ++i) 6. if (pat.match(a[i])) 7. sum += Sample.parseNumber(pat.getParen(0)); 8. System.out.println("sum = " + sum); 9. }10. static void goo(String [] a) throws RESyntaxException {11. RE exp = new RE("[0-9,]+");12. int sum = 0;13. for (int i = 0; i < a.length; ++i)14. if (exp.match(a[i]))15. sum += parseNumber(exp.getParen(0));16. System.out.println("sum = " + sum);17. }

static void foo ( ) { String a

[ ] = new String [ ] { "123,400" ,

"abc" , "orange 100" } ;

int sum = 0

; for ( int i = 0 ; i <

a . length ; ++ i )

+= pat . getParen 0

; System . out . println ( "sum = "

+ sum ) ; }

throws RESyntaxException

Sample . parseNumber (

if pat

. match a [ i ]( ) )

org . apache . regexp

. RE pat = new org . apache . regexp

. RE ( "[0-9,]+" ) ;

static void goo (

String

int sum = 0

; for ( int i = 0 ; i <

a . length ; ++ i )

System . out . println ( "sum = " + sum

if exp

. match a [ i ]( ) )

new RE ( "[0-9,]+" ) ;

+= exp . getParen 0

parseNumber ( ) )(

[ ] = new String [ ] {

int sum = 0

; for ( int i = 0 ; i <

a . length ; ++ i )

+= pat . getParen 0

+ sum ) ; }

if pat

. match a [ i ]( ) )

pat = new

RE ( "[0-9,]+" ) ;

static void goo (

String

int sum = 0

; for ( int i = 0 ; i <

a . length ; ++ i )

if exp

. match a [ i ]( ) )

new RE ( "[0-9,]+" ) ;

+= exp . getParen 0

parseNumber ( (

static void foo ( ) { String athrows RESyntaxException

$ . ) )

Lexical analysis

Transformation

Token sequence

Match detection

Formatting

[ ] = new String [ ] {

int sum = 0

; for ( int i = 0 ; i <

a . length ; ++ i )

+= pat . getParen 0

+ sum ) ; }

if pat

. match a [ i ]( ) )

pat = new

RE ( "[0-9,]+" ) ;

static void goo (

String

int sum = 0

; for ( int i = 0 ; i <

a . length ; ++ i )

if exp

. match a [ i ]( ) )

new RE ( "[0-9,]+" ) ;

+= exp . getParen 0

parseNumber ( ) )(

static void foo ( ) { String athrows RESyntaxException

[ ] = [ ] {

; for ( = ; <

. ; ++ )

; . . (

+ ) ; }

. [ ]( ) )

static (

) {[ ]

; ( = ; <

. ; ++ )

. . ( +

throws

. [ ]( ) )

new ( ) ;

( ) )(

static $ ( ) {throws

$ $ $ $

$ $ $ $ $

$ $ $ $

$ $ $ $ $

$ $ $ $

$ $ $ $ $

$ $ $ $

$ $ $ $ $

forfor

[ ] = [ ] {

; for ( = ; <

. ; ++ )

; . . (

+ ) ; }

. [ ]( ) )

static (

) {[ ]

; ( = ; <

. ; ++ )

. . ( +

throws

. [ ]( ) )

new ( ) ;

( ) )(

static $ ( ) {throws

$ $ $ $

$ $ $ $ $

$ $ $ $

$ $ $ $ $

$ $ $ $

$ $ $ $ $

$ $ $ $

$ $ $ $ $

Lexical analysis

Transformation

Token sequence

Match detection

Formatting

1. static void foo() throws RESyntaxException { 2. String a[] = new String [] { "123,400", "abc", "orange 100" }; 3. org.apache.regexp.RE pat = new org.apache.regexp.RE("[0-9,]+"); 4. int sum = 0; 5. for (int i = 0; i < a.length; ++i) 6. if (pat.match(a[i])) 7. sum += Sample.parseNumber(pat.getParen(0)); 8. System.out.println("sum = " + sum); 9. }10. static void goo(String [] a) throws RESyntaxException {11. RE exp = new RE("[0-9,]+");12. int sum = 0;13. for (int i = 0; i < a.length; ++i)14. if (exp.match(a[i]))15. sum += parseNumber(exp.getParen(0));16. System.out.println("sum = " + sum);17. }

Lexical analysis

Transformation

Token sequence

Match detection

Formatting

Our Refactoring Support Method: First Step: CCFinder (Clone Detection Process)

Definitions:Clone Pair and Clone Set

Clone Pair: a pair of identical or similar fragmentsClone Set: a set of identical or similar fragments

CCFinder detects code clones as clone pairsAfter detection process, clone pairs are transformed into clone sets

Clone Pair Clone Set

(C1, C4) {C1, C4, C5}

(C1, C5) {C2, C3}

(C2, C3)

(C4, C5)

Our Refactoring Support Method: Second Step: Extract Structural Code Clones

Structural code clones are regarded as the target of refactoring

Extract structural parts as structural code clones from the detected clone sets

What is structural code clone ?

example: Java language

Declaration: class declaration, interface declaration

Method: method body, constructor, static initializer

Statement: do, for, if, switch, synchronized, try, while

609: reset();610: grammar = g;611: // Lookup make-switch threshold in the grammar generic options612: if (grammar.hasOption("codeGenMakeSwitchThreshold")) {613: try {614: makeSwitchThreshold = grammar.getIntegerOption("codeGenMakeSwitchThreshold");615: //System.out.println("setting codeGenMakeSwitchThreshold to " + makeSwitchThreshold);616: } catch (NumberFormatException e) {617: tool.error(618: "option 'codeGenMakeSwitchThreshold' must be an integer",619: grammar.getClassName(),620: grammar.getOption("codeGenMakeSwitchThreshold").getLine()621: );622: }623: }624:625: // Lookup bitset-test threshold in the grammar generic options626: if (grammar.hasOption("codeGenBitsetTestThreshold")) {627: try {628: bitsetTestThreshold = grammar.getIntegerOption("codeGenBitsetTestThreshold");

623: }624:625: // Lookup bitset-test threshold in the grammar generic options626: if (grammar.hasOption("codeGenBitsetTestThreshold")) {627: try {628: bitsetTestThreshold = grammar.getIntegerOption("codeGenBitsetTestThreshold");629: //System.out.println("setting codeGenBitsetTestThreshold to " + bitsetTestThreshold);630: } catch (NumberFormatException e) {631: tool.error(632: "option 'codeGenBitsetTestThreshold' must be an integer",633: grammar.getClassName(),634: grammar.getOption("codeGenBitsetTestThreshold").getLine()635: );636: }637: }638:639: // Lookup debug code-gen in the grammar generic options640: if (grammar.hasOption("codeGenDebug")) {641: Token t = grammar.getOption("codeGenDebug");642: if (t.getText().equals("true")) {

fragment 1 fragment 2

Code clones which CCFinder detects

Code clones which our method detects

1007: if ( inputState.guessing==0 ) {1008: buf.append(a.getText());1009: } 1010: {1011: _loop144:1012: do {1013: if ((LA(1)==WILDCARD)) {1014: match(WILDCARD);1015: a=id();1016: if ( inputState.guessing==0 ) {1017: buf.append('.'); buf.append(a.getText());1018: }1019: }

1527: if ( inputState.guessing==0 ) {1528: t=a.getText();1529: }1530: {1531: _loop84:1532: do {1533: if ((LA(1)==COMMA)) {1534: match(COMMA);1535: id();1536: if ( inputState.guessing==0 ) {1537: t+=","+b.getText();1538: }1539: }

Code clones which CCFinder detects

Our Refactoring Support Method: Final Step: Characterize Extracted Code Clones

The following refactoring patterns[1][2] can be applied to remove clone sets including structural code clones

Extract Class,Extract Method,Extract Super Class,Form Template Method,Move Method,Parameterize Method,Pull Up Constructor,Pull Up Method,

For each clone set, our method determines which refactoring pattern is applicable by using several metrics.

[1]: M. Fowler: Refactoring: Improving the Design of Existing Code, Addison-Wesley, 1999.

[2]: http://www.refactoring.com/, 2004.

Metrics(1): Coupling Metrics for Clone SetNRV, NSV

NRV(S): represents the average number of externally defined variables referred in the fragment of a clone set S

NSV(S): represents the average number of externally defined variables assigned to in the fragment of a clone set S

Definition

Clone set S includes fragment f1, f2, ・・・ , fn

si is the number of externally defined variable which fragment fi refersti is the number of externally defined variable which fragment fi assigns

int a , b, c; … if( … ){ …;

… = b + c; a = …; …;

assignment

reference

Fragment f1

example ：・ Clone set S includes fragments f1 and f2. ・ In fragment f1 , externally defined variable b and c are referred and a is assigned to.・ Fragment f2 is same as f1.

then ， NRV(S) = ( 2 + 2 ) / 2 = 2 NSV(S) = ( 1 + 1 ) / 2 = 1

int a , b, c; … if( … ){ …;

… = b + c; a = …; …;

Fragment f2

reference

assignment

Metrics （ 2 ）： Inheritance Metric for Clone SetDCH

DCH(S): represents the position and distance between each fragment of a clone set S

Definition

Clone set S includes fragment f1, f2, ・・・， fn

Fragment fi exists in class Ci

Class Cp is a class which locates lowest position in C1, C2, ・・・，Cn on class hierarchy

If no common parent class of C1 ， C ２，・・・， Cn exists, the value of DCH(S) is -1This metric is measured for only the class hierarchy where target software exists.

example 1:・ Clone set S includes fragments f1 and f2.・ If all fragments of clone set S are included in a same class,

then ， DCH(S) = 0

class A

fragment f1

fragment f2

class A

class B class C

fragment f1　 fragment f2

example 2 ：・ Clone set S includes fragments f1 and f2.・ If all fragments of clone set S are included in a class and its direct child classes,

then ， DCH(S) = 1

fragment f1　 fragment f2

class A class B

example 3 ：・ Clone set S includes fragments f1 and f2.・ If all classes which include f1 and f2 don’t have common parent class,

then ， DCH(S) = ∞

Aries: Refactoring Support ToolOverview

Target: Java programs

Runtime environment: JDK1.4 or above

Implementation

Analysis component: Java 32,000 Lines

CCFinder is used as code clone detection component

JavaCC is used to construct syntax and semantic analysis component

GUI component: Java14,000 Lines

User can specify target clone sets through GUI operations.

Contents

Background

Step1, Step2, Step3

Case Study

Overview

Evaluation Method

Result

Conclusion

Case Study: Overview

Business application developed by a software company in Japan

LOC: 70,000

Number of classes: 309

Process of case study

Step1: Extract clone set which can be refactored using the following refactoring patterns

Extract Super Class

Move Method

Extract Method

Step2: Qualitatively evaluate the results of the refactorings by a developer from the viewpoint of size, design, cohesion, coupling, understandability, and reusability

Case Study: Filtering Conditions(1/3)

Conditions for “Extract Super Class”

The target is Class-Unit clone sets

Cloned Classes have no parent

Department

Employee

getTotalAnnualCost()

getName()

getHeadCount()

getAnnualCost()

getName()

getId()

Before Refactoring

Employee

getHeadCount()

Department

getHeadCount()

getAnnualCost()

getName()

After Refactoring

Similar classes

Case Study:Filtering Conditions(1/3)

Conditions for “Move Method”

The target is Method-Body-Unit clone sets

Cloned methods use no other resource of its class

ClassA

Before Refactoring

ClassB

ClassC

ClassD

clonedMethod()

ClassA

After Refactoring

ClassB

ClassC

ClassD

UtilityClass

clonedMethod()

Case Study:Filtering Conditions(1/3)

Conditions for “Extract Method”

The target is Statement-Unit clone sets such as “if”, “while”, “for” …

Cloned statements assign any value to 1 or no externally defined variable

All statements of a clone set are in a same class

void methodA(int i){methodZ();

System.out.println(“name:” + name);System.out.println(“amount:” + i);

void methodB(int i){methodY();

System.out.println(“name:” + name);System.out.println(“amount:” + i);

void methodA(int i){methodZ();methodC(i);

}void methodB(int i){

methodY();methodC(i);

void methodC(int i){System.out.println(“name:” + name);System.out.println(“amount:” + i);

Before Refactoring After Refactoring

Case Study:Result of Filtering

Number of detected clone set

Declaration Unit: 4

Method Unit: 13

Statement Unit: 49

Number of filtered clone set

Extract Super Class: 4

Move Method: 5

Extract Method: 12

Case Study:Evaluation Method(1/3)

A developer of the target system evaluated refactorings of filtered clone sets from the following viewpoints

About status of clones (He judged each following term of the status on a scale of (a) or (b))

size of module

design of software

cohesion of class

coupling among classes

(a) deteriorating, (b) making no impact

Case Study: Evaluation Method(2/3)

The continuation

About effectiveness of refactorings

(He judged each following term of the effectiveness on a scale from (a) to (d) )

design

cohesion

coupling

understandability

reusability

(a) improving ， (b) preventing problems for the future ，(c) having no impact ， (d) deteriorating

Case Study:Evaluation Method(3/3)

The continuationAbout costs of refactorings

(He judges two terms of the cost on a scale from (a) to (c)) modifying source codetesting for regression

(a) very easy, (b) a little troublesome ， (c) complicated

Comprehensive Evaluation ( He judged the comprehensive evaluation on a scale from (a) to (d))

This clone set …(a) must be refactored as soon as possible(b) should be refactored for the future(c) is no matter about refactoring(d) should not be refactored

The status of “Declaration-Unit” clones

All clone sets deteriorate size and designAll clone sets have no impact on cohesion, and coupling

Case Study:The Result - Extract Super Class （ 1/4 ）

　 (a) (b) Total

Size 4 0 4

Design 4 0 4

Cohesion 0 4 4

Coupling 0 4 4

(a) Deteriorating

(b) Making no Impact

The effectiveness of “Declaration-Unit” refactorings

Refactorings of all clone sets are effective in terms of size, design, understandability, and reusabilityRefactorings of all clone sets have no impact in terms of cohesion, and coupling

　 (a) (b) (c) (d)Tota

Size 2 2 0 0 4

Design 2 2 0 0 4

Cohesion 0 0 4 0 4

Coupling 0 0 4 0 4

Understandability

1 3 0 0 4

Reusability 1 3 0 0 4

(a) Improving

(b) Preventing problems for the future

(c) Having no impact

(d) Deteriorating

The cost of “Declaration-Unit” refactorings

Modifying requires a little troubleTesting becomes complicated in some cases

The comprehensive evaluation of “Declaration-Unit” refactoring

All clone sets must or should be refactored

Case Study: The Result - Extract Super Class （ 3/4 ）

　 (a) (b) (c) Total

Modifying

1 3 0 4

Testing 1 1 2 4

　 (a) (b) (c) (d) Total

Refactoring …

2 2 0 0 4

(a) Very Easy(b) A little Troublesome(c) Complicated

(a) Must be refactored(b) Should be refactored(c) No matter(d) Shouldn’t be refactored

AcceptHeadChecker SakubanHeadChecker

doStartTag()

doAfterBody()

doEndTag()

doStartTag()

doAfterBody()

doEndTag()

New Class

doAfterBody()

doEndTag()

AcceptHeadChecker SakubanHeadChecker

doStartTag() doStartTag()

Before Refactoring After Refactoring

Methods “DoAfterBody” and “DoEndTag” are exactly identical respectivelyMethods “DoStartTag” provides different functions

The status of “Method-Unit” clones

All clone sets deteriorate size and designMost of them(80%) deteriorate cohesionAll clone sets have no impact on coupling

Case Study:The Result - Move Method （ 1/4 ）

　 (a) (b) Total

Size 5 0 5

Design 5 0 5

Cohesion 4 1 5

Coupling 0 5 5

(a) Deteriorating

The effectiveness of “Method-Unit” refactorings

Refactorings of all clone sets are effective in terms of size, design, cohesion, understandability, and reusabilityRefactorings of all clone sets have no impact on coupling

Case Study: The Result - Move Method （ 2/4 ）

　 (a) (b) (c) (d) Total

Size 5 0 0 0 5

Design 5 0 0 0 5

Cohesion 5 0 0 0 5

Coupling 0 0 5 0 5

Understandability

4 1 0 0 5

(a) Improving

(d) Deteriorating

The cost of “Method-Unit” refactorings

Both modifying and testing are very easy

The comprehensive evaluation of “Method-Unit” refactorings

All clone sets must or should be refactored

Modifying

5 0 0 5

Testing 5 0 0 5

　 (a) (b) (c ) (d) Total

Refactoring … 4 1 0 0 5

The following method exist in 9 classes

This method converts ResultData to HashMap

This method uses no other fields or methods of its class

Moving to other class is very easy

The developer judged that this refactoring is effective

private HashMap convertToMap(ResultData rd){String[] names = rd.getNames();HashMap map = new HashMap();for( int i = names.length ; i++ ){

map.put([i], rd.getObject(names[i]));}return map;

The status of “Statement-Unit” clones

Part of clone sets deteriorates size and designAll clone sets have no impact on cohesion, and coupling

Case Study: The Result - Extract Method （ 1/4 ）

　 (a) (b) Total

Size 2 10 12

Design 2 10 12

Cohesion 0 12 12

Coupling 0 12 12

(a) Deteriorating

The effectiveness of “Statement-Unit” refactorings

The effectiveness of refactorings is divisive in terms of size, design, understandability, and reusabilityRefactorings of all clone set have no impact on cohesion , and coupling

Case Study:The Result - Extract Method （ 2/4 ）

　 (a) (b) (c) (d) Total

Size 0 5 4 3 12

Design 0 3 3 6 12

Cohesion 0 0 12 0 12

Coupling 0 0 12 0 12

Understandability

0 4 3 5 12

(a) Improving

(d) Deteriorating

The cost of “Statement-Unit” refactorings

Almost half of clone sets require a lot of cost on both modifying and testing

The comprehensive evaluation of “Statement-Unit” refactorings

Almost half of clone sets shouldn’t be refactored

Modifying

3 5 4 12

Testing 2 6 4 12

　 (a) (b) (c) (d) Total

Refactoring …

0 3 3 6 12

The following if-statement appeared 5 times in a class

Red variables mean that they are defined outside this if-statement

They must be added to the argument of the extracted method.

This statement can be extracted as a new method in the same class, but …

The argument of the new method become long.

Extracting 5 times requires a lot of cost.

if( vomrgpersonSyuningishi.getName() != null && !vomrgpersonSyuningishi.getName().equals(“”)) {

checkOwnSettlement(vomrgpersonSyuningishi.getShimeiNo(), vomrgpersonSyuningishi.getName(), vomanager, voprojectinfo, vodocumentinfo, vomrgpersonSyunigishi.getBusyomei());

Some “Statement-Unit” clones were judged as not suited to be refactored

Extracting manually a part of a method is complicated

Making the filtering conditions stricter

Helping modification working

Some clone sets depend on the framework used in the system

The following table represents the ratio of refactorings judged as effective ((a) improving, or (b) preventing problems for the future)

Case Study:Discussions

Extract Super Class

Move Method

Extract Method

Size 4/4 (100%) 5/5 (100%) 5/12 ( 42%)

Design 4/4 (100%) 5/5 (100%) 3/12 ( 25%)

Cohesion 0/4 ( 0%) 5/5 (100%) 0/12 ( 0%)

Coupling 0/4 ( 0%) 0/5 ( 0%) 0/12 ( 0%)

Understandability 4/4 (100%) 5/5 (100%) 4/12 ( 33%)

Reusability 4/4 (100%) 5/5 (100%) 4/12 ( 33%)

“Extract Super Class” and “Move Method” refactorings are effective

Those clone sets deteriorate some software qualities

Refactorings of them can improve qualities

They don’t require much cost

Contents

Background

Step1, Step2, Step3

Case Study

Overview

Evaluation Method

Result

Conclusion & Future Works

Conclusion & Future works

We haveproposed refactoring support method implemented a refactoring support tool, Ariesconducted a case study to a business application

“Declaration-Unit” and “Method-Unit” refactorings are effectiveSome “Statement-Unit” clones are not appropriate to be refactored

As future works, we are going toevaluate refactoring effectiveness quantitativelyadd an effectiveness measurement function to Aries for comprehensive supporting refactoring of code clones

Code clone detection for refactoring:Related Works

Detect similar sub-graphs as clone on program dependency graph [1].

High accuracy: This approach finds out data-dependence and control dependence in source codes.

High time complexity: It takes O(n2) time to construct program dependency graph.

Detect similar methods and functions as clone using metrics [2].

Low accuracy: if the size of target method or function is small, the values of metric make no difference.

detection unit restriction: only method and function unit clone can be detected.

[1] R. Komondoor and S. Horwitz, “Using slicing to identify duplication in 　 source code”, In Proc. of the 8th International Symposium on Static Analysis, Paris, France, July 16-18, 2001.[2] Magdalena Balazinska, Ettore Merlo, Michel Dagenais, Bruno Lague, and Lostas Kontogiannis, “Advanced Clone-Analysis to Support Object-Oriented System Refactoring”, WCRE 2000, pp. 98-107

Properties

Size: means the number of lines or token of modules

Design: means the structure of hierarchy of classes or encapsulation of classes

Cohesion: means the responsibility of classes. If a classes provide no or various functions, the cohesion of the class becomes wrong (low)

Coupling: means usage of fields or method of other classes. If fields or methods are defined in an inappropriate class, the coupling becomes wrong (high)

ARIES: Refactoring Support Tool for Code Clone

Documents