SSBSE 2012 Keynote

transcript

SBSE Meets Software Maintenance:

Achievements and Open Problems

Massimiliano Di PentaUniversity of Sannio, Italy

dipenta@unisannio.it

Friday, September 28, 12

AimWhy SBSE is important for software maintenance

Tasks to which SBSE can be applied

Challenges in supporting maintenance through SBSE

Some possible solutions....

Definition

The process of modifying a software system or component after delivery to correct faults, improve performance or other attributes, or adapt to a changed environment.

IEEE Glossary of Software Engineering Terminology

SBSE topics breakout

Source: SEBASE Repositoryhttp://crestweb.cs.ucl.ac.uk/resources/sbse_repository/

SBSE is not only Testing!

SBSE @ ICSM 2012Ameni Ben Fadhel, Marouane Kessentini, Philip Langer and Manuel Wimmer. Search-based Detection of High-level Model Changes

Ali Ouni, Marouane Kessentini, Houari Sahraoui and Mohamed Salah Hamdi. Search-based Refactoring: Towards Semantics Preservation

Mathew Hall, Neil Walkinshaw and Phil McMinn. Supervised Software Modularisation

However, if you look at many other papers,some of the problems could have been nicely solved using

search based methods too!Friday, September 28, 12

Maint. @ SSBSE 2012Roberto Erick Lopez-Herrejon, Jose Galindo, David Benavides, Sergio Segura and Alexander Egyed. Reverse Engineering Feature Models With Evolutionary Algorithms: An Exploratory Study

Gergõ Balogh, Ádám Zoltán Végh and Árpád Beszédes. Prediction of Software Development Modification Effort Enhanced by Genetic Algorithm

Alex Tomasi, Alessandro Marchetto and Chiara Di Francescomarino. Domain-driven Optimization of Recovered Business Processes

Shadi Ghaith and Mel Ó Cinnéide. Improving Software Security using Search-Based Refactoring

Mathew Hall and Phil McMinn. An Analysis of the Performance of the Bunch Modularisation Algorithm s Hierarchy Generation Approach

Gabriele Bavota, Filomena Carnevale, Andrea De Lucia, Massimiliano Di Penta and Rocco Oliveto. Putting the Developer in-the-loop: an Interactive GA for Software Re-Modularization

Why Software Maintenance

needs SSBSE?

Large space to explore...

Presence of conflictingobjectives

Maintenance tasks have to fulfill some design goal / non-functional requirements

Design goals are likely conflictual

Performance

Examples• maintainability vs performance

• time to market vs reliability

• functionality vs power consumption

• functionality vs cost

• customers satisfaction vs resource requirements

• portability vs performance

Batteryconsumption

Code size

Customer’s satisfaction

Reducing power consumption

Nasir Ali, Wei Wu, Giuliano Antoniol, Massimiliano Di Penta, Yann-Gaël Guéhéneuc, Jane Huffman Hayes: MoMS: Multi-objective miniaturization of software.

ICSM 2011: 153-162

Three objectives

Size vs. consumption vs. customer satisfaction

cust.satisf.

consumption

Maintenance Activities (std. IEEE 1219)

Plan changes, bug triaging, staff activitiesPlan

Analysis

Design

Implementation

Regression testing

System testing

Delivery

Balance across conflicting goals

Support to program comprehension

Support to change, automatic bug fixing self-change

(Regression) test generation, prioritization, test suite repairing

Dynamic binding, self-healing

Maintenance Activities (std. IEEE 1219)

Plan changes, bug triaging, staff activitiesPlan

Analysis

Design

Implementation

Regression testing

System testing

Delivery

Balance across conflicting goals

Support to program comprehension

Support to change, automatic bug fixing self-change

(Regression) test generation, prioritization, test suite repairing

Dynamic binding, self-healing

Two dimensionsLe

Presence of human-centric rationale

Project management

Regressiontesting

Support tochange

Self-repair

Self-evolving

Support toprogram

comprehension

Automation vs. human-centric

• SBSE has been very successful in activities that can be (almost) completely automated

• Think about test data generation, or non-functional test

• Many maintenance activities are highly human-centric

• Cannot fully and properly be captured by a fitness function

Where and how SBSE can help in software maintenance in its different phases

Challenges, difficulties, open problems

Challenge IManaging Maintenance

Activities

Some research problems

• Schedule maintenance activities over time

• Staff software tasks

• Release planning

• Bug triaging

Some work in the areaJ. Francisco Chicano, Francisco Luna, Antonio J. Nebro, Enrique Alba: Using multi-objective metaheuristics to solve the software project scheduling problem. GECCO 2011: 1915-1922

Ahilton Barreto, Márcio de Oliveira Barros, Cláudia Maria Lima Werner: Staffing a software project: A constraint satisfaction and optimization-based approach. Computers & OR 35(10): 3073-3089 (2008)

Jian Ren, Mark Harman, Massimiliano Di Penta: Cooperative Co-evolutionary Optimization of Software Project Staff Assignments and Job Scheduling. SSBSE 2011: 127-141

Massimiliano Di Penta, Mark Harman, Giuliano Antoniol: The use of search-based optimization techniques to schedule and staff software projects: an approach and an empirical study. Softw., Pract. Exper. 41(5): 495-519 (2011)

Stefan Gueorguiev, Mark Harman, Giuliano Antoniol: Software project planning for robustness and completion time in the presence of uncertainty using multi objective search based software engineering. GECCO 2009: 1673-1680

Massimiliano Di Penta, Mark Harman, Giuliano Antoniol, Fahim Qureshi: The Effect of Communication Overhead on Software Maintenance Project Staffing: a Search-Based Approach. ICSM 2007: 315-324

Chao C, Komada J, Liu Q, Muteja M, Alsalqan Y, Chang C. An application of genetic algorithms to software project management. Proceedings of the Ninth International Advanced Science and Technology, Chicago, IL, U.S.A., 1993; 247–252.

Chang CK, Christensen MJ, Zhang T. Genetic algorithms for project management. Annals of Software Engineering 2001; 11(1):107–139.

Alba E, Chicano F. Software project management with GAs. Information Sciences 2007; 177(11):2380–2401.

Problem statementMaintenance phases

Rework

Abandonment

People distribution

WP1 WP2 … WPn

Project

WP1 WP2 WP3

WP11 WP12 WP31 WP32

Project WBS

WP Ordering

Multi-objective project scheduling

1 2 3 4 5 6 7 8

Configuration

Database GUI Networking MiddlewareCore

Compl. Time (person months)

58 5653

49 4840 39 37

Configuration

Staffing

Handling specialized teams

Issues and open problems

• Project management is a human-centric activity

• A mathematical model seldom captures personal capabilities

• Skill, experience, synergy, capability to cooperate with others

• Even if we capture such variables in a model, how do we measure them?

So where can we get this data from?

Measure fitness through software repositories

MailsVersioning

Bugtracking

Mylyntraces

Capture expertise

Capture communication and mentoring skills

Identify emerging groups

RuntimeLogs

Capture skills

• Given an incoming change request, who will be the most suitable developer for fixing it?

• See who, previously, fixed very similar change requests

• Textual similarity between bug reports

John Anvik, Gail C. Murphy: Reducing the effort of bug report triage: Recommenders for development-oriented decisions. ACM Trans. Softw. Eng. Methodol. 20(3): 10 (2011)

Identify mentors ...Messages exchanged between newcomers and senior developers used to identify mentors

Similarly to what ArnetMiner does for academics

Gerardo Canfora, Massimiliano Di Penta, Rocco Oliveto, Sebastiano Panichella, "Who is going to Mentor Newcomers in Open Source Projects?", To appear in Proceedings of FSE 2012, November

10-17 2012, North Carolina, USAFriday, September 28, 12

Who induced fixes?f 1.8f 1.7f 1.6f 1.5

bug fixing

SZZ algorithm based on CVS annotate or SVN/Git blameJacek Sliwerski, Thomas Zimmermann, Andreas Zeller: When do changes induce fixes? MSR 2005

Sunghun Kim, Thomas Zimmermann, Kai Pan, E. James Whitehead Jr.: Automatic Identification of Bug-Introducing Changes. ASE 2006: 81-90

bug fixing

bug fixingbug introbug intro

Mine your fitness!

Software repositories provide you quite a lot of data to build a fitness function

e.g., capturing people attitude to perform a maintenance task, or to work together

Beyond project staffing

• Make project scheduling and staffing that self-adapt over time

• Developers availability, different workloads, request for unplanned skills

• Evolutionary models should learn from project managers’ experience

• Embedded in the fitness functions

Challenge II - Supporting Program

ComprehensionFriday, September 28, 12

A keynote from ICPC 2007...Search Based Software Engineering for Program Comprehension

Mark HarmanCREST,

King’s College LondonStrand, London, WC2R 2LS

United Kingdom

Abstract

Search Based Software Engineering (SBSE) is an ap-proach to software engineering in which search basedoptimization algorithms are used to identify optimal ornear optimal solutions and to yield insight. SBSE tech-niques can cater for multiple, possibly competing objec-tives and/or constraints and applications where the po-tential solution space is large and complex. Such situa-tions are common in software engineering, leading to anincreasing interest in SBSE. This paper provides a briefoverview of SBSE, explaining some of the ways in whichit has already been applied to program–comprehensionrelated activities. The paper also outlines some possi-ble future applications of and challenges for the furtherapplication of SBSE to Program Comprehension.

1. Introduction

Search Based Software Engineering (SBSE) aims to ap-ply search based optimization algorithms to problemsdrawn from software engineering. This approach to op-timization is a natural one, because so many engineer-ing applications are characterised by many complex andcompeting objectives in large search spaces. In these sit-uations, automated optimization techniques are naturalcandidates.

Search based optimization techniques are widely usedin other engineering domains, for example mechanicalengineering [46], chemical engineering [12], biomedicalengineering [67, 69, 79], civil engineering [4, 10, 27, 42]and electronic engineering [9, 22, 63]. Software engineer-ing is coming of age as a mature engineering discipline.It is, therefore, natural to ask:

“Why not apply search based optimization tosoftware engineering?”

This question was posed by Harman and Jones in2001 [34], in a ‘manifesto paper’ that drew attentionto the huge potential of automated search based opti-mization for software engineering problems and also in

an initial survey of early work on Search Based Test-ing, Modularisation and Cost Estimation by Clark etal. [19]. However, work on search based approaches tosoftware engineering problems dates back much further,with work on optimization for testing starting as earlyas 1976, when Miller and Spooner [60] used classical op-timization techniques for test data generation. In 1992,Xanthakis et al. were the first1 to apply meta heuristicsearch to software engineering, when they used searchbased optimization for test data generation [81].

Since the 2001 manifesto paper, there has been agreat deal of activity in this area, with SBSE beingincreasingly applied to a wide range of diverse areaswithin software engineering. There have also been sev-eral workshops, conferences and special issues on SBSE.

Search based optimization techniques have proved tobe highly applicable in software engineering. For exam-ple, in the past five years, Search Based Software En-gineering has been applied to requirements engineering[5, 82], project planning and cost estimation [2, 3, 44],testing [6, 7, 13, 16, 33, 48, 58, 78], automated main-tenance [14, 26, 32, 61, 62, 66, 71, 72], service-orientedsoftware engineering [18], compiler optimization [20] andquality assessment [15, 43]. The application of op-timization techniques to software testing has recentlywitnessed intense activity. In 2004 there was su!cientmaterial to warrant a survey paper on this sub-area ofactivity [57]. However, as the list of application areasabove indicates, optimization can be applied right acrossthe spectrum of software engineering activity.

This paper accompanies the author’s keynote at the15th International Conference on Program Comprehen-sion. It explores ways in which SBSE ideas and tech-niques can be applied to problems in Program Com-prehension. Following the explosion of work in searchbased optimization for software engineering in general,this paper asks the more specific question:

“Why not apply search based optimization toProgram Comprehension?”

1So far as the author is aware.

What did Mark discuss?• Optimized pretty-printing

• Search-based amorphous program slicing

• Optimizing code and design for program comprehension

• Self-evolving visualizations

• Co-evolving program comprehension

• Linguistic evolution

• ...

Comprehension-oriented optimization

• Many problems are similar to those encountered in modularization and refactoring

• Different objective: comprehension

• Different persistence of produced artifacts: may be used for a comprehension task only

• High interaction with humans

What’s the primary source of information for program comprehension?

Ideally, high-level artifacts, models...

At least, you have source code...

and source code is text!Friday, September 28, 12

Leveraging Natural Language Analysis of Software: Achievements,

Challenges, and Opportunities

Keynote by Lori Pollock @ICSM 2012

Applications of textual analysis

• Traceability link recovery

• Feature/concern location

• Code search

• Clone detection

• Building quality models / defect predictors

• Refactoring

• Redocumentation

• ....

How can search-based techniques be useful in this context?Friday, September 28, 12

IR process

• Various choices for each step of the process

• Each technique requires a careful calibration

• E.g. number of concepts for LSI, number of topics, α, β for LDA

• Choices are highly dependent on the corpus and on the problem

• That’s a search problem too!

Lexicalanalysis

Stopwords

removal

Stemming/lemmatization

TermIndexing

AlgebraicmodelDocuments

Identifier splitting expansion

OpenFile

Open File

OpenFile

prmcntr

Open File

OpenFile

prmcntr

pr mcntr

Open File

OpenFile

prmcntr

pr mcntr

prm cntr

Open File

OpenFile

prmcntr

pr mcntr

prm cntr

prmc ntr

Open File

OpenFile

prmcntr

pr mcntr

prm cntr

prmc ntr parameter control

Open File

OpenFile

prmcntr

pr mcntr

prm cntr

prmc ntrprogram control

parameter control

Open File

OpenFile

prmcntr

pr mcntr

prm cntr

prmc ntr

parameter counter

program control

parameter control

Open File

OpenFile

prmcntr

pr mcntr

prm cntr

prmc ntr

program counter

parameter counter

program control

parameter control

Open File

Identifier splitting/expansion as a search based problem• Search among possible words in dictionaries

• Context dictionary, English dictionary, domain terms, known abbreviations/acronyms

• Techniques: Dynamic Time Warping, Hill Climbing

L. Guerrouj, P. Galinier, Y.G. Guéhéneuc, G. Antoniol, M. Di Penta - TRIS: a Fast and Accurate Identifiers Splitting and Expansion Algorithm - To appear in proceedings of WCRE 2012, Oct. 2012

L. Guerrouj, M. Di Penta, G. Antoniol, and Y. G. Guéhéneuc. TIDIER: An identifier splitting approach using speech recognition techniques, Journal of Software Maintenance - Research and Practice, p. 31, 2011.

Challenge III Recommending

Changes

Recommendation overload!

Maintenance-related recommenders provide useful suggestions to developers

The number of possible improvement for a system snapshot can be huge

Recommending all of them constitutes just overload of information for the developer [Murphy 2007]

How SBSE can help?

• Searching for a (near) optimal set of recommendations

• With respect to what?

• Cost

• Performance

• Code quality...

• Security

• ... conflicting objectives, hence it’s a multi-objective optimization problem

Search-based modularization/refactoring

Maximizing inter-package/dependencies

Minimizing intra-package dependencies

Minimizing CK metrics

...use other metric profiles, e.g. security-related

Examples of work in the area

Brian S. Mitchell, Spiros Mancoridis: On the Automatic Modularization of Software Systems Using the Bunch Tool. IEEE Trans. Software Eng. 32(3): 193-208 (2006)

Kata Praditwong, Mark Harman, Xin Yao: Software Module Clustering as a Multi-Objective Search Problem. IEEE Trans. Software Eng. 37(2): 264-282 (2011)

Mark Harman, Robert M. Hierons, Mark Proctor: A New Representation And Crossover Operator For Search-based Optimization Of Software Modularization. GECCO 2002: 1351-1358

Mark O'Keeffe, Mel Ó Cinnéide: Search-based refactoring: an empirical study. Journal of Software Maintenance 20(5): 345-364 (2008)

Mark Kent O'Keeffe, Mel Ó Cinnéide: Search-based refactoring for software maintenance. Journal of Systems and Software 81(4): 502-516 (2008)

Shadi Ghaith, Mel Ó Cinnéide: Improving Software Security Using Search-Based Refactoring. SSBSE 2012: 121-135

Traditional modularization...Create initial population

Evaluate individuals

Select individuals to reproduce

Recombination

Mutation

stop?no

Recombination

Mutation

stop?no

Recombination

Mutation

stop?no

ModularizationQuality Metrics

Recombination

Mutation

stop?no

Do the obtained refactoring make sense

for developers?...also discussed yesterday during the refactoring session @ICSM

Hybrid interactive GACreate initial population

Recombination

Mutation

stop?no

Recombination

Mutation

stop?no

Recombination

Mutation

stop?no

Recombination

Mutation

stop?no

HumanFeedback

Recombination

Mutation

stop?no

HumanFeedback

Want to learn more?Mathew Hall, Neil Walkinshaw and Phil McMinn. Supervised Software Modularisation. In Proceedings of ICSM 2012

...and... wait for the next session of SSBSE 2012!

Gabriele Bavota, Filomena Carnevale, Andrea De Lucia, Massimiliano Di Penta, Rocco Oliveto: Putting the Developer in-the-Loop: An Interactive GA for Software Re-modularization. SSBSE 2012: 75-89

Issues in interactive GAToo many interactions required ⇒ high effort required

• Learn the fitness function

• Use a mixed interactive and non-interactive, with periodic feedback

• Achieve a tradeoff between number of feedbacks (cost) and quality of results (effectiveness)

How to experiment/evaluate it?

• Simulate the human feedback

• Run a user study

Challenge IVAutomatic, Run-time,

Self ChangingFriday, September 28, 12

Achievements: automatic program repair

• Use of GP to modify code so that test suite passes

• Not really automatic enacted, but it produces a patch for a bug automaticallyClaire Le Goues, Michael Dewey-Vogt, Stephanie Forrest, Westley Weimer: A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each. ICSE 2012: 3-13

Claire Le Goues, ThanhVu Nguyen, Stephanie Forrest, Westley Weimer: GenProg: A Generic Method for Automatic Software Repair. IEEE Trans. Software Eng. 38(1): 54-72 (2012)

Andrea Arcuri: Evolutionary repair of faulty software. Appl. Soft Comput. 11(4): 3494-3514 (2011)

Westley Weimer, ThanhVu Nguyen, Claire Le Goues, Stephanie Forrest: Automatically finding patches using genetic programming. ICSE 2009: 364-374

Achievements: QoS-aware binding

Liangzhao Zeng, Boualem Benatallah, Anne H. H. Ngu, Marlon Dumas, Jayant Kalagnanam, Henry Chang: QoS-Aware Middleware for Web Services Composition. IEEE Trans. Software Eng. 30(5): 311-327 (2004)

Gerardo Canfora, Massimiliano Di Penta, Raffaele Esposito, Maria Luisa Villani: A framework for QoS-aware binding and re-binding of composite web services. Journal of Systems and Software 81(10): 1754-1769 (2008)

Gerardo Canfora, Massimiliano Di Penta, Raffaele Esposito, Maria Luisa Villani: An approach for QoS-aware service composition based on genetic algorithms. GECCO 2005: 1069-1075

Gerardo Canfora, Massimiliano Di Penta, Raffaele Esposito, Francesco Perfetto, Maria Luisa Villani: Service Composition (re)Binding Driven by Application-Specific QoS. ICSOC 2006: 141-152

AS1 AS2

hotelSearch

getAddress

Receive

getCabPricegetShuttleTicketPrice

getDistance

getMetroCardPrice

getHotelInfo

checkFlight

getCarPrice

[enoughTime ] [not enoughTime ]

[bigDistance ][not bigDistance ]

cost=15resp . time=10

Service automatic binding

Estimating the aggregate QoS

cost=2 avail=0.9

cost=3 avail=0.8

cost=2+3=5avail=0.9 · 0.8=0.72

cost=2 avail 0.9

Estimated Iterations=5

cost=2 · 5=10avail=0.95=0.59

70%30%

cost=2 avail=0.9

cost=3 avail=0.8

cost=0.3 · 2+0.7 · 3=2.7avail=0.3 · 0.9+0.7 · 0.8=0.83

cost=2 avail 0.9r.time=3

cost=3 avail=0.8r.time=2

cost=2+3=5avail=0.9 · 0.8=0.72r.time=max(3,2)=3

Sequence Loop Fork (flow)Switch

QoS Attribute Sequence Switch Flow Loop

Time (T)

T (ti)

pai · T (ti) Max{T (ti)i2{1...p}} k · T (t)

Price (P)

P (ti)

pai · P (ti)

P (ti) k · P (t)

Availability (A)

pai ·A(ti)

A(ti) A(t)

Reliability (R)

pai ·R(ti)

R(ti) R(t)

Custom Attr. (F) fS(F (ti)i2{1...m}) fB((pai, F (ti))i2{1...n}) fF (F (ti)i2{1...p}) fL(k, F (t))

10.0000

20.0000

30.0000

40.0000

initial estimate replan. value new estimate final value

Run-TimeRe-binding

Re-binding introducesoverhead

Beyond dynamic binding...Beyond automatic bug repair...

“From off-line to continuous on-line maintenance”

Keynote by Mauro Pezzè @ ICSM 2012

Various contexts...

Run-time replacement of failing behavior

• Identification of sequences of methods that can replace failing sequences with equivalent behavior

• This can be done at design time but...

• It can be a search-based problem too!

Open problems...

• Humans are not in the loop, hence the provided solution must be reliable enough

• Applicable possibly without a human check

• Techniques like GA do not guarantee a convergence

• For most of the SBSE applications, we did not care (a lot) about time needed to find a solution

• In run-time adaptation we do

Parallelize it!Metaheuristics such as GA are intrinsically parallel, however (nearly) all implementation do not exploit such a parallelism

Multi core and GPU provide great opportunities to improve the performances of your search-based technique

See tutorial by Simon Poulding tomorrow!

Carefully select the technique to be used

Recall the trace segmentation problem

Often the most obvious technique is not the best one

Also.. sometimes you don’t even need a search-based optimization technique!

Example: GA vs. Dynamic Programming

GA has a median convergence time of 1650 s

DP of 0.93 S

DP decomposes the problem in sub-problemsmemorizing cohesion/coupling for different parts of the trace

Task: splitting execution traces in conceptually cohesive segments

Conclusions

Taking out

Taking outMaintenance is human-centric

Need to balance conflicting goals

Configuring analysis techniques crucial to achieve good performance

Performance and quality of solutions crucial for run-time applications

Configuring analysis techniques crucial to achieve good performance