QUALGEN: Modeling and Analysing
the Quality of Evolving Software Systems
Tom Mens, Leandro Doctors{tom.mens | leandro.doctors}@umons.ac.be
Naji Habra, Benoît Vanderose, Flora Kamseu{naji.habra | benoit.vanderose | flora.kamseu}@fundp.ac.be
15th European Conference on Software Maintenance and ReengineeringMarch 1–4, 2011, Oldenburg, Germany
2 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Overview
• Context• Quality Metamodel • Evolving Software Ecosystem• Research Methodology • Future work
3 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Context
• Funded by the European Regional Development Fund (ERDF) – 2007-2013 program– consortium
• 4 universities from Wallonia (FUNDP, UMONS, UCL and ULB)
• 3 research centers (CETIC, CENAERO and MULTITEL) • several small- and medium-sized local companies
involved in ICT
• Supported by Wallonia
4 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Context
• Project lead by the Center of Excellence in Information and Communication Technologies (CETIC).– aimed at developing a portfolio of innovative techniques
related to the engineering of information systems, allowing local companies to master the diversity, complexity, quality and rapid evolution of these systems
• Two virtual centers of expertise :– CEIQS: Center of expertise in engineering and quality of
systems – CELLAVI: Center of expertise on the use of libre software
in industry
5 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Context
• QUALGEN workpackage– carried out in the context of the theme “Methodologies
for developing and evolving (software-intensive) systems”
• focus on open source software projects and software distributions
– close collaboration between Belgian universities (FUNDP and UMONS) since 2010 in the context of the CEIQS center
• 1 full-time person for 4 years (UMONS)• 1 full-time person for 2 years (FUNDP)
– in relation with the research activities of the CELLAVI center (libre software)
6 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Context
• QUALGEN workpackage
– modeling and evolution of quality from different points of view
development and subsequent instantiation and validation of a quality metamodel
study and control of quality evolution of software ecosystems
7 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Quality Metamodel
• Model-Centric Quality Assessment (MoCQA) framework– theoretical framework – based software measurement principles– quality model approach– continued quality assessment of software
• Generation of customised quality models– rely on a quality metamodel
8 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Quality Metamodel
Quality-related– Definition or reuse of a
hierarchy of quality factors
Measurement-related– Definition or reuse of
measurement method – Associated to the quality
factors
Project-related concepts – Project vs product– Software ≠ black-box– Scope = project ≈
ecosystem
9 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Evolving Software Systems
• Ecosystem-centered approach – “no project is an island. Software projects exist in
larger contexts [, . . . ] ecosystems” [Lungu09]
• Software ecosystem (in our context)– artifacts (source code, documentation, mailing list
archives, packages, distributions)– stakeholders (developers, users, managers)– entities (communities, projects),– processes (development and business models) – any other influential aspect
10 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Evolving Software Systems
• Different stakeholders– different /contradictory points of view on quality – different / contradictory quality priorities
• User communities : interested in popularity, usability and lack of defects
• Developer communities : interested in readability, reusability, maintainability and portability (software packages)
11 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Evolving Software Distributions
• Focus on libre and open source software distributions– collections of “assemblable” software components
that work seamlessly together (software packages)– tools to manage and configure the entire system– examples : OS based on the Linux and/or *BSD
kernels
• Important but understudied • Also “probably the most complex type of
software ecosystem” [Lungu09]
12 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Evolving Software Distributions
• Application of quality assessment methods (GQM, Qualipso) on Ubuntu and Debian– Ubuntu and Debian are “co-evolving” software
distributions (“coopetition”)– availability of the Ultimate Debian Database (UDD)
13 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Research Methodology
• Objectives and collaboration- Analysis of evolving software distributions
- development of a customised quality model - taking into account the specificities of stakeholder - taking into account the notion of ecosystem
- Model-Centric Quality Assessment framework- validation of the quality metamodel instantiation
process - validation of the usability of the generated models
14 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
• Objectives and collaboration- Analysis of evolving software distributions
- Model-Centric Quality Assessment framework
Research Methodology
provides a framework to help formalize the quality hypothesis
provides a concrete case to apply and validate the generation process
15 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Research Methodology
Iterative approach
• Structure a candidate quality hypothesis hierarchy
– (a) stakeholders (e.g., user, developer)
– (b) quality factors (e.g., popularity, readability)
– (c) relationships and influences quality factors (i.e., influences).
• Define the measurable artifacts (e.g., packages, source code, bug tracking data, mailing lists)
• Define the metrics and how they relate to the quality factors.
• Validate the candidate model• Refine the candidate model
16 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Future Work
• Validation of the customised quality model (and the quality metamodel) on the Debian and Ubuntu case studies
• Refinement of the MoCQA framework • Generalization of the customised quality
model to other case studies of evolving software distributions
17 / 17 CSMR 2011 - March 1–4, 2011, Oldenburg, Germany
Future Work
• Expected outcome – Tool support for the generation of
Customized Quality Assessment Model • graphical quality model editor • relying on the MoCQA metamodel
– Tool support for the analysis of the quality and evolution of software distributions
• relying on the generated CAQM• adapting to the needs of each type of
stakeholder/user