Parastoo Mohagheghi- 21 Sept.20041
The Impact of Software Reuse and Incremental Development on the Quality
of Large Systems
Parastoo Mohagheghi Dept. Computer and Information Science (IDI)University of Science and Technology (NTNU)
Trondheim, [email protected]
Doctoral thesis presentation, 21 September 2004
Parastoo Mohagheghi- 21 Sept.20042
Motivation of the study• Software organizations (and researchers) have always been
looking for effective strategies to develop software faster, better and cheaper: – OO analysis, CASE tools, formal methods, software reuse, agile
methods etc. are proposed, but there is no “silver bullet”.– There is an absence of models and theories that can help to reason
about the software without building it.– Rapid evolution of technologies does not allow proper assessment.– Collecting context-independent knowledge or representing context
is difficult.– Software development is inherently complex, especially for large
systems.• There is a lack of empirical studies in industry on
technologies that can answer what they are good at, and not so good at.
• The INCO- INcremental and COmponent-based Software Development project (2001 to 2004) tries to advance the state-of-the-art and practice for these technologies.
Parastoo Mohagheghi- 21 Sept.20043
Definitions• “Software reuse is the systematic practice of developing
software from a stock of building blocks..” [Morisio,2002]. – Reusable assets can take many forms: Component libraries, COTS
and OSS components, or entire software architectures and components in a product family approach.
– A reusable component in this study is a component that is reused in more than one product.
– Two distinct activities: development for and with reuse. • Component-Based Development (CDB) covers all the
aspects of developing (reusable, executable?) software components and assembling system from them. Components adhere to a component model.
• Incremental development is covering (or discovering) requirements gradually and developing systems in increments that accumulate functionality. Iterative development is recursive application of development activities and recursive elaboration of artifacts.
Parastoo Mohagheghi- 21 Sept.20044
Company background
• Ericsson has developed two large-scale telecom systems (470 KSLOC or 1000 KSLOC in equivalent C each) that share software architecture, components and a component model, development environment and software process.
• Systems are developed incrementally (5 releases of one system and 2 of the other). In periods, over 200 developers in different countries were working with design and test.
• Lots of data has been gathered in 5 years in different repositories. Data from several releases are analyzed in this study. It covers trouble reports, change requests, effort, size of components, measures from internal measurement programs and inspections.
• Personnel turnover has traditionally been small, but software development in Norway stopped in mid-2002.
Parastoo Mohagheghi- 21 Sept.20045
Overview of the Ericsson packet-switched core network- GPRS
Backbonenetwork
SGSN GGSN
BGW
IP network
Other networks
MSC/VLR HLR
SMS-GMSCSMS-IWMSC EIR
OtherSGSN
BSC/RNC
Packet-switchedcore network
Parastoo Mohagheghi- 21 Sept.20046
Evolution of the GSN software architecture
Wireless Packet Platform (WPP)
4
1 3
65
2
Rev. & Re-eng.
Wireless Packet Platform (WPP)
GPRS for GSMGPRS for GSM
3’
6’
5’
1’
2
4
7
8
The original architecture The hierarchical architecture
GPRS for GSM
GPRS for WCDMA
Reused
Application-specificlayer
Business-specificlayerCommon servicesLayer (includingcomponent framework)
Adaptation of RUP
Initial Process Adapted RUP
and the software process model
Parastoo Mohagheghi- 21 Sept.20047
Characteristics of the system
• The approach to initiate a product family has been an extractive one and involved high risks regarding cost, quality, training, coordination and management support.
• Quality requirements: Performance, availability, scalability and maintainability (or evolvability) are of great importance. There are no direct user interfaces.
• Large systems face challenges in all phases of development that small systems do not: difficulties in iteration planning, complexity of design, integration and test and maintenance costs. Large systems are designed to be long-lived.
• Components are developed in-house by different Ericsson organizations.
Parastoo Mohagheghi- 21 Sept.20048
Research questions
• RQ1. Why is a reuse program initiated and how is it implemented?
• RQ2. What is the impact of software reuse, CBD and incremental development on the quality? The impact of development approaches on product quality metrics and on project attributes such as schedule or effort are sought.
• RQ3. How to improve the practice of incremental development of product families in some aspects?
• Research questions are further refined in studies. – Some research questions and hypotheses were identified from
earlier work (top-down style) -> replication in new context and confirmatory studies.
– Other questions and hypotheses are results of exploratory work on available data and practices in the industry, in a bottom-up style -> explorative and descriptive studies.
Parastoo Mohagheghi- 21 Sept.20049
Research methods
• Quantitative studies by exploring (‘data mining’) company databases, studying distributions and assessing hypotheses.
• Qualitative data collected from web pages, textual documents, discussions with personnel and own experience on the process and practices are combined with results from quantitative studies.
• Several case studies (post-mortem), a small survey and a controlled experiment are performed.
• The rationale for mix of methods has been:– The impact of introducing reuse or incremental development is
widespread.– It is useful to take benefit of all available data.– Triangulation of data/methods.– Answering questions that are not possible to answer otherwise.
Parastoo Mohagheghi- 21 Sept.200410
Overview of all studies
Study of Software Process & RUP
2001-2002
P9
P6
P2Survey
2002
Study ofReuse Practice
2002
P1
P7
Experiment onInspection
2002P4
Study of MDA2003
Prototype
P3
Study ofTrouble Reports
2003
P8
Study ofChange Requests
2003
P10
Study ofEffort
2003-2004
P12
Phase 1 Phase 2
Developing EffortEstimation Method
2003-2004P13
Developing Data Mining Method
2004
P11
Assessing the Impact ofDev. Approaches & Identifying Metrics
2003-2004
P11
P5
Combining results in Phase 3
July 2004March 2001Quantitative studyQualitative study
P Paper InputContribution
C6b
C6a
C1
C3
C1 C2
C5
C4
C3
C
Parastoo Mohagheghi- 21 Sept.200411
Contributions and research questionsGroup Contribution RQ1 RQ2 RQ3
Reuse C1. Empirical verification of reuse benefits. √
Incremental dev.
C2. Increased understanding of the origin and type of changes.
√
Incremental dev.
C3. Developing an effort estimation method √
Both C4. Identifying metrics for a combination of reuse and incremental development.
√
Reuse C6a. Adaptation of the Rational Unified Process for reuse.
√ √
Method C5. Developing a data mining method. √
Incremental dev.
C6b. Improving techniques for incremental inspection of UML models.
√
Parastoo Mohagheghi- 21 Sept.200412
Contributions in software reuse
• C1. Empirical verification of reuse benefits (RQ2):– Quantitative studies of Trouble Reports (TRs) and Change
Requests (CRs). 13 000 TRs and 169 CRs were analyzed.– Reused components have significantly lower defect-density than
non-reused ones, and are more stable between releases (less modified). Data from 3 releases.
– There is no difference in change-proneness between reused and non-reused components (#CRs/KLOC). Data from 4 releases.
– First large-scale industrial study on reuse benefits.• C6a. Adaptation of the Rational Unified Process (RUP)
regarding reuse (RQ1, RQ3):– Proposing changes in workflows and activities for development for
and with reuse. – No empirical studies have been found on evaluating or adapting
RUP in this aspect. Proposals are generic and may be reused in other adaptations of RUP.
Parastoo Mohagheghi- 21 Sept.200413
Reuse and defect-density, release 3
Defect-density of blocks Mean Median Variance
#TRs/KLOC, Reused 1.32 0.76 1.70#TRs/KLOC, Non-Reused 3.01 2.44 4.39#TRs/MKLOC, Reused 3.50 1.78 21.26#TRs/MKLOC, Non-Reused 5.69 3.73 21.76
•H0: Reused components have the same defect-density as non-reused ones. Rejected.•Reused components have lower defect density, but they have proportionally more defects with severity A (highest severity).•Reused components have proportionally less defectsafter delivery -> more reliable.
Parastoo Mohagheghi- 21 Sept.200414
Size vs. defect-density
0
1
2
3
4
5
6
7
8
0 5 10 15 20 25 30 35 40 45
KLOC
#TR
s/K
LOC
Reused
Non-reused
H0: There is no relation between defect-density and component size for all/reused/non-reused components. Not rejected.
Parastoo Mohagheghi- 21 Sept.200415
Contributions in incremental development • C2. Increased understanding of the origin and type of
changes in requirements or artifacts in incremental development (RQ2):– A quantitative analysis of CRs showed that perfective changes
(enhancements and optimizations) are most common. Of these, non-functional changes to improve “quality attributes” are more frequent than pure functional changes.
– Most CRs (66%) are issued by the organization itself (and not external actors). Only 34% of CRS are adaptive or customer-initiated.
– Hypotheses are grounded in data. Generalization needs further study.
Perfective/Functional
Perfective/Quality Attr.
Adaptive Preventive Other (cost)
CRs 40 74 35 30 8
Parastoo Mohagheghi- 21 Sept.200416
Contributions in incr. dev.- cont.• C3. Developing an effort estimation method using use
case specifications and effort distribution in different phases of incremental software development (RQ3):– The method was adapted for incremental changes in (large) use
cases. It also includes a factor to estimate effort for building on a previous release.
– The impact of effort-breakdown profile on the estimation results is discussed.
– Method adapted to the context and scaled up.• C6b. Improving techniques for incremental inspection of
UML models (RQ3):– A small experiment (first in industry) was performed, comparing
company’s inspection technique with the new and adjusted OORTs(Object-Oriented Reading Techniques). The results showed no difference in cost-efficiency, but difference in the types of detected defects.
– Useful for improvement of OORTs and adapting these to the context.
Parastoo Mohagheghi- 21 Sept.200417
Contributions in software reuse and incr. dev., and research methods
• C4. Identifying metrics for a combination of reuse of software components and incremental development (RQ3):– Other literature discusses metrics for CBD. However, these metrics
should be adapted for incremental development and for reuse.
• C5. A data mining method for exploring industrial data repositories (RQ3):– A data mining method is developed that is based on experience
from the quantitative studies. It combines top-down theory search with bottom-up hypotheses generation.
– Challenges in integrating the results of several studies with one another are classified into physical and conceptual ones.
Parastoo Mohagheghi- 21 Sept.200418
Assessing development approaches
Requirement Modification Planning PrecisionIncremental & Iterative
Development
Reusable Artifacts/Components
Development Approaches Practices Product/process Quality
Needed Effort
Reduced Defect-Density
Changeability
Incremental DeliverySuccess
Incremental Integration
Incremental Planning
Solution Modification
Software Reuse
Component-BasedDevelopment
Component Stability
Leads to
+/-
+/-
+
+
+
Parastoo Mohagheghi- 21 Sept.200419
The Impact of development approaches and context on practices
Reuse Incremental dev. CBD Large systems
Software evolution
Change-proneness and evolvability.
Increased change rate,Earlier releases not evolved.
Component granularity is large for CRs.
QA are improved incrementally.Long-lived.
Effort estimation
ROI? No metrics.
Software from a previous release is evolved.Incr. changes in requirements, Large CM.
No data for effort spent on components.
Large CM and testing effort.
Data collection
Data should be collected for increments.
Different granularity.
No common repository.
Parastoo Mohagheghi- 21 Sept.200420
Answers to research questions• RQ1. Why a reuse program is initiated and how is it
implemented?– A product family is initiated because of the similarity between
requirements of the emerging system and an existing system. Software architecture is evolved.
– A lightweight or extractive approach to product family adoption was chosen. Management support, common goals and common infrastructure, and experienced personnel have been critical for the success of reuse. The software process model is not adapted to the same degree.
• RQ2. What is the impact of software reuse, CBD and incremental development on the quality?– Lower defect density and higher stability of reused components.– Product is getting more change-prone (in %of accepted CRs and
reduced requirement stability). Quality is improved incrementally, and functionality is both enhanced and improved. Earlier releases are not evolved.
– High share of integration and testing effort.
Parastoo Mohagheghi- 21 Sept.200421
Answers to research questions- cont.
• RQ3. How to combine the qualitative and quantitative results to improve the practice in some aspects?– The effort estimation method, metrics for a combination of
development approaches, the research method for data mining, andproposals for adapting RUP.
• Relation to INCO goals:– G1. Advancing the state-of-the-art of software engineering. Better
understanding of approaches is achieved. Contributions C1, C2 and C4.
– G2. Advancing the state-of-the-practice in software-intensive industry and for own students. Some feedback is given to Ericsson. Contributions C3 to C6 are reusable.
– G4. Disseminating and exchanging the knowledge gained. Most results are published. 11 students were involved in the studies.Teaching in courses.
Parastoo Mohagheghi- 21 Sept.200422
Discussion• Case study research:
– Studies are performed in a real context. Context-independent knowledge is rare in SE and generalization is difficult.
– Affiliation in the company provided access to data (revelatory case).
– The system is a critical case for verifying reuse benefits (extractive approach, only two products and high risks).
– The size of the system allows evaluating techniques for large-scale development.
– In some cases, an example of industrial practice, e.g. RUP.
• Incremental development is combined with reuse of software in multiple products and CBD.– All aspects of software development need adaptation; e.g.
estimation method, software process model, data collection routines etc.
Parastoo Mohagheghi- 21 Sept.200423
Conclusions
1. Describing different aspects of software development; i.e. the power of example.– The work also identified areas for improvement like
the process model and the data collection routines.2. Verifying existing theories and assessing existing
methods in new contexts; i.e. the power of replication: – Reuse benefits in a large industrial system, – Estimation method and inspection techniques in a
large project and with incremental development, – RUP in the context of reuse.
Parastoo Mohagheghi- 21 Sept.200424
Conclusions- cont.
• Generating new theories, hypotheses or methods by analyzing data from new perspectives (as in grounded theory) or combining the results of several studies; i.e. the power of generalization:– The origin of change requests and the distribution
over functional and quality attributes,– The distribution of effort over development phases for
a large project with incremental development,– The impact of development approaches on quality
attributes,– Metrics for a combination of development approaches,– A data mining method based on experience.
Parastoo Mohagheghi- 21 Sept.200425
Why such studies are important for research community and industry?
• Methods should be tested in real context and for development in large.
• Companies are increasingly using mainstream methods, tools and programming languages. Results are of interest to a larger community.
• The studies analyzed data that was hardly studied before: evaluation of data collection routines and metrics for the company, in addition to concrete results.
Parastoo Mohagheghi- 21 Sept.200426
Future work
• Study of RUP adaptations.• Empirical studies on software evolution. • Study of effort distribution. • Estimation of incremental projects• Use cases for effort estimation.• Relevant metrics for incremental development of
large systems.
Parastoo Mohagheghi- 21 Sept.200427
Acknowledgements
• Thanks to INCO, Simula and NTNU for financial support.
• Thanks to my supervisor, Prof. Reidar Conradi, and colleagues at INCO and IDI.
• Thanks to Ericsson for the permission to perform studies and to publish the results.