Studying the Evolution of Build
SystemsShane McIntosh
Queen’s University
What is the build system?
2
Build systems help practitioners
3
OK Testers
Developers
Managers
Build systems require 12%
of a developer’stime (on average)
4
Kumfert, G., and Epperly, T.Software in the DOE: The
Hidden Overhead of the “Build”
Address Bar Search Bar
5
4 months beforethe issue was fixed!
Research hypothesisBuild system maintenance imposes a
significant degree of overhead on software development
6
Studied projects
PLplot7
>30 MLOC!
Thesis Overview
1.0 1.1 2.01.High level
evolution analysis
2. Fine-grainedevolution analysis
3. Maintenance Overhead analysis
8
Acceptance Ratio:16/51 = 31%
1.0 2.0
Acceptance Ratio: 62/441 = 14%
1.0 2.0
High level evolution
1.Code analysis: Do the static size and complexity of source code and build system evolve similarly?
2.Runtime analysis: Does the build-time complexity evolve?
91.0 1.1 2.0
Our approach
10
Code analysis(Static)
1.Do the static size and complexity of source code and build system evolve similarly?
2.Does the build-time complexity evolve?
111.0 1.1 2.0
12
ANTLRAbstraction
Module Restructuring
Module Restructuring
Source & Build Change Together!
Runtime analysis (Dynamic)
1.Do the static size and complexity of source code and build system evolve similarly?
2.Does the build-time complexity evolve?
131.0 1.1 2.0
Libs builtfrom source
Build length increases over time
14
High level evolution results1.Do the static size and complexity of
source code and build system evolve similarly?
2.Does the build-time complexity evolve?
15
Yes, but events may impact build and
source differently.Yes, length tends to
grow as projects age.
Release granularity is too
coarse!1 2 31.1 1.21.1.1 1.1.2 1.1.3
Fixed bug #1234
Added feature #1212
Issue #4444: restructured
module16
Fine-grained evolution
1.How many files does a typical build system consist of?
2.How much does a typical build system churn?
3.How large are typical build system changes?
171.0 2.0
Revision tagging
1.1.1 1.1.2
Fixed bug #1234 Added feature #1212
Issue #4444: Restructured
module18
Revision tagging
Fixed bug #1234
19
Size of the build system
1.How many files does a typical build system consist of?
2.How much does a typical build system churn?
3.How large are typical build system changes?
201.0 2.0
Build accounts for 9% of files (median)
21
Build system rate of change
1.How many files does a typical build system consist of?
2.How much does a typical build system churn?
3.How large are typical build system changes?
221.0 2.0
High churn may result in post-release bugs!
Normalized churn is on par with source code
23
Size of build changes
1.How many files does a typical build system consist of?
2.How much does a typical build system churn?
3.How large are typical build system changes?
241.0 2.0
Build changes add and delete 3-4
linesSource changes add and delete 6-
11 lines25
26
Fine-grained evolution1.How many files does a typical build
system consist of?
2.How much does a typical build system churn?
3.How large are typical build system changes?
9% of filesOn par with
sourceAdds and deletes 3-4
lines
How do projects cope with build maintenance?
27
Build maintenance overhead
1.How often are build changes required to complete development tasks?
2.How do projects distribute build maintenance work?
281.0 2.0
Logical coupling
29
LC(Source ⇒ Bld)
= 2 ÷ 4 = 50% 1.0 2.0
Logical coupling at revision level is
low!
30
Source ⇒ Build coupling well below 12% estimation!
Work item grouping
1.1.1 1.1.2
Fixed bug #1234 Added feature #1212
Issue #4444: Restructured
module31
Work item grouping
1.1.1 1.1.2
Fixed bug #1234Oops, forgot somecode for bug
#1234
Work item grouping
Bug #1234
33
Fixed bug #1234Oops, forgot somecode for bug
#1234
Task-centric analysis
1.How often are build changes required to complete development tasks?
2.How do projects distribute build maintenance work?
341.0 2.0
Work item coupling to the
build is high
35
Mozilla’s Source ⇒ Build coupling is very high!
Developer-centricanalysis
1.How often are build changes required to complete development tasks?
2.How do projects distribute build maintenance work?
361.0 2.0
If you produce source code, do you produce
build code?
37
Dispersed Ownership:
Build maintenance is
distributed amongst most team members
38
Concentrated Ownership:
A small team isresponsible for most
of the build maintenance
Build maintenance overhead1.How often are build changes
required to complete development tasks?
2.How do projects distribute build maintenance work?
Up to 27% require build changes
Dispersed or concentrated
Potential bias Studiedprojects
Allprojects
40
41
Questions?