(1)

Introduction toSoftware Build Technology

Philip Johnson

Collaborative Software Development Laboratory

Information and Computer Sciences

University of Hawaii

Honolulu HI 96822

(2)

Objectives Understand motivation for build and packaging technologies.•Why was running your classmate’s Robocode robot so hard?

Become familiar with a sample of build technologies.

Get introduced to the build technologies and standards to be used in this class.

(3)

Motivation

Modern team-based, cross-platform development involves:•Multiple developers and users.•System is developed/used on multiple platforms:

-Win2K, Linux, Win98, Solaris• Developers/users have different environments:

-c:\jdk1.5.0_08, d:\jdk1.5, c:\myjava•Differences between user and developer distributions:

-Source distribution for developers-Binary distribution for users

•Developers may prefer different IDEs:-Eclipse, JBuilder, Emacs, RationalRose

•Users may not have any IDE at all.• Frequent releases and distributions.•Dependence on third party libraries (which may themselves be

released frequently).

These issues are not unique to Java development!

(4)

Motivation (cont.)

Build/packaging techniques should allow:•Users to build and test the system without IDE.•Developers to build and test the system using their IDE of choice.•Developers to build and test the way users build and test.•Configuration to different platforms and different environments

within a platform.•Multiple release configurations•Automated build to prevent errors•Rapid release, deployment, testing•Simplified installation and updating of third-party libraries

Our (Java-specific) solution:•Ant • Ivy•Build/packaging standards

(5)

A quick tour of build tools

(6)

(7)

Make ‘Make’ is the canonical build tool.

Make features:• File-level dependency analysis

-Example: compile only changed files•Unix-based (but ported to other platforms)• Language independent (Java, C++, etc.)•ASCII configuration file syntax

-Tab characters required—Yikes!•Supports invocation of any shell command.

•Does not support library-level dependency management.

•Related: imake, cmake, scons

(8)

(9)

Ant Java-based build tool by Apache•http://ant.apache.org/

Features:•Java-based•Cross-platform•Extensible•XML configuration files•Open source•Defacto standard for Java projects

-Maven is making headway.

•File-level, but not library-level dependency management.

(10)

Ant vs. Make Advantages of Make (and related tools):•More popular and widespread•Supports any programming language.•Provides ‘glue’ for invoking any shell command.

Advantages of Ant:• Java-based, active development community•Built-in support for Java and its tools.• build.xml simpler and easier than Makefile.•Simplifies cross-platform Java development•Easier to extend in a portable manner.

Current recommendation:•Always use Ant over Make for Java-based development•Use Make for other development, at least for now.

(11)

(12)

Maven “Project Management” technology•Build based on Ant.•Documentation and reporting plugins.•File-level dependency management•Library-level dependency management•Library repository definition & management

“Convention over configuration”•Maven builds in conventions for project layout and workflow to simplify use.-Great if you like them.

(13)

(14)

Ivy A “standalone” library-level dependency management tool.

Ant + Ivy =•Much of what Maven offers.

-No built-in reporting/documentation •Without Maven “conventions”.•Better representations for dependencies

-configurations

(15)

Ant+Ivy vs. Maven Very strong feelings on both sides.

Maven advocates:•Maven is the future; deal with it.

Ant+Ivy advocates:•Maven is a black box; customization is much harder than Mavenites portray.

My opinion:•If you can live with Maven conventions, it provides more built-in functionality.

(16)

Our approach We will use Ant+Ivy•Ant is still more popular than Maven, so it is useful for you to gain experience with it.•Moving from Ant+Ivy -> Maven should not be too bad.

You will also learn some “best practices” for Ant-based projects.•Support for useful third party tools.•Ivy-based library-level dependency management.

(17)

Basic Ant concepts Every project requires an Ant build file•named build.xml by default.

Each build file is composed of targets.•Typical targets: compile, junit, etc.

Each target is composed of tasks•Ex: copy .java files to build/src, invoke javac...

Targets can have dependencies•Ex: modified sources must be recompiled before testing.

(18)

Ant targets Dependent targets are executed exactly once.•Example:

-test depends upon compile-deploy depends upon compile-all depends upon test, deploy•Compile will be executed only once.

Some targets are executed only when needed.•Sources recompiled only if changed.

(19)

Example Ant target <target name="compile" depends=“init" description="Compiles code.">

<javac srcdir="${src.dir}" destdir="${build.dir}/classes“/>

</target>

(20)

Best practices Ant provides a simple, powerful language for building Java-based systems.

How do we use this language effectively?

The next section describes my best approach so far on how to employ Ant to achieve the Three Prime Directives of Open Source Software Engineering.

The “robocode-pmj-dacruzer” example project provides you with a template structure to get you off and running quickly.

(21)

robocode-pmj-dacruzer: An example Ant-based build system

Implements a pretty lame Robocode robot.•Not very interesting.

It’s the build system that’s interesting:•Provides a template

-Easy to adapt to your own projects.• Integrates third-party build tools

-Checkstyle, PMD, FindBugs, etc.• Is modular and extensible

-You can grow or shrink it to your needs.•Well suited to build automation and project hosting:

-Continuous integration (Hudson)•Uses Ivy to download/install/manage third-party libraries.

Result is Maven-ish, but without the “black box”• If you want to change something, it’s just Ant code.

(22)

Top-level directory contents build.xml•top-level build file.

*.build.xml•“module” build files, generally one per tool.

src/•the system source files

lib/•library files: maintained by Ivy.

build/•derived files: generated by build process

(23)

Build operations compile•compile system

jar•create jar file

junit• run junit tests

javadoc•build javadoc docs

clean•delete build dir.

dist•create zip dist file.

checkstyle• analyze source code

findbugs• analyze byte code

pmd• analyze source code

sclc• compute size of system

jacoco• compute coverage of test

cases.

(Others to be added later in semester!)

(24)

Build System Scalability Issues Modularity:•putting all targets in one build.xml file creates a very large build.xml file that is difficult to understand and maintain.

Incrementality:•you may not want to require all developers to have to install every possible third party tool.

Coupling:•dependencies between target and property definitions should be minimal and easily visible.

(25)

Build system architecture

build.xml

pmd.build.xml

findbugs.build.xml

dist.build.xml

checkstyle.build.xml

javadoc.build.xml

sclc.build.xml

*.build.xml

build.xml containsmost project-specificdefinitions.

*.build.xml files import the build.xml file and provide an implementation of a single extension.

common.build.xml

boilerplate code

(26)

build.xml build.xml provides most (but unfortunately not all) of the project-specific definitions:•version number•dependency libraries•project name•etc.

Most other *.build.xml files can be used with little to no changes.•Typical exceptions: junit.build.xml, emma.build.xml,

jar.build.xml

(27)

*.build.xml responsibilities Implements support for a single tool/function:•Checkstyle, Jar, JavaDoc, PMD, Distribution, etc.

Typical tasks (assume a tool called 'foo'):•Download foo using Ivy, install in lib/foo•Download a configuration file for foo, store in

lib/configfiles.•Provide three standard tasks:

-foo.tool: runs foo tool, generates data.-foo.report: creates HTML file.-foo: runs foo.tool and foo.report.

•Make 'foo' the default task.

(28)

User Interface

To do initial installation and compilation:•ant

For other operations, use -f <filename>:•ant -f dist.build.xml•ant -f junit.build.xml•ant -f checkstyle.build.xml•ant -f pmd.build.xml•ant -f emma.build.xml•etc.

(29)

Considerations This approach provides the following:•Minimal coupling: *.build.xml files depend only on definitions in build.xml•Modularity: individual *.build.xml files are small. •Reporting: "report" targets provide details on what the "tool" targets generated.

One problem:•How to "verify" that all analyses "passed"?

(30)

verify.build.xml Provides “one stop shop” for system analysis:•Imports not just build.xml, but other *.build.xml files as well. •Runs all "tool" analyses, failing build if they generate any warnings/errors. -Compile, Junit, PMD, Checkstyle, JavaDoc, FindBugs

User interface:•ant -f verify.build.xml

(31)

The lib/ directory Contains third-party libraries required by project:•robocode•checkstyle•pmd•findbugs•etc.

Contains configuration files for analysis tools.•lib/configfiles/checkstyle.modules.xml•lib/configfiles/pmd.rulesets.xml•lib/configfiles/findbugs.filter.xml

(32)

The build/ directory

Contains files derived from build process.

Can be deleted and reconstructed.

Not included in version control or distribution .zip.

Build subdirs: Targets creating them:•build/javadoc/ javadoc•build/classes/ compile•build/dist/ dist•build/sclc/ size•build/junit/ junit

(33)

build/ vs. bin/ Both Ant and Eclipse (or any other IDE) compiles files.

It causes problems if Ant and Eclipse are using the same output directories for compilation/etc.

Therefore, Eclipse writes to the bin/ directory, not the build/ directory.•Set in Project | Properties | Java Build Path | Source

(34)

Testing Notes There are two kinds of “tests” in this system:•Manual QA using JUnit

-Ensures your code satisfies its specifications (if you write good test cases.)

•Automated QA using Checkstyle, FindBugs, PMD-Ensures your code obeys best practices for coding style.

Encourages incremental testing and formatting.

(35)

Traditional release numbering

Traditional approach (i.e. jakarta-tomcat-5.1.0.zip)•MM.mm.bb, where:

-MM = Major release number(major redesign or incompatible changes)

-mm = Minor release number(minor enhancements, backward compatible)

-bb = Bugfix release number(bug fix, no new functionality)

•All numbers maintained manually.

Advantage: •Release number comparison indicates degree of change.

Disadvantage:•Requires manual editing of release value every time which leads to

errors (easy to forget to do this.)

(36)

Our release numbering Our approach (i.e. robocode-pmj-dacruzer-1.3.2011.09.11.12.32.zip)•Major.minor.timestamp, where:

-Major = major release number-Minor= minor release number-Timestamp= time of bugfix release (yyyy.mm.dd.hh.mm)

•Major and minor release number maintained manually.•Bugfix timestamp generated automatically.

Advantages:•Automatic except when incrementing major/minor release.•Release number provides timestamp of release.

Disadvantages:•Number of bugfix releases not indicated by release number.•Very long version number.

(37)

IDE integration Important to maintain independence from IDE.•All build functions should be possible without any IDE.

But it’s fine to provide IDE-specific files to facilitate development.•Example: .class and .project files in top-level directory for Eclipse.

Provide additional files/directories for other IDEs if desired.

(38)

robocode-pmj-dacruzer as a standard

From now on, all projects must be developed, packaged, and submitted using the standard build conventions embodied in this package.

(39)

robocode-pmj-dacruzer as a template

You can use robocode-pmj-dacruzer as a template for your own projects:•Download a fresh version.•Remove dacruzer-specific info•Add your project info.

See the template instruction page in the course website for step-by-step instructions.

(40)

Ant/Eclipse integration

Eclipse has nice support for Ant, but:

•You will also need to tell Eclipse about ANT_HOME:-Window | Preferences | Ant | Runtime | Ant Home…

•Make sure Eclipse writes to the bin/ directory-Project | Properties | Java Build Path | Source

(41)

Limitations of example The robocode-pmj-dacruzer template provides a framework plus basic build functionality, but has limitations:

•No SVN integration•No support for web app development.•No support for automated metrics collection.

We will enhance this initial template during the semester.

(42)

Meta-level goal Ant provides an efficient solution to satisfying both •PD#2 (users can install system) and •PD#3 (developers can understand and enhance system)

Other approaches exist:•InstallAnywhere for PD#2 (more work)•IDE-specific solution for PD#3 (limited)

With good installation documentation, I believe Ant-based build can satisfy both PD#2 and PD#3.

(43)