Neil Bartlett

Weigle Wilczek UK

http://neilbartlett.name/blog

Managing Change

Mirko Jahn

InterComponentWare AG, Germany

http://osgi.mjahn.net

Agenda

• Where are we?

• What we are working with

• Versioning in practice

• Tool support

• Open challenges

Heraclitus Ephesius

(* 535 BC; † 475 BC)

“Nothing endures but change.“

Unknown Chinese Dude

“May you live in interesting times.”

Dealing with Change

How Microsoft does it

+ Unique version numbers

+ Fixed dependencies

+ Assembly cache

- Assembly qualified names

- Versioning only on assemblies (not namespaces)

- No flexible dependencies

- No dynamism

Doing it the Java way

• Jars as unit of reuse (packaging)

• Assumed always backwards compatible

• No enforced dependency model

• No built-in multi version support

• No runtime (except for Applets and JEE)

• No dynamism anticipated, but possible with limitations

OSGi’s answer (in 2 slides)

• Thin layer on top of (plain) Java

• Enriched meta data to thoroughly describe the unit of reuse called bundle aka jar with:

- Required dependencies

- Exposed APIs

- Runtime requirements

- Strong version support

- …

• Sophisticated runtime to enforce modularity

OSGi’s answer (in 2 slides)

• In JVM services (Object references) to decouple API from implementation

• Module dependencies designed on bundle and/ or package level

- Import-Package vs. Require-Bundle header

- Exposed API with Export-Package Header

• Versioning theme: [[[[Major].Minor].Build].Revision] like: 1.3.2.iter1234

• Ranges to flexibly define dependencies

OSGi in Action

a use case scenario

Evolving APIs

• Simple sample of a dictionary service

• Taken from the Eclipse PDE sample

• starting with 2 interfaces in package dictionary…

• Implementation details:

- Separation in two bundles (impl and api)

- Dependencies on package level

- User/ Client will be a simple command extension to proof the point

V.1.0.0

Evolving APIs

The implementation looks like this:

dictionary.api 1.0.0

dictionary 1.0.0

dictionary.impl 1.0.0[1.0.0,1.1.0)

cl.client 1.0.0

[1.0.0,2.0.0)register

s: dictionary.DictionaryService

bind

Evolving APIs

• Small changes were introduced in the second version…V.1.1.0

Evolving APIs

dictionary.api 1.0.0

dictionary 1.0.0

dictionary.impl 1.0.0[1.0.0,1.1.0)

cl.client 1.0.0

[1.0.0,2.0.0) register

bind

dictionary.api 1.1.0

dictionary 1.1.0

dictionary.impl 1.1.0[1.0.0,1.2.0)

or

register

cl.client 1.1.0

[1.1.0,2.0.0)

maybind

[1.0.0,2.0.0)

or

or

And in “real world”

- Few people are using OSGi in such a highly dynamic fashion (yet)

- Mostly the “is this bundle compatible with me” feature of versions is used and bundles get statically deployed as a product.

- Getting versions right is tough!!! Currently we only version exports. Imports only if we really know what we are doing and require this feature

- Typical problems you’re facing on a daily basis:

- What dependency is it? API, Impl or Client?

- Syntactic vs. Semantic incompatibilities

- Method/ property shadowing in forward compatibility

- Wrong/ incompatible versioned artifacts in official repositories

OSGi Issues and Questions

Gotcha Number 1

• Any qualifier is “higher” than no qualifier.

• Example: ROME (RSS/Atom library), an OSGi bundle developed by Sun Microsystems.

• Release Candidate: Bundle-Version: 1.0.0.RC1

• Release: Bundle-Version: 1.0.0

• This default is surprising for new OSGi users.

• Other systems (e.g. Maven, Jigsaw) use the opposite default.

Related Gotcha: Remember your A,B,Cs

• Release Candidate: Bundle-Version: 1.0.0.RC1

• Release: Bundle-Version: 1.0.0.Final

• Using “RC” doesn't leave enough space.

• Recommendation: use Alpha, Beta, Candidate and Final.

• May clash with existing corporate processes and policies for naming releases.

• Or, just use a date/time stamp on each build. This is the Eclipse approach, but makes it hard to identify release versions.

Gotcha Number 2

• Beta 1: Bundle-Version: 1.0.0.Beta1

• Beta 2: Bundle-Version: 1.0.0.Beta2

• …

• Beta 10: Bundle-Version 1.0.0.Beta10

• Oops, 10 < 2!

• Recommendation: zero-pad numbers in the qualifier, e.g. Beta01.

• Are 99 betas enough...?

• .Net does not allow alphanumeric characters in versions at all.

Gotcha Number 3

• Import-Package: org.foo; version=”1.0.0”

• Accepts 1.0.0 to Infinity!

• Do we really want to accept ALL future versions of org.foo?

• Is this an appropriate default?

Choosing an Import Range (1)

• Import-Package: org.apache.log4j;version=”[1.2,1.3)”

• Informally 1.2.*. Will match all future versions up to but not including 1.3.

• The most common choice. But this is a forward-looking statement.

• But what if 1.2.32 (released in 2014 after the acquisition of Switzerland by OraGoogSunBay) breaks compatibility?

• Cannot retrospectively change the released bundle. Must release a new version of our bundle with different import ranges.

Choosing an Import Range (2)

• Play it safe? Versions 1.2.13, 1.2.14 and 1.2.15 have been tested and are known to work, so:

• Import-Package: org.apache.log4j;version=”[1.2.13,1.2.15]”

• What if 1.2.16 is released tomorrow and is absolutely fine?

• What if another bundle requires version 1.2.12?

• Tight version ranges cause fragmentation.

No Disjoint Ranges

• Cannot say Import-Package: org.foo;version=”[1.0.0,2.0.0)” excluding 1.2 because it is buggy

• Should we allow definition of disjoint ranges?

• E.g. “[1.0,1.2) && [1.3,2.0)”

• Makes a complex area even more complex.

• Again cannot retrospectively add the exclusion.

Lack of Semantics

• The semantics of a version change are arbitrary.

• Is version 1.3 of Log4J compatible with version 1.2?

• There are suggested semantics but every library must be checked for compliance (and may stop complying at any time).

• Only the authors of a library truly know whether 1.3 is backwards compatible.

Idea: Reverse the Ranges?

• Idea originally suggested by Peter Kriens to CPEG

• Make exporters specify a range, not importers

• E.g. Export-Package: “[1.0.0, 1.4.0]”

• Meaning: “this is version 1.4 and it is compatible back to 1.0”.

• Should importers still import a range, where any intersection in ranges will match?

Non-Standard Semantics on Versions

• Usually higher means better, but not always

• E.g. Linux kernel, 1.1 and 1.3 are unstable, 1.0 and 1.2 are stable.

• In production we should favour 1.2.18 over 1.3.1.

• But in an lab/development environment we may favour 1.3.1

• Do we need system policies defined by a deployer or administrator?

• How on Earth could we manage such a policy if we have 1000s of packages?

• A rule-based system may be necessary.

• .Net allows limited overriding using a “publisher policy file”.

Bundle Repositories

Getting Bundles

• Most 3rd party libraries are STILL not OSGi bundles

• Even if they are, they may have poor-quality metadata (e.g. ROME)

• OSGifying libraries ourselves is a major drag

Bundle Repositories

• Some vendors have created bundle repositories with many open source libraries (e.g. SpringSource, Sonatype)

• But how reliable are they?

• If a bundle says Import-Package: org.foo;version=”[1.0,2.0)”, did the bundle creator really test every version in that range?

• If not, where does this range come from? Did they just guess?

• Who can I sue if it breaks??

Verified Repositories

• There is scope for a vendor to provide a truly verified repository backed by testing.

• Probably needs to be a paid service – somebody who can be sued

• What does it mean to test a library?

• Need to support revocation when bundles are found to have problems

Enhanced Querying

• Existing repositories answer queries based on bundle name, package name, etc.

• Can we partially offload resolving to the repository?

Tooling support

BND

• Created by Peter Kriens, now under the Apache Felix umbrella (ASF 2.0 license)

• Swiss Army Knife for bundle generation

• Can be used as:

- Command line tool

- Eclipse plug-in

- Ant task

- Maven goal

• Templating mechanism for version ranges

• Source code analysis for imports

• DS support

• Macros

BND

A sample configuration for the Maven plug-in (shortened):

...

<plugin>

<groupId>org.apache.felix</groupId>

<artifactId>maven-bundle-plugin</artifactId>

<extensions>true</extensions>

<configuration>

<manifestLocation>META-INF</manifestLocation>

<instructions>

<!-- enable simple spring xml file analysis -->

<_plugin>aQute.lib.spring.SpringComponent</_plugin>

<Export-Package>${osgi.export.package}</Export-Package>

<Private-Package>${osgi.private.package}</Private-Package>

</instructions>

</configuration>

</plugin>

...

Bundlor

• SpringSource’s tool to create Manifest files only

• Mainly a template mechanism

• The tool to use when contributing artifacts to their repo

• Can be used as:

- Command line tool

- Eclipse plug-in

- Ant task

- Maven goal

• Templating mechanism for version ranges

• Source code analysis for imports

• OSGi Profiles

Bundlor

A sample configuration for the Maven plug-in (taken from SpringSource):

Manifest-Version: 1.0

Bundle-ManifestVersion: 2

Bundle-Name: GreenPages Service

Bundle-SymbolicName: greenpages

Bundle-Vendor: SpringSource Inc.

Bundle-Version: 1.0

Import-Template:org.springframework.*;version="[2.5.6.A,3.0)

Excluded-Exports: greenpages.internal

Eclipse PDE

• Manifest centric development of bundles

• Many wizards and custom views to get you started

• Live validation of your configuration based on the target platform definition

• Integrated runtime to test you bundles without packaging them

• Issues with fragments during development

• Headless built possible

• API tooling support…

Eclipse PDE – API analysis support

• Baseline centric API comparison mechanisms

• Enriched with annotations

• Eclipse and Ant support

• Automated version increase according to the Eclipse versioning scheme possible

• Identity leakage of non-API types into API

• Many features are currently developed and will be included in Eclipse 3.5

• Based on source code and generated meta data

• OSGi aware

Eclipse PDE – API analysis support

jDiff

• Source code based analysis

• Can be used as Ant task, Maven goal or Command Line tool

• Different types of reports like JavaDoc and XML

• No OSGi awareness

jDiff

Remaining Questions

• Versioning is not entirely explored

• There is no suitable version scheme officially defined (no standard)

- What is binary compatible

- What is back-/ upwards compatible

- In the context of OSGi how do dependency affect compatibility?

• How to ensure the correctness of the applied versions?

• Dynamic applications need new strategies for testing. How could one ensure the compatibility with the future versions and rule out incompatibilities for an almost infinite set of bundle combinations

• How to ensure a bundle satisfies its version definition? (Certification?)

State of the Art

• NO existing tool can manage:

- 1000s of packages

- Multiple versions of each

- Sensible import ranges

- Different import range policies for different providers

Conclusion

• OSGi opens new opportunities, not imaginable before

• Sophisticated versioning support available

• A runtime that leverages using different versions, while guaranteeing compatibility as far as possible (if done right)

• Still, the field is not entirely explored and needs further research

- What implies a version is not satisfyingly defined

- How to ensure software quality in a dynamic world

• Tools are not there yet to remove the burden from the developer

Questions & Answers

Neil Bartlett

Weigle Wilczek UK

http://neilbartlett.name/blog

Mirko Jahn

InterComponentWare AG, Germany

http://osgi.mjahn.net

?