The Eclipse Virgo Experience at CME Group Jan Fetyko 03/29/2012.

The Eclipse Virgo Experience

at CME Group

Jan Fetyko03/29/2012

© 2012 CME Group. All rights reserved 2

CME Group

CME Group, formerly known as Chicago Mercantile Exchange Holdings Inc., was founded in 1898 and operates the CME, CBOT, NYMEX, and COMEX regulatory exchanges worldwide.

It is the world’s leading derivatives marketplace with ~2500 full time employees and a market capitalization of over $15B.

http://www.cmegroup.com/


• About our application

• Development strategy

• Getting it running in Virgo

• Web Layer

• Impact on our team

Topics


• Surveillance tool – monitors the state of exchange and the market

• Rewrite of an existing (legacy J2EE) app

• Many services

• Web application(s) that use the services

• Uses: Spring, JDBC, LDAP, EHCache, H2, c3p0, Apache Tiles, snaps

About our application

Development strategy

© 2012 CME Group. All rights reserved

Split all functionality into services / bundles

• Defined interfaces for services

• Skeletons only

6


Implemented the interfaces

• NO OSGi

• Not running in Virgo

• Unit tests with mocks

7

Getting it running in Virgo


Running in Virgo

Most bundles deployed and started without problems

Exposed and used implementations

3rd party libs a big pain

9


Breaking in Virgo Exposed and used implementations

10

Unable to satisfy dependencies of bundle 'impl-use' at version '1.0.0': Cannot resolve: impl-use

Resolver report:

An Import-Package could not be resolved. Caused by missing constraint in bundle <impl-use_1.0.0>constraint: <Import-Package: com…some.internal; version="0.0.0">

Solution

• Anything internal needs to stay internal• Even a factory should be a “service”: Interface + Impl• Should prevent others to see what is inside the bundle


• Not OSGi-fyied• Works great as long as no replication is needed• Replication using RMI probably works fine• Replication setup fails if using jgroups – it is loaded using

Class.forName(…)cacheEventListenerFactory.setClass("net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory");

• Ehcache-jgroups classes are not in Import-Package, it will throw ClassNotFoundException

• Also, ehcache-jgroupsreplication.jar is not OSGi-fyied

Solution

• Use bnd tool to modify ehcache jar(s)• Create a bnd tool accepted settings file• Push the new ehcache jar into common place (maven repo, git)

Breaking in Virgo 3rd party libraries : EHCache


Bundle-SymbolicName: ehcache-core-2.3.1Bundle-Version: 2.3.1Bundle-Name: ehcache-core-2.3.1Bundle-ManifestVersion: 2Implementation-Version: 2.3.1Implementation-Title: EH Cache CoreExport-Package: *;version=2.3.1Import-Package: org.jgroups.util,*


ehcache bnd tool properties


Bundle-SymbolicName: net.sf.ehcache.jgroupsreplicationBundle-Version: 1.4Bundle-Name: net.sf.ehcache.jgroupsreplicationImplementation-Version: 1.4Implementation-Title: EH Cache jGroups ReplicationExport-Package: *;version=1.4


bnd tool properties for ehcache-jgroupsreplication


What is happening in caching service

CachingService

On Node 2

XYZService

On Node 1

ObjectsObjects

Serialize

010011001010110010011001010110

Deserialize

ClassNotFoundException in ehcache bundle

CachingService

On Node 1


• Replication is possible only for objects known to ehcache• Deserialization of keys and values fails with ClassNotFoundException,

because your classes are not in the Import-Package

Solution

• Store only objects that are known to ehcache – serialize and deserialize them manually



Solution - continued

Public interface CachingService { CacheElement get(CacheKey key)}

public interface CacheElement { Object getValue(Class<?> classLoadingClass);}



• It is easier than bnd tool• Faster results• You will learn a lot about MANIFEST.MF which is essential to know OSGi

anyway• Most changed settings

• Import-Package• Export-Package• Version(s)

Breaking in Virgo 3rd party libraries : MANIFEST.MF

After hacking the MANIFEST.MF

• Use bnd tool to have repeatable results

Web Layer


Web Layer

host

Virgo snaps


Web Layer

host

Apache Tiles & Spring MVC

Shared Tiles Definitions

Snap Tiles definitionsextend hostdefinitions

Fragments: content


Web Layer Apache Tiles

Host

TileTile

Tile fragmentTile fragment


• Design for modularity• Understand class loading• Understand Manifest• If possible use 3rd party libs that are OSGi ready• Avoid serialization across bundles

Running in Virgo Mini Lessons learned

• For Virgo 2.1, we had to dig through the snaps source and request a tag for a version that worked in Virgo 2.1 (currently on Virgo 3.0.1)

• Snaps + Tiles not the best combination• Provides good flexibility• Snaps provides a single point of entry (filters, security, etc.)

Web Layer

ToolingDevelopmentDeploymentetc.


Tools

• STS (SpringSource Tool Suite)

• Apache maven + bundlor plugin (SpringSource)

• maven archetypes to create services and web bundles

• Flyway for database versioning

• Firefox + firebug

• No Virgo tooling or Libra

24


Developer Workflow

• Maven + profiles for easy deployment

• Custom plans in virgo while working on a bundle

<plan name="com.cme.security.plan” ….>

<artifact type="bundle" name="com.cme.web.security" version=”…" />

</plan>

25


Deployment

• QA and up: Package and deploy everything including Virgo every time

• Everything is scripted including DB schema update (flyway)

• QA time is reduced because changes are only made to a well defined set of bundles

• Risk of running the wrong code in production is reduced to zero by not including the bundle in the deployment package

– We have different deployment targets for 2 different internal customers. Their code cannot impact or risk being deployed together in production.

26


Team

• OSGi is not the easiest to understand for monolithic app developers

• Virgo is just like any other container to most users

• Once basic concepts are established, there are no issues

• There are few experts who can help everybody else

• 99% of code is not different from a non-OSGi code

• We started with 2 people using Virgo, now there are 16 at CME and 6 in Brazil

• Today we have 37 service bundles and 23 web bundles

27

Conclusions of the Eclipse Virgo experience


Virgo experience conclusions

• Did not hit any Virgo bugs (yet), stable, didn’t experience any crashes

• Problems only come from our code

• Virgo 3.x release improved memory consumption comparing to 2.1

• The learning curve is steep

• Design for modularity upfront is important

• Cannot go back to monolithic app

29

Q&A

Jan Fetyko03/29/2012

[email protected]

@jfetyko


Give Feedback on the Sessions

1 Sign In: www.eclipsecon.org

2 Select Session Evaluate

3 Vote


Futures trading is not suitable for all investors, and involves the risk of loss. Futures are a leveraged investment, and because only a percentage of a contract’s value is required to trade, it is possible to lose more than the amount of money deposited for a futures position. Therefore, traders should only use funds that they can afford to lose without affecting their lifestyles. And only a portion of those funds should be devoted to any one trade because they cannot expect to profit on every trade.

The Globe Logo, CME®, Chicago Mercantile Exchange®, and Globex® are trademarks of Chicago Mercantile Exchange Inc. CBOT® and the Chicago Board of Trade® are trademarks of the Board of Trade of the City of Chicago. NYMEX, New York Mercantile Exchange, and ClearPort are trademarks of New York Mercantile Exchange, Inc. COMEX is a trademark of Commodity Exchange, Inc. CME Group is a trademark of CME Group Inc. All other trademarks are the property of their respective owners.

The information within this presentation has been compiled by CME Group for general purposes only. CME Group assumes no responsibility for any errors or omissions. Although every attempt has been made to ensure the accuracy of the information within this presentation, CME Group assumes no responsibility for any errors or omissions. Additionally, all examples in this presentation are hypothetical situations, used for explanation purposes only, and should not be considered investment advice or the results of actual market experience.

All matters pertaining to rules and specifications herein are made subject to and are superseded by official CME, CBOT, NYMEX and CME Group rules. Current rules should be consulted in all cases concerning contract specifications.

Date post:	27-Dec-2015
Category:	Documents
Upload:	ambrose-bridges
View:	217 times
Download:	0 times

The Eclipse Virgo Experience at CME Group Jan Fetyko 03/29/2012.

Documents