Date post: | 29-Oct-2014 |
Category: |
Technology |
Upload: | charles-moulliard |
View: | 11 times |
Download: | 1 times |
Integration with Apache ServiceMix, Camel, CXF and ActiveMQ Charles Moulliard – Gert Vanthienen Apache Committer FuseSource
Speaker : Charles Moulliard
• Engineer in Agronomy & Master in Zoology – 18 years of experience in IT world development (J2EE, .NET), specialized in new technologies web/Internet & integraNon, Project manager in bank, financial, telco, insurance and transport world
• SoluNon Architect at FuseSource • CommiSer on projects : Apache ServiceMix, Apache Karaf (PMC) & Apache Camel
TwiSer : hSp://twiSer.com/cmoulliard
LinkedIn : hSp://www.linkedin.com/in/charlesmoulliard
My blog : hSp://cmoulliard.blogspot.com
Speaker : Gert Vanthienen
• @Apache SoYware FoundaNon – Apache Camel, Karaf, ServiceMix – Committer / PMC member – Apache Member
• @FuseSource – Member of the engineering team – Mainly working on FUSE ESB
Agenda
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
Agenda
• Integra(on – introduc(on
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
IntroducNon (1)
• IntegraNon projects are really hard and pain`ul
• We have to integrate different technologies/systems/infrastructures
• Lot of standards/products/tools exist (SOA,EAI,…) and formats (EDI, Fixedlength, XML, CSV, FIX,
• Their implementaNons are resource and Nme consuming
• SOA, BPEL and EAI have not been successfull
IntroducNon (2)
Apache foundaNon proposes alternaNve projects to simplify « integraNon project development »
• Apache Camel – integraNon « Swiss knife » toolbox
• Apache CXF – webservice/REST stack
• Apache AcNveMQ – middleware layer
• Apache ServiceMix – Container for IntegraNon project
Apache ODE (BPEL),
Apache Tuscany (SCA)
Apache Synapse (ESB – XML)
Not covered
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
Camel
• Birthdate -‐ March 2007
• OpenSource IntegraNon Framework implemenNng
EIP paSerns (book published in 2005 by Gregor Hohpe)
• Why EIP To provide word definiNon, grammar and language when designing integraNon projects
• Camel uses a Domain Specific Language transposing the EIP paSerns into Java, XML
Camel
• Some of its features/characterisNcs – In memory bus AlternaNve to JBI using NMR
– Route objects can be XML, File, Stream, Bytes
– TransacNonal architecture
– Sync/Async exchanges
– Threads Management
– Error and excepNon handling
Camel (1)
• More than 50 paSerns are implemented
hSp://camel.apache.org/enterprise-‐integraNon-‐paSerns.html
Camel (2)
• More than 100 components
hSp://camel.apache.org/components.html
Camel (3)
• 18 data formats to simplify data exchange between layers, applicaNons connected
• Example : JAXB, Bindy, SOAP
• Uses marshaler (xml object) and unmarshaler (object xml)
hSp://camel.apache.org/data-‐format.html
Camel (4)
The model Content Based
Router
And its DSL language
Camel (5)
Camel (6)
Camel (7)
Camel (8)
URI corresponds to component name +
properNes
Camel (9)
• Transpose the DSL using Java
• First the configure() method must created
Camel (10)
• Next we create a java class extending RouteBuilder class
Camel (11)
• Language is flexible and proposes
different ways to define the Endpoints and predicates
Camel (12)
• Integrate easily with IntelliJ, Eclipse,
NetBeans
Camel (13)
• Instead of Java, you can use
Spring / XML DSL
Camel (14)
• IDE assistance
DEMO 1
Apache Camel
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
CXF
• Merge of 2 projects (CelNc and Apache Xfire)
• Simplify the creaNon and deployment of webservices
• 2 approaches “java to wsdl” or “wsdl to java”.
• Support the following standards : – JAX-‐WS : Web Services (XML/SOAP)
– JAX-‐RS : RESTfull service (JSON)
– SOAP 1.1, 1.2, WSDL 1.1
– WS-‐Security, WS-‐Addressing, WS-‐RM
CXF (1)
<plugin> <groupId>org.apache.cxf</groupId> <artifactId>cxf-codegen-plugin</artifactId> <version>${cxf-version}</version> <executions> <execution> <id>generate-sources</id> <phase>generate-sources</phase> <configuration> <sourceRoot> ${basedir}/target/generated/src/main/java </sourceRoot> <wsdlOptions> <wsdlOption> <wsdl> ${basedir}/src/main/resources/report.wsdl </wsdl> </wsdlOption> </wsdlOptions> </configuration> <goals> <goal>wsdl2java</goal> </goals> </execution> </executions> </plugin>
To use it, simply add cxf maven plugin to the pom of the project
CXF (2)
• … and configure a spring config file <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:cxf="http://camel.apache.org/schema/cxf" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://camel.apache.org/schema/cxf http://camel.apache.org/schema/cxf/camel-cxf.xsd">
<import resource="classpath:META-INF/cxf/cxf.xml"/>
….
<cxf:cxfEndpoint id="reportIncident" address="http://localhost:9080/incident" wsdlURL="etc/report_incident.wsdl" serviceClass="org.apache.camel.example.ReportIncidentEndpoint"> </cxf:cxfEndpoint>
… </beans>
CXF (3)
• Plugin will generate Java classes – JAXB2
– Interface + @Webservice
@WebService(targetNamespace = "http://reportincident.devoxx.fusesource.org", name = "ReportIncidentEndpoint")!@XmlSeeAlso({ObjectFactory.class})!@SOAPBinding(parameterStyle = SOAPBinding.ParameterStyle.BARE)!public interface ReportIncidentEndpoint {!
@WebResult(name = "outputReportIncident",! targetNamespace = "http://reportincident.devoxx.fusesource.org",! partName = "out")! @WebMethod(operationName = "ReportIncident",! action = "http://reportincident.devoxx.fusesource.org/ReportIncident")! public OutputReportIncident reportIncident(! @WebParam(partName = "in”,! name = "inputReportIncident”,! targetNamespace = "http://reportincident.devoxx.fusesource.org")! InputReportIncident in! );!}!
DEMO 2
Apache CXF
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
AcNveMQ (1)
Clients connect to the broker using connectors
with simple URIs
File & JDBC-‐based message stores supported.
… all based on a flexible Spring-‐based
core.
Topics and queues created dynamically.
Network connectors control how the
broker interacts with other brokers for
discovery, replicaNon, failover, clustering and store
& forward.
• A high performance, reliable messaging fabric, supporNng JMS, C, C++,.Net, Stomp clients. Offers persistence for messages and bridge between ServiceMix instances
• Point to Point – Sender/producer sends messages to a queue.
– Receiver/consumer listens for messages from the queue
– Messages are stored unNl read (or expired)
– Messages can be persisted on disk, are read only once.
c1:Consumer p:Producer
<<jvm>>���frodo:ActiveMQ
Foo:File m
1: Producer sends a message to the broker
m 2: Consumer reads
the message
AcNveMQ (2)
• Publish to subscribe – A JMS client sends message to the topic.
– The JMS broker sends message to all subscribers that are currently alive. • AlternaNvely, durable subscrip@ons can be used so that all subscribers will receive message even if not alive at Nme of sending
– Messages are consumed x Nmes (relaNon 1 to many)
s1:Subscriber
p:Producer
<<jvm>>���frodo:ActiveMQ
Foo:Topic m
m s2:Subscriber
m
AcNveMQ (3)
DEMO 3
Apache Ac(veMQ
Agenda
• IntegraNon – introducNon
• Presenta(on of projects -‐ Camel, CXF, Ac(veMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
ESB – ServiceMix (1)
DemysNfy the concept Enterprise Service Bus
• → This is not
• → Or a
But an exchanging pla`orm of messages, objects moving within transport layers
ESB -‐ ServiceMix (2)
• Allow to separate the business layer from those that will transport the informaNon and transform (marshal) it
• Provide connectors(= endpoints) to collect informaNon and send it on the bus as messages
ServiceMix (3)
• ServiceMix – Project of the Apache FoundaNon
• Started in 2005 to implement the Java Business IntegraNon specificaNon
• Decoupling of the components (business, technical) by using XML messages
• Define a model to package/deploy applicaNons (SU/SA)
• Layer transport is called Normalised Messaging Router
ServiceMix (4)
Message is ACK or ERR
Routing is defined into
the SU
Normalized = XML
exchanges
ServiceMix (5)
• JBI approach is interesNng but restricNve
– Everything must be XML format
– EncapsulaNon of the rouNng – Components dvlpt (25 vs more than 100 for camel)
– Few support from IT actors (IBM, Oracle, …) to improve exisNng spec 1.0 2.0
– Parallel with EJB story ….
ServiceMix (6)
• Decision has been taken 3 years ago by members of project to
– Become independent of JBI specificaNon
– Use « modularity » for jars management, versioning based on OSGI, OSGI EE specificaNons
– Adopt Camel as the new rouNng engine
ServiceMix (7)
• ServiceMix 4
• Based on an OSGI kernel – Apache Felix – Eclipse Equinox
• Becomes an IntegraNon Pla`orm for open-‐source
– Camel
– CXF – AcNveMQ,
– Aries (JPA, TransacNon, JNDI, Web)
Karaf – Heart of ServiceMix (8)
• Birthdate - 16th of June 2010 • Runtime using OSGI • Provide a lightweight container where
Java applications Camel routes WebServices
can be deployed
Karaf – Heart of ServiceMix (9)
<<jvm>>���:ServiceMix4
Java Boot (Register as service - Wrapper)
Lightweight container (Apache Karaf)
OSGI Kernel (Apache Felix / Eclipse Equinox)
Container Container Container Container Container Container
Karaf – Heart of ServiceMix (10)
• The pla`orm is « modular » because we can select the librairies, containers that we would like to use
Integration
WebService
Web
IoC
Middleware Java
Camel, JBI Jetty, Pax Web
CXF
ActiveMQ
Spring, Blueprint
Karaf – heart of ServiceMix (11)
• CharacterisNcs :
AdministraNon console (local, remote, ssh, web, jmx)
Provisioning system (features)
Hot deployment and configuraNon management
Instances management
Security integraNon (JAAS → ldap, jdbc, file)
Logging management (log4j, logger, commons logging, ….)
DEMO 4
Apache Karaf/ServiceMix
ServiceMix
ActiveMQ
Camel
CXF
<<supports jbi endpoints>> <<deploy into>>
<<supports cxf endpoints>>
<<supports activemq endpoints>> <<deploy into>>
<<deploy into>>
<<supports transport>> <<supports transport>>
<<implements flow with>>
The big picture
Agenda
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presenta(on
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
<<jvm>> ServiceMix4
Architecture -‐ Messaging
FTP Web Service
Bean<<MyBean>>
from(«Yp://server/dir») .beanRef(«myBean»);
from(«cxf:bean:myWS») .beanRef(«myBean») ;
• The bus operates the exchanges between the endpoints using Camel, NMR, JBI
BUS
<<jvm>> ServiceMix4
Architecture -‐ Java
• The OSGI pla`orm proposes a Service Registry (comparable to JNDI) where "interfaces" are registered
• Clients access to the service using lookup mechanism
« Interface » DAO
A
OSGI registry B
<bean id="incidentServiceTarget" class="org.apache.camel.service.impl.IncidentServiceImpl"> <property name="incidentDAO"> <osgi:reference interface="org.apache.camel.dao.IncidentDAO"/> </property> </bean>
Service PoJo
C
Hibernate/JPA
<<jvm>> ServiceMix4
Architecture -‐ Messaging + Java
• Decoupling of the "services" from integraNon layer
from(« cxf:bean:myWS ») .beanRef(« myBean », « validate ») .beanRef(«org.devoxx.jpaService») .beanRef(« myBean », « generateResponse »);
C
Web Service
B
Bean A
« Interface » DAO
OSGI registry
Service PoJo Hibernate/JPA
<<jvm>> ServiceMix4
« Interface » DAO
OSGI registry
Service PoJo
Hibernate/JPA
Architecture -‐ Messaging + Java + Web
• Can be web pla`orm, it uses JeSy Web ApplicaNon Server, Pax Web – a tooling for war and OSGI HTTP Service to register Servlet, …
Web ApplicaNon
OSGI RunNme
Bundle
Web Service Bean
A
Architecture – OSGI limitation • CamelContext are isolated due bundles
classloader • They can’t communicate directly using direct://,
vm:// component
56
<<jvm>> :ServiceMix4
BUS
FTP
from(Yp) .to(direct://order)
from(direct://order) .beanRef(“order”)
WebService
Bundle A Bundle B
<<jvm>> :ServiceMix4
Architecture -‐ LimitaNon
• NMR is required to bridge routes
NMR
Camel Context
FTP
from(Yp) .to(NMR://order)
from(NMR://order) .beanRef(“order”)
Bean Order
Bundle A Bundle B
Architecture -‐ Extend
• When we need asymetric deployment because CPU, memory is not sufficient then we use AcNveMQ to interconnect the instances and persist messages
Agenda
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
High-‐availability -‐ AcNveMQ
• 2 mechanisms pure Master/Slave & Shared message store
<<jvm>> samwise:ActiveMQ
Master state replicated with the slave
<<jvm>> frodo:ActiveMQ
<<jvm>> samwise:ActiveMQ
The brokers compete to acquire lock on the shared system (file or DB)
<<jvm>> frodo:ActiveMQ
Server 2 Server 1
Master
• A clustered, highly available approach with 2 machines
Slave
<<jvm>> Frodo:ActiveMq
<<jvm>> Frodo:ActiveMq
<<jvm>> Frodo:ActiveMq
<<jvm>> Frodo:ActiveMq
Network Of
Brokers
AcNveMQ -‐ Network of brokers
High Availability -‐ ServiceMix
• Only one mechanism is available – Lock based (File, DB)
• Bundles can be started or not
Master
<<jvm>>���frodo:SMX4
<<jvm>> samwise:SMX4
Slave
Master
<<jvm>>���frodo:SMX4
<<jvm>> samwise:SMX4
Slave
AcNve/Passive
AcNve/AcNve
High-‐Availability – All
• Combine AcNveMQ & ServiceMix + Network of brokers
Slave
Headquarter
queue:order BUS
Camel -‐ FTP
from(“amq:queue:order”) .beanRef(“order”)
BUS
Bean
queue:order
Master
queue:order
BUS
SMX4 -‐ Master
queue:order
from(“Yp”) .to(“amq:queue:order”)
Regional offices
Camel -‐ WS
BUS queue:order
from(“cxf”) .to(“amq:queue:order”)
from(“amq:queue:order”) .beanRef(“order”)
Bean
SMX4 – Slave
NetWork of Broker
Fuse Fabric Strategy
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
END OF PART I
A break is welcome !
Par t I
Fuse IDE & Fabric New Strategy to build integration projects
Fuse Fabric Strategy
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• Presenta(on of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
DEMO
Fuse IDE
Fuse Fabric Strategy
• IntegraNon – introducNon
• PresentaNon of projects -‐ Camel, CXF, AcNveMQ and ServiceMix
• Architectures presentaNon
• High availability, scalability and clustering
• PresentaNon of Fuse IDE
• Fabric new Strategy for provisioning and cloud
Par t I
Par t II
Fuse Fabric Strategy
• Why Fuse Fabric ? – “Integration” projects are really hard to install, configure, run and specially on OSGI platforms
– Distribution of workload/process is not an easy task
– Packaging and deployment must be simplified
– We want to make it easy!
What is Fuse Fabric? • http://fabric.fusesource.org/ • Open source software for configuring, provisioning
& running Fuse and Apache software on any machines – physical, virtual, private, public, private+public cloud or not
etc
• Keeps you DRY from those rainy clouds :) • Weave your containers into an easy to manage
fabric
Fuse Fabric: What does it solve? • Fact :
– Pay twice “dependencies” creation (maven and OSGI)
– maven-felix-pluglin simplifies bundle creation (*)
BUT this is still a complex process • Solution :
• Fab (Fuse Archive Bundle) • Use pom.xml dependencies instead of
MANIFEST to deploy bundles
P A C K A G I N G
Fuse Fabric: What does it solve?
• Fact – OSGI : Karaf – ServiceMix allows to create new instances and administrate them
BUT
DEPLOYMENT
• Instances are not “cloned”
• Configurations must be updated (manually)
• No Central Deployment platform
• Fabric will help us to create instances and manage them – Locally – Remotely (ssh) – In the cloud
DEPLOYMENT
Fuse Fabric: What does it solve?
Fuse Fabric: What does it do? • Scalability of HTTP, WebServices, RESTfull
services depend on external infrastructure (web farm, loadbalancer, hardware, …)
• Configuration changes are required (IP address, port number, …) to deploy new servers/services
• Solution : – Virtualization of services, endpoints, brokers – Discovery of services, endpoints and
message brokers
SCALAB I L I TY
What do we need ?
• Registry = repository of information(s) organised in a tree model
• Agent = process waiting instructions about things to do on servers (karaf, servicemix, activemq)
• Profile = list of tasks to do like – Insert component(s) – Setup a repository – Configure endpoint
Registry • Based on Apache ZooKeeper
– Is a Centralized Coordination Service, distributed and highly reliable
– Uses a directory & file based abstraction for looking at entries
Agent
• Feature deployed per Karaf/ServiceMix container • Agent connects to the registry to discover “tasks,
instructions” • Agent registers important informaNon into the registry • its locaNon, how to manage it via JMX etc
• Agent can control child Karaf containers or child processes
Profiles • Allow for easy group configuration & control
– like using Access Control Lists for security or LDAP trees
• Contains artefacts definition • An agent is started with one or more profiles
associated with it • You can then easily control a group of processes • Profiles support multiple inheritance to simplify
configuration work • Profiles are versioned for easy rolling update /
rollback
Profiles • Allow for easy group configuration & control
– like using Access Control Lists for security or LDAP trees
• Contains artefacts definition – Bundle – Features = collection of bundles – Repository = features xml file – Configuration = parameters of the endpoints, …
• An agent is started with one or more profiles associated with it
Profiles • Registry entries for a given agent:
– /fabric/configs/agents/{agent} – /fabric/configs/versions/{version}/agents/{agent} – /fabric/configs/versions/{version}/profiles/{profile}
• Profiles support multiple inheritance to simplify configuration work
• Profiles are versioned for easy rolling update / rollback
Camel Fabric • Expose / Virtualise a endpoint into the fabric space • Allow to invoke a remote endpoint • Loadbalance requests
DEMO
Fabric, Zookeeper. Provisioning and loadbalancing (ElasNc services)
D-OSGi Fabric • Implementation of distributed OSGI defined in
Enterprise OSGI Specification v4.2 • Allow to register “Interfaces” and look-up them
from remote instances • Use a TCP channel to send bytes over the wire
between jvm of fabric karaf instances • Services can be distributed everywhere and
easily faciltating the scalability
D-OSGi Fabric (next)
DEMO
Distributed OSGI services
ActiveMQ Fabric • Message brokers automatically discovery each
other • Clients use connection URI “fabric:default” to
connect to a message broker • If you have multiple logical ActiveMQ clusters just
name them in the URI e.g. – fabric:us-west, fabric:us-east, fabric:europe
• Workload can be shared between brokers (Apache Apollo)
ActiveMQ Fabric (next)
Other Fabrics • CXF Fabric
– adds a feature to the CXF bus for discovery & load balancing
• ServiceMix Fabric – allow local NMR to use remote endpoints when no local
NMR endpoint available
• Process Fabric – start & monitor processes – keep processes running across machines
References
• IntegraNon -‐ Camel : hSp://camel.apache.org
• ESB -‐ ServiceMix : hSp://servicemix.apache.org
• RunNme -‐ Karaf : hSp://karaf.apache.org
• WebService -‐ CXF : hSp://cxf.apache.org
• Broker -‐ AcNveMQ : hSp://acNvemq.apache.org
• OSGI EE -‐ Aries : hSp://aries.apache.org • FuseSource : hSp://fusesource.com
• EIP : hSp://www.enterpriseintegraNonpaSerns.com
References • Fabric web site :
– hSp://fabric.fusesource.org/ – hSps://github.com/fusesource/fabric
• Apache Zookeeper project – hSp://zookeeper.apache.org/
• Demo
– hSps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐cluster-‐loadbalancing
– hSps://github.com/fusesource/fabric/tree/master/fabric-‐examples/fabric-‐camel-‐dosgi
A Progress SoYware Company
Any Questions?
• twitter: @cmoulliard@gertv #fusenews • http://fusesource.com