Hiram Chirino1
Hiram ChirinoConsulting Software EngineerRed HatAug 2013
Red Hat JBoss A-MQ is the Hotness
redhatHiram Chirino2
Presenter: Hiram Chirino Blog: h%p://hiramchirino.com/blog/
Twitter: @hiramchirino
GitHub: h%ps://github.com/chirino
Engineer at Red Hat
Apache Member and Ac?veMQ PMC Chair
Apache Commi%er on: Ac?veMQ, Camel, Karaf, ServiceMix, Felix, and Aries
Lead of STOMP 1.1 Specifica?onCo-‐Founder of many other OS projects
redhat
Hiram Chirino3
Messaging: The BasicsIt’s simple stuff!
redhatHiram Chirino4
Messaging Basics
• Uses Messaging oriented Middleware (MoM)
• Applications communicate by sending each other messages
• The MoM provides Applications a rich messaging API to control the QoS used to deliver messages
• At least once
• At most once
• Once and only Once
redhatHiram Chirino5
Destinations
App: A App: B
Consumer
Consumer
Consumer
QueueProducer
redhatHiram Chirino6
Point-to-Point: JMS Queues
Consumer
Consumer
Consumer
TopicProducer
redhatHiram Chirino7
Publish/Subscribe: JMS Topics
redhatHiram Chirino8
Selectors
redhatHiram Chirino9
Why use Messaging?
• Robustness to change• Time Independence• Loca?on Independence• Hide Latency• Scalability• Event driven• Simplicity• Configurable Quality of
Service• PlaXorm and Language
Integra?on• Fault tolerant
Hiram Chirino10
About JBoss A-MQLets dig in...
redhatHiram Chirino11
Based on Apache ActiveMQ
• Top Level Apache Software Foundation Project
• Wildly popular, high performance, reliable message broker
• Connects to nearly everything
• Native Java, C/C++, .Net,
• AMQP 1.0, MQTT 3.1, STOMP (1.0-1.2) and OpenWire
• STOMP enables Ruby, JS, Perl, Python, PHP, ActionScript …
• Embedded and standalone deployment options
• The key component of JBoss A-MQ!
redhatHiram Chirino
JBoss A-MQ
12
A small-footprint, high-performance, open source messaging platform
redhatHiram Chirino13
Fuse Fabric – Key Features
Fuse IDE
FMC
JBoss A-MQCluster
CamelEndpoints JBoss Fuse
Runtime Registry
• Supports Hybrid deployments
• Distributed Configura?on
• Run?me registry
• Centralized Management
redhatHiram Chirino14
Deploying JBoss A-MQ
• Standalone broker
• Highly Available message broker cluster
• Embedded – easy to use an en?re broker in JUnit tests (no need to Mock)
• In Tomcat, deployed as a war
• In JEE Server – either co-‐locate – or use client with JCA• Ac?veMQ distribu?ons contain a rar for this purpose
18
redhatHiram Chirino15
Smart Grids …
redhatHiram Chirino16
Next Generation Air Traffic Control …
NextGen Applications
FTI IP Backbone
En RouteControllers
TerminalControllers
Non-FAA Users(e.g., Airlines, DoDDHS, etc.) FAA
Command Center
SWIM Enterprise Infrastructure
redhatHiram Chirino17
And helping run the biggest science experiment – ever!
redhatHiram Chirino18
Btw - Why is Integration and Messaging Important ?
Head Office
STORE
24 houra
Enterprises Need to know Everything!
24 houra
24 houra
www
Hiram Chirino19
JBoss A-MQ Features every MoM should have
But some don’t…
redhatHiram Chirino20
Destination Wildcards
redhatHiram Chirino21
Composite Destinations
Producerqueue:BEER,WINE
Queue:BEER
Queue:WINE
Consumerqueue:BEER,WINE
redhatHiram Chirino22
Exclusive Consumers
redhatHiram Chirino23
Message Groups• Like Exclusive Consumers – but in parallel
• Guaranteed ordering of related messages across a Queue
• But – load balancing of messages across mul?ple consumers
• All messages with the same JMSXGroupID go to the same consumer
• How you group messages is down to the applica?on’s producer
• To explicitly close a group, set the JMSXGroupSeq to -‐1
redhatHiram Chirino24
Message Groups
Hiram Chirino25
Protocols
redhatHiram Chirino26
Open Standards Supported
AMQP 1.0 MQTT 3.1 STOMP OpenWire
Wire
Pro
toco
lsTr
ansp
orts
Clie
nt A
PIs
TCP/SSL HTTP/HTTPS WS/WSS
JMS CMS NMS
redhatHiram Chirino27
OpenWire
• Advantages:
• Fast – op?mized for Ac?veMQ
• client failover
• automa?c reconnect
• Client load balancing
• Flow control
• Many advanced features
Disadvantages:
• Not a recognized standard
• Only Java, C/C++/.Net
http://activemq.apache.org/openwire.html
redhatHiram Chirino28
MQTT http://mqtt.org
Advantages:
•M2M/”Internet of Things” transport
•Proposed as an OASIS standard
•Extremely light weight
•Growing support from vendors and OS products• WebSphereMQ
• Ac?veMQ + Apollo
• Mosqui%o
• RabbitMQ
Disadvantages:
• 3.1 does not support Queues
• Advanced features not standard• Flow control• Failover etc.
redhatHiram Chirino29
AMQP – see www.amqp.org
Advantages:
•AMQP 1.0 OASIS standard
•Proposed as an OASIS standard
•Commodi?zes the Broker
Disadvantages:
•One size doesn’t really fit all
•Currently no plans for IBM or Tibco to adopt it
redhatHiram Chirino30
STOMP – http://stomp.github.com
Advantages:•Easy to use text based protocol•Can use telnet as a client•Defacto standard:• Ac?veMQ + Apollo• HornetQ• RabbitMQ• PocoMQ• StompServer• OpenMQ• Many more …
Disadvantages:
• Not as fast as binary formats
redhatHiram Chirino31
But there’s more: WebSockets STOMP is a natural wire protocol for WebSockets
And so is MQTT!
Hiram Chirino32
Networks of Brokers
redhatHiram Chirino33
• Link Brokers together
• Use Store and Forward
• Are uni-‐direc?onal by default
• All Des?na?ons are global
Network of Brokers
redhatHiram Chirino34
Head Office
Example of a Network of Brokers
STORE
STORE
outlet
shop
24 houra
STORE
shop
BrokerClusters
redhatHiram Chirino35
Apache ActiveMQ – Broker Topologies
Store and Forward
ActiveMQLocal broker
ActiveMQRemote broker
redhatHiram Chirino36
Apache ActiveMQ – Broker Topologies
Bi-directional network
ActiveMQLocal broker
ActiveMQRemote broker
redhatHiram Chirino37
Broker Networks (Store and Forward)
Hiram Chirino38
Embedded Camel
redhatHiram Chirino39
Embedded Camel
Client
Client
Advanced rou?ng in the Broker – reduce latency and improve performance
Route between different brokers
redhatHiram Chirino40
<camelContext errorHandlerRef="errorHandler" xmlns="http://camel.apache.org/schema/spring">
<route> <from uri="activemq:queue:test.queue"/> <choice> <when> <xpath>$foo = 'bar'</xpath> <to uri="activemq:topic:topic.bar"/> </when> <when> <xpath>$foo = 'cheese'</xpath> <to uri="activemq:topic:topic.cheese"/> </when> <otherwise> <to uri="activemq:topic:topic.all"/> </otherwise> </choice> </route></camelContext>
Example Route
Hiram Chirino41
Message Storage
redhatHiram Chirino42
Message Delivery Mode
Messages can be persisted – allowing for decoupled applica?ons
For Queues, any message delivered as PERSISTENT must be stored on long term storage before being delivered to a consumer
For Topics, a message delivered as PERSISTENT will only be stored if there exists a durable subscriber
NON-‐PERSISTENT messages may also be stored on disk if the memory limits of the broker have been reached
redhatHiram Chirino43
Message stores supported by JBoss A-MQ
■ SQL (JDBC)
■ SQL (JDBC) with journal
■ AMQ Message Store
■ Kaha
■ KahaDB
■ LevelDB
Slow but popular
Don’t use (deprecated)
The “current” default
The “best yet”
redhatHiram Chirino44
LevelDB Store vs KahaDB
• Fewer index entries per message than KahaDB
• Faster recovery when a broker restarts
• LevelDB index out-‐perform Btree index at sequen?al access .
• LevelDB indexes support concurrent read access.
• Pauseless data log file garbage collec?on cycles.
• Fewer IOPS to load stored messages.
• It exposes it's status via JMX for monitoring
redhatHiram Chirino45
ActiveMQ: LevelDB Store
Hiram Chirino46
High Availability
redhatHiram Chirino47
• Java• C++• C#
Seamless Client Failover Support in
BrokerCluster
Client
failover://(tcp://host1:61616,tcp://host2:61616)
Polyglot Support
redhatHiram Chirino48
BrokerCluster
BrokerCluster
master
slave
master
slave
Combined HA and Networks
redhatHiram Chirino49
• JDBC Master/Slave
• Shared File System Master/Slave
• Replicated LevelDB Master/Slave
Master Slave Configurations
redhatHiram Chirino50
BrokerSlave
BrokerMaster
Client
BrokerSlave
Database or Shared FS
JDBC or Shared File SystemMaster Slave
Larry’s Removals
redhatHiram Chirino51
• Extreme reliability – but not as fast
• Recommended if already using an enterprise database
• No restric?on on number of slaves
• Simple configura?on
• Configurable lockKeepAlivePeriod
JDBC Master Slave
redhatHiram Chirino52
• Recommended if you have a SAN, or DRDB
• No restric?on on number of slaves
• Simple configura?on
• Ensure file locking works – and ?mes out – NFSv4 good!
Shared File System M/S
redhatHiram Chirino
Local File System Local File SystemLocal File System
53
BrokerSlave
BrokerMaster
BrokerSlave
ZooKeeperCluster
Replicated LevelDBMaster Slave
Client
redhatHiram Chirino
ZooKeeperCluster
Local File System Local File SystemLocal File System
54
BrokerSlave
BrokerMaster
Client
BrokerSlave
Larry’s Removals
Replicated LevelDBMaster Slave
redhatHiram Chirino55
• Will be available in JBoss A-‐MQ 6.1
• Requires a HA ZooKeeper Cluster
• No Single Point of Failure
• Dynamic number of slaves
• Simple configura?on
• Sync or Async Replica?on
Replicated LevelDBMaster Slave
redhatHiram Chirino56
Demo Time!
redhatHiram Chirino57
Useful Resources:
h%p://ac?vemq.apache.org
h%p://camel.apache.org
h%p://fuse.fusesource.org/fabric/docs/overview.html
h%p://fuse.fusesource.org/mq/docs/mq-‐fabric.html
h%ps://github.com/fusesource/fuseide
h%p://hawt.io
Messaging and Integration
Developer tooling
Management
Configuration and provisioning
redhatHiram Chirino58
Questions
and Discussion