Date post: | 18-Nov-2014 |
Category: |
Technology |
Upload: | dave-syer |
View: | 2,307 times |
Download: | 1 times |
Advanced Messaging with AMQPDave Syer, SpringSource, 2011
Agenda
● Overview of AMQP● Example use cases● What makes AMQP different?● Comparison with JMS● Spring AMQP
What is Messaging?
What can I use Messaging for?How does AMQP help?
What is Messaging?
What is a Messaging Protocol?What is it not?
Messaging Use Cases
Things a Message Broker Can Do ● Work distribution
● Pipelining
● Store-and-forward of messages
● Absorbing spikes of activity (again, decoupling)
● Routing to different consumers based on message properties
Messaging Use Cases
Patterns:● spatial decoupling (aka communication)
● temporal decoupling (aka buffering)
● logical decoupling (aka routing)
Promote: flexibility, resilience, performance, and scale
Conclusion: There is an enormous range of applications and problems to which messaging is an effective solution.
AMQP
AMQP = Advanced Message Queueing Protocol
Why AMQP?
AMQP is Especially Nice Because● All resources are dynamically created and destroyed by clients as
they need them – no static pre-configuration
● A clean and simple model: just three key nouns to learn
● Open standard, developed by the AMQP Working Group (VMware is a member)
● Lots of client libraries available in many languages, for free
● An excellent, freely available, open source broker implementation, called RabbitMQ.
AMQP 101
Create an exchange...
“my_exchange”type = fanout
Advanced != Complicated
X
AMQP 101
...create a queue...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
...add a binding...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
...all inside a broker...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
...publish a message...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
...message sits in queue.
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
Publish another message...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
...it also sits in queue.
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
Consumer retrieves messages in order...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
Consumer retrieves messages in order...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
With no binding, the message is discarded(producer can ask to be notified)
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
AMQP 101
Publish several messages...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
ED
CB
A
AMQP 101
and they are distributed amongst consumers on the same queue...
“my_exchange”type = fanout
Advanced != Complicated
X
“my_queue”
E
D
C
B
A
AMQP 101
Let's create another queue and bind it to the same exchange...
“my_exchange”type = fanout
Advanced != Complicated
X“my_queue”
“your_queue”
AMQP 101
..with a fanout exchange...
“my_exchange”type = fanout
Advanced != Complicated
X“my_queue”
“your_queue”
BA
AMQP 101
..all messages go to every queue...
“my_exchange”type = fanout
Advanced != Complicated
X“my_queue”
“your_queue”
B A
B A
AMQP 101
...and can be consumed...
“my_exchange”type = fanout
Advanced != Complicated
X“my_queue”
A
“your_queue”B A
B
AMQP 101
...at different rates...
“my_exchange”type = fanout
Advanced != Complicated
X“my_queue”
A
“your_queue” B A
B
AMQP 101
...depending on the consumer, network and broker
“my_exchange”type = fanout
Advanced != Complicated
X“my_queue”
A
“your_queue” B A
B
AMQP 101
Let's change the exchange to be a direct type...
Advanced != Complicated
X“my_queue”
“your_queue”“my_exchange”type = direct
AMQP 101
...then the bindings have to have a key...
“my_exchange”type = direct
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “vodka”
bk = “beer”
bk = Binding Key
AMQP 101
...and the messages have to have a routing key...
“my_exchange”type = direct
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “vodka”
bk = “beer”
rk = “vodka”
rk = Routing Keybk = Binding Key
AMQP 101
...a direct exchange matches therouting key to a binding key...
“my_exchange”type = direct
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “vodka”
bk = “beer”
rk = Routing Keybk = Binding Key
rk = “beer”
AMQP 101
...a direct exchange matches therouting key to a binding key...
“my_exchange”type = direct
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “vodka”
bk = “beer”
rk = Routing Keybk = Binding Key
AMQP 101
...messages with no matching binding...
“my_exchange”type = direct
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “vodka”
bk = “beer”
rk = Routing Keybk = Binding Key
rk = “tequila”
AMQP 101
...are discarded
“my_exchange”type = direct
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “vodka”
bk = “beer”
rk = Routing Keybk = Binding Key
AMQP 101
Let's change the exchange to be a topic type...
Advanced != Complicated
X“my_queue”
“your_queue”“my_exchange”type = topic
AMQP 101
...permits wildcard in bindings...
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
bk = Binding Key
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
EDCBArk = “a”
rk = “a.b”rk = “a.b.c”
rk = “a.e”rk = “a.b.e”
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
EDCBArk = “a”
rk = “a.b”rk = “a.b.c”
rk = “a.e”rk = “a.b.e”
X
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
EDCBrk = “a.b”
rk = “a.b.c”
rk = “a.e”rk = “a.b.e”
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
EDCBrk = “a.b”
rk = “a.b.c”
rk = “a.e”rk = “a.b.e”
X
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
EDCrk = “a.b.c”
rk = “a.e”rk = “a.b.e”
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
EDrk = “a.e”
rk = “a.b.e”
C
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
Erk = “a.b.e”
C
D
AMQP 101
Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements
“my_exchange”type = topic
Advanced != Complicated
X“my_queue”
“your_queue”
bk = “a.*.c”
bk = “#.e”
rk = Routing Keybk = Binding Key
C
DE
Other Stuff the AMQP Broker Does
Yes, all the standard things
● Errors can be raised for messages that do not get routed to any queues
● Messages can be consumed so that the broker does not forget about the message until the client explicitly acknowledges the message
● Messages can be published with a property indicating whether the message should be written to disk
● Transactions: making publication and acknowledgement of several messages atomic
● Flow control: e.g. used to stop publishers from overwhelming the broker in extreme situations
Conclusion: Why AMQP?
An Open Protocol has some advantages...
● Makes it easier to have multiple implementations that interoperate at the wire-level
● Avoids vendor lock-in: easy to rip out and replace individual components with alternative implementations
● Allows third-parties to write client libraries for other languages
● Decouples flag-day upgrades for both client libraries and broker
● Allows third-party traffic analysis tools to inspect and decode interactions between the clients and brokers
● Promotes similarities in APIs presented by different client libraries
Introducing RabbitMQ
RabbitMQ, AMQP and Messaging
● RabbitMQ consists of the broker (server) and several clients
● (Java, .Net, plus others)
● They speak the AMQP protocol to each other
● Anyone can write (and many have) other clients which also speak AMQP and work with RabbitMQ
● RabbitMQ is freely available and open source, licensed under the Mozilla Public License v1.1
● It’s already in many popular Linux distributions (Ubuntu, Debian, Fedora, Gentoo) and can be easily installed on OS X both through MacPorts and Homebrew
AMQP and JMS
How does AMQP compare to JMS?When would I use one or the other?
AMQP and JMS
AMQP JMSProtocol API
No language Java
Industry committee, mainly finance JEE
Interoperability from client libraries Interoperability from proprietary features
Resources can be managed by clients Resources often centrally managed
Not ubiquitous in enterprise Ubiquitous through JEE
Flexible usage patterns, emphasise routing
Two basic producer patterns (Queue and Topic)
Designed for large numbers of queues and exchanges
Hard to manage large numbers of Destinations
AMQP and JMS
JMS• standard in the Java space
• lots of open source options
• provided with Java EE application servers
• Spring JMS
Java producers
C
C
C
P
P
queue
Java consumers
queue
Message broker
@Component public class MessageSender { @Autowired private volatile JmsTemplate jmsTemplate; public void send(String message) { this.jmsTemplate.convertAndSend("example.queue", message); }}
Sending JMS Messages• Inject an instance of Spring's JmsTemplate.
• Provide the JMS ConnectionFactory in the JmsTemplate bean definition.
@Bean public ConnnectionFactory cf (){ return new CachingConnectionFactory(...); }@Bean public JmsTemplate template(){ return new JmsTemplate(this.cf()) ;}
JMS
AMQP and JMS
● AMQP• real standard
• a product of the the companies with the most mission critical requirements
• Spring AMQPAMQP producers
C
C
C
P queue
AMQP consumers
queue
Message broker
X
P X
exchanges
Spring AMQP
● Java and .NET
● AMQP core abstraction plus RabbitMQ implementation (built on rabbit client libraries)
● Higher level patterns for clients: admin, producers and consumers● ConnectionFactory – extra abstraction on top of Rabbit client (e.g. for re-
connection)
● RabbitAdmin – declare exchanges, queues, bindings
● RabbitTemplate – convenience methods for send and receive
● MessageListenerContainer – POJO message handler, asynchronous
● Spring Integration support
http://www.springsource.org/spring-amqp
Spring AMQP: RabbitTemplate
public class MyComponent { private RabbitTemplate rabbitTemplate;
public MyComponent(ConnectionFactory connectionFactory) { this.rabbitTemplate = new RabbitTemplate(connectionFactory); } public void read() throws Exception { ... String value = rabbitTemplate.receiveAndConvert("myQueueName"); ... }
}
public class MyComponent { private RabbitTemplate rabbitTemplate;
public MyComponent(ConnectionFactory connectionFactory) { this.rabbitTemplate = new RabbitTemplate(connectionFactory); } public void read() throws Exception { ... String value = rabbitTemplate.receiveAndConvert("myQueueName"); ... }
} Convenience methods
Spring AMQP: SimpleMessageListenerContainer
● Asynchronous message receiver● POJO handlers● Handles re-connection and listener failure (rollback, redelivery)● Message conversion and error handling strategies
<listener-container connection-factory="connectionFactory"> <listener ref="handler" method="handle" queue-names="my.queue"></listener-container>
<listener-container connection-factory="connectionFactory"> <listener ref="handler" method="handle" queue-names="my.queue"></listener-container>
Q&A