Syer amqp-s2gx-2011

transcript

Advanced Messaging with AMQPDave Syer, SpringSource, 2011

Agenda

● Overview of AMQP● Example use cases● What makes AMQP different?● Comparison with JMS● Spring AMQP

What is Messaging?

What can I use Messaging for?How does AMQP help?

What is Messaging?

What is a Messaging Protocol?What is it not?

Messaging Use Cases

Things a Message Broker Can Do ● Work distribution

● Pipelining

● Store-and-forward of messages

● Absorbing spikes of activity (again, decoupling)

● Routing to different consumers based on message properties

Messaging Use Cases

Patterns:● spatial decoupling (aka communication)

● temporal decoupling (aka buffering)

● logical decoupling (aka routing)

Promote: flexibility, resilience, performance, and scale

Conclusion: There is an enormous range of applications and problems to which messaging is an effective solution.

AMQP = Advanced Message Queueing Protocol

Why AMQP?

AMQP is Especially Nice Because● All resources are dynamically created and destroyed by clients as

they need them – no static pre-configuration

● A clean and simple model: just three key nouns to learn

● Open standard, developed by the AMQP Working Group (VMware is a member)

● Lots of client libraries available in many languages, for free

● An excellent, freely available, open source broker implementation, called RabbitMQ.

AMQP 101

Create an exchange...

“my_exchange”type = fanout

Advanced != Complicated

AMQP 101

...create a queue...

“my_queue”

AMQP 101

...add a binding...

“my_queue”

AMQP 101

...all inside a broker...

“my_queue”

AMQP 101

...publish a message...

“my_queue”

AMQP 101

...message sits in queue.

“my_queue”

AMQP 101

Publish another message...

“my_queue”

AMQP 101

...it also sits in queue.

“my_queue”

AMQP 101

Consumer retrieves messages in order...

“my_queue”

AMQP 101

Consumer retrieves messages in order...

“my_queue”

AMQP 101

With no binding, the message is discarded(producer can ask to be notified)

“my_queue”

AMQP 101

Publish several messages...

“my_queue”

AMQP 101

and they are distributed amongst consumers on the same queue...

“my_queue”

AMQP 101

Let's create another queue and bind it to the same exchange...

X“my_queue”

“your_queue”

AMQP 101

..with a fanout exchange...

X“my_queue”

“your_queue”

AMQP 101

..all messages go to every queue...

X“my_queue”

“your_queue”

AMQP 101

...and can be consumed...

X“my_queue”

“your_queue”B A

AMQP 101

...at different rates...

X“my_queue”

“your_queue” B A

AMQP 101

...depending on the consumer, network and broker

X“my_queue”

“your_queue” B A

AMQP 101

Let's change the exchange to be a direct type...

X“my_queue”

“your_queue”“my_exchange”type = direct

AMQP 101

...then the bindings have to have a key...

“my_exchange”type = direct

X“my_queue”

“your_queue”

bk = “vodka”

bk = “beer”

bk = Binding Key

AMQP 101

...and the messages have to have a routing key...

X“my_queue”

“your_queue”

bk = “vodka”

bk = “beer”

rk = “vodka”

rk = Routing Keybk = Binding Key

AMQP 101

...a direct exchange matches therouting key to a binding key...

X“my_queue”

“your_queue”

bk = “vodka”

bk = “beer”

rk = “beer”

AMQP 101

...a direct exchange matches therouting key to a binding key...

X“my_queue”

“your_queue”

bk = “vodka”

bk = “beer”

AMQP 101

...messages with no matching binding...

X“my_queue”

“your_queue”

bk = “vodka”

bk = “beer”

rk = “tequila”

AMQP 101

...are discarded

X“my_queue”

“your_queue”

bk = “vodka”

bk = “beer”

AMQP 101

Let's change the exchange to be a topic type...

X“my_queue”

“your_queue”“my_exchange”type = topic

AMQP 101

...permits wildcard in bindings...

“my_exchange”type = topic

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

bk = Binding Key

AMQP 101

Binding keys are period(.)-separated:* matches 1 element# matches 0 or more elements

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

EDCBArk = “a”

rk = “a.b”rk = “a.b.c”

rk = “a.e”rk = “a.b.e”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

EDCBArk = “a”

rk = “a.b”rk = “a.b.c”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

EDCBrk = “a.b”

rk = “a.b.c”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

EDCBrk = “a.b”

rk = “a.b.c”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

EDCrk = “a.b.c”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

EDrk = “a.e”

rk = “a.b.e”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

Erk = “a.b.e”

AMQP 101

X“my_queue”

“your_queue”

bk = “a.*.c”

bk = “#.e”

Other Stuff the AMQP Broker Does

Yes, all the standard things

● Errors can be raised for messages that do not get routed to any queues

● Messages can be consumed so that the broker does not forget about the message until the client explicitly acknowledges the message

● Messages can be published with a property indicating whether the message should be written to disk

● Transactions: making publication and acknowledgement of several messages atomic

● Flow control: e.g. used to stop publishers from overwhelming the broker in extreme situations

Conclusion: Why AMQP?

An Open Protocol has some advantages...

● Makes it easier to have multiple implementations that interoperate at the wire-level

● Avoids vendor lock-in: easy to rip out and replace individual components with alternative implementations

● Allows third-parties to write client libraries for other languages

● Decouples flag-day upgrades for both client libraries and broker

● Allows third-party traffic analysis tools to inspect and decode interactions between the clients and brokers

● Promotes similarities in APIs presented by different client libraries

Introducing RabbitMQ

RabbitMQ, AMQP and Messaging

● RabbitMQ consists of the broker (server) and several clients

● (Java, .Net, plus others)

● They speak the AMQP protocol to each other

● Anyone can write (and many have) other clients which also speak AMQP and work with RabbitMQ

● RabbitMQ is freely available and open source, licensed under the Mozilla Public License v1.1

● It’s already in many popular Linux distributions (Ubuntu, Debian, Fedora, Gentoo) and can be easily installed on OS X both through MacPorts and Homebrew

AMQP and JMS

How does AMQP compare to JMS?When would I use one or the other?

AMQP and JMS

AMQP JMSProtocol API

No language Java

Industry committee, mainly finance JEE

Interoperability from client libraries Interoperability from proprietary features

Resources can be managed by clients Resources often centrally managed

Not ubiquitous in enterprise Ubiquitous through JEE

Flexible usage patterns, emphasise routing

Two basic producer patterns (Queue and Topic)

Designed for large numbers of queues and exchanges

Hard to manage large numbers of Destinations

AMQP and JMS

JMS• standard in the Java space

• lots of open source options

• provided with Java EE application servers

• Spring JMS

Java producers

Java consumers

Message broker

@Component public class MessageSender { @Autowired private volatile JmsTemplate jmsTemplate; public void send(String message) { this.jmsTemplate.convertAndSend("example.queue", message); }}

Sending JMS Messages• Inject an instance of Spring's JmsTemplate.

• Provide the JMS ConnectionFactory in the JmsTemplate bean definition.

@Bean public ConnnectionFactory cf (){ return new CachingConnectionFactory(...); }@Bean public JmsTemplate template(){ return new JmsTemplate(this.cf()) ;}

AMQP and JMS

● AMQP• real standard

• a product of the the companies with the most mission critical requirements

• Spring AMQPAMQP producers

P queue

AMQP consumers

Message broker

exchanges

Spring AMQP

● Java and .NET

● AMQP core abstraction plus RabbitMQ implementation (built on rabbit client libraries)

● Higher level patterns for clients: admin, producers and consumers● ConnectionFactory – extra abstraction on top of Rabbit client (e.g. for re-

connection)

● RabbitAdmin – declare exchanges, queues, bindings

● RabbitTemplate – convenience methods for send and receive

● MessageListenerContainer – POJO message handler, asynchronous

● Spring Integration support

http://www.springsource.org/spring-amqp

Spring AMQP: RabbitTemplate

public class MyComponent { private RabbitTemplate rabbitTemplate;

public MyComponent(ConnectionFactory connectionFactory) { this.rabbitTemplate = new RabbitTemplate(connectionFactory); } public void read() throws Exception { ... String value = rabbitTemplate.receiveAndConvert("myQueueName"); ... }

public class MyComponent { private RabbitTemplate rabbitTemplate;

public MyComponent(ConnectionFactory connectionFactory) { this.rabbitTemplate = new RabbitTemplate(connectionFactory); } public void read() throws Exception { ... String value = rabbitTemplate.receiveAndConvert("myQueueName"); ... }

} Convenience methods

Spring AMQP: SimpleMessageListenerContainer

● Asynchronous message receiver● POJO handlers● Handles re-connection and listener failure (rollback, redelivery)● Message conversion and error handling strategies

<listener-container connection-factory="connectionFactory"> <listener ref="handler" method="handle" queue-names="my.queue"></listener-container>

Syer amqp-s2gx-2011

Technology