© 2014 Sungard AS. All rights reserved.

AMQP and RabbitMQ with CloudStackMay 2014

Sonal Ojha

2

In next 45 minutes …

• Why MOM architecture?

• Messaging broker like RabbitMQ in brief

• RabbitMQ and CloudStack

• AMQP – What is it ?

• A small demo

3

Message Oriented Middleware Architecture

• Method of communication between different applications or software components between distributed systems.

• The predominant approaches

RMI, CORBA, SOAP web services

Calls remote methods with parameters …. And waits for response

• Used for

• Scalability and load balancing

• Decoupling

• Reliability

• Availability

• Asynchronous hand off

• Queuing and buffering

• Monitoring and management

4

CloudStack Event Notification

5

RabbitMQ- Multi-protocol messaging server

• Born in 2006, Released in 2007, Acquired by Spring source in 2010

• Written in Erlang / OTP.

• AMQP is the base protocol (supports 0.8 and 0.9.1).

• Features :

• Ployglot

• Management UI and API

• Plugin Architecture

• Clustering, Active/Active HA Queues

6

AMQP (Advanced Messaging Queue Protocol)

• Originated in 2003 by John O’Hara at JPMorgan Chase.

• AMQP is not JMS.

• Open internet protocols like TCP, SMTP, HTTP.

• Open wire protocol standard for message-queuing communication.

• Enables conforming client applications to communicate with conforming messaging middleware services (aka message brokers)

• Defines

• Networking protocol

• Semantics of broker services

• Use cases

• Real time feeds for constantly updating information

• Encrypted assured transaction

Functional layer defines sets of commands

Transport layer carries these methods from application to server, handles channel multiplexing, framing, content encoding, heart-beating, data representation and error handling

AMQP conti ..

8

To get there …• Queues

• Messages are consumed from queues

• Stores messages in memory or disk

• Deliver in sequence

• Exchanges

• Messages are routed to queues through exchanges through pre-arranged criteria

• Matching and routing engines, rather can do more than routing messages.

• How is the message routed ??

Messages are routed to queues by matching routing keys against binding pattern.

Routing key Binding key

9

Queues

• Stores messages that are consumed by applications

• Attributes

• Name

• Durable (survive broker restart)

• Exclusive (used by only one connection and queue will be deleted when that connection closes)

• Auto-delete (deleted when last consumer unsubscribes)

• Arguments (for implementing additional features like message TTL)

10

Exchanges

Attributes

• Name

• Durability (survive broker restart)

• Auto-delete (deleted when all queues have finished using it)

• Arguments (these are broker-dependent)

• Note : User defined exchange types must be prefixed by “x-”.

Name Pre-declared namesDefault exchange Empty string queue name == routing keyDirect exchange amqp.direct Queue name and routing key

may differ (1:1 pattern)Fanout exchange amqp.fanout Ignores routing key (1:N pattern)

Topic exchange amqp.topic Binding pattern (wildcard pattern)

Headers exchange amqp.headers or amqp.match Header attributes (value of header == value of the binding)

11

Messages

• Carry content (header + body)

• Content body is opaque block of binary data.

• Broker never modifies the content body.

• Have client specified properties

• Content-type, Encoding, Timestamp, App-Id, User-Id, Headers

• Delivery mode

• Non-Persistent

• Persistent

• Producers send messages to exchanges with a routing key e.g. “abc”, or ordered set of keys e.g. “abc.cloud.aws”

• Exchanges route messages to queues whose binding pattern matches the message routing key or keys.

• Consumers create queues; these buffer messages for push to consumers

• Queues are stateful, ordered, and can be persistent, transient, private, shared. Exchanges are stateless routing tables.

• Consumers tell queues to bind to named exchanges; each binding has a pattern e.g. “abc” or “*.cloud.*”

Example of pubsub message flow

Messages

One more thing …

• Exchange -to- Exchange binding

• Extension over AMQP specification.

• Introduced in RabbitMQ 2.1.1 version.

• Works with applications functioning with RabbitMQ and not other AMQP 0-9-1 broker implementations.

• One message hops to multiple exchanges of different types.

• Sophisticated routing topologies than earlier.

13

14

Acknowledgements• Message Acknowledgements (aka acks)

• server removes a content from a message queue as soon as it delivers it to an application – Auto acknowledgement

• After application sends back an acknowledgement – Explicit acknowledgement

• Negative Acknowledgements (aka nacks)

• Rejected with basic.reject AMQP method

• Cannot reject multiple messages as done with acknowledgement

AMQP versions0-8, 0-9, 0-9-1, 0-10 (0 = major, 9 = minor, 1 = revision)

Newer version 1.0

15

DEMO&

Code Walk through

16

References

• https://www.rabbitmq.com/tutorials/amqp-concepts.html

• http://www.rabbitmq.com/documentation.html

• http://tryrabbitmq.com/

• http://www.amqp.org/

Thank You!