October 18, 2007 1

AMQP, Erlang, and RabbitMQ

OOI Cyberinfrastructure Design Meeting

San Diego, 17th-19th October 2007

Matthias Radestockmatthias@lshift.net

October 18, 2007 2

So you want to build a computing infrastructure for ocean observation …

distributed reliable & resilient manageable secure

works,so it needs to be

… like an enterprise messaging system!

scaleable portable pluggable interoperable

evolves,so it needs to be

…that transfers, routes, transforms, processes and stores a variety of data streams, and

October 18, 2007 3

AMQP

“business dialtone”

Everyone uses email without thinking, so then why is universal commercial business messaging so hard?

AMQP aims to become THE standard for enterprise messaging

→ need an Open Standard Protocol for Message Oriented Middleware

Made to satisfy real needs:• created by users and technologists working together • in development for 3+ years, went public on June 20th 2006

October 18, 2007 4

AMQP’s place in the “stack”

Network(TCP/IP, UDP, SCTP)

Content based routing(1-Many, 1-1)

Guaranteed Delivery

Store & forward

Transaction participant

“Rich Object Model”

“Document Model”

ESB / SOA / EDA

CDL-BPM-CEP

User Applications

ticks

orders

send/ack

trades

positions

reports

settlement

treasury

bank

FIX, FAST

FpMLWS-*

dataCisco

Utility Servicese.g. exchangeaccess links

MQ

AMQP

Product StandardUseLayer

AMQPTibco

JMS

WCF

October 18, 2007 5

Comparison with some other protocols SMTP – unreliable, slow

HTTP – synchronous, unreliable, no routing XMPP – no delivery fidelity or queue management FTP – point to point, transient, does not work well with NAT/SSL MQ – exactly once TCP – at least once, reliable but short lived, no app level state mgmt UDP – fast but has no delivery guarantees

AMQP - can do all of the above as ‘use cases’

… and switch between them

October 18, 2007 6

How does it work?

Exchange

Queue

Queue

QueueBindings

File Transfer

Messaging

Transactions

any language (C, C++, Java, C#, Python, Javascript, Erlang, Lisp, Ruby, Tcl, PHP, …)

any model (native, JMS, Mule, WCF, …) any payload (binary, XML, SOAP, JSON, …) any transport (TCP, SCTP, HTTP, …) any platform (desktop, router, mobile, EC2, …)

reliable interoperable manageable performant scaleable

October 18, 2007 7

A typical AMQP client program# Addresses where a particular broker is reachable.endpoints = [amqp.Endpoint('hostname', 5672), amqp.Endpoint('alternate', 5673)]# Construct a session.session = amqp.Session("1234", 600, endpoints)# Set up some resources on the broker.session.ExchangeDeclare("xname",amqp.ExchangeTypes.DIRECT)session.QueueDeclare("qname") session.QueueBind("xname","qname","routingkey")# Publish a message.session.MessageTransfer("xname","routingkey","body")# Set up a consumer.session.MessageConsume("qname",MyConsumer(session))class MyConsumer: def __init__(self,session): self.session = session def handle_MessageTransfer(self,destination,exchange, routingkey,body): print "Received message: " + body return amqp.Constants.ACCEPT

October 18, 2007 8

Under the covers – AMQP Layering

Transport

Framing

Session

Model

Execution

Application

API calls

commands

bytes / packets

framesets

frames

command segmentation / assembly, ordering and acknowledgement

queues, exchanges, messages, transactions, …

frameset sequencingframeset (dis)assembly

frame flow controlreliability (exactly once) and failover

(de)multiplexing by channel/streamheartbeating

framing, integrity check

October 18, 2007 9

Communication between peers

Transport

Framing

Session

Model

Execution

Application

Framing

Session

Model

Execution

Applicationmessages

commands

controls

control frames

control frames

October 18, 2007 10

Under the covers: session reattachmentInitiating Peer

ChannelResponding Peer

Channel

attach(1234,ABCD)

ack(...30)

HWM=30seen=...42peer=...30

attach(1234,ABCD)

ack(...42)

HWM=44seen=...30peer=...42

replay of 43

replay of 44

HWM=30seen=...44peer=...30

ack(...44)

HWM=44seen=...30peer=...44

October 18, 2007 11

AMQP - Inclusive GovernanceJPMorgan

D/Borse

Credit Suisse

Goldman Sachs

iMatix OpenAMQ

Protocol Products

Red Hat Enterprise Messaging

Cisco Network

AMQP Working Group controls the

standard Diverse products implement the standard

29West

Envoy

Community Feedback

TWIST

Novell

RabbitMQIonaCeltix

AM

Apache

October 18, 2007 12

So you want to build a computing infrastructure for ocean observation …

distributed reliable & resilient manageable secure

works,so it needs to be

… like a telecommunications system!

scaleable portable pluggable interoperable

evolves,so it needs to be

…that transfers, routes, transforms, processes and stores a variety of data streams, and

October 18, 2007 13

Erlang History

1984:EricssonComputerScience Labformed

1987:Early ErlangPrototype projects 1991:

First fastimplementation

1993:DistributedErlang

1996:Open Telecom Platform

No language was well suited for telecom

systems development

1993:The FirstBook

1998:Open SourceErlang

1994:First Product

2007: A NewBook!

October 18, 2007 14

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

High abstraction levelConcise readable programs

No mutable state

October 18, 2007 15

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

Light-weight processesHighly scalable

Message Passing

October 18, 2007 16

Erlang Highlights: Concurrency

Processes communicate by asynchronous message passing

Pid ! {data,12,13} receive {start} -> ……… {stop} -> ……… {data,X,Y} -> ………end

October 18, 2007 17

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

Response times in thelow milliseconds

Per-process garbage collection

October 18, 2007 18

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

Simple and consistenterror recovery

Supervision hierarchies"Program for the correct case"

October 18, 2007 19

Erlang Highlights: Robustness

Cooperating processes may be linked together

October 18, 2007 20

Erlang Highlights: RobustnessWhen a process terminates, an exit signal is sent to all linked processes

… and the termination is propagated

October 18, 2007 21

Erlang Highlights: RobustnessExit signals can be trapped and received as messages

receive {‘EXIT’,Pid,...} -> ...end

October 18, 2007 22

Erlang Highlights: RobustnessRobust systems can be built by layering

“Supervisors”

“Workers”

October 18, 2007 23

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

Explicit or transparent distribution

Network-awareruntime system

October 18, 2007 24

Erlang Highlights: Distribution

Erlang Run-Time SystemErlang Run-Time System Erlang Run-Time SystemErlang Run-Time System

B ! Msg

network

C ! Msg

October 18, 2007 25

Erlang Highlights: Distribution

{rex, Node} ! {self(), {apply, M, F, A}},receive {rex, Node, What} -> Whatend

loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop() end.

Simple Remote Procedure Call

October 18, 2007 26

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

Easily change code in a running system

Enables non-stop operationSimplifies testing

October 18, 2007 27

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

"Ports" to the outside worldbehave as Erlang processes

October 18, 2007 28

Erlang Highlights: External Interfaces

PortPort Externalprocess

Externalprocess

Port ! {self(), {command, [1,2,3]}},receive {Port, {data, Info}} -> ...end

October 18, 2007 29

Erlang runs on a Virtual Machine ported to UNIX,

Windows, VxWorks, OS X, …Supports heterogeneous

networks.

Functional Concurrent Soft real-time Robust Distributed Hot code loading External interfaces Portable

Erlang Highlights

October 18, 2007 30

Open Telecom Platform

O PT

Applications,Libraries & Tools

System DesignPrinciples

October 18, 2007 31

October 18, 2007 32

Credits Alexis Richardson, CohesiveFT –

for providing much of the AMQP material Francesco Cesarini, Erlang Consulting –

for providing much of the Erlang material Tony Garnock-Jones, LShift –

for the sample code Matthew Arrott – for inviting me

Thank you!

October 18, 2007 33

Backup slides

October 18, 2007 34

AMQP features In some ways like email but:

What goes in must come out Very fast - think big - global scale communication

In some ways like TCP and HTTP, but delivers true MESSAGING Routing and addressing -

“to:phil@cohesiveft.com”, “buy.ibm.100” Guaranteed Delivery Delegation - the concept of a middleman →

security, reliability, translation, …

October 18, 2007 35

AMQP version history and roadmapQ2’06 0-8 routing

Q4’06 0-9 clarifications & bug fixes, experimental work-in-progress extensions

Q4’07 0-10 guaranteed delivery, transport independence, general tidying up

Q1’08 0-11 management, security, federation, addressing

Q2’08 1-0 release

October 18, 2007 36

Under the covers: session creation

Initiating PeerChannel

Responding PeerChannel

1:open(1234,600)

HWM=-1seen=...-1peer=...-1

2:open-ok(1234,ABCD,600);

HWM=-1seen=...-1peer=...-1

October 18, 2007 37

Under the covers: configurationClient model Server model

Exchange.Declare(xname,direct)

exchangecreated

complete

execution-layercompletionnotification

Queue.Declare(qname)

queuecreated

complete

Queue.Bind(xname,qname,routingkey)

bindingcreated

complete

Adobe Acrobat 7.0 Document

October 18, 2007 38

Under the covers: message transferAlice Server Bob

Message.Consume(qname,tag)

complete

Message.Transfer(xname,routingkey,body)

complete

messagerouted

to queueand then toconsumer

Message.Transfer(tag,xname,routingkey,body)

complete

Adobe Acrobat 7.0 Document

October 18, 2007 39

Mule binding<mule-configuration id="RabbitMQ_Demo" version="1.0"> <model name="rabbitmqDemo"> <mule-descriptor name="Mulebot“ implementation="com.rabbitmq.examples.muledemo.Mulebot"> <inbound-router> <endpoint address="amqp://localhost/?vhost=/&amp;realm=/data&amp; exchange=chat&amp;exchange-type=fanout&amp;routing-key="/> </inbound-router> <outbound-router matchAll="true"> <router className="org.mule.routing.outbound.OutboundPassThroughRouter"> <endpoint address="amqp://localhost/?vhost=/&amp;realm=/data&amp; exchange=chat&amp;exchange-type=fanout&amp;routing-key="/> </router> </outbound-router> </mule-descriptor> </model></mule-configuration>

October 18, 2007 40

WCF bindingservice = new ServiceHost(typeof(Calculator), new Uri("soap.amq://dev.rabbitmq.com:5672/"));service.AddServiceEndpoint(typeof(ICalculator), new RabbitMQDualBinding(), "Calculator");service.Open();

fac = new ChannelFactory<ICalculator>( new RabbitMQDualBinding() { ClientBaseAddress = new Uri("soap.amq://dev.rabbitmq.com:5672/")}, "soap.amq://dev.rabbitmq.com:5672/Calculator");fac.Open();calc = fac.CreateChannel();

Console.WriteLine("{0} + {1} = {2}", x, y, calc.Add(3, 4));

((IChannel)calc).Close();service.Close();

October 18, 2007 41

Erlang Highlights: High-level Constructs

-define(IP_VERSION, 4).-define(IP_MIN_HDR_LEN, 5).

……DgramSize = size(Dgram), <<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16, ID:16, Flgs:3, FragOff:13, TTL:8, Proto:8, HdrChkSum:16, SrcIP:32, DestIP:32, Body/binary>> = Dgram, if (HLen >= 5) and (4*HLen =< DgramSize) -> OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN), <<Opts:OptsLen/binary, Data/binary>> = Body, ….. end.

Parsing an IP Datagram using the Bit Syntax