ZeroMQ: Super SocketsBy James Dennis (@j2labs)

http://j2labs.net

What is ZeroMQ?

• Message Passing– You still design the payload

– Fault tolerant

– A message IS your payload

• even when broken into multiple packets

– Many messaging “patterns” offered

What is ZeroMQ?• Super Sockets– Multiple networking protocols (tcp, ipc, whatevz!)– Language agnostic (bindings for: ruby, python,

java, c, lua, lisp, scheme and more…)– Feels like a scripting language for sockets

• Socket behaviors– Round-robin routing– Publishing to subscribers– Sending job requests

What ZeroMQ is not…

• Persistent– reasonably easy to add

• A message queue– there are literally zero mq’s here

• An out-of-the-box solution– instead, it’s easy to build any solution– remember, they’re like scripting sockets

Socket Types (transports)

• tcp :: regular TCP sockets– socket.connect(“tcp://…”)

• ipc :: interprocess communication (unix sockets)– socket.connect(“ipc://…”)

• in-proc :: in process communication– socket.connect(“in-proc://…”)

• pgm – pragmatic multicast (or epgm via udp)– socket.connect(“pgm://…”)

Socket Behaviors

• Actually called patterns

• They exist to make typical cases easy– Request-Reply– Publish-Subscribe– Pipeline– Exclusive pair• you probably don’t want this

Patterns: PUSH/PULL

• Sends messages to multiple connected hosts

• Passes messages ROUND ROBIN to connections

• Blocks PUSH socket if no connections are present

• N hosts PULL messages from pusher

• Connect/Remove new hosts as necessary

Patterns: PUSH/PULL

Server: Sends a message

import zmqctx = zmq.Context()s = ctx.socket(zmq.PUSH)s.bind("ipc://*:5678")while True: s.send(”bla bla bla") # blocks if no

receivers print “Sent one”

Patterns: PUSH/PULL

Client: Prints messages from server socket

import zmqctx = zmq.Context()s = ctx.socket(zmq.PULL)s.connect("ipc://*:5678")while True: msg = s.recv() print 'Got msg:', msg

Patterns: PUSH/PULL

A typical worker pipeline looks a bit like this.

What’s that PUB/SUB stuff?

Patterns: PUB/SUB

• PUB socket sends topical payloads– Can start streaming messages right away– ZMQ discards messages if no subscribers exist

• SUB socket subscribes to “topic”– receives all messages– reads topic and discards accordingly– happens under the hood.

Patterns: PUB/ SUB

Server: sends messages to `whatevz`

s = ctx.socket(zmq.PUB)s.bind("tcp://127.0.0.1:5566")topic = ’whatevz’message = json.dumps({'msg': 'what up dude?'})while True: s.send(topic, message) # send to topic print ‘Sent one’

Patterns: PUB/SUB

Client: demonstrates subscribing to `whatevz`

s = ctx.socket(zmq.SUB)s.connect("tcp://127.0.0.1:5566")topic = "whatevz"s.setsockopt(zmq.SUBSCRIBE, topic) # set socket

optionwhile True: msg = s.recv() print 'Got msg:’, json.loads(msg)

Patterns: REQ/REP

• One to one messaging

• Each REQ message must have a REP response

• Synchronous communication

• Will error if two REQ’s are sent before one REP

Patterns REQ/REP

Client• context.socket(zmq.REQ) # btw… REQ sockets are

blocking!

Worker (could say “server”)• context.socket(zmq.REP)

Multiple Connections• One-to-one messaging is useful for workers getting work

– Basically, pull work request & reply when done• Client may need to multiple workers

– Can’t just open more sockets… we need a new pattern…

Patterns: REQ/REP

CLIENT sends a “Hello”

SERVER sends a “world” back

The REQ socket can send a message, but must not send anything else until it gets a response or ZMQ will error.

Likewise, a REP socket cannot send a message unless it first received one.

Synchronized PUB/SUB

Handling “no broker” (aka building one)• More than one socket used• Like having OneMQ now

Hypothetical Communication Order:1. Subscriber sends sub REQ for topic2. Publisher REP - “listen <tcp://here>”3. Subscriber listens to PUB socket

Patterns: XREQ/XREP

• XREQ is a historical name for DEALER– A DEALER asynchronously sends to *REP sockets– It can send to XREP (aka: ROUTER) sockets too

• XREP is a historical name for ROUTER– A ROUTER asynchronously handles *REQ sockets– It can handle DEALER (aka: XREQ) sockets too.

Patterns: DEALER/ROUTER• CLIENT sends message to ROUTER and waits for a response

• ROUTER processes message and sends to some worker via the DEALER

• DEALER sends message to worker and waits for one response

• WORKER processes message, sends response back

• DEALER get message from WORKER, sends along to ROUTER

• ROUTER gets message and sends back to CLIENT

• CLIENT does something with it!

Complicated Network Layouts

Broker Model: aka “The Titanic”

Broker Model with ZMQ

• Every connection is a REQ socket• ROUTER socket to all connections• LRU Queue (your broker code!)

• Up to the LRU Queue handle socket identities properly

• REQ/REP gets that for free.

Flexibility Via Decentralization“ØMQ is like Git (essentially distributed) whereas AMQP is like SVN

(essentially centralized)” – James Dennis (me)

Welcome From AMQP: http://www.zeromq.org/docs:welcome-from-amqp

Mongrel2 (Zed Shaw’s second web server)

http://mongrel2.org

Brubeck• Mongrel2 handler in Python• Modeled after Tornado with ideas from Flask / Bottle• Uses Eventlet

– Coroutines!– Nonblocking I/O!

• Uses ZeroMQ– Write back end services in any language!

• Uses DictShield– I wrote this too! – Database agnostic modeling!

• ZeroMQ is language agnostic, right?!

https://github.com/j2labs/brubeck

Try it!https://github.com/j2labs/zmq_examples

http://mongrel2.orghttps://github.com/j2labs/brubeck

Read more!http://nichol.as/zeromq-an-introduction

http://zguide.zeromq.org/

Questions

?

James Dennis (@j2labs)jdennis@gmail.com