ZeroMQ with NodeJS

transcript

ØMQ @ NodeJSØMQ: Small Pieces for Scalable Software

Fernando D. Alonso

1. Facts about ØMQ [ ]

Facts about ØMQ

Low level messaging library

○ It is written in C++.○ A thin layer between application and transport

layers.○ Message-passing using in-memory queues.○ Concise API which talks over the ZMTP protocol.○ Works on top of TCP, IPC, in-memory, and

PGM/EPGM communication protocols.

○ Philosophy: horizontal scalability to reduce points of failure.

○ Network topology aware. Point to point communication reduce latencies.

○ Communication is reduced to a few patterns, which combined are powerful.

○ No need to have a multitenant service.

Facts about ØMQ

A broker is not mandatory

Facts about ØMQ

Messaged oriented communication

○ Messages as first-class citizen. No need to fight with framing or buffering.

○ You can send multipart messages.○ Messages are atomic.○ Asynchronous messaging. A single background I/O

thread does the job for ØMQ sockets."A ØMQ message is a discrete unit of data passed between applications or components of the same application. ØMQ messages have no internal structure and from the point of view of ØMQ itself they are considered to be opaque binary data."

Facts about ØMQ

Fast for development

○ Interchangeable transports: e.g, scaling from one IPC server to a bunch of TCP.

○ Automatic reconnections.○ Multicore made easy.○ Provides zmq_poll to read BSD sockets.

○ It doesn't provide message persistence.○ It doesn't provide data serialization.○ It doesn't provide data compression.○ It doesn't provide message encryption.○ It doesn't provide security mechanisms. Not in the

next version of the ZMTP protocol !

(*) ZTMP 3.0 draft protocol: http://hintjens.com/blog:39

Facts about ØMQ

Aims for simplicity

Facts about ØMQ

Platforms & Languages

○ Bindings for 30+ languages including NodeJS, Ruby, Python, Java, C, C++, Scala, Erlang, Perl, PHP, .NET.

○ Linux & Windows.○ Raspberry PI.○ Android OS.

2. Install [ , ]

Install

On Linux from source

sudo apt-get install build-essential libtool autoconf automake uuid-devwget http://download.zeromq.org/zeromq-3.2.3.tar.gztar xvzf zeromq-3.2.3.tar.gz./configuremakesudo make installsudo ldconfig

NOTES:(*) Check http://download.zeromq.org/ for more versions.(**) Avoid 3.0.x, 3.1.x, 3.2.0, 3.2.1 versions. They lack backwards

compatibility.(***) Check ./configure --help for more options on build.(****) ./configure --with-pgm

Install

On Linux using packages

Libzmq 3.2:

● Debian Sid: sudo apt-get install libzmq3-dev

Libzmq 2.2:

● Debian Wheezy, Jessie, Sid: sudo apt-get install libzmq-dev

● Debian Squeeze backport: http://packages.debian.org/squeeze-backports/libzmq1

Install

On Mac

brew install zmq

Install

NodeJS bindings

NPM Packages:

● zmq (https://github.com/JustinTulloss/zeromq.node) [RECOMMENDED]● zmq-stream (https://github.com/Schoonology/zmq-stream)

npm install zmq

> var zmq = require('zmq');

Install

Bindings in other languages

Ruby: https://github.com/chuckremes/ffi-rzmq [RECOMMENDED]

Python: https://github.com/zeromq/pyzmq

Erlang: https://github.com/zeromq/erlzmq2

Java JZMQ (uses JNI): https://github.com/zeromq/jzmq

Java JeroMQ: https://github.com/zeromq/jeromq

Scala ZeroMQ: https://github.com/mDialog/scala-zeromq

Scala ZeroMQ (official): https://github.com/valotrading/zeromq-scala-binding

3. Sockets

Sockets

Creation

(*) http://api.zeromq.org/3-2:zmq-socket

ØMQ C API

NodeJS ZMQreq ZMQ_REQ

rep ZMQ_REP

dealer ZMQ_DEALER

router ZMQ_ROUTER

push ZMQ_PUSH

pull ZMQ_PULL

ØMQ C APINodeJS ZMQ

pub ZMQ_PUB

sub ZMQ_SUB

xsub ZMQ_XSUB

xpub ZMQ_XPUB

var sock = zmq.socket("<SOCKET_TYPE>");

zmq_socket(context, <SOCKET_TYPE>);

Request

Dealer (async Request)

Router (async Reply)

Publisher

Subscriber

XSubscriber

XPublisher

Sockets

Bind & Connect

sock.bind("tcp://10.0.0.1:5555", function(err) { .. });

sock.connect("tcp://10.0.0.1:5555");

sock.close();

(*) http://api.zeromq.org/3-2:zmq-bind(**) http://api.zeromq.org/3-2:zmq-connect(***) http://api.zeromq.org/3-2:zmq-close

ØMQ C API

zmq_bind(socket, "tcp://10.0.0.1:5555");

zmq_connect(socket, "tcp://10.0.0.1:5555");

zmq_close(socket);

NodeJS ZMQ

Connect

Sockets

Bind & Connect

sock.bindSync("tcp://10.0.0.1:5555");// This line gets executed once the socket// is binded, but it breaks on error.

NodeJS ZMQ

BindSync

Sockets

Transports

TCP● usage: tcp://<address>:<port>

sock.bind("tcp://192.168.1.100:5555");

sock.bind("tcp://eth0:5555");// binds to the first eth0 interface

sock.bind("tcp://*:5555");// binds to all interfaces

(*) http://api.zeromq.org/3-2:zmq-tcp

ØMQ C API

zmq_bind(socket, "tcp://192.168.1.100:5555");

zmq_bind(socket, "tcp://eth0:5555");

zmq_bind(socket, "tcp://*:5555");

NodeJS ZMQ

Sockets

Transports

Inter-Process Communication (IPC)● usage: ipc://<address>

● Requires RW permissions on the specified path.

(*) http://api.zeromq.org/3-2:zmq-ipc

sock.bind("ipc:///tmp/mysocket");

ØMQ C API

zmq_bind(socket, "ipc:///tmp/mysocket");

NodeJS ZMQ

Sockets

Transports

In-process Communication (in-memory)● usage: inproc://<address>

● Requires to bind before connect.

● Max. address name length is 256.

(*) http://api.zeromq.org/3-2:zmq-inproc

(**) Buggy on node-zmq package.

sock.bind("inproc://queue");

ØMQ C API

zmq_bind(socket, "ipc://queue");

NodeJS ZMQ

Sockets

Transports

PGM (multicast)● usage: pgm://<address>;<multicast_address>:<port>

● Requires sudo privileges to access raw IP sockets.

● Needs ØMQ built with PGM extension (./configure --with-pgm).

(*) http://api.zeromq.org/3-2:zmq-pgm

sock.bind( "pgm://192.168.1.100;239.192.1.1:3055");

ØMQ C API

zmq_bind(socket, "pgm://192.168.1.100;239.192.1.1:3055");

NodeJS ZMQ

Sockets

Transports

Encapsulated PGM (multicast)● usage: epgm://<address>;<multicast_address>:<port>

● Needs ØMQ built with PGM extension (./configure --with-pgm).

(*) http://api.zeromq.org/3-2:zmq-pgm

sock.bind( "epgm://192.168.1.100;239.192.1.1:3055");

ØMQ C API

zmq_bind(socket, "epgm://192.168.1.100;239.192.1.1:3055");

NodeJS ZMQ

Sockets

Send & Receive

sock.send("My message");

sock.on("message", function(msg) {

console.log(msg.toString());

(*) http://api.zeromq.org/3-2:zmq-send

(**) http://api.zeromq.org/3-2:zmq-recv

Receive

sock.on("message", function(msg) {

console.log("Received " + msg);

// <msg> is a Buffer object.

Sockets

Multipart messages

ØMQ Queue

zmq_send(socket, "Como", 4, ZMQ_SNDMORE);

Messages are atomic

A ØMQ message is composed of 1 or more message parts.

zmq_send(socket, "andas", 5, 0);

4 ComoPart 1

5 andasPart 2

ØMQ ensures that peers receive either all message parts of a message or none at all.

The total number of message parts is unlimited except by available memory.

Sockets

Multipart messages

sock.send("First part", zmq.ZMQ_SNDMORE);sock.send("Second part");

sock.send(["First part", "Second part"]);

var r = sock.send("Como", zmq.

ZMQ_SNDMORE);

console.log(r);

Check the outgoing buffer and socket state:

running this will output ->

> { type: 'req', _zmq: { state: 0, onReady: [Function] }, _outgoing: [ [ <Buffer 43 6f 6d 6f>, 2 ] ], _shouldFlush: true, _events: { message: [Function] } }

Sockets

Multipart messages

sock.on("message", function(first_part, second_part) {

console.log(first_part.toString());

console.log(second_part.toString());

sock.on("message", function() {

for(var key in arguments) {

console.log("Part " + key + ": " + arguments[key]);

Receive

sock.on("message", function() {

var messages = Array.prototype.slice.call(arguments);

messages.forEach(function(msg) {

console.log("Part:" + msg);

Sockets

Batched messaging

ØMQ Queue

"ØMQ batches messages in opportunistic manner. It sends all the messages available at the moment in one go. Latency of subsequent messages will be improved because sending single batch to the card is faster then sending lot of small messages."

sock.send("Part A",

zmq.ZMQ_SNDMORE);

sock.send("Msg");

sock.send("Part

6 Part A

6 Part B

ØMQ Socket

Client Thread ØMQ I/O Thread

send batch

Patterns

Plugging multiple transports

Example

var zmq = require('zmq'), pub = zmq.socket('pub');

pub.bindSync('tcp://127.0.0.1:5555');pub.bindSync('ipc:///tmp/zmq.sock');

setInterval(function() { pub.send("I am polyglot!");}, 1000);

sub = zmq.socket('sub');sub.connect('ipc:///tmp/zmq.sock');sub.subscribe('');sub.on('message', function(msg) { console.log("Received: " + msg);});

var zmq = require('zmq'), sub = zmq.socket(sub');

sub.connect('tcp://127.0.0.1:5555');sub.subscribe('');

sub.on('message', function(msg) { console.log(Received: ' + msg);});

Sockets

Setting Socket Options

(*) http://api.zeromq.org/3-2:zmq-setsockopt

Setsockopt● usage (NodeJS ZMQ API): sock.setsockopt(<option>, <value>);

or sock.<option> = <value>;

sock.identity = 'monitor-1';sock.setsockopt('identity', 'monitor-1');

Identity:

// value can be any string up to 255 length.// identity is required on sockets// connecting to a Router.

Sockets

Setting Socket Options

Setsockopt

subscriber.subscribe('MUSIC');subscriber.setsockopt('subscribe', 'MUSIC');

Subscribe:

// sets a message filter for subscriber sockets.

subscriber.unsubscribe('MUSIC');susbscriber.setsockopt('subscribe', 'POP');

Unsubscribe:

// sets off a message filter for subscriber sockets.

4. Patterns [ ]

Patterns

Request / Reply

Synchronous task distribution

Ø REP #1

Ø REP # 2

Ø REQ

< round robin >

< synchronous >

Used when each message needs to be matched with a response. Handles only one message at time. Strict send-recv cycle.

Patterns

Request / Reply

Ø REP #1 Ø REP # 2Ø REQ

recv (wait response)

recv (dont wait response)

Patterns

Request / Reply

Ø REP #1 Ø REP # 2Ø REQ

-> REP #1 is downFAILURE

REQ #1 will not recover !

Patterns

Request / Reply

Basic Request / Reply

var zmq = require('zmq');

reply = zmq.socket('rep');

reply.bind('tcp://127.0.0.1:5555', function(err) { if (err) throw err; });

reply.on('message', function(msg) { console.log('Received: ' + msg); reply.send('Pong!');});

var zmq = require('zmq');

request = zmq.socket('req');

request.connect('tcp://127.0.0.1:5555');

request.send('Ping');

request.on('message', function(msg) { console.log('Response: ' + msg);});

Patterns

Request / Reply

Dealer socket message deliveryMessages have to be multipart, consisting on: an empty delimiter header, followed by the content body parts.

sock.send("Body");

Outgoing Queue

6 Body

sock.send(new Buffer([]), zmq.

ZMQ_SNDMORE);

Patterns

Request / Reply

Dealer socket message deliveryOutgoing messages are round-robined among all connected peers. However, sending depends on the availability of the receiver.

sock.send(

["", "Msg"]);

Outgoing Queue

6 Part A

Peer Socket A Peer Socket B

sock.send(

["", "Bla");

sock.send(

["", "Msg2"]);

4 Msg2

Patterns

Request / Reply

Dealer socket message deliveryReconnections are handled automatically: messages are asynchronously delivered. Response time does not block sending more messages.

Outgoing Queue Peer Socket A Peer Socket B

-> Up againsock.send(

["", "Msg"]);

sock.send(

["", "Msg2"]);

4 Msg2

Patterns

Request / Reply

Ø REP #1 Ø REP # 2Ø DEALER

enqueue < Msg A >

send < Msg B >

-> REP #1 is down

enqueue < Msg C >

send < Msg A, Msg C >

send < Msg D >

Dealer socket handles peer reconnection automatically. It will send messages that queued for that peer once it has established connection.

Patterns

Request / Reply

> Response from: 5001> Response from: 5001> Response from: 5002> Response from: 5001> Response from: 5001> Response from: 5002> ...

Dealer example

var dealer = zmq.socket('dealer');dealer.connect('tcp://127.0.0.1:5001');dealer.connect('tcp://127.0.0.1:5002');

setInterval(function() { dealer.send(["", "Msg"]);}, 1000);

dealer.on('message', function(h, msg) { console.log('Response from: ' + msg);});

running this 'might' output ->

Patterns

Request / Reply

Router socket messagingMessages have to be multipart, consisting on: an identity frame, an empty delimiter frame, followed by the content body parts. This applies to both incoming and outgoing messages.

sock.send("Body");

Outgoing Queue

6 Body

sock.send("", zmq.

ZMQ_SNDMORE);

sock.send(

"worker-1", zmq.

ZMQ_SNDMORE); 8 worker-1

Patterns

Request / Reply

Router is asynchronousIncoming messages are fair-queued among all connected and available peers.

Peer Socket A Peer Socket B ØMQ Router

Incoming Queue

Socketsend

Patterns

Request / Reply

Router is asynchronousReceiving depends on the availability of the sender.

Peer Socket A Peer Socket B ØMQ Router

Incoming Queue

Socketsend

Patterns

Push / Pull

Unidirectional data distributionThe Push / Pull pattern fits well on pipelined communication, i.e., when no response is required for a given message.

Patterns

Push / Pull

Binding the Push socketOutgoing messages are round-robined among all connected and available peers.

push.send("Msg");

push.send("Part

push.send("Part A",

zmq.ZMQ_SNDMORE);

push.send("Msg2");

6 Part A

6 Part B

4 Msg2

Patterns

Push / Pull

Binding the Push socketOutgoing messages are round-robined among all connected and available peers.

-> Down3 Msg

6 Part A

6 Part B

4 Msg2

push.send("Msg");

push.send("Part

push.send("Part A",

zmq.ZMQ_SNDMORE);

push.send("Msg2");

Patterns

Push / Pull

Binding the Push socketDisconnections and reconnections are handled automatically.

4 More

6 Part 1

6 Part 2

5 Other send

push.send("More"); send

push.send("Part

push.send("Part 1",

zmq.ZMQ_SNDMORE);send

push.send("Other");send

Patterns

Push / Pull

Parallel task distribution

Ø PULL #1

Ø PULL # 2

Ø PUSH

< round robin >

Patterns

Push / Pull

Passive agents / Monitor

Ø PULLØ PUSH

...Ø PUSH

Patterns

Push / Pull

Binding the Pull socketPush's outgoing messages are round-robined among all connected peers.

push.send("Msg");

push.send("Part

push.send("Part A",

zmq.ZMQ_SNDMORE);

push.send("Msg2");

6 Part A

6 Part B

4 Msg2

Patterns

Push / Pull

Binding the Pull socketPush's outgoing messages are round-robined among all connected peers.

push.send("Msg");

push.send("Part

push.send("Part A",

zmq.ZMQ_SNDMORE);

push.send("Msg2");

sendsend

6 Part A

6 Part B

4 Msg2

Patterns

Push / Pull

Binding the Pull socketDisconnections and reconnections are handled automatically.

-> Down4 More

6 Part 1

6 Part 2

5 Other

push.send("Part

push.send("Part 1",

Patterns

Push / Pull

Binding the Pull socketSending is asynchronous, it depends on the availability of the receiver.

-> Up again

5 OtherTo B

push.send("Part

push.send("Part 1",

Patterns

Push / Pull

Ø PULLØ PUSH #1

...Ø PUSH #2

Active agents / collector pattern

Patterns

Publish / Subscribe

Broadcast data distribution

Ø SUB #1

Ø PUB #1

...Ø SUB #2

Ø SUB #2

Message distribution

Patterns

Publish / Subscribe

sock.send("Msg");

Outgoing Queue

Subscriber A

Subscriber B

sendsend

Outgoing messages are send to each connected and available peers.

Subscriber C

Message distribution

Patterns

Publish / Subscribe

sock.send("Msg");

Outgoing Queue

Subscriber A

Subscriber B

Outgoing messages are send to each connected and available peers.

Subscriber C

5 Othersock.send("Other");send

Subscribers can register to a specific topic.

Data filtering

Patterns

Publish / Subscribe

ØMQ Publisher

Subscriber A

Subscriber B

Subscriber C

Outgoing Queue

subscribe AR

subscribe VE

Socket

subscribe VE

Data filtering

Patterns

Publish / Subscribe

ØMQ Publisher

Subscriber A

Subscriber B

Subscriber C

Outgoing Queue

subscribe AR

subscribe VE

Socket

subscribe VE

sock.send("AR

news");

send7 AR news

Data filtering

Patterns

Publish / Subscribe

ØMQ Publisher

Subscriber A

Subscriber B

Subscriber C

Outgoing Queue

subscribe AR

subscribe VE

Socket

subscribe VE

sock.send("VE

news");

send7 VE news

Patterns

Publish / Subscribe

Data filtering example

pub = zmq.socket('pub');

pub.bindSync("tcp://10.0.0.12:3055");

count = 0setInterval(function() { pub.send("TEST " + count++);}, 1000);

sub = zmq.socket('sub');

sub.connect("tcp://10.0.0.12:3055");sub.subscribe("TEST");

sub.on("message", function(msg) { console.log("Received: " + msg);});

// older messages won't be// received

> TEST 6> TEST 7> TEST 8> TEST 9> TEST 10

running this 'might' output ->

Patterns

Publish / Subscribe

Multicast example

var zmq = require('zmq'), pub = zmq.socket('pub');

pub.bind( "epgm://10.0.0.12;239.192.1.1:3055", function(err) { if(err) throw err; })

setInterval(function() { pub.send("From pid: " + process.pid);}, 1000);

var zmq = require('zmq'), sub = zmq.socket('sub');

sub.connect( "epgm://10.0.0.13;239.192.1.1:3055");

sub.subscribe("");

sub.on('message', function(msg) { console.log("Received " + msg);});

(*) See running demo at http://youtu.be/NQrH0SATPk0

"XSUB and XPUB are exactly like SUB and PUB except they expose subscriptions as special messages."

Data distribution proxy

Patterns

XSUB / XPUB

XSUB/XPUB Proxy

Subscriber A

Subscriber B

Subscriber CXPUB Socket

XSUB Socket

Publisher A

Publisher B

Publisher C

ribe P

forward subscription

Messages are forwarded by the proxy to the registered subscribers.

Data distribution proxy

Patterns

XSUB / XPUB

XSUB/XPUB Proxy

Subscriber A

Subscriber B

Subscriber CXPUB Socket

XSUB Socket

Publisher A

Publisher B

Publisher C

forward message

send "PY <msg>"

5. Useful links [ ]

Useful Links

Workshop examples

http://github.com/krakatoa/node_zmq_workshop

Useful Links

The Guide

http://zguide.zeromq.org/page:all

Useful Links

Pieter Hintjens personal blog

http://hintjens.com/

Useful Links

API References

http://api.zeromq.org/3-2:zmq-socket

http://api.zeromq.org/3-2:zmq-setsockopt

Preguntas ?fedario@gmail.com

github.com/krakatoa

Fernando Dario Alonso

Gracias !

ZeroMQ with NodeJS

Technology