Building a multi protocol broker for the internet of things using nodejs

transcript

@matteocollinamatteocollina.com

Node.js Conference - Brescia 2012/11/09

Building a multi-protocol broker for the Internet of Things using node.js

How many people will own a smartphone by 2014

1.000.000.000

Did you see it coming

http://www.flickr.com/photos/12738000@N00/360231193/

...They didn’t!

What's next(hint: it's a big number)

50.000.000.000

1.000.000.000

50.000.000.000

"Things"

Let's experiment

I want to chat with my house

>: hi househi matteo

>: what's the temperature?20

>: 4 8 15 16 23 42

If you don’t understand this, you are not a geek!

Enter Hubot

A programmable robotthat is controlled through chat

We can supercharge our house with hubot

module.exports = (robot) ->

robot.respond /what’s the temperature?/i, (msg) -> msg.http("http://mchouse.it/temperature") .header("Accept", "application/json") .get() (err, res, body) -> msg.send JSON.parse(body)

We can supercharge our house with hubot

In order to ask our temp to hubot, we need to:

1. sense the temp2. get the temp on the web3. build a web API

We are building an API

In Italy,“API” means

“bees”http://www.flickr.com/photos/theseanster93/4056815767

What do we need to sense my house temperature

We add a sensor to an Arduino

We add an ethernet shield to connect to the Internet

http://www.flickr.com/photos/snootlab/6052465980/

What protocol do we use to push our temperature to our API?

http://www.flickr.com/photos/clearlyambiguous/48185613/

HTTP is slow and safe

We need a fast, binary protocol

http://www.flickr.com/photos/grrphoto/305649629

M2M protocols are mostly ad-hoc, and researchers and businesses focus on low level problems.

“Things” should interact with our lives, and all the

technology should be built to make them easy

to use.

• “things” exposed with binary protocol

• publish/subscribe

• topics as the naming system

• “things” exposed to the web

• request/response

• URIs as the naming system

QESTDiscover

qest.me

• MQTT broker

• REST interface

• HTTP semantics

• no QoS

• built on node.js and redis

QESTREST Server

MQTT Server

Data Layer

HTTP Clients MQTT Clients

Web App

DeviceSC

TXRX ON

9 8DIGITAL

7 6 5 4 3 2 1 0

5V GndPOWER

www.arduino.cc

ANALOG INVin 0 1 2 3 4 5

Arduino UNO

IoTBroker

state-of-artapproach to IoT apps

QEST-basedsolution to IoT apps

Bridge

Web App

Device

Web App

Device

IoTBroker

state-of-artapproach to IoT apps

QEST-basedsolution to IoT apps

Bridge

Web App

DeviceSC

TXRX ON

9 8DIGITAL

7 6 5 4 3 2 1 0

5V GndPOWER

www.arduino.cc

ANALOG INVin 0 1 2 3 4 5

Arduino UNO

QEST• retains every message received

: MQTT to REST

curl -H "Accept: txt" http://qest.me/topics/temp

client = PubSubClient(server, 1883, callback);client.publish("temp", "30");

• every topic has its own URI: /topics/<NAME>

QEST• transform every HTTP PUT received to a MQTT message

: REST to MQTT

void callback(char* topic, byte* payload, int length) { ...}PubSubClient(server, 1883, callback);client.subscribe("temp");

curl -X PUT -d '{ "payload": 42 }' \ -H "Content-Type: application/json" \ http://qest.me/topics/temp

• devices can listen directly to MQTT topics

How to scale

HTTP/MQTT Clients

REST Server MQTT Server

QESTData Layer

Load Balancer

REST Server MQTT Server

QESTData Layer

How much time took me to write the first version of QEST?

A day.

Thanks

How much time took me to write the first version of QEST?

• Express

• Socket.io

• node_redis

• MQTT.js

• Coffeescript

A day.

Thanks

What happens if you write a broker in a day?

(Hint)

Spaghetti code!

How to untangle it?

Refactoring!!!

http://www.flickr.com/photos/adewale_oshineye/2933030620/

1.A single file can go really far!

Well, the first improvement were two files

Coffee-scriptis awesome for prototyping!

Coffee-scriptis a real pain to debug

2.Extracting a data layer!

REST Server

MQTT Server

Data Layer

HTTP Clients MQTT Clients

Discover my roots

Ruby Language

Ruby on Rails

The ActiveRecord

pattern is AWESOME!

ActiveRecord

1. Best pattern for storing data2. Its main responsibilities are

CRUD operations3. I used that pattern to build a

cross-process pub/sub system

3.Extracting an inter process pub/sub library

AscoltatoriDiscover

github.com/mcollina/ascoltatori

ltator

i• Redis

• ZeroMQ

• RabbitMQ

• MQTT (Mosquitto)

• Memory (EventEmitter)

is a multi-process pub/sub library backed by

ltator

ivar ascoltatori = require('ascoltatori');

// you can use RedisAscoltatore, ZeromqAscoltatore,// RabbitAscoltatore, MQTTAscoltatorevar ascoltatore = new ascoltatori.MemoryAscoltatore();

ascoltatore.subscribe("hello/*", function() { // this will print { '0': "hello/42", '1': "a message" } console.log(arguments); process.exit(0);});

ascoltatore.publish("hello/42", "a message", function() { console.log("message published");});

ltator

100 us

1.000 us

10.000 us

100.000 us

1 10 100 1000Listeners

Memory Redis ZeroMQ RabbitMQ Mosquitto

ltator

iallowed me to refactor QEST as just an MQTT wrapper!

I am se

ibutors

github.com/mcollina/qest

github.com/mcollina/ascoltatori

Source NASA

Matteo Collina

Software Engineer @ Mavigex

Ph.D. Student @ University of Bologna

matteocollina.com

Matteo Collina (matteo.collina2@unibo.it)

Thank You!

@matteocollina

http://www.flickr.com/photos/axel-d/479627824/

Building a multi protocol broker for the internet of things using nodejs

Technology