SCALING MQTTFrom Trial to Millions of MQTT Clients

WHAT IS MQTT?

➤ IoT Messaging Protocol

➤ Minimal Overhead

➤ Publish / Subscribe

➤ Easy

➤ Binary

➤ Data Agnostic

➤ Designed for reliable communication over unreliable channels

USE CASES

➤ Push Communication

➤ Unreliable communication channels (e.g. mobile)

➤ Constrained Devices

➤ Low Bandwidth and High Latency environments

➤ Communication from backend to IoT device

PROTOCOL DETAILS

➤ MQTT requires TCP/IP

➤ IPv4 and IPv6 supported

➤ Persistent TCP connections

➤ Heartbeat mechanism

➤ Clean & Persistent Sessions

➤ QoS levels for message delivery guarantees

➤ TLS in the mix for professional deployments

MQTT

TCP

IP

5-7

ISO/OSI

Layer

4

ISO/OSI

Layer

3

ISO/OSI

Layer

PUBLISH / SUBSCRIBEScalable communication paradigm for the IoT

PUBLISH / SUBSCRIBE

SCALING MQTTA solution

THE SCALING

CHALLENGEBillions of devices

CHALLENGE 1:

HUGE AMOUNT OF TCP CONNECTIONS

➤ MQTT uses persistent TCP connections

➤ Number of concurrent TCP connections limited (hard to scale > 1-2 Million on a single machine)

➤ “Reconnect Storms”

CHALLENGE 2:

SECURITY

➤ TLS for secure communication

➤ TLS handshake is resource intensive

➤ Brokers accessible over the Internet

➤ Scalable backend systems for authentication & authorization needed

CHALLENGE 3:

MQTT IS STATEFUL

➤ MQTT allows sessions

➤ Subscriptions, Queued Messages, Message Flow remembered for offline client

➤ Retained Messages for every topic possible (dynamic topics!)

CHALLENGE 4:

HIGH AVAILABILITY

➤ MQTT Brokers are a single point of failure

➤ Cloud Environments WILL fail

➤ Bridged Brokers (brokers that are chained via MQTT) do not support session migration

A SCALABLE MQTT REFERENCE ARCHITECTURE

A solution to the challenges for scaling MQTT

IoT Devices

elastic beam

MQTT with TLS 1.2

elastic beam

MQTT BrokerCluster

OtherEnterprise Applications

APIProxyCluster

Application Servers

Databases Microservices

Datacenter

MQTT over Websocketswith TLS 1.2

SCALING WITH HIVEMQ CLUSTERING

Advantages & Best Practices

HIVEMQ OVERVIEW

➤ Enterprise MQTT Broker

➤ Elastic Clustering

➤ Linearly scalable

➤ Best-in-class Performance

➤ Open Source Plugin System

➤ Scalable horizontally & vertically

➤ > 1.000.000 MQTT connections per broker possible

HIVEMQ CLUSTER ADVANTAGES

➤ Elimination of the Single Point of Failure

➤ Sophisticated message distribution across cluster nodes

➤ Clients can connect to any node to resume their sessions

➤ Linearly scalable

➤ High Availability due to resilience and fault-tolerance

➤ Zero-Downtime Upgrades

WHY IS HIGH AVAILABILITY IMPORTANT FOR MQTT?

Distributed Systems are hard.

FALLACIES OF DISTRIBUTED COMPUTING

➤ The network is reliable.

➤ Latency is zero.

➤ Bandwidth is infinite.

➤ The network is secure.

➤ Topology doesn't change.

➤ There is one administrator.

➤ Transport cost is zero.

➤ The network is homogeneous.

“The Eight Fallacies of Distributed Computing”

Peter Deutsch

THE CAP THEOREM

➤ Applicable to all distributed systems with persistent state

➤ “Pick two”

➤ You can’t trade Partition Tolerance (see Fallacies of Distributed computing)

➤ CP would render the whole broker cluster unusable in case no consensus is achieved

➤ HiveMQ is AP and has self-healing mechanisms for reconciliation => The broker cluster is always available and never drops client connections

Image Credits: Benjamin Erb

http://berb.github.io/diploma-thesis/original/061_challenge.html

HIVEMQ CLUSTER DETAILS

➤ Masterless Architecture

➤ Highly Resilient

➤ Always Available (favors AP in the CAP theorem)

➤ Elastic. You can add and remove nodes at runtime

➤ Network partitions are supported

➤ Data distribution across cluster nodes

➤ Configurable Replica Count

➤ Self-Healing mechanisms when merging network partitions

Broker 2 Broker 3

Broker 1

MQTT Broker Cluster

IoT Devices

elastic beam

MQTT with TLS 1.2

elastic beam

MQTT BrokerCluster

OtherEnterprise Applications

APIProxyCluster

Application Servers

Databases Microservices

Datacenter

MQTT over Websocketswith TLS 1.2