Towards standardization of MQTT-alert-based sensor networks:protocol structures formalization and low-end node security

Georgios Vrettos1 Evangelos Logaras2

1 Dept. of Information and Communication Systems Eng., University of the Aegean, Samos, Greece2Infineon Technologies, Graz, Austria

Email: [gvrettos, kalliger]@aegean.gr, evangelos.logaras@infineon.com

Emmanouil Kalligeros1

Abstract—We present a small scale sensor network applicationas a testbench to explore different setups in terms of hardware/software, network protocol and data processing/storage schemeoptions, focusing on alert-based sensor systems with long idletimes. Our specifications required the deployment of the networkusing the MQTT protocol over an encrypted TLS connection.We focused on using sensor nodes with low-power consumptionprofile, as well as on the formalization of the MQTT protocol’sbasic elements, by clearly defining the topic/message scheme usedacross the network. In addition, we experimented on how tocombine locally stored information with data from popular cloud-based platforms and also acquired results regarding the pro-cessing performance of the nodes using different data exchangeformats and database technologies. Work in progress includesdata preprocessing on the network edge targeting distributionof the processing power across the network and network-trafficlimitation, and also big data post-processing on server side or ondedicated high performance nodes to reveal hidden data patterns.

Keywords—sensor network, MQTT, hw/sw, cloud, security

I. INTRODUCTION

Within the next few years, billions of different types ofconnected computing and sensor devices will be used to mon-itor life in different environments, like home/car applications,industrial/metropolitan areas, remote wildlife areas and in theoceans. With the technological progress that has been achievedduring the last decades in domains like networking and In-ternet of Things (IoT), telecommunications and embeddedelectronics, it is now possible for all these devices to forma huge network acting like a “skin” for our planet [1]. It ishighly likely that even single chip self-powered sensor deviceswith networking capabilities can be found in the most remoteareas, pushing data to the Internet. While adding devices tothe network will be a trivial process, computer engineersalready face questions on: i) what kind of network protocoltechnologies are suitable for the next years and decades tosupport all these new devices, ii) where to store and processthe vast amount of collected data, and iii) how to secure allthese data transmissions, with nearly every computing devicehaving networking capabilities.

In this paper, by using a small scale sensor networkfor environmental and infrastructure monitoring, we try toprovide/improve solutions on the following hw/sw and networkrelated topics: a) the use of the Message Queuing Teleme-try Transport (MQTT) machine-to-machine protocol and theformalization of its data structures (topics/commands format,use of XML/JSON data format). We chose MQTT becauseit has a very small footprint [2] and its implementation isfeasible, even with encrypted data transmissions, on verysmall devices. b) Different hw/sw options for the network

nodes to efficiently handle data in a secure way, c) databasemanagement options to store and share data, d) combininglocal data traffic with data stored/retrieved to/from the cloud,e) data preprocessing on the field to reduce network traffic, andf) data post-processing (online/offline) on server side to extracthidden data patterns, using dedicated hardware co-processors,e.g., Field Programmable Gate Arrays (FPGAs), GraphicsProcessing Units (GPUs), etc. The main contribution of ourwork is twofold: i) we improve the formalization of the MQTTprotocol data structures, since no standardized syntax exists atthe moment [3]. Also, since security is the biggest concernof IT and semiconductor industry [4], as billions of deviceswill be connected to the network in the next few years, ii)we experimented on the encryption mechanism of the MQTTprotocol and how feasible is to apply it, in terms of processingand power consumption, to very small and resource restricteddevices at network edge. To the best of our knowledge, noconcrete results exist in the literature in this area.

A. Network Topology, Motivation and Design GoalsThe topology of our network architecture is presented in

Figure 1. Locally, the highest level of the network is a Linuxserver running the MQTT broker (level #2), and managingthe database, which collects incoming data, and an efficientdashboard to present these data, in a meaningful way, to systemadministrators. The server might also have high performancecomputing capabilities to process data and may also push datato other cloud platforms (level #3), like ThingSpeak, GoogleCloud, etc. The server may collect data directly from low levelMQTT client nodes at the edge of the network (level #0), orintermediate levels with more powerful nodes (level #1) mayfilter/compress data to reduce network traffic. Nodes at level#0 do not establish long-lived connections with the brokerbut rather communicate in an alert-based manner to reportdata periodically or based on interrup/timeout events. Althoughthese nodes make use of very low-end hardware, we want themto be able to: a) connect to the MQTT broker over local/longrange WiFi or 3G/4G networks, b) get into low-power/sleepmode to save power, but also include hardware that cansupport MQTT message encryption using the Transport LayerSecurity (TLS) protocol, c) support software execution usingthreading and/or interrupts to handle MQTT traffic, as well asinterrupt events from connected sensors, d) support, if possible,Operating System (OS) and file based software, and e) supportOver the Air (OtA) re-programming and configuration.

One main concern of our work was to be able to efficientlymanage and store incoming data from edge nodes, so that itbecomes easier to process and share the information either inlocal level or in the cloud as open data. Shared data, relating to

978-1-5386-4155-2/18/$31.00 c©2018 IEEE

Fig. 1. Sensor network topology

local weather, building surveillance, traffic information, etc., incloud platforms, can really improve every day life, especiallyin small communities like the island of Samos in Greece,where IT infrastructure is limited. Having open data as a highlevel goal, we tried to establish some hard criteria for the topicnames and the content of data messages, the two main datastructures of the MQTT protocol, for which there are no well-established and accepted formats [3]. Of course, the MQTTprotocol has been utilized in many previous Wireless SensorNetwork (WSN) applications but, most of the times, along withquite complex and power demanding hardware at the edge ofthe network and also without really specifying or providing aschema of the MQTT data format [5].

In our application, we support the idea that at the edgeof the network, the lowest possible level of hardware shouldbe used that supports encrypted network connectivity withlow-power consumption. Although MQTT WSN solutionsbased on 8-bit uC architectures [6] really favor low power,unfortunately cannot handle TLS encryption. The use of aPython based framework along with the Advanced MessageQueuing Protocol (AMQP) has been presented in [7] and whilethe protocol favors faster communication throughput usinglong-lived node connections, the authors do not provide clearperformance results on the maximum number of supportednodes or data throughput through the network. Using ournetwork architecture, we try to provide experimental resultsabout the MQTT message network throughput supported byour hardware and also about the maximum number of nodesthat our server/broker can support, by measuring the servermessage processing time (refer to Section IV). We haveplanned also to measure and report the power profile of ourdevices, related to MQTT message traffic and throughput.

II. NETWORK LAYER USING THE MQTT PROTOCOL

In our application, we wanted to establish a machine-to-machine communication, where sensor nodes send shortmessages (less than 1KB) to report conditions in a certainindoor or outdoor area. The MQTT protocol [8] has been usedalready in many signaling and control telemetry applications,so that companies can get millions of products connected to

their servers, and even as a short message chatting platform(Facebook messenger).

The protocol standard provides the means for the brokerand the client nodes (CN) to create message topics, to whicheach node can subscribe and listen to incoming messages orpost data in the form of short messages. The protocol alsospecifies three different levels of Quality of Service (QoS)for the transmitted messages and also provides an encryptionmechanism based on TLS. A connection is made to the brokerand a CN may transmit data or may just listen to some topics.So, the broker is not aware about the state of the CNs, incase they are connected but constantly in a passive/listeningmode. Also, despite the fact that there are some restricted/ad-minsitrative topics, the CNs may freely create new topics andpublish data to them. In terms of message content, the protocolspecifies a type of retained messages (last published data onthe topic) and also a type of last-will message, broadcasted bythe broker in case a CN disconnects in an abnormal way.

In our network, an XML node configuration file is associ-ated and stored in each network node, including the broker.This file holds important information about each node: a)geolocation information, b) type of hardware and attachedsensors, c) type of software and OS, d) type of data typesprovided by the sensors, and e) network parameters like IDand IP address. The content of the configuration file is parsedduring hardware boot by every node and also transmitted byeach CN to the broker during a connection/validation process,where the broker validates the XML file against a definedschema. The CN validation process is further explained in thefollowing sections. After a successful CN node validation bythe broker, the XML configuration file is stored in the databaseand associated to the specific CN, while work in progress willalso provide a mechanism so that the server can modify andupdate OtA the configuration file of a CN.

Handling of the XML configuration file in our applicationis closely related to the formalization of the MQTT protocoldata structures. Each CN, according to the number and typeof attached sensors, is able to automatically create the nameof the topic that will use to publish its data, by extractingthe related fields from its configuration file. In this way, eachCN subscribes to: a) a control topic used to send and receivecommands to/from the broker, b) as many as needed data-related topics to publish data from its sensors, and c) a numberof system topics to receive broadcasted information from thebroker, e.g., time, network status, etc. The format of these threetopics is the following (system topics start with a ‘$’):/networkName/nodeID/country/districtState/cityTown/areaDe-

scription/area/buidling/room/control [(str) command*, (str)parameter1, (str) paramter2, (str) parameter3]/networkName/nodeID/country/districtState/cityTown/areaDe-

scription/area/buidling/room/sensorDataName [(str) dataType,(str) dataRangeUnits, (str, int, float) data]$/networkName/nodeID/country/districtState/cityTown/areaD-

escription/area/buidling/room/sysTopicName [(str) dataType,(str, int, float) data]where in the control topic the supported command may containup to three parameters, and in the sensor data topic the datatype, units and range are provided for each piece of data.Using these topics’ formats, data from CNs are tagged andcharacterized in terms of location and sensor type, while node

administration is very modular, since to add more sensors orchange the location of a CN, only an update to the XMLconfiguration file is required. By using a standardized format,collecting data from the topic name or the message contentand storing information to a database on the server becomeseasier, using powerful text processing languages like Python.

III. CLIENT NODES (CNS) SPECIFICATION

Our network configuration on the edge consists of twokinds of client nodes: a) ESP8266 and b) Raspberry Pi, bothequipped with software that has a rigid but easily expandablestructure. The low-power and efficient Espressif ESP8266 WiFiSoC integrates a 32-bit single core RISC processor clocked at80MHz (up to 160MHz), combined with 160KB of RAMand External Flash support (up to 16MB). Considered asopen hardware, ESP8266 modules come in many variationsfrom different vendors. For our implementation, the NodeMCUdevelopment kit was used with an ESP8266 (ESP-12E) WiFimodule from Ai-thinker, providing an additional 4MB ofFlash memory (ROM). The SoC has various interfaces includ-ing GPIO and ADC for digital and analog inputs and out-puts. As measurement devices, a digital temperature/humidityDHT22 sensor was used, along with an analog/digital flamesensor. Raspberry Pi 3 Model B was our board of choice as asecond edge node. This version of the popular board computerutilizes a 64-bit quad core 1.2GHz processor and 1GB ofRAM for more demanding tasks. Combined with RaspberryPi Camera V2 module, this node handles images and videostreams for further analysis and object recognition. In terms ofsoftware, we adopted an OS based approach for all the nodes,so that we can easily handle encryption certification files andother file-based operations, and also be able to support, usinginterrupts/threads, at least the two concurrent operations of: i)MQTT message traffic and ii) sensor data acquisition.

The nodes are constantly connected and able to respondto commands in the control topic. We introduced three clientfunctional modes (INITial, ACTIVE, BLOCKED) that definethe connection status of the CNs. During INIT mode, eachnode is validated by the server according to the content ofits XML file and the server sets its functional mode either toACTIVE or BLOCKED. While in ACTIVE mode, a severitymode should also be set by each CN, related to the status of theacquired measurements. The default severity mode after bootis NORMAL, in which the CN acquires measurements every60sec. Every change in the severity mode is reported to theserver using dedicated commands in the control topic. Changesof the severity mode are triggered by the CNs, according todata preprocessing functions applied on acquired data. Thisprocess allows critical conditions already identified on thefield, to be indicated by the CNs with severity mode changes,while further data post-processing is done on server side. Thesensor sampling interval is decreased to 40sec in WARNINGmode and to 20sec in DANGER mode.

For the ESP boards we used the MicroPython OS [9].This lite version of Python 3 has many CPython functions ona small package, well suited for uCs. MicroPython providesa Python programming (cross-)platform (all nodes runningPython) with a dedicated file system. It features OtA fileupdates, making prototyping extremely versatile. File systemoffers multi-directory distributed coding with separated Pythonmodules that increases flexibility and code maintenance. Dur-

Fig. 2. MQTT main topics and available functional/severity modes for CNs

ing code development we came across major RAM memoryissues, solved with memory optimized programming and ker-nel recompilation to increase memory utilization. A lot ofeffort was required, using multiple interrupt service routines,to implement a pseudo-thread mechanism in Python (MicroPy-thon does not support real threads) to handle concurrentlythe MQTT client and sensor data acquisition. Using also adata regression function, ESP generates a linear model of thetemperature measurements, to handle severity mode transitionsaccording to the temperature increase/descrease rate.

The Raspberry board hosts a Linux Debian 9 OS (RaspbianStretch), equipped with Python packages to run our project’sscripts. As a programming language we used Python 2.7 totake advantage of its cross-platform compatibility and get allof our CNs running similar Python code. The main purpose ofthe Raspberry node is to capture images on a timed intervaland communicate directly with cloud services, like the GoogleVision API, for basic image analysis. Analysis result arefiltered and provided to the MQTT server for storage andfurther analysis. In this way, the Raspberry node providesdata to the local MQTT network, but also uses powerfulprocessing resources from the cloud. Work in progress includesa mechanism to be developed, so, if required, images can betransferred from the Raspberry board to the server for furtheranalysis, using an SFTP connection (MQTT does not favor filetransfers). Severity mode transitions are handled according tothe type and number of identified objects on the images, whilethe mode transition rules can be changed on the fly (type ofidentified objects that trigger transitions) using commands onthe control topic. Figure 2 summarizes the main MQTT topicsand the functional/severity modes of the CNs.

Both nodes in our setup exchange TLS encrypted messages,using locally stored certificates. On the ESP8266 board we hadto heavily optimize our code and recompile the MicroPythonkernel in order to save some KBs of memory. To the best of ourknowledge, the ESP8266 board (NodeMCU) is the smallest,very low-power, embedded CPU plus WiFi transceiver com-bined System on Chip (SoC), that, along with the MicroPythonOS, can handle concurrently the MQTT encrypted messages,sensor data acquisition and support OtA software updates.

IV. SERVER NODE (SN) SPECIFICATION

In our network topology, we use a server PC that actsboth as the MQTT network broker and a network client. A64-bit 3.3GHz Intel i3 processor with 3GB of RAM canhandle all the tasks of our small scale network. The serverhosts an Ubuntu 16.04 OS and is used as: a) MQTT broker,

b) database (DB) and DB management, and c) Web Server fordata visualization and cloud services communication.

As MQTT broker, we used the Mosquitto Broker with theMQTT protocol ver. 3.1. The broker forces TLS encryptionto all the clients, as well as password protected connection.Encryption is achieved with server self-signed certificates. Tostore and manage our data efficiently, we chose the Post-greSQL database that offers the advantages of relational DBs(indices, triggers, structured data) and some non-SQL DBcharacteristics (JSON data type), without any major sacrificeon speed. For data visualization and other web services inte-gration, our node acts also as a web server, providing opendata in the widely acceptable JSON format.

The server’s software is also implemented in Python 2.7as a state machine, to realize the several states related tothe CNs functional/severity modes implementation model. Aspreviously described, a configuration file in XML format thatfollows a specific XML schema is used for every node, asnode identity. There is a strict process of node validation uponconnection to the broker that strengthens node integrity. Theconfiguration files of all nodes are stored in our database alongwith live data, such as the functional and the severity mode ofeach node. In the event of an invalid configuration file, the nodeattempting the connection is blocked. The main task of theSN is to receive all incoming messages from clients, performmessage analysis, handle the data and respond accordingly.

A multi-threaded approach was used in Python to developthe software on the server, demonstrating the potential of thelanguage in parallel processing environments. In our imple-mentation, we handle the message load by using two buffersand a function that checks for messages periodically. The mainbuffer holds all incoming messages by continuously checkingthe broker, while a parallel thread ensures constant availabilityby moving the captured messages to a second (temporary)buffer for further actions. The content of this buffer is filteredaccording to message topic and sensor data are stored into theDB using 1 thread per message. With this approach, our low-end server can easily process and store large bursts of sensormeasurements from a single client, averaging on a satisfying60 messages/sec without message loss. To simulate multi-client conditions, we developed some benchmarks that includesimultaneous bursts from more clients. For our benchmarks,we used a sample sensor data message (<1KB) that we sendto the server repeatedly, using QoS of level 2, to ensure that themessage is delivered exactly once. As we can see from Table I,throughput tends to decline either with increased client countor burst size. The last case with 500 connected clients sending1 msg. each is the most realistic one, as it simulates a commoncondition in alert-based networks. We simulated the 500 clientsusing separate threads running in parallel, achieving an averagedelay of 14msec between messages. It is very encouraging thatour server manages to maintain its peak performance underthese circumstances. Experiments with level 1 QoS messagesshowed a 40% increase in performance, due to the fastertransmission process. Work in progress includes hardware up-grades and software optimizations. Python code, state machinediagrams (SN, CN), and the CN XML configuration file andschema can be found in our code repository [10].

V. CONCLUSIONS AND WORK IN PROGRESS

We presented a small scale MQTT-based sensor networkimplementation to i) explore different setups on the formaliza-

TABLE I. SERVER MESSAGE THROUGHPUT FOR DIFFERENT # OF CNSAND SIZE OF MESSAGE BURSTS (MESSAGE SIZE <1KB)

# of CNs # of messages (burst size) / CN Stored messages / sec1 1000 701 2000 651 10000 44

10 1000 34500 1 71

tion of the protocol data structures. We experimented also ii) onthe use of very low-power sensor nodes to handle concurrentlyencrypted message traffic and sensor data acquisition. Morepowerful nodes are used to iii) combine locally available datawith powerful hardware cloud-based services. Moreover, ourwork includes iv) experimenting with database setups, usingnon-SQL and relational characteristics, to efficiently storesensor measurements and expose them as public open data.Working on these four different areas of a sensor networksetup, we believe that we have established a standardized wayon how to setup a network with the use of the MQTT protocol,based on alert-based sensors and on how to store, expose andcombine acquired data with cloud based information and ser-vices. Work is in progress to increase the size of our network,using FPGA-based nodes to implement data preprocessingand reduce network traffic and server processing load. Also,work is in progress to finalize the server dashboard, connectit to popular cloud platforms to improve data visualizationand alert triggering (emails, sms, etc.), implement big datapost-processing algorithms using dedicated server hardware todiscover hidden data patterns in the acquired measurements,and optimize the use of the network by small communities interms of infrastructure and environmental monitoring.

REFERENCES[1] Alexander Malaver, Nunzio Motta, Peter Corke, and Felipe Gonzalez,

“Development and integration of a solar powered unmanned aerialvehicle and a wireless sensor network to monitor greenhouse gases,”Sensors, vol. 15, no. 2, pp. 4072–4096, 2015.

[2] A. Niruntasukrat, C. Issariyapat, P. Pongpaibool, K. Meesublak,P. Aiumsupucgul, and A. Panya, “Authorization mechanism for MQTT-based Internet of Things,” in 2016 IEEE International Conference onCommunications Workshops (ICC), May 2016, pp. 290–295.

[3] N. Tantitharanukul, K. Osathanunkul, K. Hantrakul, P. Pramokchon,and P. Khoenkaw, “MQTT-topic naming criteria of open data forsmart cities,” in 2016 International Computer Science and EngineeringConference (ICSEC), Dec. 2016, pp. 1–6.

[4] R. Mahmoud, T. Yousuf, F. Aloul, and I. Zualkernan, “Internet of Things(IoT) security: Current status, challenges and prospective measures,”in 2015 10th International Conference for Internet Technology andSecured Transactions (ICITST), Dec. 2015, pp. 336–341.

[5] R. Brzoza-Woch, M. Konieczny, P. Nawrocki, T. Szydlo, and K. Zielin-ski, “Embedded systems in the application of fog computing - Leveemonitoring use case,” in 2016 11th IEEE Symposium on IndustrialEmbedded Systems (SIES), May 2016, pp. 1–6.

[6] Gianluca Barbon, Michael Margolis, Filippo Palumbo, Franco Rai-mondi, and Nick Weldin, “Taking Arduino to the Internet of Things:The ASIP programming model,” Computer Comm., vol. 89-90, pp. 128– 140, 2016, Internet of Things: Research challenges and Solutions.

[7] A. Azzar, D. Alessandrelli, S. Bocchino, M. Petracca, and P. Pagano,“PyoT, a macroprogramming framework for the Internet of Things,”in Proceedings of the 9th IEEE International Symposium on IndustrialEmbedded Systems (SIES 2014), June 2014, pp. 96–103.

[8] “MQTT Version 3.1.1 - OASIS standard,” 2015, http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/mqtt-v3.1.1.html.

[9] “MicroPython - Python for uCs,” 2017, https://micropython.org.[10] “Git repository,” 2018, https://github.com/evlog/mqtt sensor network.