Message Queuing Telemetry Transport (MQTT)

Message Format

Khamdamboy Urunov, a Ph.D. student.

Special Communication Research

Center.,

Graduate School of Financial

Information Security., Kookmin

University

MQTT model (1/3)

2

The core elements of MQTT are clients, servers (=brokers), sessions, subscriptions and topics.

http://indigoo.com/dox/wsmw/1_Middleware/MQTT.pdf

MQTT model (2/3)

3

MQTT model (3/3)

4

5

MQTT message format

Ongoing Message format

Discussed

CONNECT message format:

6

MQTT message format

The CONNECT message contains many session-related information as optional header fields.

CONNECT 1 Client request to connect to Server

7

Overview CONNECT message fields:

MQTT message format (cont…)

8

MQTT message format (cont…)A good-natured client will send a connect message with the following content among other things:

ClientId

The client identifier (short ClientId) is an identifier

of each MQTT client connecting to a MQTT broker.

As the word identifier already suggests, it

should be unique per broker.

The broker uses it for identifying the client

and the current state of the client.

If you don’t need a state to be hold by the

broker, in MQTT 3.1.1 (current standard) it is also

possible to send an empty ClientId, which results

in a connection without any state.

A condition is that clean session is true,

otherwise the connection will be rejected.

http://www.hivemq.com/blog/mqtt-essentials-part-3-client-broker-connection-establishment

Conceptual idea:

Topic name has an upper length limit of 32,767 characters.

http://blog.opensensors.io/

Developing a workflow

9

Clean Session

The clean session flag indicates the broker, whether the client wants to establish a persistent session or

not.

A persistent session (CleanSession is false) means,

• that the broker will store all subscriptions for the client and also all missed messages

• when subscribing with Quality of Service (QoS) 1 or 2

If clean session is set to true,

• the broker won’t store anything for the client and will also purge all information from a previous

persistent session.

MQTT message format (cont…)

True = 1

False=0

10

MQTT message format (cont…)

11

MQTT message format (cont…)

12

Username & Password flag

MQTT allows to send a username and passwordfor authenticating the client and also authorization.

However, the password is sent in plaintext,

• if it isn’t encrypted or hashed byimplementation

• or TLS is used underneath.

We highly recommend to use username and

password together with a secure transport of it.

In brokers like HiveMQ it is also possible toauthenticate clients with an SSL (secure socketslayer) certificate, so no username and password isneeded.

MQTT message format (cont…)

http://slides.com/disk91/mqtt#/13

MQTT Will

Message

• The will message is part of the last will and testament feature of MQTT.

• It allows to notify other clients, when a client disconnects ungracefully.

• A connecting client will provide his will in form of an MQTT message and topic in the

CONNECT message.

• If this clients gets disconnected ungracefully, the broker sends this message on behalf

of the client. We will talk about this in detail in an individual post.

13

MQTT message format (cont…)

14

If a message is published with a topic, the broker distributes it to all subscribers following the topic.

• If nobody is currently following that topic, the message is lost – it isn't cached because there is no queue.

• To prevent this behavior, users can set a retained flag when publishing.

This flag prompts the broker to store the message and to offer it to future subscribers.

• If, for example, the outside temperature is published with temp/terrace and a client subscribes to this topic hours

later,

• this client is immediately sent this recently published temperature value (last-known value) if the retained flag is

set.

What happens if a client unexpectedly fails?

• LWT (Last Will and Testament) is an important feature of MQTT.

• A client tells the broker during the connection setup that: "In case of my death, please send the following message

with this topic …" If the broker then detects that this client is no longer alive, it does as it was told and publishes the

message with the desired topic.

• MQTT thus has monitoring installed: It's easy to write to clients that respond to the passing of other clients, such

as by sending alerts.

LWT messages are often used together with the retained flag in order to permanently describe the state of a client:

• When starting, a client will,

• for example, publish a status message with a retained flag with the clients/id/status topic.

• If this client deceases, the broker would distribute the dead message with a retained flag and the same topic on its

behalf.

• A query on clients/id/status thus always reveals whether the client is online or offline.

MQTT message format

(cont…)Last Will and Testament

http://www.admin-magazine.com/Archive/2015/30/Using-the-MQTT-IoT-protocol-for-unusual-but-useful-purposes

15

Example: Subscribers receive last known temperature value from the temperature data

topic. RETAIN=1 indicates to subscriber B that the message may have been published

some time ago

MQTT message format

(cont…)Will Retain: If set to 1 indicates to server that it should retain a Will message for the

client which is published in case the client disconnects unexpectedly.

16

MQTT message format

(cont…)Will QoS- Specifies the QoS level for a Will message.

17

MQTT message format

(cont…)Will Flag- Indicates that the message contains a Will message in the payload

along with Will QoS flags.

MQTT Message Keep Alive

• The keep alive is a time interval, the clients commits to by sending regular

PING Request messages to the broker.

• The broker response with PING Response and this mechanism will allow

both sides to determine if the other one is still alive and reachable.

• That are basically all information that are necessary to connect to a MQTT

broker from a MQTT client.

• Often each individual library will have additional options, which can be

configured

• Protocol includes support for client and server to detect failed connections

– At connection time, a keep alive can be specified

• Maximum keep alive interval of 18 hours

– Can specify a value of 0 to disable keep alive18

http://www.slideshare.net/henriksjostrand/devmobile-2013-low-latencymessagingusingmqtt 19

MQTT Message Keep Alive (cont…)

MQTT Low Power Usage

• Restriction on some device, such as cell phone may simply be available power,

how does MQTT use power ?

• HTC Desire Android mobile phone

• Protocol allows tuning to suit devices

MQTT & HTTP

20http://www.slideshare.net/henriksjostrand/devmobile-2013-low-latencymessagingusingmqtt

MQTT uses (much) less bandwidth that HTTP

21

CONNACK message format:

MQTT message format (cont…)

CONNACK 2 Connect Acknowledgment

22

MQTT message format (cont…)PUBLISH 3 Publish message

Byte 2 (At least one byte) contains the Remaining Length field. The fields are described in the following sections.

All data values are in big-endian order: higher order bytes precede lower order bytes. A 16-bit word is presented

on the wire as Most Significant Byte (MSB), followed by Least Significant Byte (LSB).

23

MQTT message format (cont…)PUBACK 4 Publish Acknowledgment

PUBREC 5 Publish Received (assured delivery part1)

24

MQTT message format (cont…)

PUBREL 6 Publish Release (assured delivery part 2)

PUBCOMP 7 Publish Complete (assured delivery part 3 )

25

MQTT message format (cont…)

SUBSCRIBE 8 Client Subscribe request

26

MQTT message format (cont…)

SUBACK 9 Subscribe Acknowledgment

27

MQTT message format (cont…)

UNSUBSCRIBE 10 Client Unsubscribe request

28

MQTT message format (cont…)UNSUBACK 11 Unsubscribe Acknowledgment

PINGREC 12 PING Request

PINGREST 13 PING Response

DISCONNECT 14 Client is Disconnecting

http://www.slideshare.net/PeterREgli/mq-telemetry-transport

MQTT connection message flow

29http://www.sharetechnote.com/html/IoT/App_Protocol_MQTT.html

MQTT message

30

MQTT example

31http://www.sharetechnote.com/html/IoT/App_Protocol_MQTT.html

< CONNECT >

MQTT example

32

< CONNECT ACK>

MQTT example

33

< PUBLISH >

MQTT example

34

< SUBSCRIBE >

MQTT example

35

< SUBACK >

MQTT format (cont…)

36

Payload Message identifier MQTT and UTF-8 Unused bits

The following types of MQTT command message have a payload:

CONNECT

The payload contains one or more UTF-8 encoded strings.

They specify a unqiue identifier for the client, a Will topic and message and the User Name and Password to

use.

All but the first are optional and their presence is determined based on flags in the variable header.

SUBSCRIBE

The payload contains a list of topic names to which the client can subscribe, and the QoS level. These strings are

UTF-encoded.

SUBACK

The payload contains a list of granted QoS levels. These are the QoS levels at which the administrators for

the server have permitted the client to subscribe to a particular Topic Name.

Granted QoS levels are listed in the same order as the topic names in the corresponding SUBSCRIBE

message.

The payload part of a PUBLISH message contains application-specific data only.

No assumptions are made about the nature or content of the data, and this part of the message is treated as a

BLOB.

MQTT format (cont…)

37

Payload Message identifier MQTT and UTF-8 Unused bits

The message identifier is present in the variable header of the following MQTT messages: PUBLISH,

PUBACK, PUBREC, PUBREL, PUBCOMP, SUBSCRIBE, SUBACK, UNSUBSCRIBE, UNSUBACK.

The Message Identifier (Message ID) field is only present in messages where the QoS bits in the fixed header

indicate QoS levels 1 or 2.

The Message ID is a 16-bit unsigned integer that must be unique amongst the set of "in flight" messages in a

particular direction of communication. It typically increases by exactly one from one message to the next, but is not

required to do so.

A client will maintain its own list of Message IDs separate to the Message IDs used by the server it is connected

to.

It is possible for a client to send a PUBLISH with Message ID 1 at the same time as receiving a PUBLISH with

Message ID 1.

The ordering of the two bytes of the Message Identifier is MSB, then LSB (big-endian).

Do not use Message ID 0. It is reserved as an invalid Message ID.

MQTT format (cont…)

38

Payload Message identifier MQTT and UTF-8 Unused bits

UTF-8 is an efficient encoding of Unicode character-strings that optimizes the encoding of

ASCII characters in support of text-based communications.

In MQTT, strings are prefixed with two bytes to denote the length, as shown in the table below.

String Length is the number of bytes of encoded string characters, not the number of characters. For example, the string OTWP is encoded in UTF-8 as shown in the table below.

MQTT format (cont…)

39

Any bits marked as unused should be set to zero (0).

Payload Message identifier MQTT and UTF-8 Unused bits

Schedule of next plan

• Next step compare MQTT and CoAP protocols

• Analysis source code steps

• Make those u_CoAP, u_MQTT protocols message format

40

1. https://github.com/aoabook/WroxAccessories/blob/master/src/com/wiley/wroxaccessories/MQTT.java2. https://www.crossdart.info/p/mqtt/1.1.0/mqtt_message_publish.dart.html3. http://iicb.org/viewpaper/14. http://www.javased.com/?source_dir=Racenet-for-Android/src/com/albin/mqtt/message/RetryableMessage.java5. http://www.programcreek.com/java-api-examples/index.php?api=org.dna.mqtt.moquette.proto.messages.AbstractMessage.QOSType6. http://indigoo.com/

Link of the source code:

Thank you

hamdamboy.urunov@gmail.com

41