Towards Ranking IoT Middleware Platforms Based on Quantitative and Qualitative Metrics

Mauro A. A. Da Cruz1, Joel J. P. C. Rodrigues1,2,3, Kashif Saleem4, Andre L. L. Aquino5

1 Instituto Nacional de Telecomunicações (INATEL), Santa Rita do Sapucaí-MG, Brazil 2 Instituto de Telecomunicações, Portugal

3 University of Fortaleza (UNIFOR), Fortaleza-CE, Brazil 4 Center of Excellence in Information Assurance (CoEIA), King Saud University (KSU), Riyadh, Kingdom of Saudi Arabia

5 Laboratório de computação Científica e Análise Numérica (LaCCAN) – Instituto de Computação (IC), Universidade Federal de Alagoas (UFAL), Maceió-AL, Brazil

maurocruzter@gmail.com; joeljr@ieee.org; ksaleem@ksu.edu.sa; alla@laccan.ufal.br

Abstract— With so many resource-constrained objects

(“things”) interacting autonomously in Internet of Things (IoT) solutions, the surrounding environment becomes smarter. A software, called middleware, plays a key role as it is responsible for most of the intelligence in IoT, acting as a “brain”, integrating data from devices, allowing them to communicate, and making decisions based on collected data. By nature, smart environments are heterogeneous, considering the plethora of available technologies, and middleware can thrive, playing even a more relevant role in large environments, such as smart cities. This paper explores the differences between connecting conventional and IoT devices to the Internet, while highlights the theoretically ideal operation of IoT middleware modules. Then, in order evaluate the performance comparison and rank this type of platforms across different scenarios, quantitative and qualitative metrics are proposed and detailed described.

Keywords— Internet of things; IoT; Middleware; Platform; Performance metrics; Qualitative; Quantitative

I. INTRODUCTION When the Internet was envisioned, the primary concern was

connectivity among computer networks and human-generated data. This paradigm prevailed for many years but is expected to shift with the uprising of the Internet of Things (IoT) paradigm where 50 Billion Internet-enabled devices are expected by 2020 [1]. These devices, called “things,” can belong to both virtual and real world (virtual and physical things, respectively) [2]. These things make machine-generated data more relevant, as the number of Internet-connected devices contrasts with the 7.7 billion human population that is estimated in the same period [3]. The number of IoT-enabled devices is expected to grow exponentially, so the amount of machine-generated data will follow the same path. As the number of devices increase, it is predicted that surrounding environment will be smarter.

Human-generated data refers to the data that is produced through the recording of human choices or is pre-processed by a human (e.g., when clicking a “like” button on social media, computer stores the data). In another perspective, machine-

generated data refers to data that is produced entirely by a machine as a result of its decisions. These data can also be generated by observing humans instead of recording only their actions (e.g., data analytics and big data). Figure 1 illustrates two scenarios of human-generated and machine-generated data. In (a), a camera records a video and saves that data to a hard disk, this counts for human-generated data. In (b), a camera is recording video, stores it in a hard disk and, then, analyses the footage, extracting meaning and context from the recording, this counts for machine-generated data.

Fig. 1. Illustration of (a) human-generated data and (b) human-generated data

transformed into machine-generated data.

The amount of data produced by large IoT environments, such as Smart Cities, is unprecedented as they heavily rely on machine-generated data. This data increasing poses significant challenges for researchers and industry. One of these difficulties is related to machine-to-machine (M2M) communications. The main principle of communication implies that each collocutor must “speak” the same language. In IoT, this is a big issue since there is a plethora of devices, each one with its own language that does not follow the standards (as well as vendors that are not concerned with compatibility of their product devices with

2017 IEEE First Summer School on Smart CitiesNatal, Brazil, August 6-11, 2017

978-1-5386-1063-3/17/$31.00 ©2017 IEEE 67

others). This compatibility problem is solved through a middleware, a software that provides interoperability between devices and networking technologies. Otherwise, they should not communicate. IoT environments, such as smart cities, are tremendously heterogeneous in nature, and IoT middleware is one of the technologies that enables them. In the literature, IoT middleware solutions are also referred to as IoT platforms or IoT middleware platforms. This occurs because platforms provide an ecosystem where things are built in and available to users [4]. So, in this paper, the three terms are used interchangeably. Currently, there are many IoT middleware proposals but there is not an objective way to compare their performance. Then, this paper recognizes the key responsibility of IoT platforms and studies their role, presenting a simplified middleware architecture, and defines the best operation of each module. This work also proposes qualitative and quantitative metrics to evaluate, compare the performance, and select an IoT middleware for a given scenario.

The remainder of the paper is organized as follows. Section II provides a background on how Internet connectivity is slightly different in IoT comparing to the conventional Internet due to their requirements. Section III describes the operation of relevant modules of an ideal IoT middleware to meet IoT requirements, while Section IV introduces qualitative and quantitative performance metrics to evaluate IoT middleware. Section V presents a brief demonstration on how the metrics can be used. Open issues and research challenges identification on IoT middleware are discussed at Section VI and Section VII concludes the paper.

II. CONNECTING TO THE INTERNET IN IOT ENVIRONMENTS In IoT scenarios most objects are minuscule; therefore, they

have constrained resources. The current Internet protocol stack does not take these limitations into account. Wi-Fi (IEEE 802.11 a/b/g/n/ad/ac) is one of the most common ways to access the Internet, and its protocol stack is not suitable for providing wide area coverage, low power consumption on end-devices, or supporting a high number of end-devices. For this reason, alternatives have been developed and are being used on IoT, such as the Bluetooth 5 and the IEEE 802.15.4 that supports both ZigBee and 6LoWPAN (IPv6 over Low Power Wireless Personal Area Networks). Bluetooth 5 is the latest iteration of the popular Bluetooth standard. Similar to Bluetooth 4.2, Bluetooth 5 also supports IP networks [5]. Unfortunately, users rarely explore the IP capabilities provided by Bluetooth. IEEE 802.15.4 is a standard for Low-Rate Wireless Personal Area Networks (LR-WPANs) that specifies the physical and MAC layers of the OSI model [5]. 6LoWPAN and ZigBee deployments use IEEE 802.15.4. 6LoWPAN is an Internet Engineering Task-Force approach that compresses and encapsulates the IPv6 headers and accommodate them on the frame IEEE 802.15.4 [5]. ZigBee was developed and maintained by the ZigBee Alliance and it is mostly known for its mesh topology, but it supports other topologies, such as star and tree [5]. The advantage of 6LoWPAN is the use of the well-known Internet Protocol that,

unlike ZigBee and traditional Bluetooth, it is not needed a complex gateway when communicating to the Internet, which requires further security mechanisms and increased overhead. A ZigBee gateway is illustrated in Fig. 2; and a similar concept is applied to any technology that does not support IP natively. Recognizing the importance of IP networks, a modification of ZigBee, called ZigBee IP, was released. ZigBee IP uses many 6LoWPAN concepts, especially the header fragmentation and compression scheme [5].

Another famous way of accessing the Internet is through 3G/4G networks, but both of them are not suitable in IoT for the same reasons as Wi-Fi. Because of this, long-range network solutions, such as LoRa, Sigfox, and IEEE 802.11ah (HaLow) were proposed. These alternatives provide broad area coverage and consume significantly less battery. IEEE 802.11ah takes an advantage over the other two technologies because it is an IEEE 802.11 standard, then, it is IP ready. Both LoRa and Sigfox need a gateway to perform an interface with end devices, and the gateway connects to a backhaul to provide a connection to the Internet [6], as depicted in Fig. 3. Another promising solution for IoT is the recent 5G technology, which presents different performance requirements for distinct scenarios. 5G is expected to be released to public usage around 2020 [7].

Fig. 2. Illustration of a ZigBee architecture connecting to the Internet through a

TCP/IP gateway.

Fig. 3. Illustration of Sigfox/Lora overall architecture.

The Internet architecture uses HTTP in the presentation layer, but common HTTP requests consume too much energy and, for this reason, some protocols have been proposed as alternatives. Two protocols that emerged on this contact and are being used on IoT are the Constrained Application Protocol (CoAP) and Message Queing Telemetry Transport (MQTT), both expecting a TCP/IP stack. CoAP runs over UDP while MQTT runs over TCP [8, 9]. CoAP is based on the REST model, which allows resource-constrained devices to access resources through REST

68

methods. MQTT relies on the Publish/Subscribe (Pub/Sub) model; therefore, it needs a message broker. Among other aspects, the broker is responsible for sending the message to all subscribed clients. For instance, the Messenger from Facebook uses MQTT. A variation of this protocol for networks that are not based on TCP/IP is called MQTT-SN; so in theory, it can be used in Zigbee or Bluetooth. CoAP generates less overhead than MQTT for all message sizes when the packet loss is low; when the packet loss is higher, CoAP produces less overhead only when the message size is small [9]. When the message is large the probability that TCP loses the message is smaller than UDP, which causes MQTT to retransmit the entire message less times than CoAP [9]. Another particular aspect of IoT is data representation. Currently, the most used coding technic is JSON, but one of its biggest strengths (easily readable to humans) implies more computational capacity when encoding or decoding as well as transmitting, which is virtually unnoticed in the conventional Internet. Therefore, binary encodings such as Apache Thrift and Google’s Protocol buffers are better suited for most IoT devices [10]. This does not mean that they should not support JSON, it means that they should only use JSON encoding when strictly necessary.

III. MIDDLEWARE REFERENCE ARCHITECTURE MODULES When IoT is publicized, beautiful scenarios are presented

where devices learn from the user habits and react to them, improving quality of life and experience, while also reducing the user’s carbon footprint. All the scenarios that are presented finish with a sentence similar to this one: “all of this with minimal human intervention.” These scenarios are only possible because of middleware platforms that integrate data from all the devices and acts upon it. Most devices are small and resource constrained to make complex decisions; therefore, the platforms are responsible for most of the intelligence in IoT. To fulfill their goals, the modules of an IoT platform architecture should necessary reflect IoT requirements, as follows: i) interoperability, ii) persistence and analytics, iii) context, iv) resource and event, and v) security. The modules of a considered ideal IoT platform are presented in Fig. 4 and described below.

Interoperability module. The IoT is a heterogeneous environment and the platform is the integrator. Therefore, it should provide an API (application programming interface) that allows software to expose certain functions without sharing code for applications and things to communicate [11]. API requests made by things/applications can be performed through any protocol, so the middleware should, at least, support the most popular IoT application protocols, such as CoAP, MQTT, and HTTP [11]. The module should also support standard data representation methods, like XML and JSON, as well as binary encodings (Apache thrift, Google protocol buffer).

Persistence and Analytics module. IoT produces huge amount of data, which needs to be continuously stored for chronological consultation and further processing [12]. One vision of IoT assumes that things learn from the user habits, then, in practice, middleware will learn from the collected data.

Therefore, at least it should provide basic analytics, such as averages, simple graphs, or max/min values [13]. However, the best (ideal) is further data processing through data warehousing, big data, or even feeding it to deep/machine learning algorithms as the collected data is highly valuable.

Context module. Context is the idea that understanding the background of a certain aspect changes the way an individual phrase is perceived, as sentences carry no meaning by themselves. In a communication, context provides meaning to a conversation. IoT environments are expected to adapt to surroundings and context will play a significant role in this regards [14]. Semantic interpretation and ontologies are expected in this module because people communicate semantically and the same is expected when humans interact with machines in IoT environments. Unfortunately, semantic interpretation and context-awareness are hard to achieve without artificial intelligence techniques (one of the most challenging fields in this technology), but the platform can use external APIs to achieve this goal.

Resource and Event module. For things being efficient in their actions, they must know what they are able to perform and their internal operation status (battery level, internal/external temperature, current memory usage), so they can advertise their resources and discover resources from others. Multiple devices are expected to communicate among them simultaneously; they can even offer the same service, and better devices are supposed to be requested more often than the others. This means that they will not always be able to provide the best service due to memory constraints (too many requests being processed), or even constraints from the physical world (distance). These issues are a concern related to multiplicity of actions and the tiny devices constraints [15]. Platforms minimize these problems by managing and optimizing these interactions. When connecting for the first time to a middleware platform, devices and external applications should announce their capabilities through some sort of text message (e.g., in JSON). Then, the context module semantically interprets the capabilities and when a device or application needs an individual service, the middleware understands all capabilities provided by its environment and is able to generate the proper events. Middleware should also facilitate events update through devices [11], as it is not expected a person can manually manage every single device in a smart city environment.

Security module. IoT will not become popular without plug-and-play. This means that middleware should be flexible enough for the average user to handle. Unfortunately, ease of use (usability) is difficult to achieve with the level of security needed by a middleware. If the data could be tampered or retrieved by a malicious user or application, the threats would be limitless. IoT devices are not known for being secure, IoT platforms should not follow the same trend because they are the brain of IoT. The amount and value of the collected data is significant and must be secure. “With great power comes great responsibility,” a quote from the famous movie Spider-Man that defines IoT platforms,

69

as they should be updated using the state-of-the-art security mechanisms.

Fig. 4. Definition of ideal IoT platform modules.

An IoT environment is characterized by its heterogeneity considering different technologies and data collected used across its many verticals. However, some scenarios are broader than others. Small solutions like weather stations will just consider data collection and storage, as most of its data is predictable and repetitive; then, it will most likely perform basic analytics and expose data for external consultation. In big verticals, such as smart cities, that can include energy management, smart parking, smart transportation and mobility, etc., data is unpredictable. The platform should be equipped with AI mechanisms to analyze broader scenarios. In practice, this means that not all possible scenarios require all the presented modules since in small scenarios a simple platform that facilitates data consultation and storage might be enough. An example of a platform that is broad enough for many scenarios is IBM Bluemix, which integrates its AI assistant called Watson [16].

IV. PERFORMANCE METRICS FOR IOT MIDDLEWARE The first step towards selecting an IoT platform is defining

the scenario because a considerable part of the available platforms are built for specific scenarios. The same principle is applied when weighting performance metrics, some scenarios are more sensitive to certain parameters than others. Performance metrics may be both qualitative and quantitative. Qualitative metrics are metrics that cannot be translated into numbers because the way they are perceived depends on the person that is analyzing (e.g., beauty). Quantitative metrics are easily translated into numbers and can be quantified. Individuals can disagree on who is more beautiful (qualitative), but if one is taller than the other, or possesses longer hair, it is undeniable (quantitative). There are many available IoT platforms; therefore, qualitative metrics can be used to pre-filter the ones that will be compared and quantitative metrics can be used to rank them according to different scenarios and requirements.

Qualitative metrics. When individuals/organizations adopt a software solution, it is a long-term commitment. So, if the chosen

software is discontinued, the repercussions can be extremely big. Platforms play a huge role in IoT, so the first concern regarding the solution to choose should take into account its popularity because it is less likely that a successful software is discontinued. When a software is dying updates and promotion events are less frequent, so users should pay attention to both. Another indicator of dying software is the amount of spam in the comment sections. Everyone hates spam, so if there is lots of undeleted spam, moderators probably do not visit the forums regularly; therefore, it is likely the project is dying. The quality of the support is also an important factor. Many available solutions for free have trouble imposing themselves on the market because their support is through public forums and replies can take a long time. A considerable number of un-replied issues can also be an indicator of bad documentation, weak support quality, or the solution may present many problems. Usually, good documentation includes illustrations (figures or videos). A common mistake presented by free solutions is that they do not update documentation after releasing updates. Usability aspects should also be considered; some platforms do not provide basic aspects like a Graphical User Interface turning data visualization very unpleasant. The last element to consider (but not the least) is the functionalities provided by the platform, as they must be compatible with the scenario. For instance, in a smart city scenario, multiprotocol support or a mobile app for easy data visualization might be interesting functionalities.

Quantitative metrics. IoT platforms can receive data and access from distributed places around the globe, then, it is interesting to know what is the threshold to start interpreting legitimate sensor data as a DDoS (distributed denial of service) attack or even if it can notice the difference between legitimate requests and malicious attacks. Understanding this aspect is highly important because malicious individuals (or applications) could potentially shut down or damage smart cities by hijacking legitimate devices. Response times will also play a huge role, as some applications are more sensitive than others. Response time refers to the time that a software needs to process information. The packet size of requests is also relevant as it helps to manage critical device resources, such as the battery level. Another significant aspect is the variation of latency and response times during a stress test, as it will assist in determining the scalability of the solution. Stakeholders always consider the price of the solution. So, it is also an important aspect to consider when evaluating a middleware solution.

A fair comparison among different solutions implies that conditions for all of them are the same for all the aspects to consider. For software, this means that they will have the same available resources (memory, processing power, disk space, etc.). For IoT platforms, this pre-condition turns the performance comparison complicated regarding those that are only available in the cloud with local instances because it is not possible to determine what resources are allocated to the cloud instance. In practice, this means that with more resources, the local instance can perform better in comparison with fewer resources. Summarizing, Figure 5 presents the proposed qualitative and

70

quantitative metrics for IoT middleware performance evaluation and comparison.

Fig. 5. Summary of the (a) qualitative (filter) and (b) quantitative (ranking) metrics for IoT middlewares.

V. DEMONSTRATING THE METRICS To demonstrate the proposed metrics, imagine a simple IoT

scenario where a platform needed for the sole purpose of sending, receiving, and storing device data. Furthermore, the solution should be broad enough to attend a multitude of devices. There are also no financial constraints in the scenario, so price should not be a factor when selecting a platform. The candidate platforms were selected randomly and are i) Webinos [17], ii) Tago [18], and iii) ARTIK Cloud [19], which are descrbed below.

Webinos is a Service Platform that is part of the EU FP7 ICT Programme. Its primary goal is the creation of an open source platform for the future internet. Webinos uses the concept of Personal Zones, which allows communication between services and devices. Personal zones are divided into two parts: i) PZH (Personal Zone Hub) and ii) PZP (Personal Zone Proxy). A PZH possesses a public IP address and runs in the cloud [17]. The PZP is a device that is able to run Webinos services. Tago, and ARTIK Cloud are Platforms that are only available in the cloud. The main strength of cloud platforms is that the examples are simple to execute and visualize through a graphical interface. The disadvantage is that most platforms that are only available in the cloud are paid.

The main goal of IoT platforms is to simplify the interactions between users and devices, allowing everyone to focus on what is important. In theory, all presented solutions are scalable, interoperable, are able to persist and analyze data, generate events, and provide a certain degree of security. However, they are not able to understand context or interpret data semantically. This means that the difference between these three platforms (and most platforms) is minimal and is provided in Table I, which shows the supported protocols and the number of updates in the year 2017 (till July)6, 7, 8. Regarding the number of updates, more than five (> 5) is used, because it indicates that a platform was updated at least once every 2 months (in a year), it is the view of

this paper that it is a reasonable number. After the qualitative comparison, Webinos is excluded due to the lack of updates in PZP and PZH modules (last update was in 2015), and a quantitative comparison is demonstrated in Table II, where the packet size of an HTTPS POST and GET methods is analyzed using a software called Fiddler. In both POST and GET methods, the sent and received bytes are analyzed because RESTFUL methods generally end with a reply code. Tago is slightly more efficient than ARTIK Cloud regarding packet size.

Table I. QUALITATIVE COMPARISON.

Platform Protocols Updates in 2017 Webinos HTTP(S), Websocket 0

Tago HTTP(S), MQTT > 5

Artik Cloud HTTPS, Websocket, MQTT, CoAP > 5

Table II. QUANTITATIVE COMPARISON.

Platform Post (Bytes) Get (Bytes)

Sent Received Sent Received

Tago 182 183 182 183

Artik Cloud 190 183 190 183

6 - https://github.com/webinos

7 - https://api.tago.io/changelog 8 - https://developer.artik.cloud/documentation/changelog.html

VI. OPEN ISSUES AND RESEARCH OPPORTUNITIES A platform can significantly impact the performance of an

IoT solution. Currently, many middleware solutions are available in the market and comparing all of them consumes a significant amount of time. Qualitative metrics can be used as pre-filter to shorten the list, but this type of metrics is subjective in nature. For this reason, it is advisable to detail as much as possible why an individual platform was considered better or worse than another. Comparing cloud platforms is further complicated, namely because it is hard to directly access the servers and, besides that, cloud platforms generally limit aspects such as the number of devices in their free service, and charge for monthly requests, analysis, records stored, and sent e-mails. For this reason, large-scale experiments might be costly and difficult to execute without proper authorization from the service providers.

Security and privacy are a big issue in IoT as most devices are unsecured. This aspect is a key issue and raises many questions and endangers the future of IoT. IoT solutions providers should dedicate more resources towards security. Privacy issues should also be considered. A significant amount of Facebook and Google revenue comes from collecting browsing history on their websites and selling to advertisers (it is consented by the user in the service agreement), but it is not clear what data

71

they indeed collect. This problem is more serious when using VoiceLabs, devices that are always listening [20], such as the Amazon Alexa and Google assistant. There is no way to know what data this type of devices collects. Do they just start collecting data when specific words are said or are they periodically doing it? An IoT platform escalates the risks even more with the amount of collected data. Governments should create regulation related to privacy for IoT devices and platforms; even mainstream Internet connectivity methods such as 3G, 4G (and soon 5G), and IoT alternatives (LoRa and Sigfox) should be dealt with.

VII. CONCLUSION The Internet of Things (IoT) priorities are different in

comparison to the regular Internet (as it is known today), as they reflect how IoT devices connect to the Internet and exchange data. By definition, IoT is smart and most of its devices are constrained in terms of computational, storage, energy, and processing resources. This means that its intelligence does not come from the devices themselves, but from an external and powerful controller. This controller is a piece of software identified as IoT middleware (also known as IoT middleware platform or, simply, platform) that allows interoperability between devices, stores and retrieves data, understands the context, generates events, and perform analytics. Choosing the right platform for a particular scenario can be the difference between a good and bad IoT solution.

Choosing the right IoT middleware platform is important for a successful IoT solution and the market is currently overloaded with many available solutions. Therefore, in order to identify the best criteria to choose an IoT middleware for a given IoT scenario, this paper analyses the most relevant IoT requirements and proposed a set of qualitative and quantitative metrics to evaluate their performance.

Future works should focus on narrowing the list of available platforms using the metrics proposed in this paper. This can be achieved by filtering all the available platforms through qualitative metrics, followed by an objective ranking using quantitative metrics.

ACKNOWLEDGMENTS This work was supported by National Funding from the FCT

- Fundação para a Ciência e a Tecnologia through the UID/EEA/50008/2013 Project, by Finatel through the Inatel Smart Campus project, and by Finep, with resources from Funttel, Grant No. 01.14.0231.00, under the Centro de Referência em Radiocomunicações - CRR project of the Instituto Nacional de Telecomunicações (Inatel), Brazil.

REFERENCES [1] D. Evans, “The Internet of Things - How the Next Evolution of the Internet

is Changing Everything,” Cisco white paper, pp. 1-11, April 2011.

[2] International Telecommunication Union, “Recommendation ITU-T Y.2060: Overview of the Internet of things,” p. 1-22, June 2012.

[3] United Nations Department of Economic and Social Affairs, “World Population Prospects: The 2015 Revision, Key Findings and Advance Tables,” pp. 1–59, 2015.

[4] A. Tiwana, Platform Ecosystems Aligning Architecture, Governance, and Strategy, Elsevier, 2013.

[5] R. K. Ghosh, Wireless Networking and Mobile Data Management. Springer, Singapure, 2017.

[6] M. Centenaro, L. Vangelista, A. Zanella, and M. Zorzi, “Long-Range Communications in Unlicensed Bands: the Rising Stars in the IoT and Smart City Scenarios,” IEEE Wireless Communications, vol. 23, no. 5, pp. 60-67, Oct. 2016.

[7] J. G. Andrews et al., “What Will 5G Be?,” IEEE Journal on Selected Areas in Communications, vol. 32, no. 6, pp. 1065–1082, June 2014.

[8] S. Bandyopadhyay and A. Bhattacharyya, “Lightweight Internet protocols for web enablement of sensors using constrained gateway devices,” International Conference on Computing, Networking and Communications(ICNC 2013), San Diego, CA, USA, Jan. 28-31, 2013, pp. 334–340.

[9] D. Thangavel, X. Ma, A. Valera, H.-X. Tan, and C. K.-Y. Tan, “Performance evaluation of MQTT and CoAP via a common middleware,” IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP 2014), Singapore, April 21-24, 2014, pp. 1–6.

[10] H. Lampesberger, “Technologies for Web and cloud service interaction: a survey,” Service Oriented Computing and Applications, vol. 10, no. 2, pp. 71-110, June 2016.

[11] M. A. Razzaque, M. Milojevic-Jevric, A. Palade, and S. Cla, “Middleware for internet of things: A survey,” IEEE Internet of Things Journal, vol. 3, no. 1, pp. 70–95, Feb. 2016.

[12] G. Fersi, “Middleware for internet of things: A study,” IEEE International Conference on Distributed Computing in Sensor Systems (DCOSS 2015), Fortaleza, Brazil, June 10-12, 2015, pp. 230–235.

[13] A. H. H. Ngu, M. Gutierrez, V. Metsis, S. Nepal, and M. Z. Sheng, “IoT Middleware: A Survey on Issues and Enabling technologies,” IEEE Internet of Things Journal, vol. 4, no. 1, pp. 1–20, February 2017.

[14] L. Atzori, A. Iera, and G. Morabito, “The Internet of Things: A survey,” Computer Networks, vol. 54, no. 15, pp. 2787–2805, October 2010.

[15] M. A. Chaqfeh and N. Mohamed, “Challenges in middleware solutions for the internet of things,” International Conference on Collaboration Technologies and Systems (CTS 2012), Denver, Colorado, USA, May 21-25, 2012, pp. 21–26.

[16] M. Kim, A. Mohindra, V. Muthusamy, R. Ranchal, V. Salapura, A. Slominski, R. Khalaf, “Building scalable, secure, multi-tenant cloud services on IBM Bluemix,” IBM Journal of Research and Development, vol. 60, no. 2–3, pp. 8:1-8:12, March 2016.

[17] D3.7: Final webinos specification. [Online]. Available: http://webinos.org/files/2012/09/webinos-phase_II_architecture_and_components.pdf. [Accessed: 11- Jul- 2017].

[18] Tago.io. [Online]. Available: https://tago.io/. [Accessed: 11- Jul- 2017]. [19] IoT Cloud Platform - Samsung ARTIK Cloud. [Online]. Available:

https://artik.cloud/. [Accessed: 11- Jul- 2017]. [20] P. Dempsey, “The Teardown: Google Home personal assistant,”

Engineering and Technology, vol. 12, no. 3, pp. 80–81, April 2017.

72