1. IoT, IN4.0 & Co. DAMIR DOBRIC damir.dobric@daenet.com b-dadobr@microsoft.com Microsoft PTSP (Partner Technical Solution Specialist) Microsoft Most Valuable Professional Blog Twitter CloudAppsCloud CloudConsultingCloudAppsMiddlewareCloudAppsMiddlewareSoftware 2. Big Data & Machine Learning Olivia Klose http://oliviaklose.com @oliviaklose olivia.klose@microsoft.com Technical Evangelist Microsoft Deutschland GmbH 3. IoT TechCamp Olivia Klose Damir Dobric Microsoft daenet 4. AGENDA Things NETMF (Netduino)/ Gadgeteer (FEZ Spider) WinRT (SBSDK Managed) Win10 UWP (Pi2) Arduino Yun Services Connectivity Approaches Service Bus Queues/Topics/EventHub Azure Stream Analytics Machine LearningFrameworks/SDKs EventHub WORKSHOP 5. IN4.0 - Value Added 2013-2015 Mechanical Engineering & Construction (EUR +23 Mrd.) Kraftwagenteile (EUR +15 Mrd.), Information & Communication Technology (EUR +14 Mrd.), Electrical Industry (EUR +12 Mrd.) Chemical Industry (EUR +12 Mrd.) Land- und Forstwirtschaft (EUR +3 Mrd.). Source: Deutsche Bank Research: http://www.dbresearch.de/PROD/DBR_INTERNET_DEPROD/PROD0000000000335241.pdf 6. OEM Revenue Opportunity | Market Forecast CY17 7. 2003201020152020 50* Milliarden Things 8. Home Transportation Health Buildings Cities 9. berwacht Gesundheit Nimmt Herzinfakt wahr Ziel Startet Klimaanlage Reserviert Arbeitsplatz Sendet Notdienst bermittelt Frhdiagnose an Krankenhaus Informiert Notdienst ber optimale Route Started Reinigung Verringert Temperatur Verfgbare Parkpltze Position Home Transportation Health Buildings Cities 10. Wartungs- zeiten verringern Zeit sparen Verschiedene Systeme Integrieren Entscheidungs findung optimieren Prozesse auto- matisieren Unternehmerische Probleme mit IoT lsen 11. In Focus Self organizing Industry Smart & Connected Factory Intelligent Products Internet of Things Driven by Advanced ICT Communication between products & plants 12. Why now? Huge number of connected devices Devices produces data (big-data) We can build information from big-data High Level Software in Microcontrollers Lower development costs Huge developers community (almost) Any solution/app can compatibly participate New connectivity approaches Web provides out of the box connectivity Cloud platform provides new possibilities 13. What do we think about IoT? USB, TCP, UDP, Direct WiFi, MQTT, HTTP,.. This is not a new approach?! Not really an issue?! This thinking is about things but not about internet of things!! There is no internet of things without Internet!! 14. AllJoyn IoT Device Interoperability AllJoyn http://channel9.msdn.com/Events/Build/2015/2-623 15. Consumer (client) Discovers and connects to one or more producers. Controls thing and consumes data from thing. AllJoyn Software Roles Producer (server) AllJoyn SDK Source Code:https://allseenalliance.org/developers/download 16. High-Level software for small devices FEZ Spider (NET MF) Raspberry Pi (.NET/UAP) SOM Win10/Pi 17. DEMOS 18. DEMO Gadgeteer Board with Microcontroller .NET Microframework Change light color with Joystick 19. Demo: Windows UWP on Pi2 Board with Microprocessor and operative system (Windows10) Universal Windows Platform (UWP) Control GPIO in C# 20. Molecule https://github.com/MoleculeDotNet/Molecule s 21. Arduino Uno 22. Advanced: Arduino Yun 23. Arduino Yun 24. Arduino Yun 25. Arduino Yun 26. Full board: Raspberry Pi 2 27. Windows 10 on Raspberry Pi 2 Setup: http://www.hanselman.com/blog/SettingUpWin dows10ForIoTOnYourRaspberryPi2.aspx Windows 10 Machine Visual Studio 2015 Preview Microsoft IOT SDK 28. IoT Frameworks and SDKs Azure Service Bus Client https://www.nuget.org/packages/WindowsAzure.ServiceBus Service Bus JavaScript SDK https://www.nuget.org/packages/SbJsSdk/ https://www.nuget.org/packages/SbJsSdkSample/ AMQP Lite https://amqpnetlite.codeplex.com/ Iot Open SDK (work in progress) http://daenet.eu/share/notforpublic.zip Device Registry + SAS Token Service with OAuth 29. Connecting Things ? 30. COMMON CONNECTIVITY APPROACHES DO NOT WORK WITH THINGS IN INTERNET! 31. Solution: Service Assisted Communication SERVICE BUS No need for infrastructure. Use existing cloud platform. Bus talks many protocols (AMQP, MQTT, HTTP/S,..) No need for Public Addresses. No need for VPN More secured No any device is listening. You never talk directly to device. Only Service Bus knows how to reach device. Huge Scale 32. IoT Demos Desktop Java Script NETMF WinRT 33. Scan code and send messages http://microsoft-apiapp21b5a352204646f082e10baceabab6e9.azurewebsites.net 34. Event Hubs 35. How big is Bigdata? 36. Get speed with partitioning Model that removes locks Scales out with partitions (no scale up) Logical opposite of Competing Consumer ( = traditional queueing model) 37. EventHub Example 38. EventProcessorHost 39. Multiple Workers 40. Partitions and TUs 41. How much can we scale? 42. EventHub Demos Ingress Data Egress with single host Egress with multiple host (high scale) 43. AZURE STREAM ANALYTICS 44. What is Streaming Data? 45. Stream Analytics Pattern 46. Real-time analytics Real-time Analytics Intake millions of events per second (up to 1 GB/s) Low processing latency, auto adaptive (sub-second to seconds) Correlate between different streams, or with reference data Find patterns or lack of patterns in data in real-time Fully Managed Cloud Service No hardware acquisition and maintenance No platform/infrastructure deployment and maintenance Easily expand your business globally leveraging Azure regions 47. SELECT FROM WHERE GROUP BY HAVING CASE WHEN THEN ELSE INNER/LEFT OUTER JOIN UNION CROSS/OUTER APPLY CAST INTO ORDER BY ASC, DSC SAQL Language & Library WITH PARTITION B OVER DateName DatePart Day Month Year DateTimeFromParts DateDiff DateAdd TumblingWindow HoppingWindow SlidingWindow Sum Count Avg Min Max StDev StDevP Var VarP Len Concat CharIndex Substring PatIndex Lag IsFirst CollectTop 48. Scenario Twitter Analytics ID CreatedAt UserName TimeZone Text Language Topic 1 2015-04-30T20:45:30 Joshua X Eastern Time (US & Canada) Oh, joy! More forced @Xbox Live updates en XBox 2 2015-04-30T20:45:31 Cristabel Y London RT @verge: Streaming Xbox One games .. en XBox A news media website wants to increase site traffic by covering trending topics on social media. To determine which topics are immediately relevant to customers, they need real-time analytics about the tweet volume and sentiment for each topic. TwitterStream 49. Filters SELECT UserName, TimeZone FROM InputStream WHERE Topic = 'XBox' Show me the user name and time zone of tweets on the topic XBox "Haroon, Eastern Time (US & Canada) "XO", London Zach Dotseth, London, Football,() "Haroon, Eastern Time (US & Canada) XBox,() "XO",London, XBox, () 50. Windowing Concepts Windows can be tumbling, hopping, or sliding Windows are fixed length Must be used in a GROUP BY clause Output event will have the timestamp of the end of the window 1 5 4 26 8 6 4 t1 t2 t5 t6t3 t4 Time Window1 Window2 Window3 Aggregate Function(Sum) 18 14OutputEvents 51. Hopping Windows SELECT Topic, Count(*) AS TotalTweets FROM TwitterStream TIMESTAMP BY CreatedAt GROUP BY Topic, HoppingWindow(second, 10, 5) Every 5 seconds give me the count of tweets over the last 10 seconds 1 5 4 26 8 6 A 10-secondHoppingWindowwith a 5-secondHop 4 26 8 6 5 3 6 1 1 5 4 26 8 6 5 3 6 15 3 52. Joining multiple streams {XO, 4, Win10} {Jo, 0, Surface} {Foo,4, Bing}{Dip, 2, XBox} {XO, 0, Win10} {Dip, 0, Xbox}{Jo, 4, Surface} {Foo, 0, Bing}Twitter Stream: (same stream, further down the timeline) SELECT TS1.UserName, TS1.Topic FROM TwitterStream TS1 TIMESTAMP BY CreatedAt JOIN TwitterStream TS2 TIMESTAMP BY CreatedAt ON TS1.UserName = TS2.UserName AND TS1.Topic = TS2.Topic AND DateDiff(second, TS1, TS2) BETWEEN 1 AND 60 WHERE TS1.SentimentScore != TS2.SentimentScore List all users and the topics on which they switched their sentiment within a minute 53. Detecting absence of events Show me if a topic is not tweeted for 10 seconds since it was last tweeted SELECT TS1.CreatedAt, TS1.Topic, TS1.UserName FROM TwitterStream TS1 TIMESTAMP BY CreatedAt LEFT OUTER JOIN TwitterStream TS2 TIMESTAMP BY CreatedAt ON TS1.Topic = TS2.Topic AND DateDiff(second, TS1, TS2) BETWEEN 1 AND 10 WHERE TS2.Topic IS NULL {XO, 4, Win10} {WAA, 2, Microsoft} {AB, 0, Bing}{Dip, 4, Xbox} {Foo, 0, Win10} {Tim, 2, Microsoft} {AB, 0, Bing}Twitter Stream: (same stream, further down the timeline) 54. Reference Data Seamless correlation of event streams with reference data Static or slowly-changing data stored in blobs CSV and JSON files in Azure Blobs; scanned for new snapshots on a settable cadence JOIN (INNER or LEFT OUTER) between streams and reference data sources Reference data appears like another input:SELECT myRefData.Name, myStream.Value FROM myStream JOIN myRefData ON myStream.myKey = myRefData.myKey 55. Scaling using Partitions Partitioning allows for parallel execution over scaled-out resources SELECT Count(*) AS Count, Topic FROM TwitterStream PARTITION BY PartitionId GROUP BY TumblingWindow(minute, 3), Topic, PartitionId Query Result1 Query Result2 Query Result3 Event Hub 56. Multiple steps, multiple outputs WITH Step1 AS ( SELECT Count(*) AS CountTweets, Topic FROM TwitterStream PARTITION BY PartitionId GROUP BY TumblingWindow(second, 3), Topic, PartitionId ), Step2 AS ( SELECT Avg(CountTweets) FROM Step1 GROUP BY TumblingWindow(minute, 3) ) SELECT * INTO Output1 FROM Step1 SELECT * INTO Output2 FROM Step2 SELECT * INTO Output3 FROM Step2 A querycanhavemultiplesteps toenablepipelineexecution A stepis a sub-querydefined usingWITH(commontable expression) Canbe usedto developcomplex queriesmore elegantlyby creatingaintermediarynamed result Createsunitofexecutionfor scalingoutwhen PARTITIONBY is used Eachstepsoutputcanbe sentto multipleoutputtargetsusing INTO 57. AZURE Machine Learning 58. Machine Learning? The goal of machine learning is to program computers to use example data or past experience to solve a given problem. Introduction to Machine Learning, 2nd Edition, MIT Press 59. Azure Machine Learning Machine Learning fr alle Unternehmen, Data Scientists, Entwickler, Analysten und Nutzer berall auf der Welt zugnglich machen 60. Azure Machine Learning HDInsight SQL Server VM SQL DB Blobs & Tabellen Cloud Desktopdateien Exceltabelle Andere Lokal ML Studio IDE fr ML Web Service M Monetarisieren Speicherkonto 61. Good to know http://daenet.eu/share/notforpublicopensdk.zip http://daenet.eu/share/PiSensorKit.zip http://daenet.eu/share/pi2.zip 62. Please give feedback: http://aka.ms/TechCampIoT_0609_fba Landing page for developers in Germany: http://aka.ms/TechWiese Damir Blog (presentation) http://developers.de/ 63. IoT Messaging Stack across platforms Windows (System32) Windows RT Windows Phone WindowsAzure.Messaging.Managed.dll WindowsAzure.ServiceBus.dll WindowsAzure.Messaging.Managed.dll Service Bus Node JS var azure = require('azure'); Java Script Service Bus JavaScript SDK .NET Micro framework Microsoft.ServiceBus.Micro.dll 64. Crossbar.io application router implements WAMP (Web Application Messaging Protocol) open source (with commercial support) used in production for IoT http://crossbar.io/iotcookbook 65. Recap IoT & Microsoft Azure a perfect fit Azure Event Hub Azure Stream Analytics Azure Machine Learning Azure Storage Azure SQL Database End-To-End IoT Solution Microsoft IoT Suite 66. Cheat Sheet Labs https://github.com/dx-ted-emea/iot-labs https://github.com/RobertEichenseer/IoT_Mas terClass https://github.com/dx-ted-emea/azure-tessel https://microsoft.hackster.io http://www.connectthedots.io 67. Cheat Sheet Windows 10 + Raspberry Pi 2 http://www.hanselman.com/blog/SettingUpWin dows10ForIoTOnYourRaspberryPi2.aspx Arduino Yun 101: http://www.arduino.cc/en/Guide/ArduinoYun Crossbar.IO: http://crossbar.io/iotcookbook/ Slide Deck: http://1drv.ms/1Gw6Qp9 Cheat Sheet: http://1drv.ms/1MDLIw1 68. References .NET Gadgeteer Research Home Page http://research.microsoft.com/en-us/projects/gadgeteer/ http://www.netmf.com/gadgeteer/ .NET Gadgeteer http://en.wikipedia.org/wiki/.NET_Gadgeteer .NET Micro Framework http://en.wikipedia.org/wiki/.NET_Micro_Framework#History SbJsSdk http://developers.de/blogs/damir_dobric/archive/2014/08/12/jum pstart-azure-service-bus-javascript-sdk.aspx Gadgeteer custom Module http://10rem.net/blog/2011/10/30/building-a-net-gadgeteer- compatible-hardware-and-software-module-der-blinkenled 69. References Moving to VS vNext: http://informatix.miloush.net/microframework/Articles/VS2013PreviewSDK.aspx Experimental: SDK for VS2013 http://forums.netduino.com/index.php?/topic/10201-experimental-visual-studio- 2013-support/ Modules with Gadgeteer https://www.ghielectronics.com/catalog/category/275 Sockets https://www.ghielectronics.com/catalog/product/471 Windows On Devices (Microsoft Embedded Team) https://dev.windows.com/en-US/iot Azure Service Bus http://azure.microsoft.com/en-gb/services/service-bus/ 70. Q&A DAMIR DOBRIC damir.dobric@daenet.com b-dadobr@microsoft.com Microsoft PTSP (Partner Technical Solution Specialist) Microsoft Most Valuable Professional Blog Twitter CloudAppsCloud CloudConsultingCloudAppsMiddlewareCloudAppsMiddlewareSoftware 71. Azure IoT Suite SDKs Device-facing For devices and field gateway Platforms RTOS (FreeRTOS) Linux (Ubuntu, Debian, Fedora, Raspbian, Angstrom) Windows 7/8/10 ARM mbed Android iOS Languages C, Java, C#, Javascript Service-facing For back-ends and cloud gateway Languages .NET C# 72. IoTHub slides excluded