The Three Musketeers, IoT, Cloud, and Big Data: One for All, and All for One

Daeyoung Kim

Director of Auto-ID Lab at KAIST

Professor, Computer Science Department, KAIST, Korea kimd@kaist.ac.kr http://resl.kaist.ac.kr http://autoidlabs.org

http://autoidlab.kaist.ac.kr http://oliot.org

Future Internet ?

Future Internet = Internet of Things

Internet of Things – Which is right direction?

Internet of things and us, Auto-ID Labs

In 1999, the Internet of Things" was first coined by Kevin Ashton who cofounded the Auto-ID Center at the MIT

Business Processes and Applications

Software and Network

Hardware The image cannot be displayed. Your

The image cannot be displayed. Your

GS1

GS1 - Essentials

GS1 – Who do we represent?

Regulation EU No 1169/2011 Food Information Regulation

http://europa.eu/legislation_summaries/consumers/product_labelling_and_packaging/co0019_en.htm

GS1 – UDI Code for Medical Devices

http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/UniqueDeviceIdentification/default.htm?utm_source=Members-Only%20Updates

GS1 Source

GS1 Digital

What is Things? (no limitation)

수동형 사물 액티브 태그형 사물 무선센서형 사물

스마트형 사물

스마트가전형 사물

지식데이터 사물

Internet of Things Vision / Architecture

Future IT Eco-System

H u m a n In f o rm a t io n

T h in g In f o rm a t io n

C o n te x t -aw a re S e rv ice s

Social Network Service

M2M networking of Home Devices

Internet Browsing

Entertainment(Multimedia & Game)

Home Surveillance

Augmented Reality

B ig D a taA n a ly sis

C lo u d C o m p u t in g

In te rn e t o f T h in g s

Cloud Computing

Inte rne t o f Thing s

User log, Business activity logs, etc.

Internet contents, SNS, etc.

Everyday objects Multimedia (video, audio), etc.

S o c ia l N e tw o r k S e rv ice f o r Io T

One for All, and All for One

Internet of Things Research

Based on GS1 Philosophy and Technology at Auto-ID Lab KAIST

Oliot, Snail, Seahaven, Lilliput

IoTMaaS, (GPUCLOUD, CRADHN)

Dr. M Project, (Food SCM and Safety System Project, SoC Management System Project)

Oliot (Open Language for Internet of Things)

Oliot (Open Language for Internet of Things)

•  GS1 EPCglobal Network on the Cloud for Groceries Trace Framework since 2013

–  Complete Implementation of GS1 EPCglobal Network

–  EPCIS Enhanced with NOSQL DB –  Cloud Support –  ELFIN: Enhanced LLRP- enabling Framework for the INternet of Things –  Geo-discovery Service –  ONS 2.0.1 implemented

•  Smart Thing Information Service (STIS) since 2005, 2011

–  Expand GS1 EPCglobal Network to integrate various sensor network protocols –  Successor of EPC Sensor Network –  ZigBee, 6LoWPAN/CoAP integration –  Obix integration, etc.

RFID Reader(e.g., passive

tags)

Sensor Network (e.g., smartphone

phone)

Wireless ID and Sensor networks (e.g., active tags)

RFID Filter & Collection Middleware

Sensor F&C&CEP middleware

LLRP LLRP Sensor Interface Sensor Interface

Actuator Networks (e.g., consumer

electronics, appliance)

Sensor & Actuator Networks

Actuation interface

Object orchestration and organization management

Domain-specific applications

Sensor Interface

Sensor Interface

Smart Things Information Service (static and dynamic information)

ALE

Actuation Interface

ObjectObject

Actuation Interface

ONS

Geo-distance Discovery Services

Federated Systems

Decentralized System

s

Oliot Open Source Project (release on July 7, 2014)

Document-based NoSQL DB for Geo-distance and text search, analytics, etc.

For time-series data with frequent insertion

Relational DB for structed MasterData

Oracle Spatial and Graph Graph database organization relationship management

IoT Connectivity – SNAIL(Sensor Networks for All IP World) Project Since 2007

Internet of Things

SNAIL Border Router (6LBR)

SNAIL Node (6LN)

SNAIL Node (6LN)

SNAIL Node (6LN)

SNAIL Node (6LN)

IEEE 802.15.4Btle

IEEE 802.15.4Btle

Entertainment & Social Net. ServiceDevice Browsing & Mashup

Big Data AnalysisUser Experience with IoT Service

•  SNAIL (Sensor Networks for an All-IP worLd) •  an IP-based Wireless Sensor Networks platform •  Supported Protocols

•  Interoperability between IPv4/v6 domains and the IEEE 802.15.4 •  Lightweight IPv6, ICMPv6, MIPv6, NEMO, UDP, TCP, SSL •  Dual-Mode gateway for WiFi AP and IP-WSN edge router •  CoAP, HTML5, Web browsing (HTTP/TCP) •  Mesh routing in adaptation layer, Addressing •  DTLS/BLE ongoing

IoT Connectivity – SNAIL(Sensor Networks for All IP World) Project Since 2007

IoT Connectivity – SNAIL(Sensor Networks for All IP World) Project Since 2007

저전력 블루투스 기반 초경량 IP 네트워크 프로토콜 기술 개발

•  6LoWPAN 표준 기반 초경량 / 저전력 IPv6 over Bluetooth low energy 프로토콜 개발

§  안드로이드 기반 6LoWPAN 게이트웨이 소프트웨어 모듈 개발

§  저전력 블루투스 기반 IP 네트워킹을 통하여 다양한 헬스케어 서비스 구현을 위한 6LoWPAN-ble API 개발

Internet

SmartphoneMedicine-care device

Battery-less Bio Sensor

Blood Pressure Sensor  

Smart Glass Device

Healthcare Watch

Cloud Computing

Overload Internet

SmartphoneMedicine-care device

Battery-less Bio Sensor

Blood Pressure Sensor  

Smart Glass Device

Healthcare Watch

Cloud Computing

Web-based Visualization/WOT PAAS

•  Smart devices and consumer electronics are equipped with web/CoAP servers that can response

directly to requests from the Internet •  Presentation Cloud provides rich web contents to s

upport those embedded web servers •  Sensing data and Actuation commands/results ar

e retrieved directly from web browser and display on top of rich web interface, either in numbers or in graphs (HTML5)

Nam Giang, Minkeun Ha, and Daeyoung Kim, "Cross Domain Communication in the Web of Things, A New Context for the Old Problem," The 23rd International World Wide Web Conference (WWW) 2014, Demo Session, Seoul, S. Korea, Apr. 2014.

No app. Download Any medical devices and Sensors either old or new

Seahaven - Visual Sensor Networks Platform

Seahaven - Visual Sensor Networks Platform

IoTMaaS – IoT Mashup as a Service Cloud

•  IoT Mashup as a Service Cloud •  사물 매쉬업 서비스 모델을 기반으로 런타임 IOT 서비스 생성을 지원하는 클라우드 기반 플랫폼 •  사물 매쉬업 서비스 모델: 사물, 소프트웨어, 컴퓨팅 리소스의 조합으로 이루어진 서비스 모델로서 엔드 유저가 런타임에 세 가지 요소를 선택하여 커스터마이징이 가능함

•  엔드 유저, 매쉬업 서비스 디자이너, 클라우드 제공자, 소프트웨어 개발자, 사물 소유자 등으로 이루어진 에코시스템

Low Traffic

Thing Selection

Processing Logic

Selection

Computation Resource Selection

Janggwan Im, Seong Hoon Kim, and Daeyoung Kim, "IoT Mashup as a Service: Cloud-based Mashup Service for the Internet of Things," 10th IEEE International Conference on Services Computing (SCC 2013), CA, USA, Jun. 2013.

Lilliput – IoT Social Network Service

Two  Space  Reflection  Manager

Online  Social  Networks

Io TS oc i a l G rap h

Lilliput  APIs

Lillipu

t  Architecture

IoTSocial Model

IoT  Platforms

ApplicationApplicationApplication

Real  world  contextMetadata,  etc

Smart  Thing

Personal  profilessocial  relationships,  etc

Web  Service  Interface          

Graph  BuilderElement  Extractor

Querying  Manager

Access  Controller

Thing  ServiceInvoker

Service  Invocatione.g.  Air-­‐conditioning

Publish ing  messageUpdating  relationships

Graph  Utilization  Manager

Security  Manager

Lilliput  Reflection  Manager

OSN  Publisher

IoT-­‐Social  Graph  Manager

ModificationManager

xAcceptable  Request Unacceptable  Request

Reasoning  Manager

Rules

Smart  ThingService  Manager

External  Com

ponents

Entity  Manager

Relationship  Manager

Sync.  Manager

Query  Engine

Reasoning  Engine

Notification  Receiver

Chg.  Notif.  Manager

•  Lilliput – IOT SNS •  Sorcerer’s Book – IoT SNS Application, Lilliput API를 이용한 응용 서비스

Lilliput Architecture

Sorcerer’s Book

Dr. M Design Concept

Health Monitoring Medical Assistance

D rM D a ta b a se

2 0 1 32 0 1 22 0 1 1

R e a l- t im e M o n it o r in g D a ta

H ist o r ica l D a ta

D rM D a ta b a se

B io O p t ic S e n so r

B io O p t ic S e n so r

HealthcareWatch

HealthcareWatch

E E G b io t e lem e try

Blood Pressure

Blood Pressure

st ick -o nH e a rt R a te S e n so r

V iru s M o n ito rin gV iru s

M o n ito rin g

F o o t S e n so rF o o t Se n so r

Smart SensorsSmart Sensors

E C G S e n so rE C G S e n so r

E E G b io t e lem e try

s t ick -o nH e a rt R a te S e n so r

M a c h in e L e a r n in gM a c h in e L e a r n in gB ig A n a ly t icsB ig A n a ly t ics

P re d ic tio n

D ise a se k n o w le d g e

(1) 일반인/환자 헬스 모니터링/원격검진 (2) 병원/의사 의료 지원

생체신호 센싱

실시간 분석

질병분석 및 예측

의료 지식 발견

KAIST 전산/전자/산공/지식/산디 28명 교수진 참여

Dr. M Platform Overview

D r . M S e rv ic e P la t f o rm

B io H e a lth ca re /M e d ica l S e n so r

T C P /IP su ite

B lu e to o th L o w E n e r g y Z ig B e e W iF i 6 L o W PA N

S o c ia l N e tw o rk S e rv ice p la t f o rm

S o c ia l N e tw o rk S e rv ice p la t f o rm

C o m m u n ica t io n T e ch n o lo g y

In t e r n e tW B A NW PA N

W LA N /W A N

B ig D a ta

D a ta P r o ce ss in g

D a ta F u st io n

D a ta M in in g

C lo u d C o m p u t in g

In t e r n e t

Ia a S P a a S S a a S

V irtu a liz a t io n

C lu s te r e d N e two rk s

C lo u d / Io T

Privacy / Security

P a t ie n t M o n ito rin g

U se r In te ra c t io n / U se r in te r f a ce

R em o te M e d ica l- ca r eU se r

E x p e rie n ce

D r .M T e stb e d

M a ch in e L e a rn in g

P r e d ic t iv e A n a ly t ics E n g in e

H ig h -D im D a ta V isu a liz a t io n

P a tte r n m in in g

B ig D a ta /M a c h in e L e a rn in g

Io T

T h in g re so lu t io n

D isco v e ry Se r v ice

F ilte r in g / C a p tu r in g

K n o w le d g e B a se

Dr. M Fitness

Dr. M

Cloud Computing

E C G S e n so rE C G S e n so r

E E G b io t e lem e try

E E G b io t e le m e try

HealthcareWatch

HealthcareWatch

Blood Pressure

Blood Pressure

G P S S a t e lliteG P S

S a t e llite

G P S T ra ck in gG P S

T ra ck in gH e a lt h

M o n ito r in gH e a lt h

M o n ito r in gG la ss- ty p e D isp la yG la ss- ty p e D isp la y

다양한 카이스트 센서 및 UX 활용 머신러닝/빅데이터 처리 결과 실시간 피드백 활용

Dr. M EagleEye

K A IS T_s tu 1 K imK A IS T_s tu 1 K im

N am e : K A IS T_s tu 1 K imD e p t . o f C SH e ar t R at e : 2 3S p e e d : 1 m /s

C C T V 2P re d ic te d D ise a se- H e a rt A t t a ck- S ym p to m s . D is c o m f o r t , p re ss u r e , h e av in e s s, o r p a i n i n t h e c h es t . D is c o m f o r t ra d i a t in g t o t h e b ac k . R ap id o r i rr e g u l a r h e a rt b e a ts

N e a r e s t H o sp i ta ls- 튁됴랬＿널셴쨉 (N o t if i e d )- 꼈혹뵘랬＿널 랬첫앞뺌셴쨉- 쫴캐랬＿셴쨉- 랬첫앍셴쨉

C C T V 1

M u lt i- V isio n D isp la y

E E G b io t e le m e try

E E G b io t e le m e try E C G S e n so rE C G S e n so r

Cloud Computing

F itb it se n so rF itb it se n so r

G P S S a te lli teG P S S a te lli te

M u lt i- V isio n D isp la y

K A IS T C lin ic

C C T VC C T V

M a c h in e L e a r n in gM a c h in e L e a r n in gB ig A n a ly t icsB ig A n a ly t ics

캠퍼스내 학생(원격모니터링)에 웨어러블 디바이스 지급 실시간 위치/생체신호 모니터링/빅데이터화 심장질환등 긴급 상황 경보

Dr. M Patient Browsing/Operations Archiving/Dr. Assistance

fitbitfitbit

E E G b io t e le m e try

s t ick -o nH e a rt R a te S e n so r

Smart SensorsSmart SensorsE C G S e n so rE C G S e n so r

E E G b io t e le m e try

s t ick -o nH e a rt R a te S e n so r

6 L o W P A N -b le

Cloud

App

B

uild

er/D

eplo

yer

Services

ThingDNS

ThingID

ThingProxy

ThingSocial

WoT Apps

JS CSS HTML

Images, audio, video

AppInit

M a c h in e L e a r n in g

P r e d i ct iv e A n a l y t ic s E n g in e

H i g h -D im D a t a V is u a liz a ti o n

P a t te rn m in in g

Big Data AnalysisBig Data Analysis Machine Learning / Big Data Analysis

Distributed Database

Cloud Computing Platform

N am e : S am u e lB lo o d T y p e : AA g e : 6 4D o c t o r : D r . MS ym p to m : H y p e r te n s io n

V isu a liz a t io n

P re d ic te d D ise a se- H e a rt A t t a ck- S ym p to m s . D is c o m f o r t , p re ss u r e , h e av in e s s, o r p a i n i n t h e c h es t . D is c o m f o r t ra d i a t in g t o t h e b ac k . R ap id o r i rr e g u l a r h e a rt b e a ts

A m b u la n c e H o sp ita l R o o mA m b u la n c e H o sp ita l R o o m

M o b ileG a t e w a y

S ta t icG a t e w a y

M o b ileG a t e w a y

In t e r n e tIn t e r n e t

환자 착용 웨어러블 디바이스 생체신호+추가정보+응용 포함한 클라우드 기반 Browsing