Tom Zastrow, 7th December 2017

A Backend for Sensor Data

Tom Zastrow, 7th December 2017

Overview

• Hardware– Sensors, Arduino & Raspberry Pi integration

• Software– Sending data around

– Storing data

– Access, analysis and visualization

Tom Zastrow, 7th December 2017

Hardware

Dust

MQ2

MQ135

DHT11

Temp.

Humidity

BMP180

Temp.

Pressure

Tom Zastrow, 7th December 2017

GUIs

Data storage

Sensors

Arduino

Raspberry Pi

RESTstyle

webservice

Influx DB

Yii based

web interface

USB Serial

InfluxDB

line

format

RESTstyle calls

Chronograf,

Grafana, ...

Any other

sensor data

Json via

Internet

Ard

uin

o C

Pyth

on

PHP,

JavaScript

& InfluxDB

SQL

dialect

Tom Zastrow, 7th December 2017

Ingest: Data Flow

RESTstyle

webservice

Influx DB

Json via

Internet

InfluxDB

line

format

{"esp8266id": "tomsensors", "software_version": "arduino2.py",

"sensordatavalues":[{"value_type":"dust",

"value":"736.41"},{"value_type":"mq2",

"value":"356.0"},{"value_type":"mq135",

"value":"247.0"},{"value_type":"DHT11temperature",

"value":"19.0"},{"value_type":"DHT11humidity",

"value":"19.0"},{"value_type":"BMP180pressure",

"value":"947.66"},{"value_type":"BMP180temperature",

"value":"20.4"},{"value_type":"BMP180altitude", "value":"560.98"}]}

sidtomsensors

dust=736.41,mq2=356.0,mq135=247.0,DHT11temperature=19.0,DHT11humidity=19.0,B

MP180pressure=947.66,BMP180temperature=20.4,BMP180altitude=560.98

1512053085000000000

Tom Zastrow, 7th December 2017

InfluxDB

• NoSQL database specialized on time series

• Part of bigger software stack (including Chronograf)

• SQL like query syntax:

select max(BMP180pressure), min(BMP180pressure) from sidtomsensors; name: sidtomsensors time max min ---- --- --- 1970-01-01T00:00:00Z 949.35 946.68

curl -G "http://localhost:8086/query" --data-urlencode "q=select max(BMP180pressure),min(BMP180pressure) from sidtomsensors" --data-urlencode "db=feinstaub"

{"results":[{"statement_id":0,"series":[{"name":"sidtomsensors","columns":["time","max","min"],"values":[["1970-01-01T00:00:00Z",949.35,946.68]]}]}]}

Tom Zastrow, 7th December 2017

GUI

• Build with PHP based framework Yii

• Similar to Django

• Bootstrap already on board

• User management

• Forms

• Diagrams & charts

• Current features:– CSV import from Madavi server

– Generic query execution (inc. CSV export)

– Geo visualization done with JLeaflet JavaScript library

– Chart & graph visualizations

Tom Zastrow, 7th December 2017

Demo!

Tom Zastrow, 7th December 2017

Thanks!

Tom Zastrow, 7th December 2017

Some more things ...

Tom Zastrow, 7th December 2017

Tools, libraries, frameworks, ...

BagIt: File Packaging Formathttps://tools.ietf.org/html/draft-kunze-bagit-14

Swagger: The Worlds most popular API Toolinghttps://swagger.io/

Yii: The Fast, Secure and Professional PHP Frameworkhttp://www.yiiframework.com/

Colorbrewer2: Color advice for cartographyhttp://colorbrewer2.org/

PyCharm: Python IDEhttps://www.jetbrains.com/pycharm/

JLeaflet: JavaScript Library for interactive mapshttp://leafletjs.com/

JsonAPI: A specification for building APIs in Jsonhttp://jsonapi.org/

Slack: Online collaborationhttp://www.slack.com

Orange: Data Mining Fruitful and Funhttps://orange.biolab.si

InfluxDB: Time Series DBhttps://www.influxdata.com/

Affinity Photohttps://affinity.serif.com/de/photo/

Sensor stuffArduino & Raspi,...

Tom Zastrow, 7th December 2017

Orange: Data Mining Fruitful and Fun

• Data analytics

• Data visualization

• Machine learning

„Open source machine

learning and data

visualization for novice

and expert. Interactive

data analysis workflows

with a large toolbox.“

https://orange.biolab.si/

Tom Zastrow, 7th December 2017

Features

Interactive Data Visualization Visual Programming

Add-ons Extend Functionality

Teachers and Students Love It