A Backend for Sensor Data - thomas-zastrow.de · Tom Zastrow, 7th December 2017 InfluxDB • NoSQL...

transcript

Tom Zastrow, 7th December 2017

A Backend for Sensor Data

Overview

• Hardware– Sensors, Arduino & Raspberry Pi integration

• Software– Sending data around

– Storing data

– Access, analysis and visualization

Hardware

Humidity

BMP180

Pressure

Data storage

Sensors

Arduino

Raspberry Pi

RESTstyle

webservice

Influx DB

Yii based

web interface

USB Serial

InfluxDB

format

RESTstyle calls

Chronograf,

Grafana, ...

Any other

sensor data

Json via

Internet

JavaScript

& InfluxDB

dialect

Ingest: Data Flow

RESTstyle

webservice

Influx DB

Json via

Internet

InfluxDB

format

{"esp8266id": "tomsensors", "software_version": "arduino2.py",

"sensordatavalues":[{"value_type":"dust",

"value":"736.41"},{"value_type":"mq2",

"value":"356.0"},{"value_type":"mq135",

"value":"247.0"},{"value_type":"DHT11temperature",

"value":"19.0"},{"value_type":"DHT11humidity",

"value":"19.0"},{"value_type":"BMP180pressure",

"value":"947.66"},{"value_type":"BMP180temperature",

"value":"20.4"},{"value_type":"BMP180altitude", "value":"560.98"}]}

sidtomsensors

dust=736.41,mq2=356.0,mq135=247.0,DHT11temperature=19.0,DHT11humidity=19.0,B

MP180pressure=947.66,BMP180temperature=20.4,BMP180altitude=560.98

1512053085000000000

InfluxDB

• NoSQL database specialized on time series

• Part of bigger software stack (including Chronograf)

• SQL like query syntax:

select max(BMP180pressure), min(BMP180pressure) from sidtomsensors; name: sidtomsensors time max min ---- --- --- 1970-01-01T00:00:00Z 949.35 946.68

curl -G "http://localhost:8086/query" --data-urlencode "q=select max(BMP180pressure),min(BMP180pressure) from sidtomsensors" --data-urlencode "db=feinstaub"

{"results":[{"statement_id":0,"series":[{"name":"sidtomsensors","columns":["time","max","min"],"values":[["1970-01-01T00:00:00Z",949.35,946.68]]}]}]}

• Build with PHP based framework Yii

• Similar to Django

• Bootstrap already on board

• User management

• Forms

• Diagrams & charts

• Current features:– CSV import from Madavi server

– Generic query execution (inc. CSV export)

– Geo visualization done with JLeaflet JavaScript library

– Chart & graph visualizations

Thanks!

Some more things ...

Tools, libraries, frameworks, ...

BagIt: File Packaging Formathttps://tools.ietf.org/html/draft-kunze-bagit-14

Swagger: The Worlds most popular API Toolinghttps://swagger.io/

Yii: The Fast, Secure and Professional PHP Frameworkhttp://www.yiiframework.com/

Colorbrewer2: Color advice for cartographyhttp://colorbrewer2.org/

PyCharm: Python IDEhttps://www.jetbrains.com/pycharm/

JLeaflet: JavaScript Library for interactive mapshttp://leafletjs.com/

JsonAPI: A specification for building APIs in Jsonhttp://jsonapi.org/

Slack: Online collaborationhttp://www.slack.com

Orange: Data Mining Fruitful and Funhttps://orange.biolab.si

InfluxDB: Time Series DBhttps://www.influxdata.com/

Affinity Photohttps://affinity.serif.com/de/photo/

Sensor stuffArduino & Raspi,...

Orange: Data Mining Fruitful and Fun

• Data analytics

• Data visualization

• Machine learning

„Open source machine

learning and data

visualization for novice

and expert. Interactive

data analysis workflows

with a large toolbox.“

https://orange.biolab.si/

Features

Interactive Data Visualization Visual Programming

Add-ons Extend Functionality

Teachers and Students Love It

A Backend for Sensor Data - thomas-zastrow.de · Tom Zastrow, 7th December 2017 InfluxDB • NoSQL...

Documents