Monitoring in a Scalable World

Jacob TomlinsonSenior Solutions Engineer

Agenda

● The Informatics Lab (10 mins)

● What makes a monitoring system ? (10 mins)

● Overview of technologies (10

mins)

● Making it scale (10 mins)

● Questions (10 mins)

Demystify Buzzwords

Prototypes

www.informaticslab.co.uk

What is a monitoring

system?

Logs vs Telemetry

Twelve-Factor AppXI. LogsTreat logs as event streams

12factor.net

Components

Collection

Storage

Aggregation &

Dashboards

Alerting

Monitoring technologies

Managed services

Telegraf cAdvisor FilebeatTelegraf cAdvisor Filebeat

NGINX

Grafana Kibana

InfluxDB ElasticSearch

Logstash

Telegraf cAdvisor Filebeat

Lab Monitoring Stack v1

Kapacitor

Scaling

Scalability vs Availability

NGINX

Grafana Kibana

InfluxDB ElasticSearch

Logstash

Telegraf cAdvisor Filebeat

Kapacitor

NGINX

Grafana Kibana

InfluxDB ElasticSearch

Logstash

Telegraf cAdvisor Filebeat

Kapacitor

NGINX

Grafana Kibana

InfluxDB ElasticSearch

Logstash

Telegraf cAdvisor Filebeat

Kapacitor

Telegraf cAdvisor Filebeat

InfluxDB ElasticSearch

Kapacitor

Logstash

NGINX

Grafana Kibana

InfluxDB ElasticSearch

Logstash

Telegraf cAdvisor Filebeat

Kapacitor

Telegraf cAdvisor Filebeat

InfluxDB ElasticSearch

Logstash

NGINX

Grafana Kibana

NGINXGrafana Kibana

InfluxDB ElasticSearch

Logstash

cAdvisor Filebeat

Lab Monitoring Stack v2

Kapacitor

Telegraf

ELB

PostgreSQL

Questions

Jacob Tomlinsonwww.informaticslab.co.uk

@_jacobtomlinson

inlb.co/tech-exeter-2016