How to Monitor Microservices

transcript

Jorge Salamero Sanz@bencerillo @sysdig

How toMonitor Microservices

How to Monitor Microservices?

Health Checks

JVM/JMXCustom metrics

Metrics Processing Unicorns, rainbows And cute dashboards

% whoami

Jorge Salamero Sanz

<jorge.salamero@sysdig.com>

• Working on OSS last 12 years

• Working on monitoring last 3 years

• Containers gamer @sysdig

@bencerillo

@sysdig

Agenda

• Challenges of container infrastructures

• Traditional monitoring limitations

• Best practices monitoring Microservices

• Sysdig, container native monitoring & troubleshooting

Breaking traditional model

Microservices and containers break traditional monitoring and troubleshooting models

Traditional deployment

Full host OS

kernelsystemdsyslogd

App services

MySQLNginx

OpenSSLJava

Ops Devs

Application workflow

Database App Cache

backend middleware frontend

The Docker revolution

Servers Virtual Machines Containers

Unit: machine

Orchestration: no

Architecture: monolithic

Unit: machine

Orchestration: externalArchitecture: monolithic

Unit: (micro)service

Orchestration:nativeArchitecture: distributed

Containerized deployment

Full host OS

kernel

+Docker

MySQL App A

Ops DevOps

Nginx + OpenSSL App B

Java 8.0 build XXX App C

… but in reality

Database App Cache/Frontend

Computing node

Computing node Computing node

Computing node

… but in reality

Computing node

Database App Cache

New organization structures

Container monitoring

New challenges:

1. How do we get the metrics?

2. How do we shape all this amount of metrics?

3. Analysis and troubleshooting

4. Teams on Microservices infrastructure

New challenges:

1. Metric collection

• We containers, because:

– are simple

– are small

– are isolated

– less dependencies

• … but they are an opaque blackbox

“Workarounds”

Agent in the Docker container

Agent in the Kubernetes pod

Export metrics through an external agent

App Agent App Agent

1. Complex instrumentation (x2 because just the monitoring) plus service monitoring configuration

2. Limited and pre-established metric collection (Docker API, etc)

Kernel instrumentation

Kernel

Docker

Container1

Container2

Container3

App Apprkt LXC

Sysdig

Docker

Why this is cool?

• Just one instrumentation per host:– spawning or destroying a container is instrumentation-less

• Full visibility: all the system calls:– automatic service discovery

– all metrics collection (no filtering)

– application monitoring without instrumentation (magic of

decoding protocols)

New challenges:

Remember... but in reality:

Computing node

2. Information aggregation

• Infrastructure monitoring should be transparent and

automatic (no instrumentation no configuration)

• You should handle your custom/biz metrics

• All metrics should be tagged automatically

• All metrics should be aggregated and segmented on a

service level basis

Database App Cache

Orchestration platforms knows already...

New challenges:

Real example

https://github.com/kubernetes/kubernetes/issues/14051

3. Analysis & troubleshooting

• Imagine:

strace + wireshark + htop + lsof + iostat + vmstat + *

• Not available on containers, don’t understand

namespaces

• Metrics and logs can bite your in the ass, system

calls have all the truth

• Infrastructure gets more complex and volatile

New challenges:

Teams on Microservices

Francesc Zacarias, SRE @ Spotify

4. Teams by service

• Tags/Metadata from the orchestration platform, eg

Kubernetes:

– namespaces (dev, prod)

– services, deployments, RCs, pods

– custom tags

• ACLs out of the box (dashboards, alerts, etc) on

multi-tenant/PaaS scenarios

New challenges:

Sysdig

• 100% open-source

• 1M+ downloads

• Host analysis

• sysdig.org

• SaaS & on-prem

• 200+ customers

• Cluster analysis

• Dashboards, alerts,

events, teams

• sysdig.com

¡Danke MicroXchg!

@bencerillo@sysdigwww.sysdig.orgwww.sysdig.com

How to Monitor Microservices

Software