Monitoring at Scale over diverse data sources @ PayPal – Druid,
TSDB, Hadoop Grafanacon, December 1, 2016
Senthil Pandurangan - Platform Architect Abhishek Sant – Senior Software Engineer
©2016 PayPal Inc. Confidential and proprietary.
Agenda • Introduction
• Monitoring Platform Architecture
• Grafana Druid Plugin
• Q & A
3
PayPal Command CenterPowered by Monitoring Team
4
PayPal Monitoring Ecosystem - Sherlock
APIs
Logs Processing
Metric Processing Insights Engine
Alerts
Visualization
Events Stream
Metrics Stream
Topology
Logs Stream
Pools & Hosts
PaaS
IaaS
Network
Databases
CAL (Logging)
Remediation OrchestrationConfiguration
Sherlock is a unified monitoring, alerting and logging platform that exposes real time data and insights, enabled by an extensible visualization platform.
Developers
Analysts
Command Center
SRE
©2016 PayPal Inc. Confidential and proprietary.
Tools & Technologies
Storage
Processing
Visualization
6Ingest Queue Process Store Query
PayPal Monitoring Platform - Architecture
Message Bus
TSDBApp
Logs
App
App
Services
DBLB
NO
SQ L
Met
ric s
Stream processing
Met
ric sLo
gs
Log Views
Hadoop Reports
Custom Visualizatio
ns
Alerts
©2016 PayPal Inc. Confidential and proprietary.
Agenda • Introduction
• Monitoring Platform Architecture
• Grafana Druid Plugin
• Q & A
©2016 PayPal Inc. Confidential and proprietary.
Grafana Druid Pluginhttps://grafana.net/plugins/abhisant-druid-datasource/installation
©2016 PayPal Inc. Confidential and proprietary.
Grafana - Adding Druid Data source
©2016 PayPal Inc. Confidential and proprietary.
Grafana Druid Plugin – Query Editor
©2016 PayPal Inc. Confidential and proprietary.
Druid Query Types
Time Series
Top N
Group by
Select
©2016 PayPal Inc. Confidential and proprietary.
Multiple Data sources
Time Series
Top N
Group by
Select
TSDB
Druid
Hadoop Reports
©2016 PayPal Inc. Confidential and proprietary.
Thank You