Open PostgreSQLMonitoring
A brand new & free monitoring suite
Jean-Paul Argudo / Julien Rouhaud
Dalibo - www.dalibo.org
PostgreSQL Conference Europe 2014 - October, 22
1 / 26
OriginsComparison with Oracle
I Multimaster : Bi-Directional Replication (BDR)(2014-07)
I Materialized views : version 9.3 (2013-09)I Easy partitionning : TODOI Parallel queries : TODOI AWR : PoWAI Oracle Enterprise Manager, Oracle Grid : OPM!
2 / 26
State of the ArtA tool is missing
I proprietary softwareI existing Open Source projects to assembleI no real Open Source tool dedicated to PostgreSQL
3 / 26
Main ideas
To be accepted by the community the tool must beI completely free and Open SourceI crafted for PostgreSQLI extensible with a ”pluggable” architecture
4 / 26
Objectives
I a powerful monitoring platformI the standard for PostgreSQL monitoringI must compete with proprietary software
5 / 26
Functionalities
I an extensible architectureI Suitable probesI graphsI alerts
6 / 26
An active project
I OPM has 5 releases alreadyI 13 releases for check_pgactivityI fast pace dev : release often, release early
7 / 26
HistoryVersion 1.X
I creation of the architecture (opm_core and wh_nagios)I first metrics in check_pgactivityI graphs management
I aspect (Stacked, filled, etc)I human-readable unitsI export graphs into images
8 / 26
HistoryVersion 2.0 to 2.3
I new designI new authentication and grant/revoke methodsI pluggable architecture to accept pluginsI tags managementI bug fixes
9 / 26
OPM in details1/2
I A long projectI Composed of some new projectsI Still using some well-known ones, like Nagios
10 / 26
OPM in details2/2
I A graphing engine opm_coreI Modules to store data, at the moment, only
wh_nagiosI Probes dedicated to PostgreSQL : check_pgactivity
11 / 26
Graphing engine1/3
I PostgreSQL extension (opm_core) written in pl/pgsqlI web GUI in Perl (Mojolicious) and JavascriptI access control
12 / 26
Graphing engine2/3
I discovers automatically the data found (servers,services, etc)
I generates automatically the associated graphs, andassociates them to the correspond server
I tags to aggregate various graphs
13 / 26
Graphing engine3/3
I an API to facilitate writing to the warehouseI sampling functionI storing tables to inheritI . . .
14 / 26
Authorizations
I groups and UsersI 1 server = 1 groupI each user is part of one or many groupsI two levels of users : Administrator and User
15 / 26
wh_nagiosStorage module
I standard warehouse dedicated to Nagios data(perfdata)
I storing and archiving of all the valuesI contains a dispatcher to handle Nagios data
16 / 26
check_pgactivityNagios probe (1/3)
I standard Nagios probeI very simple code, and an APII performance data oriented to graphing
I managing of units in the perfdatasI the right metric is sent, no transformation needed
17 / 26
check_pgactivityNagios probe (2/3)
I better or new probes, based on our experienceI local file storage to compute deltasI probe on bgwriterI writing speed into WALsI . . .
18 / 26
check_pgactivityNagios probe (3/3)
I still many probes to invent or addI roadmap
I make it more modularI create a dedicated probe for the system
19 / 26
Overview
20 / 26
Create a warehouse1/3
I specify a schemaI table inheritance
I servicesI metricsI series
I And all the warehousing necessary
21 / 26
Create a warehouse2/3
I implement functionsI get_metric_data()I (grant/revoke)_dispatcher()I purge_service()
I and all the warehousing necessary
22 / 26
Create a warehouse3/3
I optional, add elements in the interfaceI decorateors, e.g. :
I details_server()I details_service()
I or dedicated screensI e.g. : Administration screens wh_nagios
23 / 26
Roadmap
I remove the dependency on Nagios or another toolI push and pull modes supportI have the dashboard customizableI metrics correlationI and much more
24 / 26
Some numbers
I more than 100 servers generatesI more than 2000 servicesI more than 17000 metrics
I over 9 monthsI represents 8.6 Gb of data
25 / 26
Demo
Because a good demo is better than boring slides !I demo.opm.ioI login : opmI pass : demo
More informations on opm.io
26 / 26