Post on 09-Jan-2017
transcript
Measure your app internals with InfluxDB(time-series database)Symfony Day 2015 - Reggio Emilia
Trust me
Why we measure things?
Measure - Learn - Build - Measure
Measure for a change
Measure for discovery
Measure for compare
What we need?* A data provider (our application)
* A database that will collects data
* A dashboard for representing information
What is a time-series database?A database optimized for handling time series data:
arrays of numbers indexed by time(a datetime or a datetime range)
sign elem is 4 -> @10:13 elem is 4
A time series is a sequence of data points, measured typically atsuccessive points in time spaced at uniform time intervals
InfluxDBEasy to useWritten in Golang (no external dependencies)ready for bilion of rows (scalable)UDP/IP and HTTP network adaptersSeveral client libraries (PHP, Golang, etc...)Integrated query web panelReady for external dashboards (grafana, chronograf, etc...)
Write information in your InfluxDBcurl XPOST 'http://localhost:8086/write?db=mydb' \ d 'cpu load=2 1434055562000000000'
Read data back from InfluxDBcurl G http://localhost:8086/query?pretty=true dataurlencode "db=mydb" \ dataurlencode "q=SELECT * FROM cpu WHERE host='server01' AND time < now() 1d"
For example:
InfluxDB tagsYou can mark new points with a tag information
sign @10.32 key=value tag1=value tag2=value
sign @10.32 temperature=21.3 location=home env=test
sign @10.32 cpu=21.3,mem=123.3 location=euwest1 env=dev instance=ia91afcd1
Use tags in your queriesExample
SELECT * FROM home_sensors WHERE location="home" AND temperature > 19.2 AND time >= now() 10m
Example for websites:SELECT * FROM index_page WHERE env="prod" AND load_time > 19.2 AND time >= now() 1d
Just use them like string column values
GRAFANA DASHBOARD
An open source, feature rich, metrics dashboard and graph editorfor several time series databases (InfluxDB, Graphite, etc.)
Annotate your graphs
Thanks to annotations you can use a serieas an annotation list
Mark a new software release in your graph (Continuous Delivery)Mark any critic event in all your graphs (Information Point)
CHRONOGRAF DASHBOARD
A new data visualization tool for InfluxDB.
How to send my application data?* Send every single point directly in real-time
* Collect data during the application life-cycle and send thosepoints once per request
InfluxDB supports two network adapters
HTTP or UDP/IPWhich one we should have to use?
UDP/IP must be enabled by configuration
HTTP* TCP/IP based
* Data point delivery guaranteed
* Error handling
* ~ 374 Ops/second (for compare with UDP/IP)
UDP/IP* No delivery guarantees
* No error handling
* ~ 22917 Ops/second (for compare with TCP/IP)
HTTP* Sensitive data
* Good for once at a time strategy (or if we do not care about monitor impact on our application)
UDP/IP* Good for immediate delivery
InfluxDB-PHP-SDKhttps://github.com/corley/influxdb-php-sdk
Supports both: HTTP and UDP/IPStar 46 Fork 13
composer require corley/influxdb-sdk
How it works$client>mark("serie_name", [ "customer_name" => "walter", "item_price" => 100.12, //...]);
Concise Format (good for immediate delivery)
How it works (2)$client>mark([ "points" => [ [ "measurement" => "instance", "fields" => [ "cpu" => 18.12, "free" => 712423, ], ], [ "measurement" => "page_load", "fields" => [ "listeners_speed" => 1.12, "counted_elements" => 341143, ], ], ]]);
Extended Format (good for a bulk strategy)
InfluxDB ClientUDP/IP + HTTP
$httpOptions = new Options();$reader = new Http\Reader($httpOptions);$udpOptions = new Options();$writer = new Udp\Writer($udpOptions);$client = new Client($reader, $writer);
HTTP only
$http = new \GuzzleHttp\Client();
$options = new Options();$reader = new Http\Reader($http, $options);$writer = new Http\Writer($http, $options);$client = new Client($reader, $writer);
Supports Guzzle versions: ~4, ~5, ~6
Symfony: Dependency Injection Containerparameters: influxdb_host: "localhost" influxdb_port: 4444
services: influxdb: class: InfluxDB\Client arguments: ["@influxdb_reader", "@influxdb_writer"]
influxdb_writer: class: InfluxDB\Adapter\Writer\Udp arguments: ["@influxdb_udp_options"]
influxdb_reader: class: InfluxDB\Adapter\Http\Reader arguments: ["@influxdb_http_options"]
Symfony: Dependency Injection Container#Only options
influxdb_udp_options: class: InfluxDB\Options calls: ["setHost", ["%influxdb_host%"]] # InfluxDB host ["setPort", ["%influxdb_udp_port%"]] # UDP/IP Port ["setTags", ["env": "%kernel.environment%"]] # tag every point with env
influxdb_http_options: class: InfluxDB\Options calls: ["setHost", ["%influxdb_host%"]] # InfluxDB host ["setPort", ["%influxdb_http_port%"]] # HTTP Port
parameters: influxdb_host: "influxdb.domain.tld" influxdb_udp_port: 4444 influxdb_http_port: 8086
Don't try this at homeclass DefaultController extends Controller public function indexAction(Request $request) $client = $this>container>get("influxdb");
$client>mark(...);
Goals: single-responsibility, reduce coupling, simplifing testing
Controllers as a service?For a monitor? Make sense?
With eventsclass DefaultController extends Controller public function indexAction(Request $request) //... $dispatcher>dispatch("my_event_name", $event);
Event listener
class MyMonitorAwareForMyEventListener public function __construct(Client $client) ...
public function onMyEvent(Event $event) $this>client>mark(...);
Deregister this listener in order to stop data collection
Compose your listenermonitor_listener: class: AppBundle\Listener\MonitorListener arguments: ["@influxdb", "@debug.stopwatch"] tags: name: kernel.event_listener, event: kernel.request, method: startStopwatch name: kernel.event_listener, event: kernel.response, method: stopStopwatch
The stopwatch will help us to collect page timing information
A listener example with stopwatchclass MonitorListener public function startStopwatch(GetResponseEvent $event) $routeName = $event>getRequest()>attributes>get("_route"); $this>stopwatch>start($routeName); public function stopStopwatch(FilterResponseEvent $event) $routeName = $event>getRequest()>attributes>get("_route"); $events = $this>stopwatch>stop($routeName); foreach ($events>getPeriods() as $measure)
$this>client>mark($routeName, [ "memory" => $measure>getMemory(), //... ]);
Check your data via InfluxDB
Prepare your dashboard with grafana
Checkout out the app examplehttps://github.com/wdalmut/symfonyday2015-example