About Me> Eduardo Silva

● Github & Twitter: @edsiper● Personal Blog : http://edsiper.linuxchile.cl

> Treasure Data, Inc.

● Open Source EngineerFluentd / FluentBit

> Projects

● Monkey Server / monkey-project.com● Duda I/O / duda.io

Logging

Logging Matters

● Applications status

● Debugging

● Record specific events

● General information about anomalies: errors

● Troubleshooting / Support

● Local or Remote (network)

Pros

Logging and otheruse cases

Logging

More use cases

● Internet of Things (IoT)

● Embedded Applications

When resources are available, Logging can be done pretty straightforward with professional solutions such as Fluentd, but in more restrictive environments the scenario is quite different:

The fun begins ;)

Internet of Things (IoT)

Internet of Things

Facts

● IoT will grow to many billions of devices over the next decade.

● Now it's about device to device connectivity.

● Different frameworks and protocols are emerging.

● It needs Logging.

Internet of Things

FrameworksIn order to solve the device to device connectivity and proper messaging, multiple framework/stacks are emerging, all of them backed by different companies and vendors. The most remarkable at the moment are:

Brillo, by GoogleJust announced, no logo..sorry!

Internet of Things

Logging

● How do we approach data collection for IoT ?

● How can we improve our IoT based product if we have data from thousand of devices with multiple sensors (many inputs) ?

Fluentd for Embedded ?

Fluent Bit

Objective

● Sensors

● Services

● Signals / Radios

● Operating System Info

● Automotive / Telematics

Open Source data collection tool for Embedded Linux:

Fluent Bit

Requirements

● Lightweight

● Written in C Language.

● Customizable, pluggable architecture.

● Integrate with Fluentd

An Embedded environment requires special handling, specifically on performance and resources utilization, for hence it needs to be:

Fluent Bit

Requirements

● Support custom Inputs

● Support custom Outputs

● Use Binary Serialization (MsgPack)

● Configurable (though file system files)

Features

Fluent Bit

Features

● Collection & Distribution.

● Built-in system metrics.

● C API for Developers (WIP).

● Integration with third party services.

● Open Source / Apache License v2.0

Fluentd Integration

Fluent Bit

Fluentd Integration

Direct Output

Fluent Bit

Fluentd Integration

Examples

Fluent Bit

Built-in Metrics: CPU usage

Fluent Bit

Kernel Log Messages

Fluent Bit

Configuration Example

Roadmap

Fluent Bit

Roadmap

● Library mode.

● Support a stock of sensors (inputs).

● HTTP input.

● Release first stable version.

● Documentation.

Fluent Bit

More information

● http://fluentbit.io (official web site)

● http://github.com/fluent/fluent-bit

For more details about Fluent Bit please refer to the following links:

Thanks!

For more details about Fluent Bit please refer to the following links:

Eduardo Silva, @edsipereduardo@treasure-data.com