+ All Categories
Home > Documents > IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire...

IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire...

Date post: 21-May-2020
Category:
Upload: others
View: 5 times
Download: 0 times
Share this document with a friend
13
1 1 IoT Data Platforms Processing IoT Data with Apache Kafka
Transcript
Page 1: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

1 1

IoT Data Platforms Processing IoT Data with Apache Kafka™

Page 2: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

2 2

Common Challenges with Internet of Things (IoT)

Systems are overwhelmed by dramatic increases in

data volumes

The range of connected devices is broad, no one

size-fits-all solution

Data streams must flow both in and out of devices

Page 3: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

3 3

More than 1 petabyte of data in

Kafka

Over 1.2 trillion messages per day

Thousands of data streams

Source of all data warehouse & Hadoop data

Over 300 billion user-related events

per day

Page 4: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

4 4

70% of active Kafka Committers

Founded September 2014

Technology developed while at LinkedIn

Founded by the creators of Apache Kafka

About Confluent and Apache Kafka™

Jay Kreps CEO

Cheryl Dalrymple CFO

Todd Barnett VP WW Sales

Neha Narkhede CTO, VP Engineering

Luanne Dauber CMO

Chrix Finne Head of Product

Page 5: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

5 5

Kafka Adoption Spans Many Industries

Financial Services Enterprise Tech Consumer Tech

Entertainment & Media Telecom Retail Travel & Leisure

Page 6: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

6 6

Connected Car and Swarm Intelligence Using Confluent

850 sensors feed onboard processors

Regional data centers capture swarm data in Kafka pipelines

Real-time traffic routing across an entire geographic

region

Page 7: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

7 7

IoT Value Chain and Confluent Platform

Sensors Data

M2M Protocols

Local Data Ingestion

IoT

Gateways

Data Transmission

Confluent Platform

Data

Aggregation

Data Ingestion at Scale

Stream Processing

Reporting

Actionable Insights

Page 8: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

8 8

The Confluent Advantage

Connect services via a shared distributed log

Handle unlimited stream of data

Backed by the people that created Kafka

Operate with confidence

Page 9: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

9 9

Introducing Confluent CloudTM

A Way to Liberate Developers.

Page 10: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

10 10

“ Public cloud services that require developers to program to proprietary

APIs will not succeed. ”

Neha Narkhede, Co-Founder and CTO at Confluent

https://www.confluent.io/blog/announcing-confluent-cloud-apache-kafka-as-a-service/

Page 11: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

11 11

Centrica Connected Home, Hive Home, and British Gas

Page 12: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

12 12

Kafka IoT

Page 13: IoT Data Platforms...swarm data in Kafka pipelines Real-time traffic routing across an entire geographic region 7 IoT Value Chain and Confluent Platform Sensors Data M2M Protocols

13 13

Confluent Platform: Enterprise Streaming based on Apache Kafka™

Database Changes Log Events loT Data Web Events …

CRM

Data Warehouse

Database

Hadoop

Data Integration

Monitoring

Analytics

Custom Apps

Transformations

Real-time Applications

Apache Open Source Confluent Open Source Confluent Commercial

Confluent Platform

Confluent Platform

Apache Kafka™

Core | Connect | Streams

Data Compatibility

Schema Registry

Monitoring & Administration

Confluent Control Center

Operations

Replicator | Auto Data Balancing

Development and Connectivity

Clients | Connectors | REST Proxy


Recommended