+ All Categories
Home > Documents > Solace Kafka Connectors for PubSub+€¦ · Kafka excels at log data aggregation and streaming to...

Solace Kafka Connectors for PubSub+€¦ · Kafka excels at log data aggregation and streaming to...

Date post: 03-Jun-2020
Category:
Upload: others
View: 10 times
Download: 0 times
Share this document with a friend
2
There is some functional overlap between Solace and Kafka, but if you look at the underlying architecture and operational characteristics of the two tools you’ll realize they solve different problems in very different – and frequently complementary – ways. Kafka excels at log data aggregation and streaming to data repositories. Solace complements Kafka by adding robust bi-directional event distribution between diverse applications and IoT devices running in hybrid clouds and around the world. Solace provides two free, open source connectors based on the Kafka Connector specification -- one Source and one Sink -- that make it easy to integrate Kafka and Solace so information can flow seamlessly between the environments. Source Connector The Solace Source Kafka Connector uses the Kafka Connect API to consume Solace queue or topic data events and stream them to a Kafka topic. The Solace Source Connector uses Solace’s high performance Java API to move Solace data events to the Kafka Broker. Solace PubSub+ supports popular messaging APIs and protocols like AMQP, JMS, MQTT, REST, WebSocket and more so the connector can move any message to a topic (keyed or not keyed) on the Kafka broker. Sink Connector The Solace Sink Kafka Connector uses the Kafka Connect API to consume Kafka topic records and stream the data events to Solace PubSub+ event brokers as a topic and/or queue data event. Therefore, from the single Solace Sink Connector any Kafka topic (keyed or not keyed) sink record is instantly available for consumption by any consumer that uses one of the many languages, protocols or APIs that PubSub+ supports. Source and Sink Connectors make it easy to leverage the unique strengths of Kafka and Solace PubSub+ Solace Kafka Connectors for PubSub+ Flexibility Easy access to the rich functionality and multi-protocol support of Solace PubSub+ eliminates the need to write connectors for other services. Scalability Enables connections and communications between a virtually unlimited number of applications and IoT devices. Bi-Directional MQTT The ability to not just collect data via MQTT, but to send alerts and instructions to specific devices or vehicles, enables sophisticated command and control use cases Dynamic Routing Support for hierarchical topics and wildcards, combined with dynamic intelligent routing, enables all kinds of fan-in, fan-out and filtration. Global/WAN Distribution Solace lets you establish a global network of mutually-aware event brokers that dynamically routes events from one application to another no matter where these applications are deployed, what API they use, or which specific broker they are connected to. To learn more visit bit.ly/KafkaConnectors
Transcript
Page 1: Solace Kafka Connectors for PubSub+€¦ · Kafka excels at log data aggregation and streaming to data repositories. Solace complements Kafka by adding robust bi-directional event

There is some functional overlap between Solace and Kafka, but if you look at the underlying architecture and operational characteristics of the two tools you’ll realize they solve different problems in very different – and frequently complementary – ways.

Kafka excels at log data aggregation and streaming to data repositories. Solace complements Kafka by adding robust bi-directional event distribution between diverse applications and IoT devices running in hybrid clouds and around the world.

Solace provides two free, open source connectors based on the Kafka Connector specification -- one Source and one Sink -- that make it easy to integrate Kafka and Solace so information can flow seamlessly between the environments.

Source ConnectorThe Solace Source Kafka Connector uses the Kafka Connect API to consume Solace queue or topic data events and stream them to a Kafka topic.

The Solace Source Connector uses Solace’s high performance Java API to move Solace data events to the Kafka Broker. Solace PubSub+ supports popular messaging APIs and protocols like AMQP, JMS, MQTT, REST, WebSocket and more so the connector can move any message to a topic (keyed or not keyed) on the Kafka broker.

Sink ConnectorThe Solace Sink Kafka Connector uses the Kafka Connect API to consume Kafka topic records and stream the data events to Solace PubSub+ event brokers as a topic and/or queue data event.

Therefore, from the single Solace Sink Connector any Kafka topic (keyed or not keyed) sink record is instantly available for consumption by any consumer that uses one of the many languages, protocols or APIs that PubSub+ supports.

Source and Sink Connectors make it easy to leverage the unique strengths of Kafka and Solace PubSub+

Solace Kafka Connectors for PubSub+

FlexibilityEasy access to the rich functionality and multi-protocol support of Solace PubSub+ eliminates the need to write connectors for other services.

ScalabilityEnables connections and communications between a virtually unlimited number of applications and IoT devices.

Bi-Directional MQTTThe ability to not just collect data via MQTT, but to send alerts and instructions to specific devices or vehicles, enables sophisticated command and control use cases

Dynamic RoutingSupport for hierarchical topics and wildcards, combined with dynamic intelligent routing, enables all kinds of fan-in, fan-out and filtration.

Global/WAN DistributionSolace lets you establish a global network of mutually-aware event brokers that dynamically routes events from one application to another no matter where these applications are deployed, what API they use, or which specific broker they are connected to.

To learn more visit bit.ly/KafkaConnectors

Page 2: Solace Kafka Connectors for PubSub+€¦ · Kafka excels at log data aggregation and streaming to data repositories. Solace complements Kafka by adding robust bi-directional event

Solace’s smart data movement technologies use open APIs and protocols to rapidly and reliably route information between applications, devices and people across clouds. Elite enterprises and high-growth startups around the world use Solace to modernize legacy applications and successfully pursue analytics, hybrid cloud and IoT strategies. Learn more or contact us at https://solace.com.

Applications and Advantages

High-Volume Web and Mobile ConnectivitySolace PubSub+ supports MQTT connectivity at massive scale, able to establish reliable, secure, real-time communications with tens of millions of devices or vehicles so you can collect data and hand it off to Kafka for aggregation or analytics.

Since Solace can also stream events via other popular open protocols and APIs, you can stream events from all of your applications, running in all kinds of cloud and on-premises environments, to Kafka via the same Solace Source Connector.

IoT Command and ControlIn addition to supporting the inbound aggregation of events from millions of connected devices, PubSub+ supports bi-directional messaging and the unique addressing of millions of devices through fine-grained filtering.

For example, with Solace and Kafka working together you could send a tornado warning alert to a specific vehicle, or all vehicles in or approaching the affected area. The Solace Sink Connector allows PubSub+ to send record events placed in a single Kafka topic to whatever vehicles satisfy a given condition or topic, whether that’s as general as being in the tri-county area or as specific as a single vehicle.

Connectivity Across Hybrid CloudPubSub+ lets you link applications and datastores across multiple cloud and on-premises environments without unwieldy adapters or bridges. PubSub+ keeps bandwidth consumption low by using fine-grained topic filtering to deliver exactly and only the events required.

For example, if you’re using Kafka to aggregate log data and perform offline analytics on it, but want to use a real-time analytics service running in the cloud to promote products based on sentiment analysis or real-time weather conditions, Solace PubSub+ can take the event stream from Kafka and route a filtered set of information to the analytics engine.


Recommended