+ All Categories
Home > Documents > Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and...

Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and...

Date post: 22-Apr-2018
Category:
Upload: dangdung
View: 228 times
Download: 5 times
Share this document with a friend
23
Event-based Dissemination and Processing of Geospatial Information ArcGIS GeoEvent Server Thomas Paschke Product Specialist Real-Time & Big Data Pre-conference workshop at the 20th AGILE International Conference on Geographic Information Science, Wageningen, May 9, 2017
Transcript
Page 1: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Event-based Dissemination and Processing of Geospatial Information

ArcGIS GeoEvent Server

Thomas PaschkeProduct Specialist Real-Time & Big Data

Pre-conference workshop at the 20th AGILE International Conference on

Geographic Information Science, Wageningen, May 9, 2017

Page 2: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Real-Time GIS1

Page 3: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

GIS DataWhat has happened, what is happening, what will happen

The ‘current’ snapshot is outdated almost as soon as it’s created…

Credit: iStockphoto/chris_lemmens

Page 4: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Real-Time GIS RequirementsIngestion, Analytics, Notifications and Alerting

Requirement #1

Requirement #2

Continuous

AnalysisFeatures

Position

AlertInside

Boundary

Requirement #3

Applications

Page 5: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Ingestion & Dissemination of Events2

Page 6: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

you can create

your own

connectors

Ingestionuse an existing input connector

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Poll an ArcGIS Server for Features

Poll an external website for GeoJSON, JSON, or XML

Receive Features, GeoJSON, JSON, or XML on a REST endpoint

Receive RSS

Receive GeoJSON or JSON on a WebSocket

Receive Text from a TCP or UDP Socket

Subscribe to an external WebSocket for GeoJSON or JSON

Watch a Folder for new CSV, GeoJSON, or JSON Files

Ou

t o

f th

e B

ox

REST

.csv

WS

WS

HTTP

Esri

Gallery

ActiveMQ

CAP

Instagram

Exploitation Support Data

Cursor-on-Target

RabbitMQ

NMEA 0183

MQTT

Sierra Wireless (RAP)

KML

Kafka

Trimble (TAIP)

Twitter

Part

ner

Gallery

CompassLDE

enviroCar

GNIP

FAA (ASDI)

exactEarth AIS

Waze

Valarm

Networkfleet

OSIsoft *

*

*

*

Zonar *

Amazon IoT

Azure IoT

Page 7: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

you can create

your own

connectors

Ingestionconfigure a input connector by pairing a transport & adapter together

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Feature-JSON

Connectors Transport Adapter

GeoJSON

Receive Feature-JSON from Kafka

Receive GeoJSON on a REST endpoint

Feature-JSON

Adapters

GeoJSON

JSON

RSS

Text

XML

CAP

Cursor-on-Target

GeoMessage

Instagram

NMEA

Sierra Wireless (RAP)

Trimble (TAIP)

Twitter

VMF

Esri

Gallery

Waze

Feature Service

Transports

File

HTTP

HTTP+BasicAuth

HTTP+OAuth

TCP

UDP

Waze

WebSocket

Esri

Gallery Amazon IoT

Azure IoT

IRC

Kafka

MQTT

Kafka

HTTP

RabbitMQ

Twitter

Page 8: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

you can create

your own

connectors

Disseminationuse an existing output connector

GeoEvent Server

Inp

uts

Ou

tpu

ts

GeoEvent Services

Ou

t o

f th

e B

ox

Add or Update a Feature

Publish Text to a UDP Socket

Send a Text Message

Send an Email

Push Text to an external TCP Socket

Push GeoJSON or JSON to an external WebSocket

Push GeoJSON or JSON to an external Website

Send an Instant Message

Send Features to a Stream Service

Write to a CSV, GeoJSON, or JSON File .csv

WS

im

HTTP

Add a Feature to a spatiotemporal big data store

Update a feature in a spatiotemporal big data store

ActiveMQ

Esri

Gallery

Cursor-on-Target

Hadoop

Kafka

MongoDB

MQTT

RabbitMQ

Twitter

Amazon IoT

Azure IoT

Page 9: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Real-Time Analytics3

Page 10: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Applying Real-Time AnalyticsGeoEvent Services

• A GeoEvent Service defines the flow of GeoEvents

- The Filtering and Processing steps to perform

- what input(s) to apply them to

- and what output(s) to send the results to

Page 11: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Performing continuous analytics in ArcGISProcessors

You can create

your own

processors.

• You can perform continuous analytics on events as they are received using a processor.

GeoEvent Extension

Inp

uts

Ou

tpu

ts

GeoEvent Services

Buffer Creator

Convex Hull Creator

Difference Creator

Envelope Creator

Field Calculator

Field Enricher

Field Mapper

Geotagger

Incident Detector

Intersector

Projector

Simplifier

Symmetric Difference

Track Gap Detector

Field Reducer Union Creator

Ou

t o

f th

e B

ox

Add XYZ

Esri

Gallery

Bearing

Ellipse

Event Volume Control

Extent Enricher

Field Grouper

GeoNames Lookup

Range Fan

Reverse Geocoder

Service Area Creator

Symbol Lookup

Track Idle Detector

Unit Converter

Visibility

Motion Calculator Query Report

Page 12: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Additional Real-Time Spatial AnalyticsSpatial condition evaluation

• Spatial operators:

inside outside

enter exit

intersects disjoint

touches contains

crosses equals

overlaps within

Page 13: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Additional Real-Time Spatial AnalyticsExample – Impending weather notifications

Page 14: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Additional Real-Time Spatial AnalyticsExample – Convoy separation alerting

Page 15: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Additional Real-Time Spatial AnalyticsGeoFence scope

• All GeoFences• Any GeoFence

• Specify a GeoFence scope: Any or All

Page 16: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Additional Real-Time Spatial AnalyticsExample – Territory adherence alerting

Page 17: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

ArcGIS Enterprise4

Page 18: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

ArcGIS Enterprisewith real-time & big data capabilities

ArcGIS

Enterprise

GeoEvent

Server

spatiotemporal

big data store

Big DataIoT

GeoAnalytics

Server

Page 19: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

ArcGIS Enterprisewith real-time & big data capabilities

DesktopWeb Device

visualization

live & historic

aggregates & features

map & feature service

• Ingest high velocity real-time

data into ArcGIS

• Perform continuous analytics

on events as they are received

• Store observations in a

spatiotemporal big data store

• Run batch analytics on

stored observations

• Visualize high velocity &

volume data:

- as an aggregation

- as discrete features

• Notify those who need to

know about patterns of

interest

stream service

live features

ArcGIS

Enterprise

GeoEvent

Server

spatiotemporal

big data store

storage

ingestion

analytics

GeoAnalytics

Server

analytics

Page 20: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Visualization5

Page 21: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

Two patterns

GeoEvent Server

Your

Applications

Stream Layer

Map Services

Feature Services

Feature Layer

feature layers

Update a Feature

Add a Feature

EGDB/

SBDS

Polling

(Pull)

Stream Services

Getting Real-Time Data into Web Apps

• Feature layers pull from feature services

- Web apps poll to get periodic updates

- Must be backed by an enterprise geodatabase (EGDB) or spatiotemporal big data store (SBDS)

• Stream layers subscribe to stream services

- Web apps subscribe to immediately receive data

- Low latency and high throughput

Send Features to a Stream Service

Page 22: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

10.5 enhancements

• spatiotemporal big data store & map service aggregation capabilities:

- enhanced aggregation capability to support on-the-fly square & hexagon aggregation

spatiotemporal big data store

hexagon aggregationsquare aggregation

Page 23: Event-based Dissemination and Processing of Geospatial ... · Event-based Dissemination and Processing of Geospatial Information ... Receive Feature-JSON from Kafka ... Kafka MQTT

© Esri Deutschland GmbH

Esri Deutschland, Esri Schweiz, Esri Rwanda, Esri Burundi

Esri Rwanda Ltd. Kigali

KG 5 Ave, Nr 18, Kacyiru

P.O. Box 6867

Kigali, Rwanda

Lat. 1°56'46.9" S; Long. 30°5'24.8" E

Telefon +250 788 38 19 00

[email protected]

Esri Rwanda Ltd. Kigali

KG 5 Ave, Nr 18, Kacyiru

P.O. Box 6867

Kigali, Rwanda

Lat. 1°56'46.9" S; Long. 30°5'24.8" E

Telefon +250 788 38 19 00

[email protected]

Esri Schweiz AG Zürich

Josefstrasse 218

8005 Zürich

Telefon +41 58 267 18 00

[email protected]

Esri Suisse SA Nyon

Grand-Rue 9

1260 Nyon

Telefon +41 58 267 18 60

[email protected]

Esri Deutschland GmbH

Ringstraße 7

85402 Kranzberg

Telefon +49 89 207 005 1200

[email protected]

Niederlassung Berlin

Karl-Liebknecht-Str. 5

10178 Berlin

Telefon +49 89 207 005 1560

[email protected]

Niederlassung Bonn

Rheinallee 24

53173 Bonn

Telefon +49 89 207 005 1720

[email protected]

Niederlassung Hannover

Schiffgraben 11

30159 Hannover

Telefon +49 89 207 005 1520

[email protected]

Niederlassung Köln

Konrad-Adenauer-Ufer 41-45

50668 Köln

Telefon +49 89 207 005 1760

[email protected]

Niederlassung Leipzig

Fechnerstraße 8

04155 Leipzig

Telefon +49 89 207 005 1420

[email protected]

Niederlassung Münster

Martin-Luther-King-Weg 24

48155 Münster

Telefon +49 89 207 005 1460

[email protected]

24


Recommended