Towards Efficient Semantically EnrichedComplex Event Processing and Pattern

Matching

Syed Gillani1,2 Gauthier Picard1 Frederique Laforest2Antoine Zimmermann1

Institute Henri Fayol, EMSE, Saint-Etienne, France 1

Telecom Saint Etienne, Universite Jean Monnet, Saint-Etienne, France 2

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Overview

IntroductionTraditional Vs Real-Time Data ProcessingEvent Processing Vs Time AxisComplex Event Processing

Semantic Complex Event Processing

Proposed Approach

Conclusion

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Traditional Vs Real-Time Data Processing

Database

One Shot Database Queries

E2

E4

E1E3

E5En

Continuous

Event Query

Processing

Event Arrival Time

Time-PastTime-Future

Incoming Events

Time-Current

Traditional Data Processing Real-Time Data Processing

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Event Processing Vs Time Axis

Before Event ArrivalAt Event Arrival Some Time

After the Event

After Considerable

Time e.g. 2 Hours, 1 Day, 3 Months

Time Axis

Proactive Actions

Predictive A

nalysis

(Based on H

istorical Analysis)

Real-Time

Complex Event Processing

and

Pattern Matching

Late ReactionHistorical Events

Post Processing

and

Historical Analysis

*Dr. Adrian Paschke, DemAAL Summer school 2013

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Complex Event ProcessingI Aggregation, derivation of Primitive EventsI Occurrence and non-occurrence of certain eventsI Imposing Temporal Constraints (application of certain

rules )I For Instance

I Detection of state changes based on observations (If totalconsumed electricity > 10MWatt)

I Matching sequence of events that describes a scenario (IfA<10 AND B>40 OR B<80 AND C>90)

Primitive Events

Primitive Events

Primitive Events

Complex Events

Event Source 1

Event Source 2

Event Source n

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Overview

Introduction

Semantic Complex Event ProcessingSCEPState-of-the-art SCEPFoundational Challenges for SCEP

Proposed Approach

Conclusion

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

SCEP

I Complex Event Processing +Stream Reasoning+ SemanticTechnologies (rules & ontologies) + Heterogeneous DataHandling?

I Incoming Stream Reasoning + Background Knowledge

I Distributed into TWO flavoursI Stream Reasoning (Real Time + Background Information +

Aggregation through Windows) (C-SPARQL, CQELS....)I Pattern Matching (Sequence, Optional, Negation)

(EP-SPARQL)

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

State-of-the-art SCEP

*Streaming the Web: Reasoning over Dynamic Data: Alessandro Margara, Jacopo Urbani, Frankvan Harmelen, Henri Bal

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

State-of-the-art SCEP

I Complex Pattern Matching (Approaches)I Relational Community

I NFA, EDG, RETE algorithm, Rule based systemI Semantic Web Community

I RETE algorithm, Logical Rule based systemI How about NFA and EDG in SCEP context?I NFA and EDG are proven to be the most efficient for

Pattern Matching in relational community

*Non-Deterministic Finite Automata*Event Detection Graphs

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Foundational Challenges for SCEP

I Distributed Event Processing (per Query): Moving fromcentralised push based event processing

I Distributed Temporal Pattern Matching: Dedicated languagefor Pattern Matching (Implementation of Kleene Closure,Negation in distributed manner)

I Historical Management of Events: Storing and Partitioning ofevents

I Defining Event Boundaries: Triple based to Graph basedstreaming, preserving graph model to implement Eventboundaries

I Predictive Event Processing: A new paradigm for SCEPI Stream Reasoning + CEP: Combing two different worlds

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Overview

Introduction

Semantic Complex Event Processing

Proposed ApproachEvent and Stream Data ModelQuery Model and Language Specification

Conclusion

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Event and Stream Data Model

I Considering RDF as first class citizen (even for temporalreasoning, instead relying on external engines)

I Temporally Annotated RDF Named Graph(< NG, [ts, te] >)

<http :// www.streaminginfo.com/ElecGen > [st1 ,et1]

:gen1 :hasName ‘PowGen -Sect1 ’.

:gen1 :hasLocation ‘St-Etienne ’.

:gen1 :hasCurrentPower ‘60’.

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Proposed Data Model

I Data Partitioning ==> Optimises query time

I Summarisation ==> Merging of similar NG

I Event Boundaries ==> With NG

I Access Control ==> With NG

I Provenance Tracking ==> With NG

I Fact Assignment ==> With Time Interval

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Query Model and Language Specification

I Former Query ModelsI Reliance on Triple-Based Data Model

I Uses black-box approach (delegation to external Engines)

I Overhead in query and data translation

I Query Semantics not suitable for distributed processing perquery (SPARQL Extensions...)

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Proposed Query Model

Sub-Query 1 (Event Pattern A)

Sub-Query 2 (Event Pattern B)

Sub-Query 3 (Event Pattern C)

(a)(b)

Stream Source Selection, Temporal

Operators

Pattern Duration

Temporal Pattern

Description

Rewritten Subqueries

(Stream Processing)

KB

Integration

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

System Overview

S1

S2

S3

S4

J1

J2

G1

G2

Pattern

Module

E1

E2

E3

Ru

le n

. . .Ru

le 2

Ru

le 1

DBA

∆ = P1 ⇒ True ∆ = P1 ⇒ True& P2

∆ = P1 ⇒ True& P2

∆ = P2 ⇒ True& P3

Stream 1

Stream 2

Stream 3

Stream 4

Stage 1:

Stream Selection

Stage 2:

Continuous

Query Processing

and Inference

Stage 3:

Rule or Pattern

Mapping

Stage 4:

Distributed and

Parallel Pattern

Matching

(a) EDG (b) NFA

Storage of Archived

Streams

Archived

Streams

Streami : Incoming Streams Sk : Select Operators

Ji : Join Operations Gt : Generated Events

En : Event Nodes A/B/D : NFA States

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Proposed Model

I Supports Triple based and NG based data model

I Offers event source based Filtering

I Historical management of events through summarisation(Facts Assignments)

I Provide dedicated design for SCEP (No Data or QueryTranslation unlike EP-SPARQL and other systems)

I Distributed and parallel sub-query processing with queryrewriting

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Proposed Model

I Integrating stream processing and CEP

I Offers various new operators including, Sequencing,Kleene Closure and Negation for RDF Graph patterns

I Allows NFA and EDG to be used in the context of SCEPthrough query rewriting (from Rule based to State basedsystem)

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Overview

Introduction

Semantic Complex Event Processing

Proposed Approach

Conclusion

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Conclusion

I Annotated RDF NG enables temporal reasoning at RDFlevel

I Our data/query model and query rewriting allowsI Annotated NG based event data modelI Historical management of stream dataI Integration of various new operators for RDF Graphs

(Kleene Closure, Negation )I Integration of NFA and EDG in the context of SCEPI Parallel and distributed event processing (per query)

Introduction Semantic Complex Event Processing Proposed Approach Conclusion

Questions?