1

A Middleware Platform to FederateComplex Event Processing

Fawaz Paraïso, Gabriel Hermosillo, Romain Rouvoy, Philippe Merle, Lionel Seinturier

The Sixteenth IEEE International EDOC Conference (2012)

University of Lille & Inria lille-Nord Europe (France)

2

Agenda

• Motivation

• Challenges

• Contribution

• Validation

• Conclusion & Perspectives

3

Motivation

• What do we mean by event?

– A piece of data that represents somethinghappened in the real world

• Event-driven behaviour in daily life

– Computer

– Systems

– …

4

Motivation

• Events are everywhere

Produce events

5

Motivation

• Events are useless if they are not filtered and correlated

Events

Processing

6

Motivation

• What is Complex Event Processing (CEP)?

– Real time processing

– Intelligent business applications

• What applications can benefit from CEP?

– Real-time supply chain management

– Algorithm trading

– Monitoring (transaction, network, …)

– Credit card fraud detection

7

Motivation

• The need for real-time processing of information is relevant for many systems

– Business activity monitoring

– Fraud detection

– Nuclear crisis management

8

Motivation

Population

Experts

Localauthority

Police Firemen

EmergencyMedicalService

Nuclear Central

Media

Army

DecisionOperation

RadiationSurvey Network

NationalWeatherForecast

9

Agenda

• Motivation

• Challenges

• Contribution

• Validation

• Conclusion & Perspectives

10

Challenges

• Challenge 1: Communication heterogeneity

11

Challenges

• Challenge 2: Heterogeneous CEP Engines

CEP

Esper Etalis

StreamCruncher

ruleCore Server

12

Challenges

• Challenge 3: Scalability

Performance&

Scalability

13

Challenges

• Challenge 4: Adaptability

14

Agenda

• Motivation

• Challenges

• Contribution

• Validation

• Conclusion & Perspectives

15

Contribution

• A Middleware Platform to Federate ComplexEvent Processing

– Federate distributed CEP Engines

– Supports multiple communication services

REST, JMS, WS-Notification

– The DiCEPE Platform is an SCA-based solution

– Implemented in SCA using FraSCAti

Reflective component model

Runtime adaptative system

16

Contribution

• Distributed Platform Architecture

DiCEPE

DiCEPE

DiCEPE

17

Contribution

• Platform Architecture

DiCEPEContext

Engine Statement

*

Listener

*

BindingRest

BindingJMS

LegendComposite

Service

Property

Component

Reference

Wire

18

Contribution

• Platform Architecture

– Communication heterogeneity

Orchestrate heterogenenous services

Different bindingsREST, WS, JMS, JNA, UPnP, RPC ,RMI, JGroups, etc.

– Reconfiguration capability

Dynamic reconfigurable runtime architecture

19

Contribution

• Platform architecture

– Facililate the integration of CEP engine

Compose an heterogenous piece of software to build a new service

Supports variousImplementation technologies (Java, BPEL, C, C++, Python, …)

Interface definiton language (WSDL, Java)

20

Contribution

• The Platform adresses the challenges of :

– Communication heterogeneity

– Heterogeneous CEP

– Scalability

– Adaptability

21

Agenda

• Motivation

• Challenges

• Contribution

• Validation

• Conclusion & Perspectives

22

Validation

• DiCEPE for nuclear crisis management

Available here: http://dicepe-broker.soceda.cloudbees.net

23

Validation

• The SCA validates the challenge:

– Communication heterogeneity

– Heterogeneous CEP

– Scalability

– Adaptability

24

Validation

• Integration with the Esper and Etalis engine

Overview of Esper Engine Architecture

EsperServiceProvider

Event object

Listeners

EPLStatements

Co

nfigu

ration

1 2

3 4

5

DiCEPEArchitecture

EventExecutionWorker

PrologEngineWrapper

EtalisEventListener

InputEvents

Etalis

EtalisWrapper

PrologOutputEvents

1

2

3

statement4

1 2 3

4

5

1 2

4

3

25

Validation

• The integration of Esper and Etalis CEP engine validates the challenge :

– Communication heterogeneity

– Heterogeneous CEP

– Scalability

– Adaptability

26

Validation

• DiCEPE Cost Analysis

Implementation Avg. Exec. Time SCA overhead

Esper 27 sec -

DiCEPE (Esper+ FraSCAti) 30 sec 11%

27

Validation

• DiCEPE Scalability

Firemen Events Failures Avg. Sessions Avg. response

10,000 500,000 0 89 0.113 ms

15,000 750,000 0 135 0.142 ms

+ 50% + 50% + 51% + 26%-

28

Validation

• The scalability analysis validates the challenge

– Communication heterogeneity

– Heterogeneous CEP

– Scalability

– Adaptability

29

Validation

• Dynamic reconfiguration

30

Validation

• The FraSCAti validates the challenge:

– Communication heterogeneity

– Heterogeneous CEP

– Scalability

– Adaptability

31

Agenda

• Motivation

• Challenges

• Contribution

• Validation

• Conclusion & Perspectives

32

Conclusion & Perspectives

• DiCEPE offers interoperability between CEP engines via federation

• Flexible component architecture– Successful integration and validation of CEP engines– Multiple communication protocols

• Real scalability

• Integrate a Domain Specific Language(DSL) to express rules• Deployment of DiCEPE on heterogeneous cloud

environments• Error handling capabilities for distributed environments

33

Thank you

Questions?

@email: fawaz.paraiso@inria.fr