Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 214 times |
Download: | 0 times |
Event Processing Course(Advanced topics in CS 6 -236606)
Lecture one –Introduction (relates to chapters 1 and 2)
2 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
The motto of this course
Focus on emerging technology
Dive deep inside
Internalize and retain over time
3 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Emerging technologies in enterprise computing(Gartner Hype Cycle, Summer 2009)
5 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Internalize by: Experiment
Product
Developer
View
Application
Developer
View
6 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Entry point
Architecturaland programmingPrinciples
Chapter1
PART 1:
ENTRY POINT
Event Processing Agent
Context
EventChannel
EventConsumer
EventType
EventProducer
GlobalState
chapter
3
chapter
6
chapter
6
chapter
6
chapter
5
chapter
4
chapter
7
Filteringand transformation
PatternMatching
chapter
9chapter
8
PART 2:
THE BUILDING
BLOCKS
Engineering andimplementationconsiderations
Challenging topics Emerging directions
Chapter 10
Chapter2
PART 3:
ADDITIONAL TOPICS
Chapter 11
Chapter 12
7 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Lecture outline
• The notion of event
• Event-driven computing
• Event processing and its positioning in IT enterprise computing
• Event-driven architecture
• Model driven approach and major building blocks
• The FFD example
8 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event and situation
An event is an occurrence within a particular system or domain; it is something that has happened, or is contemplated as having happened in that domain. The word event is also used to mean a programming entity that represents such an occurrence in a computing system.
A situation is an event occurrence that might require a reaction
10 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 1:Smart healthcare – Preterm Infants
Patient is hooked up to multiple monitors (in hospital or at home) - the physician can set up event- based rules on multiple measurements and patient’s history when to send an alert and to whom:
DefinedPattern
11 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 2: RFID on luggage
Orchestrate: Passenger has been re-routed to another destination – send the luggage
Orchestrate:
Bag has reached to the wrong aircraft
Notify:
Bag has been checked but did not reach the ULD within 20 minutes
Notify: Bag has been checked but did not reach the connecting flight
THE LUGGAGE PERSPECTIVE:
Across the 24 largest airlines more than 5.6 million bags went missing in 2006, this is an average of 15.7 bags per 1,000 travelers. 15% of the bags are never found.
BBC News, April 4, 2007
13 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 4: Alerts in banking
Bank Account alerts; Weekly withdrawals are more than X Balance is less than y My investment portfolio is up z% since the start of the week
14 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 5: Fraud detection
15 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 6 – Emergency first responders(presented by Dieter Gawlick from Oracle in the 4th event processing symposium)
Fire StationFire Station
County Office
Hospital
Notification Message
ABC Chemicals
16 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 7 – fairness in trading
17 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 8 - Diagnostics: root cause analysis of failure
18 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 9 – the cameras in London
CarDB
Billing
Car Theft
19 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Example 10: Presence in social network
5 members are online
20 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event processing
Event Processing is computing that performs operations on events. Common event processing
operations include reading, creating, transforming and deleting events.
Specific properties of event processing
Abstraction—The operations that form the event processing logic can be separated out from the application logic, allowing them to be modified without having to change the producing and consuming applications.
Decoupling—the events detected and produced by one particular application can be consumed and acted on by completely different applications. There’s no need for producing and consuming applications to be aware of each others’ existence, and they may be distributed anywhere in the world. An event emitted by a single producing application can be acted on by many consuming applications. Conversely you can arrange for an application to consume events produced by many different producing applications.
Real-world focus—event processing frequently deals with events that occur, or could occur, in the real world.
21 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Deterministic and approximate event processing
TOLL VILOATOR FRUSTRATED
CUSTOMER
22 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Different types of event processing IBM Haifa Research Lab – Event Processing
IBM Haifa Research Lab – Event Processing © 2008 IBM Corporation
DetectDecide
Respond
EP Solution Segments – Business Value
Real-Time Operational
Information Dissemination
Observation
Predictive ProcessingActive Diagnostics
Reactions to events are done as part of business transactions – achieving low latency decisions, and quick reaction to threats and opportunities
Getting the
right
information
in the right
granularity
to the right
person at
the right
time
Diagnose
problems
based on
symptoms and
resolve them
Quick observation into exceptional business behavior and notification to the appropriate people.
Mitigate or
eliminate
predicted events
24 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event processing positioning BPM:
Can serve as consumer and producer.
Some BPM products have ad-hoc EP capabilities.
BAM:
Typically has built-in limited EP capabilities (mostly aggregations for KPI tracking).
Two BAM trends:
Moving from KPI to more observation types;
Moving from batch observations to some on-line
BI:
Event-driven BI is still relatively minor, but the area of “operational intelligence” is event-driven and on-line
BRMS:
Often confusion between the two.
Differences: state processing vs. transition history processing.
Possible synergies:
1. BRMS as consumer: An action for an event can be rule flow.
2. Rules are used for some EP functionality, e.g. routing
3. BRMS systems can be producers of events
4. Event-driven contexts can serve BRMS
5. Patterns from event processing can be used for cross-state reasoning.
25 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event processing positioning (cont).
Network and Systems Management:
Event-driven but concentrated in limited aspects:
Event correlation to find the root cause of a problem.
Message oriented middleware:
Event may be expressed as message – but not every message has event semantics
Partial overlap in functionality – especially ESB deals with: validation, enrichment, transformation and routing.
Stream computing:
Streams are types of data that are not necessarily express events – audio, signal, video… Some overlap on “event stream” --- various positioning throughout the industry.
26 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Request-Response request
response
Web browser Internet server
Taken from http://www.eaipatterns.comBased on the Enterprise Integration patterns book byHohpe and Woolf
Synchronous interactionsIn a synchronous interaction the provider is expected to send a response back fairly promptly. This usually means a few tenths of a second at the longest, though response times of several seconds are sometimes encountered in web applications. The requestor's thread of execution blocks waiting for the response to arrive without doing anything else in the interim/
An event is an indication of something that has already happened whereas a request,
as its name implies, expresses the requestor’s wish that something specific should happen in the future.
As a result we use slightly different terminology for the participants in an interaction involving events. Instead of talking about a service requestor and a service provider, we refer to an event producer and an event consumer.
27 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Point to Point
Taken from http://www.eaipatterns.comBased on the Enterprise Integration patterns book byHohpe and Woolf
28 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Publish-Subscribe
Taken from http://www.eaipatterns.comBased on the Enterprise Integration patterns book byHohpe and Woolf
29 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Principle of decoupling
In a decoupled Event Processing System an event producer does not depend on any particular processing or course of action being taken by any event consumer.
Moreover an event consumer does not depend on any processing performed by the producer other than the production of the event itself.
31 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event vs. Message
There’s one area of possible confusion that it is worth clearing up at this stage, and that is the distinction between an event and a message.
Message passing systems are often used to distribute events between event producers and event consumers, and it’s tempting to equate events with the messages used to transmit them.
The difference here is that the event is the entity that records information about the event occurrence, whereas the message is the mechanism used to exchange that information.
There are messages that don’t contain events, and conversely you can have events that aren’t in the form of messages (for example event records held in an event log).
32 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
What Makes EDA Different from Traditional SOA?
In Event-Driven Architecture (EDA):1. Events are always pushed, not pulled.2. The timing of the event message is determined by the source (event server) not by the
event consumers. 3. The event source and the event object never specify what action the consumer will
perform — the consumer determines the operation.
Not all Event Processing is EDA: 1. Event objects may be transmitted in a request/reply message pair (for example, RPC). 2. Or event objects may be at rest in a database or file (used, for example, in a BI
database query).
Event-based programming, also called event-driven architecture (EDA) is an architectural style in which one or more components
in a software system execute in response to receiving one or more event notifications.
33 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event processing network
Event Producer 1
Event Consumer 1
Event Consumer 2
Event Producer 2
Event Consumer 3
Agent 2
Channel
Agent 1
State
Agent 3
34 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event Stream
An event stream is a set of associated events .It is often a temporally totally ordered set (that is to say that there is a well-defined timestamp-based order to the events in the stream).
A stream in which all the events must be of the same type is called a homogeneous event stream;
a stream in which the events may be of different types is referred to as a heterogeneous event stream.
35 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
PlatformSpecific
PlatformIndependent
Specific DefinitionElements Instances
Platform Specific Definition Elements
Run-time Instance
Platform Independent Definition Elements
Event TypeBuilding Block
Creator for
Translated to
36 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
PlatformSpecific
PlatformIndependent
Delivery Request 3329From the Great Flower Shop
Delivery Request Event Type)an event schema of product X(
Run-time Instance
Delivery Request Event Type
Event TypeBuilding Block
Creator for
Translated to
37 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event Processing Agent
Context
EventChannel
EventConsumer
EventType
EventProducer
GlobalState
The seven
Building blocks
38 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Event Processing Agent
Filter Transform Detect Pattern
Translate Aggregate Split Compose
Enrich Project
Event Processing Agents
39 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Flower StoreVan Driver
Ranking and Reporting
System
Bid Request
Delivery Bid
Assignments,
Bid alerts, Assign Alerts
Control System
GPS Location
Location Service
Location
Driver’s Guild
Ranking and reports
Delivery
confirmation
Pick Up confirmation
Ranked drivers / automatic assignment
Bid System
StorePreferences
Delivery Request
Assignment System
Manual Assignment Assignment
Assignments,
Pick Up Alert
Delivery Alert
Fast Flower Delivery
40 Copyright ©2009 Opher EtzionCopyright ©2009 Opher Etzion
Lecture Summary
• In this lecture we have discussed the following topics:
– The notion of event
– Event-driven computing
– Event processing and its positioning in IT enterprise computing
– Event-driven architecture
– Model driven approach and major building blocks