From Events to Situations: An
Event-web perspectiveVivek Singh
Advisor: Professor Ramesh Jain
University of California, Irvine
Event-web
• Connecting real users, events and places rather than just documents.
• Events and objects as basic organization and linking mechanism
▫ Multimodal
▫ Closer to real world
• Users gain insights and experiences
• IBM Smarter planet
Event-web (imminent signs)
• Image and video sites for sharing experiential data related to events
• Tweets about events of interest
• Multimodal news broadcast of events
• Detection of events in surveillance videos
Motivation: From events to situations…
• Given a plethora of event data. How can we:
▫ Disambiguate relevant and irrelevant events?
▫ Combine events into meaningful representations ?
▫ Allow inference and cascading effects
▫ Support different interpretations based on application domain
▫ Support Control & decision making
Situation based control: Motivations
1. Inherent support for event-based (temporal) reasoning
2. The ability of the controller to reason based on symbols (rather than just signals)
3. Explicit inclusion of domain semantics (to support multiple applications)
Applications
• Energy efficient buildings:
▫ When to switch off air-conditioner?
• Telepresence:
▫ Which camera feed to send out?
• Business analysis:
▫ What should be the correct price for iPhone?
• Earthquake rescue effort:
▫ Where to send out the next fire-fighter engine?
E2E communication: Project Overview
Sentient Information
System
Sentient Information
System
Environment 1 Environment 2
Web
Device to Device communication
Towards Environment to Environment (E2E) multimedia communication systems, in Multimedia Tools and Applications Journal, Springer Netherlands, 2009.
Also in: ACM Workshop on Semantic Ambient Media Experiences (SAME), ACM Multimedia workshop, 2008.
Network/Transmis
sion
Environment Model
Environment Server
Situation based
controller
Actuator / Presentation
Model
MMDB
Sensors
Actuators / Presentation
Devices
Physical Environment
EventBase
Environment: Node Architecture
Situation Calculus: Quick overview▫ enter(P1), startWork(P1)
▫ enter(P1), exit(P1), enter(P1), startWork(P1), stopWork(P1), startWork(P1)
- isInRoom(P1, s(k))- isWorking(P1, s(k))
isInRoom(P1, s) ˄~isWorking(P1, s) →
IncreaseMusicVolume()
isInRoom(P1, s) 0
isWorking(P1, s) 01
1
Situation = Not events , nor sequence of events, but their assimilated descriptor
Situation calculus
• Ω = {Actions, Situations, Objects, Fluents}
• Situation:
▫ “The set of necessary and sufficient world state descriptors for undertaking control decision”.
• D = Dfnd U Duna U ε U Dap U Dss U D0
▫ Precondition axioms
▫ Successor-state axioms
▫ Initial situation
• Do(action, situation): A X S → S
Control theoretic problem formulation
•
•
•
•
•
•
Loc 1: Desk Loc2: Whiteboard
Loc 3: EngineeringModel
Actions possible:
1. Work on PC
2. Work on Table
Conditions Actions
Move to location
Activity SelectedCam
Desired Volume
Desk WorkOnPC
1 1
Desk WorkOnTable
2 2
Whiteboard
- 3 3
Model - 4 4User
Situation modeling: E2E application
Situation based control for cyber physical environments, Accepted: IEEE workshop on situation management, MILCOM, 2009
Situ-itter: Large scale situations on
• Looking beyond a room:
▫ Can an entire city or country be considered a cyber physical system.
• Humans as sensors:
▫ Everywhere !
▫ Perception, Censors, Rumors, Delays
• Data has salient features: Unstructured, Noisy, Humungous, Spatial semantics
• Event detection is not well studied!
Situ-itter: First steps
• Spatio-temporal visualization for insights
• Spatio-temporal analysis for event detection
• Combining with external sources of information for decision making
• Applications
▫ Event detection
▫ Should iPhone price be increased/decreased?
▫ Where and when to launch an ATT roadshow?
Comparison with external dataAggregate interest on iPhone, Current ATT store location data
Where to have an ATT roadshow?
(using spatial-temporal convolution)
<geoname><name>Sandy Big Bend Reservoir Number 1</name><lat>42.5191149</lat><lng>-109.4681887</lng><geonameId>5837570</geonameId><countryCode>US</countryCode><countryName>United States</countryName><fcl>H</fcl><fcode>RSV</fcode><fclName>stream, lake, ...</fclName><fcodeName>reservoir(s)</fcodeName><population>120,178<population/><alternateNames/><elevation>2194</elevation><continentCode>NA</continentCode><adminCode1>WY</adminCode1><adminName1>Wyoming</adminName1><adminCode2>035</adminCode2><adminName2>Sublette County</adminName2><timezone dstOffset="-6.0" gmtOffset="-7.0">America/Denver</timezone><distance>3.3639</distance></geoname>
Location has semantics
Future directions
• Tip of the iceberg:
▫ Spatio-temporal event detection in social media
• Reasoning/inference mechanisms
• Combining spatial, temporal and social semantics into decision making
• Considering multi-modal data, user and sensor based data
• A cyber-physical event-web which connects real users and environments