Date post: | 15-Dec-2015 |
Category: |
Documents |
Upload: | norman-margison |
View: | 214 times |
Download: | 0 times |
Detecting sewer rising main events using an ontology-driven event processing system
CSIRO LAND AND WATER
Jonathan Yu | Research software engineerPaul Davis, Irina Emelyanova, Scott Gould, Kerry Taylor, Donavan Marney
22 Feb 2013
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
The problem with sewer rising mains
Sewer rising main / pressure sewers• pipeline that carries sewerage at pressure from a pumping station• transport sewage where gravity flow is not possible or practical
Failures can be severe• Direct costs: water service providers ($ mil. per event)
– Pipeline repair – rising mains can be long... ~10km+– Sewerage removal by contractor:
say 3 weeks = 12 runs/day [24/7] @ 10 kL per run
• Indirect costs: social, environmental ($10k - $1 mil. per event)– In a recent case, a burst in a relatively new pressure sewer led to
undetected sewage discharge to a nearby creek for approximately 3 months
2 |
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Addressing sewer rising main events An option: Retro-fit commercial pressure sewer monitors
Costly and time-consuming...
3 |
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Q: Can we extract value from ‘business-as-usual’ data for early detection and pre-empting of these failures?
4 |
• In most cases, data is already collected• Wet-well levels• Pressure at pump station• Flow rate
• Look for breakpoints in sewer inflow rate timeseries by analysing trend analysis
• Capture these rules/event conditions
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Our approach
1) Investigate how to detect of sewer rising main events• Trend analysis methods and algorithms• Stream processing engines to enable real-time detection
2) System for user-definition and deployment of event constraints • Capture semantics of event constraints • System for deployment of event constraints on stream processing engine• Propose ontology-driven approach
5 |
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Sewer rising mains case study
Prototyping event detection algorithmsSewer rising mains pipe burst detection – flow, pump pressure
• Simple Moving Average, Exceeded thresholds, • Breakpoint analysis and Near Real-Time Disturbance detection (Irina)
6 |
0 2000 4000 6000 8000 10000 12000 14000 160000
20
40
60
80
100
120
140
160
Time (mins)
Flow
rate
(l/s
)
Failure event at 4260 minutes
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Stepped notifications: < 5 consequtive = low, 5 - 20 consequtive = moderate, > 20 = high risk
Applying Simple Moving Average & Stepped notifications
7 |
(Low risk)
(High risk)
Stepped Notifications
Flow observations
Simple Moving Average
Looking for when flow exceeds a preset threshold over the Simple Moving average
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Applying “Near Real-Time Disturbance detection” [1]
8 |
[1]. Verbesselt J, Zeileis A, Herold M (2011). Near Real-Time Disturbance Detection in Terrestrial Ecosystems Using Satellite Image Time Series: Drought Detection in Somalia. Working Paper 2011-18. Working Papers in Economics and Statistics, Research Platform Empirical and Experimental Economics, Universitaet Innsbruck. http://EconPapers.RePEc.org/RePEc:inn:wpaper: 2011-18. Submitted to Remote Sensing and Environment.
Start of monitoring periodHistorical data (input)
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Applying Near Real-Time Disturbance detection pt. 2
9 |
Historical data (input) Monitoring period
Break-point identified
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Use of stream processing engine (GSN)
10 |
SensorMiddleware
(GSN)
Sensor Network
End users
Open source sensor middleware.Provides abstraction APIs on raw streaming
sensor data(windowing, aggregate sensor sources, low-level
processing libraries, flexible output options)
Real-time streaming sensor data
Implements event detection algorithms
(Scripting via R, Groovy)
Email, SMS, Text2Speech, Integration with existing monitoring systems
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Real-time sensor stream data processing
• High level entry for an end user e.g. Scientists and managers
• Knowledge hidden behind code or implicit in people’s heads• Possible barrier for reusability
11 |
Curation CodingAnalysis,
Monitoring, Management
SensorMiddleware
(GSN)
Sensor Network
End users
Programmers
Semantics-based approach for defining complex event rules for algal bloom detection | Jonathan Yu
Problem of data heterogeneity, integration
• Multiple datasets• Often multiple data schemas and formats
• Example: The use of the observation property “Flow rate”• Flow• FLOW_RATE• RATE_OF_FLOW_L_per_s
• Want to be able to have mechanism of translating and mapping differing fields, labels to something commonly understood• Enhance interoperability
12 |
Semantics-based approach for defining complex event rules for algal bloom detection | Jonathan Yu
“Semantics-based approach”
Ontologies• Capture semantics• Lingua franca• Machine readable/processable
Vocabulary of things you care about in your data/system• E.g. Ability to refer to ‘Flow rate’ concept, rather than FLOW_RATE
We use ontologies for:• Providing translation between fields within sensors, datasets• Defining event rules • Generating code for actioning event rules on the sensors
13 |
OntologiesSemantic
Sensor Net. Ontology
DomainOntology
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Ontology-driven event detection system
14 |
SensorMiddleware
(GSN)
Sensor Network
End users
Ontology-enabledUser Interface
OntologiesSemantic
Sensor Net. Ontology
DomainOntology
Annotates available sensors and their capabilities
e.g. Pump pressure sensor data at Location X
Generate appropriate code to perform event detection on available sensors using event constraint semanticse.g. Identify break-point in sewer inflow rate according to trend
Populates user interface elements based domain semantics and sensor network annotations.
Allow users to define event constraints using ontology semantics
Return notifications from triggered events with metadata based on ontology semantics
e.g. The sewer rising main has a problem due to <...>
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu15 |
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Future work
• Implementation of breakpoint analysis and “near real-time disturbance” algorithms in our system
• Continuing ontology engineering for sewer rising main event detection• Harmonising with standard units of measure ontologies
• User interface refinement, and user testing• Very much a prototype / proof-of-concept
• Code generation module for deploying event constraints into GSN
• Performance and load testing to handle volumes of data
16 |
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Extension idea - fusing real-time data with domain knowledge base
17 |
KnowledgeBase
Sensor NetworkReal-time data
Event of Interest
Query knowledge base(domain knowledge)
Notifications
Populate knowledge base with parameterised historical pipe failure data.
Infer likelihood of pipe failure based on physical attributes and known operating environment
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu
Summary
Value of analysing ‘business-as-usual’ data for early detection of sewer mains pipe failure
Investigated a variety of timeseries analysis methods that is suitable for breakpoint detection of sewer rising mains failure events – SMA, Breakpoint analysis, Near real-time trend detection• Does not require extensive training datasets
Ontology-driven event detection system• User interface for defining machine readable event constraints using domain-
specific ontologies• System for deploying these event constraints for detection over sensor
network
18 |
Land and WaterScott GouldResearch Projects Officert +61 3 9252 6103e [email protected] www.csiro.au/clw
ICT CentreKerry TaylorPrincipal Research Scientistt +61 2 6216 7038e [email protected] www.csiro.au/ict
Land and WaterDonavan MarneyResearch team leadert +61 3 9252 6585e [email protected] www.csiro.au/clw
LAND AND WATER
Thank youLand and WaterJonathan YuSoftware Engineert +61 3 9252 6440e [email protected] www.csiro.au/clw
Land and WaterPaul DavisResearch Scientistt +61 3 9252 6310e [email protected] www.csiro.au/clw
Land and WaterIrina EmelyanovaResearch Scientistt +61 8 9333 6243e Irina. Emelyanova @csiro.auw www.csiro.au/clw
Semantics-based approach for defining complex event rules for algal bloom detection | Jonathan Yu
Advantages of semantics-based approach• Transparent and transferrable
• Rules, vocabularies, mappings are captured in the ontologies• Can deploy to other systems as long as they are mapped
• Traceable• Alerts can attach metadata to describe triggers: what, why, when
• End users can focus on exploring real-time datasets
20 |
Curation Coding Analysis, Monitoring, Management
Curation CodingAnalysis,
Monitoring, Management
Detecting sewer rising main events using an ontology-driven event processing system | Jonathan Yu21 |
Ontology-driven event detection system1. Composes CE
Sensor Network
Ontology-enabledUser Interface
Semantic Mediator
GSN
Ontologies
SSN Ontology
DomainOntology
7.Updates UI withalert
3. Deploys CE to GSN as VSensor via translation
capture rule to sensor API mappingcapture sensor / data sources mappings
6. Matching event alert generated
2. Submits CE definition
captures alerts
captures CE definition
8. Views alert
5. Sensor streams
data
Users