Scaling distributed search for diagnostics and prognostics

applications

Prof. Jim AustinComputer Science, University of York UK

CEO Cybula Ltd.

OGF 20072

Outline Challenge to be addressed Background to Signal Data Explorer The industry problem: Aerospace The science problem: Neuroscience Scaling to other industries

The challenge

OGF 20074

Challenge

Asset orSystem

Data

Data Store

Massive data collection happeningeverywhere

Need to get knowledge from the data

Knowledge

OGF 20075

Challenge

Europe

China

US

Japan

OneData

Warehouse

OGF 20076

Challenge Data too large for a single data store Data changes too fast to keep up to date Networks too slow to transfer the data The system has a central point of failure Large processing needed on centralised

store

OGF 20077

Signal Data Explorer

PME PMC

PME PMC

PME PMC

PME PMC

SDE data architecture

Local nodes

OGF 20078

Challenge Do not move the data - move the processing

That’s much lower effort More reliability Scalable Networks need not be high bandwidth Leverages local computation for free (the

Grid vision)

The Signal Data Explorer

OGF 200710

SDE

!Event detector

Data silosAsset

Data feed

Basic SDE data process

OGF 200711

SDE Allows a user

Set up triggers on multiple, complex, real time data feeds.

Find examples of events that are seen but unknown.

Manage distributed data.

OGF 200712

SDE Signal Search

and analysis Finds

examples like previous

OGF 200713

Enterprise SDE system

Distributed Data Repositories

DataDownload

Novel

Data

En

gin

e

Da

ta

Se

rvic

eD

ata

SignalProcessing

AURA-GCBR

DecisionSupport

EngineModel

DataVisualisation

PortalSecurity management Workflow Manager

Collaborative Working

En

gin

e

Da

ta

En

gin

e

Da

ta

AU

RA

DA

TA

Middleware Services & Data Management

Resource Broker PMC & SRBGrid Service Manager Quality of Service/SLA’s

Grid Infrastructure

GT3

IndustryAerospace: DAME, BROADEN

OGF 200715

Industry Industry demonstrator for the SDE

technology Demonstrates the impact SDE can have

on the business process Allows asset management – full life

support processes

OGF 200716

Rolls-Royce

Engine flight data

Airline office

MaintenanceCentre

London AirportNew York Airport

GRID DiagnosticsCentre

Engine flight data

European data centre

Engine flight data

Airline office

MaintenanceCentre

London AirportNew York Airport

US data centre

GRID DiagnosticsCentre

Engine flight data

OGF 200717

DAME and BROADEN DAME developed basic idea – developed

in lab (2000-2004) BROADEN now has demonstrated this in

Rolls-Royce (2004-2007) Engineers can now use SDE to visualise and

analyse data from Trent engines and test rigs

OGF 200718

DAME and BROADEN The example has shown how this can be

applied in many industries.

ScienceNeuroscience: CARMEN

OGF 200720

Science Data is often not shared from experiments Individuals duplicate many expensive

experiments Should share both data and methods Distributed data and service repositories

needed to support this

OGF 200721

CARMEN Tackles the scientific application of SDE 4 year project to build a neuroscience

repository and experimental platform Project started Oct 2006 £4.5M over 4 years

OGF 200722

CARMEN

resolving the ‘neural code’ from the timing of action potential activity

determining ion channel contribution to the timing of action potentials

examining integration within networks of differing dimensions

Understanding the brain may be the greatest

informatics challenge of the 21st century

OGF 200723

NeuroscienceNeuroscience Gain

Recording from brain tissue removed from epileptic patients (scarce tissue and data rates up to 20 Gb/h)

On line analysis by distributed collaborators will enable experiment to be defined

Repository will enable integration of rare case types from different laboratories

New knowledge will lead to advances in treatment

Scaling to other sectors

OGF 200725

Other sectors Concept is to provide full asset monitoring

Provision of the complete maintenance package, not just the asset

Better value add Manufacturers of the asset are the best

people to diagnose faults, manage the maintenance.

The SDE system allows this

OGF 200726

Other sectors Rail industry

Track and carriage monitoring Oil and Gas

Pipeline monitoring – for leaks Power Generation

Monitoring of generation equipment Transport

Road traffic management

OGF 200727

Thanks DAME, BROADEN and CARMEN teams Support of EPSRC, DTI and industrial

collaborators