#TDPARTNERS16 GEORGIA WORLD CONGRESS CENTER
Mining medical device logs to improve operational efficiency at Siemens HC
Bruce Baum – SiemensMike Watzke – Teradata Labs
• Siemens Introduction• Business Problem• Technical Solution
Overview
2
Siemens – Who we are
3
Electrification, Automation and Digitalization are long-term growth fields of Siemens.
Power and Gas Wind Power and Renewables Power Generation Services
Energy Management Building Technologies Mobility
Digital Factory Process Industries and Drives Healthcare
Shifting markets drive need for answers
4
Business Problem
5
1) Business problem & data overview
2) The story so far: Classical machine learning
3) Towards pattern mining for imaging devices
4) Excursion: The Siemens compute environment
Remote Diagnostics @ Healthcare
6
• Siemens Healthineers medical imaging devices are used all across the world in a demanding market:
• Minimize downtimes. Just imagine…• … doctors puzzling over blurred images• … an ER room
• Minimize maintenance cost (personnel, material)
• Siemens answer: Remote monitoring & diagnostics• Goal: Exchange unplanned for planned downtime• Technology: Predictive maintenance (min. 3 days)• Critical constraint: False alarms not accepted
Data at a Glance (CT Example)
7
• Regulatory constraints require focus on already existing data sources:
• Device logs• Time-stamped sequence of events• >100m lines per device & year
• Parts exchange data• Calls to service center (+ exam results)• <10 faults per device & year
The story so far: Classical ML
9
timestamp source code text
2014-05-17 11:31:12 A 37 xxx
2014-05-17 11:31:12 B 42 yyy
2014-05-17 11:31:13 B 17 .. hi temp (37.5) in ..
Device Logs
The story so far: Classical ML
10
timestamp source code text
2014-05-17 11:31:12 A 37 xxx
2014-05-17 11:31:12 B 42 yyy
2014-05-17 11:31:13 B 17 .. hi temp (37.5) in ..
device episode f1 f2 … f10000
55049 4711 37 3.45 true
55049 4712 42 ? false
55049 4713 17 3.12 true
Feature Matrix
Device Logs
Feature Extraction
• Event counts• Statistics for
extracted values• Derived features
(grouping/ scaling/ trends/ …)
• Different time bins (day/ scan/ …)
The story so far: Classical ML
11
timestamp source code text
2014-05-17 11:31:12 A 37 xxx
2014-05-17 11:31:12 B 42 yyy
2014-05-17 11:31:13 B 17 .. hi temp (37.5) in ..
device episode f1 f2 … f10000
55049 4711 37 3.45 true
55049 4712 42 ? false
55049 4713 17 3.12 true
Analytical Models
Feature Matrix
Device Logs
Feature Extraction
• Event counts• Statistics for
extracted values• Derived features
(grouping/ scaling/ trends/ …)
• Different time bins (day/ scan/ …)
Model Training
• Prediction horizon• Model selection• Feature selection
Parts Exchange Data
The story so far: Classical ML
12
timestamp source code text
2014-05-17 11:31:12 A 37 xxx
2014-05-17 11:31:12 B 42 yyy
2014-05-17 11:31:13 B 17 .. hi temp (37.5) in ..
device episode f1 f2 … f10000
55049 4711 37 3.45 true
55049 4712 42 ? false
55049 4713 17 3.12 true
Analytical Models
Feature Matrix
Device Logs
Feature Extraction
• Event counts• Statistics for
extracted values• Derived features
(grouping/ scaling/ trends/ …)
• Different time bins (day/ scan/ …)
Model Training
• Prediction horizon• Model selection• Feature selection
Parts Exchange Data
High-quality modelsfor current use cases
New offers need evenfewer false alarms
X
Why temporal patterns?
13
CT log data is a “flattened” representation of overlapping processes!
AcquirePrepare StoreProcessPatient 12345
AcquirePrepare StoreProcessPatient 98765
AcquirePrepare StoreProcessPatient 506156
timestamp source code text
2014-05-17 11:31:12 A 37 xxx
2014-05-17 11:31:12 B 42 yyy
2014-05-17 11:31:13 B 17 .. hi temp (37.5) in ..
First steps to pattern mining (1/2)
14
Domain-specific pattern-mining algorithm in Java
resilience to“stray events”from parallel processes
anytime-capability
support same-timeevents, includingrandom order of
“almost same time”
user-definedquality functions
X Scalability not sufficient for use case (transfer, processing)
First steps to pattern mining (2/2)
15
Using Aster nPath features, instrumented with KNIME
X Millions of nPath calls, no generation in-DB, expensive grouping operation needed!
The Siemens Smart Data Lab
16
Dat
a m
anag
emen
t
Data analytics
Data presentation
Data Warehouse
Hadoop
Data integration
Aster
… and others
… and others
4 nodes148 virtual units11 TB storage
Per node:• 24 cores• 256 GB RAM
Technical Solution
17
1. Overview
2. Sequence mining algorithm and implementation
3. Experimental data and demographics
4. Data preparation
5. Sequence mining training
6. Pattern scoring
7. Findings
Overview
18
• Hypothesis: temporal sequences of events can be used to provide early warning of failures?
• Test hypothesis with an experiment• Computed Tomography (CT) device logs• New sequence mining machine learning algorithm• Pattern scoring function
• Prior work: With the large volume of data and large pattern search space standard sequence mining approaches failed to work
Related Sequence Mining Work
19
• Frequent pattern mining (FP-growth, FrequentPaths) and Association Rules
• Frequency is not necessarily correlated to failure• Subgroup Discovery: related to above but allows for a more
flexible definition of the quality metric (frequency, unexpected, discriminating, ..)
• Temporal ordering of events is not considered
• Pattern Matching: requires patterns as inputs, in this context a function such as nPath would be more appropriate for scoring
Sequence Mining Definitions
20
Devices
FailureTime
BDFF A
ABACDABFEBDBACCEBDFAC
Event alphabet (A-F)
BDF
Sequence 1
Sequence 2
Events Pattern
Sequence Mining Algorithm
21
• Supervised machine learning algorithm, data classes are categorized based on time to failure
• Exhaustive iterative breadth first search of event pattern space. Search space size is an exponential function of depth
• Matching based on solving a Constraint Satisfaction Problem• Searching space pruning
• Quality Metric, Positive Predictive Value (PPV)• Sequence match counts• Prior matched sequences<->patterns• Terminate expansion of patterns with PPV = 1.0 (monotonic)
Sequence Mining Algorithm
22
Iteration 2, 32 Patterns
Iteration 1, 31 Patterns|Events| = 3Search Space
PPV = 0.93, matched 93 class 1 and 7 class 0Level 4 Pruning Example
PPV = 0.48, matched 48 class 1 and 52 class 0
. . .
. . . . . .
. . .
Algorithm Matching
23
• Matching a pattern to a sequence is based on solving a Constraint Satisfaction Problem
• Is (BDF) a subsequence of input sequence?• Constraints to be solved:
• Time(B,D) < Forward Gap OR Time(D,B) < Backward GapAND • Time(D,F) < Forward Gap OR Time(F,D) < Backward Gap
• Other matching approach would be a finite automata
BDF
Teradata DBS Implementation
24
PatternsSequences
Sequences (~Map)
Patterns(Vector)
Expand/ CSPMatch
Global (by pattern) Score and Prune
DuplicatedHashed
Sequence : Pattern
Nth instance of Global Score and Prune
Nth instance of Local Expand and Match
AMP 1 AMP NHashed
Experiment Data
25
• 350,000,000 CT device events• Data Record: {device, time, event,
class label}• Training data set (60% sample),
Validation data set (40%)• ~2,000 distinct Events
Experiment Data Demographics
26
• Event distribution
• X: Daily event count per device
• Y: frequency of specific daily event count
Experiment Data Preparation
27
• Generating Episodes
• Additional transformations• Timestamp to Epoch • Event string to numeric identifier• Event pruning; know error events, only events that
occur in failure window
Experiment Model Training
28
Sequences
Quality Metrics
Iterative Search Execution
Search Control
Patterns
Pattern <-> Sequences
Inputs Outputs
Depth=15~2B patterns
Experiment Model Scoring
29
• Metrics: precision and recall• Precision (PPV) = TP / (TP + FP)• Recall = TP / (TP + FN)
• Device and Episode match counts
Sequence MatchedFP TP
FNTN Sequence Not Matched
Time
Experiment Findings
30
50 Iterations of Train and Score using 60%/40% Samples
21 NAs
Experiment Findings
31
• Very high precession values can be achieved at the expense of recall.
• Configurable PPV per search depth iteration is useful• Common subpattern elimination
• Results from events occurring at same epoch and backward / zero delta support
• Episode support metric contributes to precision• Additional use cases being considered
• sensor data from power generation devices
Thank You
Questions/CommentsEmail:
Follow MeTwitter @
Rate This Session # with the PARTNERS Mobile App
Remember To Share Your Virtual Passes
645
32