CMUJuly 29, 2006

Pittsburgh, PA, USA

Machine Learning Algorithms for

Surveillance and Event Detection

Denver Dash – Intel, Corp.Terran Lane – University of New MexicoDragos Margineantu – The Boeing CompanyWeng-Keen Wong – Oregon State Univ.

Workshops

Workshop Sponsors

The Boeing Company

Intel

Event Detection

Biosurveillance Example:

Detect if there is a disease outbreak in a city as early as possible

Approach:

Monitor the total number of Emergency Department visit in the city each day.

Event Detection• Obtain Emergency Department data from the

past year• Fit a Gaussian to this data• Raise an alert when the daily number of ED visits

exceeds a threshold

0 x5035

Event Detection

• An interesting event occurs when the # of ED visits per day exceeds the threshold.

• If it corresponds to a real disease outbreak, it is a true positive. Otherwise, it is a false positive.

Number of ED Visits per Day

0

10

20

30

40

50

1 10 19 28 37 46 55 64 73 82 91 100

Day Number

Nu

mb

er o

f E

D V

isit

s

“Interesting” Events and their Detection

Knowledge

Data

DecisionsModel Event

DetectionProcess

Problem/Environment

Problem/Environment

Our World

P(x)

0 x10050

Model learned from x1 – x100, and on expert/prior knowledge

Interesting Events With Respect To…

x101 48.2

x102 51.3

x103 48.2

x104 51.3

x105 48.2

x106 51.3

x107 48.2

x108 51.3

x109 48.2

x110 51.3

… …

Events of Interest are observations with likelihood (very small?) of occurrence with respect to

• The model M that is believed to have generated the observations

• The other observations X that are available

P( xi | M, X ) =

Interesting Events with Respect To…

Complex Forms of Data

Primary Key Date Time Prodrome Sex Age Home Location

Many

more…

100 6/1/03 9:12 Fever M 20s NE …

101 6/1/03 10:45 Diarrhea F 40s NE …

102 6/1/03 11:03 Respiratory F 60s NE …

103 6/1/03 11:07 Diarrhea M 60s E …

: : : : : : : :

JAKARTA, Indonesia (AP) -- Researchers scouring swamps in the heart of Borneo island have discovered a venomous species of snake that can change its skin color, the conservation group WWF announced Tuesday.

The ability to change skin color is known in some reptiles, such as the chameleon, but scientists have seen it rarely with snakes and have not yet understood this phenomenon, the group said in a statement.. . .

Event Detection Tasks• Intrusion detection / network security• Security monitoring• Fraud detection• Biosurveillance• Traffic incident detection• Detection of interesting differences between images• Detection of potential causes for instability in dynamic systems or control

loops• Quality control in manufacturing• Topic detection• Sensor network monitoring• Aircraft / train / vehicle maintenance monitoring• Fault detection• Activity monitoring• Supernova detection• Weather modeling• Data cleaning• Detection of regions of increased brain activity from fMRI data• And many more…

Features Shared by MostEvent Detection Tasks

• Event detection is difficult or time consuming for human experts

• Interesting events are usually rare• Detecting an interesting event can have a

significant impact• Difficult to capture all the conditions that

make an event “interesting” • Evaluation of algorithms is difficult

Not Typical Machine Learning

• Standard supervised learning approaches are unsatisfactory: – few or no positive examples, plenty of

negatives– new forms of interesting events appear

• Standard unsupervised learning approaches are unsatisfactory: – skewed distributions– in many cases, not just looking for outliers

Standard MLEvent Detection Approaches

• One-class classification of “normal” observations; every other instance considered a potential “important event”

• Unsupervised clustering + post processing• Multi-Stage Event Detection: a standard ML approach +

filtering of false positives

+• Incorporation of background knowledge

Research Questions• Event Detection approaches for complex data (video, text,

spatio-temporal, relational)• Sensor fusion• Incorporating domain knowledge into the detection

models• Validation and testing of Event Detection Algorithms &

Tools:– Statistical tests– Testbeds for anomaly detection systems

• Online Event Detection• Defining the “interestingness” of an event (active

learning?)• Explaining why an event is interesting

– Effective visualization techniques• Event Detection in adversarial environments

Session 1 (9:20-10:50) 9:20-10:00 Interactive Event Detection in Audio and Video

Rahul Sukthankar10:00 - 10:25 Framework for Anomalous Change Detection –

James Theiler, Simon Perkins10:25-10:50 Shape Outlier Detection Using Pose Preserving Dynamic Shape Models

Chan-Su Lee, Ahmed Elgammal

Coffee Break (10:50-11:20)

Session 2 (11:20-12:40) 11:20-12:00 Detection of Stepping-Stones: Algorithms and Confidence Bounds

Shobha Venkataraman12:00-12:20 Distributed Probabilistic Inference for Detection of Weak Network Anomalies

Denver Dash12:00-12:20 Learning Sequential Models for Detecting Anomalous Protocol Usage

Lloyd Greenwald

Lunch (12:40-14:05)

Schedule

Session 3 (14:05-15:45)14:05-14:45 Forecast, Detect, Intervene: Anomaly Detection for Time Series

Deepak Agarwal14:45-15:25 Bayesian Biosurveillance

Greg Cooper

15:25-15:45 A Wavelet-based Anomaly Detector for Early Detection of Disease Outbreaks

Thomas Lotze, Galit Shmueli, Sean Murphy, Howard Burkom

Coffee Break (15:45-16:15)

Session 4 (16:15-17:35)16:15-16:45 Towards a Learning Traffic Incident Detection System

Tomas Singliar, Milos Hauskrecht

16:45-17:05 Bayesian Anomaly Detection (BAD v1.0)Tim Menzies, David Allen

17:05-17:35 Discussion Panel

Schedule