Sensor and Graph Mining

Outline

• Introduction - motivation

• Problem #1: Stream Mining– Motivation– Main idea– Experimental results

• Problem #2: Graphs & Virus propagation

• Conclusions

Introduction• Sensor devices

– Temperature, weather measurements– Road traffic data– Geological observations– Patient physiological data

• Embedded devices– Network routers– Intelligent (active) disks

Introduction• Limited resources

– Memory– Bandwidth– Power– CPU

• Remote environments– No human intervention

Introduction – problem dfn• Given a emi-infinite stream of values (time

series) x1, x2, …, xt, …

• Find patterns, forecasts, outliers…

Introduction

Periodicity? (daily)

Periodicity? (twice daily)

“Noise”??

• E.g.,

Introduction

Periodicity? (daily)

Periodicity? (twice daily)

“Noise”??

• Can we capture these patterns– automatically– with limited resources?

Related workStatistics: Time series forecasting

• Main problem:

“[…] The first step in the analysis of any time series is to plot the data [and inspect the graph]” [Brockwell 91]

• Typically:• Resource intensive

• Cannot update online

• AR(I)MA and seasonal variants• ARFIMA, GARCH, …

Related workDatabases: Continuous Queries

• Typically, different focus:– “Compression”– Not generative models

• Largely orthogonal problem…– Gilbert, Guha, Indyk et al. (STOC 2002)– Garofalakis, Gibbons (SIGMOD 2002)– Chen, Dong, Han et al. (VLDB 2002); Bulut, Singh (ICDE 2003)– Gehrke, Korn, et al. (SIGMOD 2001), Dobra, Garofalakis, Gehrke

et al. (SIGMOD 2002)– Guha, Koudas (ICDE 2003) Datar, Gionis, Indyk et al. (SODA

2002)– Madden+ [SIGMOD02], [SIGMOD03]

• Adapt and handle arbitrary periodic components

• No human intervention/tuning

• Single pass over the data

• Limited memory (logarithmic)

• Constant-time update

Outline

• Conclusions

Wavelets“Straight” signal

WaveletsIntroduction – Haar

Wavelets

• So?

• Wavelets compress many real signals well…– Image compression and processing– Vision; Astronomy, seismology, …

• Wavelet coefficients can be updated as new points arrive [Kotidis+]

WaveletsCorrelations

Main ideaCorrelations

• Wavelets are good…

• …we can do even better– One number…– …and the fact that they are

equal/correlated

Proposed method

Wl,tWl,t-1Wl,t-2Wl,t l,1Wl,t-1 l,2Wl,t-2 …

Wl’,t’-1Wl’,t’-2Wl’,t’

Wl’,t’ l’,1Wl’,t’-1 l’,2Wl’,t’-2 …

Small windows suffice… (k~4)

More details…

• Update of wavelet coefficients

• Update of linear models

• Feature selection– Not all correlations are significant– Throw away the insignificant ones– very important!!

[see paper]

(incremental)

(incremental; RLS)

(single-pass)

Complexity• Model update

Space: OlgN + mk2 OlgNTime: Ok2 O1

Where– N: number of points (so far)– k: number of regression coefficients; fixed– m: number of linear models; OlgN

[see paper]

Outline

• Conclusions

• First half used for model estimation

• Models applied forward to forecast entire second half

• AR, Seasonal AR (SAR): R– Simplest possible estimation – no maximum

likelihood estimation (MLE), etc.

• … vs. Python scripts

ResultsSynthetic data – Triangle pulse

• Triangle pulse• AR captures wrong trend (or none)• Seasonal AR (SAR) estimation fails

ResultsSynthetic data – Mix

• Mix (sine + square pulse)• AR captures wrong trend (or none)• Seasonal AR estimation fails

ResultsReal data – Automobile

• Automobile traffic– Daily periodicity with rush-hour peaks– Bursty “noise” at smaller time scales

(filtered)

• AR fails to capture any trend (average)• Seasonal AR estimation fails

• AWSOM spots periodicities, automatically

• Generation with identified noise

ResultsReal data – Sunspot

• Sunspot intensity – Slightly time-varying “period”• AR captures wrong trend (average)• Seasonal ARIMA

– Captures immediate wrong downward trend– Requires human to determine seasonal component period (fixed)

ResultsReal data – Sunspot

• Sunspot intensity – Slightly time-varying “period”

Estimation: 40 minutes (R) vs. 9 seconds (Python)

Variance

• Variance (log-power) vs. scale:– “Noise” diagnostic (if decreasing linear…)

– Can use to estimate noise parameters

~ 1 hour

~Hurst exponent

Running time

stream size (N)

Space requirements

Equal total number of model parameters

Conclusion

Adapt and handle arbitrary periodic components

No human intervention/tuning

Single pass over the dataLimited memory (logarithmic)Constant-time update

Conclusion

Adapt and handle arbitrary periodic components

No human intervention/tuning

Single pass over the dataLimited memory (logarithmic)Constant-time update

no human

limitedresources

Outline

• Introduction - motivation• Problem #1: Streams• Problem #2: Graphs & Virus propagation

– Motivation & problem definition– Related work– Main idea– Experiments

• Conclusions

Introduction

Internet Map [lumeta.com]

Food Web [Martinez ’91]

Protein Interactions [genomebiology.com]

Friendship Network [Moody ’01]

► Graphs are ubiquitious

Introduction

• What can we do with graph analysis?– Immunization;– Information

Dissemination– network value of a

customer [Domingos+] “Needle exchange” networks of drug users

[Weeks et al. 2002]

“bridges”

Problem definition

• Q1: How does a virus spread across an arbitrary network?

• Q2: will it create an epidemic?

• (in a sensor setting, with a ‘gossip’ protocol, will a rumor/query spread?)

Framework

• Susceptible-Infected-Susceptible (SIS) model – Cured nodes immediately become susceptible

Susceptible/

healthy

Infected &

infectious

Infected by neighbor

Cured internally

The model

• (virus) Birth rate β : probability than an infected neighbor attacks

• (virus) Death rate δ : probability that an infected node heals

Infected

Healthy

N2Prob. β

Prob. β

Prob. δ

Epidemic threshold

Defined as the value of , such that

if / < an epidemic can not happen

• given a graph

• compute its epidemic threshold

Epidemic threshold

What should depend on?

• avg. degree? and/or highest degree?

• and/or variance of degree?

• and/or determinant of the adjacency matrix?

Basic Homogeneous Model

Homogeneous graphs [Kephart-White ’91, ’93]

• Epidemic threshold = 1/<k>• Homogeneous connectivity <k>, ie, all

nodes have ~same degree unrealistic

Power-law Networks

• Model for Barabási-Albert networks– [Pastor-Satorras &

Vespignani, ’01, ’02]

– Epidemic threshold = <k> / <k2>

– for BA type networks, with only γ = 3 (γ = slope of power-law exponent)

Epidemic threshold

• Homogeneous graphs: 1/<k>• BA (=3) <k> / <k2>

• more complicated graphs ?

• arbitrary, REAL graphs ?

• how many parameters??

Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A

Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A

largest eigenvalueof adj. matrix A

attack prob.

recovery prob.epidemic threshold

Proof: [Wang+03]

Epidemic threshold for various networks

• sanity checks / older results:

• Homogeneous networks– λ1,A = <k>; τ = 1/<k>

– where <k> = average degree– This is the same result as of Kephart & White !

• Star networks– λ1,A = sqrt(d); τ = 1/ sqrt(d)

– where d = the degree of the central node

• Infinite, power-law networks– λ1,A = ∞; τ = 0 : *any* virus has a chance!

[Barabasi et al]

• Finite power-law networks– τ = 1/ λ1,A

Outline

• Introduction - motivation• Problem #1: Streams• Problem #2: Graphs & Virus propagation

– Motivation & problem definition– Related work– Main idea– Experiments

• Conclusions

Experiments

• 2 graphs– Star network: one “hub” + 99 “spokes”– “Oregon” Internet AS graph:

• 10,900 nodes, 31180 edges

• topology.eecs.umich.edu/data.html

• More in our paper: [SRDS ’03]

β/δ > τ (above threshold)

β/δ = τ (at the threshold)

β/δ < τ (below threshold)

Experiments (Star)

Experiments (Oregon)

β/δ > τ (above threshold)

β/δ = τ (at the threshold)

β/δ < τ (below threshold)

Our prediction vs. previous prediction

• our predictions are more accurate

Oregon Star

PL3PL3

OurOur

β/δ β/δ

Conclusions

We found an epidemic threshold

√ that applies to any network topology

√ and it depends only on one parameter of the graph

Overall conclusions

• Automatic stream mining: AWSOM

• graphs and virus propagation: eigenvalue

Ongoing / related work

• Streams– how to find hidden variables on multiple

streams [w/ Spiros and Jimeng Sun]– ‘network tomography’ [w/ Airoldi +]

• Graphs– graph partitioning [w/ Deepay+]– important subgraphs [w/ Tomkins + McCurley]– graph generators [RMAT, w/ Deepay]

Thank you!

Contact info:christos @ cs.cmu.edu

spapadim @ cs.cmu.edu

deepay @ cs.cmu.edu

Main References

• Spiros Papadimitriou, Anthony Brockwell and Christos Faloutsos Adaptive, Hands-Off Stream Mining VLDB 2003, Berlin, Germany, Sept. 2003.

• [Wang+03] Yang Wang, Deepayan Chakrabarti, Chenxi Wang and Christos Faloutsos: Epidemic Spreading in Real Networks: an Eigenvalue Viewpoint, SRDS 2003, Florence, Italy.

Additional References

• Connection Subgraphs, C. Faloutsos, K. McCurley, A. Tomkins, SIAM-DM 2004 workshop on link analysis

• RMAT: A recursive graph generator, D. Chakrabarti, Y. Zhan, C. Faloutsos, SIAM-DM 2004

• iFilter: Network tomography using particle filters, Edoardo Airoldi, Christos Faloutsos (submitted)

Sensor and Graph Mining

Documents