1

Spectral Analysis of Power-Law Graphs and its Application to Internet Topologies

Milena MihailGeorgia Tech

2

The Internet Phenomenon

Routers

WWW

P2P

Open Decentralized Dynamic Market Competition Security, Privacy

Paradigm Shift :

Networks as Artifacts that we construct.

Networks as Phenomena that we study !

3

Internet Performance

Congestion (TCP/IP, )

Stability (Game Theory, )

Scalability (TCP ? Moore’s Law ?)

WWW , P2P : Index, Search

Van Jacobson 88

Kelly 99

(Kleinberg 97, Google 98)

4

Required Data & Models

Routers

WWW

P2P

Connectivity

Capacity Traffic / Demand

Internet Models, such as GT-ITM, Brite, Inet, for Analytic & Simulation based studies :

How do elements organize ?

5

The Internet Phenomenon

Routers

WWW

P2P

Open Decentralized Dynamic Market Competition Security, Privacy

Paradigm Shift :

Networks as Artifacts that we construct.

Networks as Phenomena that we study !

6

Level of Autonomous Systems

Sprint

AT&T

Georgia Tech

CNN

Topology data from BGP routing tables, collected by NLANR, looking glass-U. Oregon

Decentralized Routing !

7

The AS Graph

~14K nodes in 2002 ( ~2K nodes in 1997)

~30K links in 2002

GeorgiaTech

CNN

AT&T

Sprint

8

The Directed AS Graph

GeorgiaTech

CNN

AT&T

Sprint

Peering Relationships :

Customer – Provider

PeersGao 00, Subramanian et al 01

Five Tier HierarchySubramanian et al 01

9

The Real AS Graph

CAIDA http://www.caida.org

10

Degree-Frequency Power Law

Faloutsos et al

degree

1 3 4 5 10

freq

uen

cy

2 100

11

Rank-Degree Power Law

rank

deg

ree

1 2 3 4 5 10

Faloutsos et al 99

UUNET

SprintC&WUSA

AT&TBBN

12

Eigenvalue Power Law

rank

eig

en

valu

e

1 2 3 4 5 10

Faloutsos et al 99

13

Eigenvalue Power Law

rank

eig

en

valu

e

1 2 3 4 5 10

Faloutsos et al 99

UUNET

SprintC&WUSA

AT&TBBN

14

Heavy Tailed Degree Distribution

Departure from standard Internet Models such as Waxman, Transit-Stub Zegura et al

95

Models and techniques must be revisited

Degrees not concentrated around mean

Highly irregular graphs

Departure from Erdos-Renyi

Sharp concentration around mean, Exponential Tails

15

Power Law Graphs Which primitives drive their evolution ?

Preferential attachment , Barabasi 99, Bollobas et al 00

Multiobjective Optimization, -------------------------------------------------------------------------------------Carlson & Doyle 00, Papadimitriou 02

What are their structural properties ?

Hierarchy, Subramanian et al 01, Govindran et al 02

Clustering

16

Spectral Analysis of Matrices

Examines eigenvalues and eigenvectors.

Useful analogy to signal processing.

All eigenvectors form a basis (complete representation).

Focus on large eigenvalues and the corresponding eigenvectors.

Pervasive in

Algebra : Representation Theory

Algorithms : Markov chain sampling

Complexity : Expanders, Pseudorandomness

Datamining, Information Retrieval

Highly technical application specific adaptations

17

0. Spectral primitives : eigenvalues and eigenvectors.

1. Eigenvectors ~ Significance ,

hence HIERARCHY ( capacity / load )

2. Eigenvectors ~ Clustering

CLUSTERING impacts CONGESTION

(1.) and (2.) use normalization preprocessing of the data

3. On Eigenvectors of Eigenvalue Power Law

4. Further Directions

Outline of Results in this Talk

18

Eigenvectors & Eigenvalues

0 1 0 1 1 0 0 1 0 1 1 0 1 0 1 0 0 11 0 1 0 0 10 1 0 1 0 10 1 0 1 0 11 0 0 1 0 11 0 0 1 0 10 1 1 0 1 00 1 1 0 1 0

A =A =

Ax x=

19

Matrix as a Linear Transformation

0 1 0 1 1 0 0 1 0 1 1 0 1 0 1 0 0 11 0 1 0 0 10 1 0 1 0 10 1 0 1 0 11 0 0 1 0 11 0 0 1 0 10 1 1 0 1 00 1 1 0 1 0

A =A =

5 2

1

01

1

4 6

3

46

7

Matrix as a Linear Transformation

20

5 2

1

01

1

4 6

3

46

7

19 12

17

15

15

13

40 51

40

44

47

51

Step 0

Step 1

Step 2

Step 3

21

1

1

11

1

1 = 3

3

3

33

3

3 9

9

99

9

9

22

Stochastic Normalization

0 1/3 0 1/3 1/3 0 1/3 0 1/3 1/3 0 0 1/3 0 1/3 0 0 1/3 0 1/3 0 0 1/31/30 1/3 0 1/3 0 0 1/3 0 1/3 0 1/31/31/3 0 0 1/3 0 1/3 0 0 1/3 0 1/31/30 1/3 1/3 0 1/3 0 1/3 1/3 0 1/3 000 1/3 1/3 0 0 0 1/3 1/3 0 0 1/31/3

A =A =

Ax x=

23

The Random WalkEigenvalues between 1 and –1 ( 1 and 0 also easy).

1

0

0

00

0

1/3

1/3

1/3

0

0

0

2/91/9

1/9

1/9

2/9

2/9

24

In undirected graphs the weights of the principal eigenvector are proportional to degrees.

1. Principal Eigenvector ~ Significance

Principal Eigenvector is Stationary Distribution, corresponding to to = 1

1/6

1/6

1/6 1/6

1/6

1/6 3/16

2/16

3/163/16

2/16

3/16

25

1. Principal Eigenvector ~ Significance

In directed graphs the weights of the principal eigenvector can vary way beyond degrees.

10^-4

2*10^-4 5*10^-4 0.240.39

26

1. Hierarchy from Principal Eigenvector of Directed AS Graph

Significance by High Degree

Significance by Significant Peers and Customers

Add 5% prob. Uniform jump to avoid sinks

In WWW : Google’s pagerank

27

1. Principal Eigenvector Ranking

28

Eigenvector vs Five Tiers

29

1. Principal Eigenvector Ranking

30

0. Spectral primitives : eigenvalues and eigenvectors.

1. Eigenvectors ~ Significance ,

hence HIERARCHY ( capacity / load )

2. Eigenvectors ~ Clustering

CLUSTERING impacts CONGESTION

(1.) and (2.) use normalization preprocessing of the data

3. On Eigenvectors of Eigenvalue Power Law

4. Further Directions

Outline of Results in this Talk

31

2. Eigenvectors ~ Clustering

1/6

1/6

1/6 1/6

1/6

1/6

= 1

32

2. Eigenvectors ~ Clustering

1/6

1/6

1/6 1/6

1/6

1/6

1/6

1/6

1/6 1/6

1/6

1/6

33

2. Eigenvectors ~ Clustering

1/6

1/6

1/6 1/6

1/6

1/6

1/6

1/6

1/6 1/6

1/6

1/6

-

--

-

-

-

34

2. Eigenvectors ~ Clustering

1/6

1/6

1/6 1/6

1/6

1/6

1/6

1/6

1/6 1/6

1/6

1/6

35

2. Eigenvectors ~ Clustering

1/6

1/6

1/6 1/6

1/6

1/6

1/6

1/6

1/6 1/6

1/6

1/6

36

2. Eigenvectors ~ Clustering

1/6

1/6

1/6 1/6

1/6

1/6

1/6

1/6

1/6 1/6

1/6

1/6

-

--

-

-

-~

~ ~

~ ~

~

~ ~

~~

~ ~

Matrix Perturbation Theory

37

Spectral Filtering

1 2 3n K+1

k=1 > >>>> >

Find clusters in most positive and most negative ends of eigenvectors associated with large eigenvalues.

Heuristic :

( Kleinberg 97, Fiat et al 01 )

38

2. Eigenvectors ~ Clustering

Weig

ht

of

eig

en

vect

or

k rank

1 2 3n

K+1

k=1 > >>>> >

An Example of a Cluster

An Example of a Cluster

An Example of a Cluster

An Example of a Cluster

An Example of a Cluster

44

Additional Matrices

Similarity Matrix A*A^T, where A is directed AS graph.

Complete and Pruned AS topology.

In all cases prune leaves of very big degree nodes. Necessary frequency normalization.

Clusters consistent and evolving over time.

Synthetic Internet topologies have much weaker clustering properties.

45

Clustering and Congestion Assume 1 unit of traffic between

each pair of ASes in each direction. Route traffic in the graph (like

BGP). Compute # of connections using

each link. This is a measure of congestion.

46

Effect of intra-cluster and inter-cluster traffic to most congested link

Internet

Inet Internet Inet

0% 100% 100% 0% 100% 100%

20% 91.5% 97.7% 20% 126% 109%

40% 83.1% 95.3% 40% 153% 117%

60% 74.2% 92.9% 60% 172% 128%

80% 65.7% 90.8% 80% 191% 136%

100%

57.3% 88.5% 100% 207% 142%

47

Outline of Results in this Talk

1. Eigenvectors ~ Significance ,

hence HIERARCHY ( capacity / load )

2. Eigenvectors ~ Clustering

CLUSTERING impacts CONGESTION

(1) and (2) used normalization preprocessing of the data

Normalization preprocessing of data is necessary.

3. Eigenvectors of Eigenvalue Power Law LOCALIZED

4. Further Directions

48

Which Eigenvectors correspond to Eigenvalue Power Law ?

rank

eig

en

valu

e

1 2 3 4 5 10

Faloutsos et al 99

UUNET

SprintC&WUSA

AT&TBBN

49

Large Degrees, or“Stars” of AS Graph

Dominate Spectrum of Adjacency Matrix, Prior to Normalization

50

Principal Eigenvector of a Star

11

1

11

1

1

1

d

51

Disjoint Stars

52

“Mostly” Disjoint Stars

Proof By Matrix Perturbation Theory, Spectral Graph Theory

53

3. Explanation of Eigenvalue Power Law

Theorem : Random graphs whose largest degrees are, in expectation, d_1 > d_2 > … > d_k,

d_j ~ j ^ -a have largest eigenvalues sharply concentrated around

_j ~ j ^ -b for j = 1,…,k,

and corresponding eigenvectors localized on corresponding largest degrees, with very high probability.

54

Summary0. First Spectral Analysis on Internet Topologies.

1. PRINCIPAL EIGENVECTOR implies HIERARCHY

2. EIGENVECTORS of LARGE EIGENVALUES imply CLUSTERING

3. CLUSTERING impacts CONGESTION

4. Defined Intra-cluster and Inter-cluster Traffic.

5. Introduced Heady Tailed Specific Normalization Preprocessing.

6. Explained Eigenvalue Power Law.(1) Through (5) with C. Gkantsidis and E. Zegura

(6) with C. Papadimitriou

55

Further Directions

How does congestion scale in power law graphs ?

Other properties, such as resilience.

What are the growth primitives of power law graphs ?

Optimization tradeoff primitives translate to cost – service

Towards efficient and accurate synthetic data.

Level of Autonomous Systems :

Routing protocol (BGP) stability by game theory.