1
Spectral Graph Theory and its Applications
Lillian Dai6.454 Oct. 20, 2004
2
Outline
• Basic spectral graph theory
• Graph partitioning using spectral methods
D. Spielman and S. Teng, “Spectral Partitioning Works: Planar Graphs and Finite Element Meshes,” 1996
3
Graph and Associated Matrices
Adjacency matrix
0 1 1 11 0 0 11 0 0 11 1 1 0
GA
=
Degree matrix
3 0 0 00 2 0 00 0 2 00 0 0 3
GD
=
Incidency matrix
1 1 1 0 01 0 0 1 0
0 1 0 0 10 0 1 1 1
GB
− = −
− − −
( ),G V E=
4V n= =
5E m= =
Laplacian matrix
G G GT
G G
L D A
B B
= −
=
1
2
3
4
4
Properties of the Laplacian Matrix
3 1 1 11 2 0 11 0 2 11 1 1 3
GL
− − − − − = − − − − −
1
2
3
4
• Symmetric -> real eigenvalues; eigenspaces are mutually orthogonal
• Orthogonally diagonalizable -> an eigenvalue with multiplicity k has k-dimensional eigenspace
{ }0,2,4,4λ =
1111
01
10
−
111
3
− − −
2110
−
5
More Properties of the Laplacian Matrix
• Positive semidefinite -> non-negative eigenvalues
• Row sum = 0 -> singular -> at least one eigenvalue = 0, unity eigenvector (since row sum = 1)
• Orthogonal eigenspacesu = eigenvector of non-zero eigenvalue
{ }0,2,4,4λ =
1111
01
10
−
111
3
− − −
2110
−
( )( ) ( )( )
2
,
0TT T T T T
G G G G G i ji j E
x L x x B B x x B x B x x∈
= = = − ≥∑
( )1 2, ,..., nnx x x x= ∈
1
0n
ii
u=
=∑
1 m× 1m×
6
Star
Ring
Line
Complete
Eigenvalues
Spectrum of Some Graphs
, , ( ){ }10, nn −
( )2 2cos k nπ−
( )2 2cos 2 k nπ−1,...,k n=
( ){ }20,1 ,2n−
1,..., 2k n=
Which graphs are determined by their spectrum?
• Complete Graphs• Graphs with one edge• Graphs missing 1 edge• Regular graphs with degree 2• Regular graphs of degree n - 3
7
Graph Connectedness
For connected graphs,{ }0,2,4,4λ =
1111
01
10
−
111
3
− − −
2110
−
2λ
1 2 ... nλ λ λ≤ ≤ ≤
Fiedler Value
v Fiedler Vector
2 0λ >
Multiplicity of the 0 eigenvalueindicates # of connected components
( )( )
2
,
TG i j
i j E
x L x x x∈
= −∑Recall
If is eigenvector for eigenvalue 00GL x =
x
i jx x= ( ),i j E∈
8
Onto Graph Partitioning …
9
Graph Partitioning
• Remove as little of the graph as possible to separate out a subset of vertices of some desired “size”• “Size” may mean the number of vertices, number of edges, etc. • Typical case is to remove as few edges as possible to disconnect the graph into two parts of almost equal size
Isoperimetric problem
One of the earliest problems in geometry –considered by the ancient Greeks: Find, among all closed curves of a given length, the one which encloses the maximum area Stein, 1841
Diagram from Berkeley CS 267 lecture notes
10
Applications
• Load balancing while minimizing communication
• Sparse matrix-vector multiplication
• Optimizing VLSI layout
• Communication network design
11
Bisection and Ratio-Partition
• Divide vertices into two disjoint subsets and
• Cut Size
• Cut Ratio
• Isoperimetric Number
SS
( ),E S S
( )( )( )
,
min ,G
E S SS
S Sφ =
( )minG GS VSφ φ
⊂=
( ),E S SBisection Minimize subject to # of nodes in each partition differ by at most 1.
Ratio-Partition Minimize ( )G Sφ
NP-Complete
12
Spectral Partitioning
• Find Fiedler vector of the Laplacian matrix – map to vertices• Choose some real number s• Partition vertices given by
• Bisection, s = median of • Ratio partition, s is chosen to give the best cut ratio
{ }:L iV i v s= ≤
{ }:L iV i v s= >
{ }1,... nv v
13
Example
1
2
3
4
5
6
Fiedler vector [-1 -2 -1 1 2 1]
14
Spectral Partitioning For Planar Graphs
• Guattery and Miller – Performance of Spectral Graph Partitioning, 1995
• Spielman and Teng, Spectral Partitioning Works on Planar Graphs, 1996
• Kelner, Spectral Partitioning Works on Graphs with Bounded Genus, 2004
15
Simple Spectral Bisection May Fail(Guattery & Miller)
The simple spectral bisection method produces cut size of ( )nΘ
for kG , for any k
16
Optimal Bisector for Graphs with Bounded Genus(Kelner)
There is a spectral algorithm that produces bisector of size ( )O gn
Genus g of a graph G: smallest integer such that G can be embedded on a surface of genus g without any of its edges crossing one another. Eg. Planar graphs have genus 0
Sphere, disc, and annulus has genus 0Torus has genus 1
For every g, there is a class of bounded degree graphs that have no bisectors smaller than ( )O gn
17
Improved Bisection Algorithm on Planar Graphs(Spielman and Teng)
Bisector of size ( )O n
Why does the spectral method work?Why does it work well on planar graphs?Why does simple bisection fail even on planar graphs?
18
Another Look at Fiedler Value
Recall where
Rayleigh quotient:
Fiedler value satisfies with the minimum occurring only when is a Fiedler vector.
( )( )2
,2
T i ji j EGx T
i
x xx L xx x x
φ ∈−
= =∑
∑
( )( )
2
,
TG i j
i j E
x L x x x∈
= −∑ ( )1 2, ,..., nnx x x x= ∈
( )2 1,...,1min xx
λ φ⊥
=
x
22
T TG
x T Tx L x x xx x x x
λφ λ= = =
19
Connection Between Fiedler Value and Isoperimetric Number
Theorem 1 (Mihail ‘89) Let be a graph on nodes of
maximum degree . For any vector such that
Moreover, there is an so that the cut has ratio at most
( )( )
,min
min ,G S V
E S S
S Sφ
⊂=Recall Isoperimetric Number
is the best ratio-partition possible
G n
∆ nx∈1
0n
ii
x=
=∑2
2
TG GT
x L xx x
φ≥
∆
s
2
2 2Gφλ ≥∆
Good ratio-partition can be achieved if Fiedler value is small
{ }: ii v s≤ { }: ii v s>
( )2 2Gφ ∆
20
Upperbound on the Fiedler Value for Planar Graphs
Theorem 2 (Spielman & Teng ‘96) For all planar graphs with vertices and maximum degree
Gn ∆
28n
λ ∆≤
1On
2
28
2G
nφ
λ ∆≤ ≤
∆4
G nφ ∆
≤1On
By bounding Fiedler value of planar graphs, ratio-partitioning method is shown to work well
What about bisection?
21
Relationship Between Ratio-Partitioning and Bisection
Lemma 3 Given an algorithm that will find a cut ratio of at mostin every k-node subgraph of , for some monotonically
decreasing function . Then repeated application of this algorithm can be used to find a bisection of of size at most
G
( )1
n
xx dxφ
=∫
( )kφφ
G
( ) 1xx
φ = ( ) ( )1
2 1n
xx dx nφ
== −∫ ( )O n
Bisection can be obtained by repeated application of ratio-partitioning
22
Theorem 1
Map graph vertices to a line
2x
2
2
TG GT
x L xx x
φ≥
∆ 1
0n
ii
x=
=∑
1 2 ... nx x x≤ ≤ ≤
( )( )
,min
min ,G S V
E S S
S Sφ
⊂=
1x nx
If 2i n≤ At least Giφ edges must cross over ix
( )( ) ( )( )
22
,2 2
sum length of edge
sum length away from 0
T i ji j EGT
i
x xx L xx x x
∈−
= =∑
∑
12
3
4
23
Proof of Theorem 2
Theorem 4 (Koebe-Andreev-Thurston). Let G
be a planar graph. Then, there exist a set of disks { }1,..., nD D
in the plane with disjoint interiors such that iD touches jD
iff ( ),i j E∈
. Kissing disks
28n
λ ∆≤
24
Proof of Theorem 2 cont.
Stereographic Projection( ) ( ){ }1 ,..., nD Dπ π
Circles in the plane -> circular caps on the sphere
25
Proof of Theorem 2 cont.
Let ix be the center of ( )iDπ on the sphere.
1ix =2
1
n
ii
x n=
=∑
Let ir be the radius of the cap ( )iDπ
( ) ( )22 2 22i j i j i jx x r r r r− ≤ + ≤ + ( ),i j E∈
2 4irπ π≤∑
( )( )
( )
2 2 2 2
, ,
2 2 8i j i j i ii j E i j E i
x x r r d r∈ ∈
− ≤ + ≤ ≤ ∆∑ ∑ ∑
( )2
,2 2
8i ji j E
i
x x
nxλ ∈
− ∆≤ ≤∑
∑
26
Conclusion
Why does the spectral method work?- close relationship between Fiedler value and Isoperimetric number
Why does it work well on planar graphs?- planar graphs have nice collection of spherical cap embeddings
Why does simple bisection fail even on planar graphs?- even though good ratio-partitions can be found, the result may be unbalanced in the size of the partitions
2
2 2Gφλ ≥∆
28n
λ ∆≤