My Seminar on Linear Systems

8/8/2019 My Seminar on Linear Systems

1/51

. . . . . .

Graphs, Linear Systems and Laplacian MatricesA seminar

Dilawar SinghVersion 0.5

Department of Electrical EngineeringIndian Institute of Technology Bombay

November 29, 2010


2/51

. . . . . .

An old Chinese philosopher never said, Words about graph wortha thousand pictures.

Bending the curve On Langugae, William Sare 1

1http:

//www.nytimes.com/2009/09/13/magazine/13FOB-OnLanguage-t.html
http://www.nytimes.com/2009/09/13/magazine/13FOB-OnLanguage-t.htmlhttp://www.nytimes.com/2009/09/13/magazine/13FOB-OnLanguage-t.html


3/51


4/51. . . . . .

Outline

Solve Ax = b Quick!Goal Special case when A is Laplacian matrix of a Graph.

2Daniel A Spielman ; Proceedings of ICM Hyderabad 2010


5/51. . . . . .

Outline

Solve Ax = b Quick!Goal Special case when A is Laplacian matrix of a Graph.

Connections between spectral graph theory, and numericallinear algrabra. 2

2Daniel A Spielman ; Proceedings of ICM Hyderabad 2010


6/51. . . . . .

Why?


7/51. . . . . .

Why?

Classical Approach!


8/51. . . . . .

Why?

Classical Approach!

Recent Development


9/51. . . . . .

Why?

Classical Approach!

Recent Development

Connections!


10/51. . . . . .

Why?

Classical Approach!

Recent Development

Connections!

?


11/51

. . . . . .

Graphs and Laplacian Matrix

Denition

Graph G given by a triple (V,E,w ),

3

If w represents conductance in corresponding electrical network and x isany potential vector then x T Lx is the total power consumed in network.


12/51


13/51

. . . . . .

Graphs and Laplacian Matrix

Denition

Graph G given by a triple (V,E,w ),Laplacian Matrix L of a graph is naturally dened by the

quadratic form it induce. For a vector x V , the Laplacianquadratic form of G is x T Lx = (u,v )E wu,v (x(u ) x(v)

2).3

More the x jumps, larger the quadratic form. Thus L providesa measure of smoothness of x over the edges of G .

3

If w represents conductance in corresponding electrical network and x isany potential vector then x T Lx is the total power consumed in network.


14/51

. . . . . .

Laplacian continued ...

Dene D to be a diagonal matrix whose diagonal contains d,and dene the weighted adjacency matrix of G by

A(u, v ) = {wu,v if (u, v ) E 0 otherwiset

.a

.b

.e

.c

.d

. 1

.1

.1 .1

.2

.1 2 1 0 0 1 1 3 1 1 00 1 2 1 00 1 1 4 2

1 0 0 2 3


15/51

. . . . . .

Properties of Laplacian Matrix

2 1 0 0 1 1 3 1 1 00 1 2 1 00 1 1 4 2

1 0 0 2 3

They are symetric, have zero row-sums, and have non-positiveoff-diagonal entries.


16/51

. . . . . .


2 1 0 0 1 1 3 1 1 00 1 2 1 00 1 1 4 2

1 0 0 2 3


Positive semi-denite . Every eigenvalue is non-negative.


17/51

. . . . . .


2 1 0 0 1 1 3 1 1 00 1 2 1 00 1 1 4 2

1 0 0 2 3



Connectivity . Let 0 1 2.... n be the eigenvalues.Then 2 0 iff G is connected.


18/51

. . . . . .


2 1 0 0 1 1 3 1 1 00 1 2 1 00 1 1 4 2

1 0 0 2 3



Connectivity . Let 0 1 2.... n be the eigenvalues.Then 2 0 iff G is connected.When two graphs are NOT isomorphic?


19/51

. . . . . .

Some denitions

Denition

Given a set of vertices S V , dene the boundary of S , written (S ) to be the set of edges in V with exactly one vertex in S

It is of great interest in computer science to minimize ormaximize the (S ).


20/51

. . . . . .

Some denitions

Denition



Conductance of a graphLet (S )

def = w | (S )|min( |d(S )| ,|d(V S )|) , where d(S ) is the sum of the

weighted degree of the vertices in S then conductance of graph G ,def = min S V (S )

d


21/51

. . . . . .

Some denitions

Denition



Conductance of a graphLet (S )

def = w | (S )|min( |d(S )| ,|d(V S )|) , where d(S ) is the sum of the

weighted degree of the vertices in S then conductance of graph G ,def = min S V (S )

Iso-perimeter NumberLet i(S )

def = | (S )|min( |(S )| ,| (V S )| ) , then iso-perimeter number of G ,

iGdef = min sG i(S ).

C d


22/51

. . . . . .

Conductance


23/51

C d t


24/51

. . . . . .

Conductance

Fast solution of Linear Equations


25/51

. . . . . .


Conductance Sparcity

Conjugate Gradient Method is fast when conductance is high!



26/51

. . . . . .




Elimination is fast when conductance is low on G and allsubgraphs.



27/51

. . . . . .




Elimination is fast when conductance is low on G and all

subgraphs.PROBLEM Fast solutions when graph is in between theextremes.



28/51

. . . . . .




Elimination is fast when conductance is low on G and all

subgraphs.PROBLEM Fast solutions when graph is in between theextremes.

Sparsify the graph

Solving Linear Equations in Laplacian


29/51

. . . . . .


Conjugate Gradient Methods multiplies matrix A by somevector x at each iteration. The number of iterations dependson the eigenvalue of A . Condition number 4 .

4 Ratio of largest to smallest eigenvalue5

Gerard Meurant, Computer Solution of Large Linear Systems, NorthHolland


30/51



31/51

. . . . . .


Conjugate Gradient Methods multiplies matrix A by somevector x at each iteration. The number of iterations dependson the eigenvalue of A . Condition number 4 .

Preconditioned Iterative Methods Can one nd a matrix B

such that BA will have the same solution but have smallercondition number? We can!

B is known as pre-conditioner.They have been provedincredibly useful in practice. 5

4 Ratio of largest to smallest eigenvalue5 Gerard Meurant, Computer Solution of Large Linear Systems, North

Holland

Preconditioned Iterative Methods


32/51

. . . . . .

Preconditioned Iterative Methods

When A and B are Laplacian Matrix of connected graph,similar analysis holds. 6

Relative condition number of AB + written (A, B ) in thiscase measure how well those graphs approximate each other.B + is Moore-Penrose pseudo-inverse of B.

6 precondition number is now the ratio of largest to smallest non-zeroeigenvalue

Approximation by Sparse Graph


33/51

. . . . . .

Approximation by Sparse GraphSparsicationSparsication is the process of approximating a given graph G by a

sparse graph H . H is said to be an -approximation of G if f (L G , L H ) 1 + , where L G and L H are Laplacian matrix of Gand H .



34/51

. . . . . .

pp y p pSparsicationSparsication is the process of approximating a given graph G by a


G and H are similar in many ways. They have similareigenvalues and the effective resistance between every pair of nodes in approximately the same.


35/51



36/51

. . . . . .



G and H are similar in many ways. They have similareigenvalues and the effective resistance between every pair of nodes in approximately the same.It Solving system in dense matrix Solving in a sparsematrix. Conjugate Gradient Methods are faster on sparseMatrix.

Find H with O (n ) zero entries then solve system in L G byusing a preconditioned Conjugate Gradient with L H aspreconditioner, and solving the system in L H by ConjugateGradient.



37/51

. . . . . .



G and H are similar in many ways. They have similareigenvalues and the effective resistance between every pair of nodes in approximately the same.It Solving system in dense matrix Solving in a sparsematrix. Conjugate Gradient Methods are faster on sparseMatrix.

Find H with O (n ) zero entries then solve system in L G byusing a preconditioned Conjugate Gradient with L H aspreconditioner, and solving the system in L H by ConjugateGradient.Each solve in H takes O (n 2), and for accuracy, PCG will

take log 1. Total complexity would be O ((m + n 2)log 1).

More on Sparcication


38/51

. . . . . .

Does such good specier exists?

7 Spileman, Proceedings of ICM 2010

More on Sparcication


39/51

. . . . . .

Does such good specier exists?

Benezur and Karger seems to have developed very similar. 7 .

7 Spileman, Proceedings of ICM 2010

Subgraph Preconditioner and Support theory


40/51

. . . . . .

Vaidyas idea of precondition Laplacian matrices by the matrixof its own subgraph. The tools used to analyse them are

known as support theory. 8

8 urlhttp://www.sandia.gov/ bahendr/support.html9 Algebraic Tools for Analyzing Preconditioners, Bruce Hendrickson (with

Erik Boman). Invited talk at Preconditioning0310 Kolla et all, CoRR 99



41/51

. . . . . .


known as support theory. 8Vaidya did not publish his results. He build a software. Hisstudent used his work in his dissertation.





42/51

. . . . . .



Laplacian of maximum spanning tree is used as preconditioner.

Lower bounds were not proved. A bit inefficient.





43/51

. . . . . .




Lower bounds were not proved. A bit inefficient.Low stretch Spanning Tree were proven good precondition.

A few edges may be added to maximum spanning tree toimprove the preconditioning 9 .





44/51

. . . . . .




Lower bounds were not proved. A bit inefficient.Low stretch Spanning Tree were proven good precondition.

A few edges may be added to maximum spanning tree toimprove the preconditioning 9 .

Good preconditioner existed10

but the challenge was toconstruct them quickly.



Overview : Application of Laplacian


45/51

. . . . . .

Regression on Graphs . A function f is given on a subset of vertices of G , calculate f over remaining vertices. Minimize

xT

Lx .

11 Daniel P. Spielman and Shang Hua Tang, Smoothed analysis of termination of linear programming algorithms, Mathematical Programming, B,2003. to appear

12 Gilbert Strang, Introduction to Applied Mathematics, 1986



46/51

. . . . . .


xT

Lx .Spectral Graph Theory . Study of eigenvalues and theirrelation with graphs. Adiabatic computing!





47/51

. . . . . .


xT

Lx .Spectral Graph Theory . Study of eigenvalues and theirrelation with graphs. Adiabatic computing!Interior Point Method and Maximum Flow Problem.Equivalent to solving system of linear equations that canbe reduced to restricted Laplacian Systems .11

Resistor Networks. Measurement of effective resistancebetween two vertices.





48/51

. . . . . .


xT

Lx .Spectral Graph Theory . Study of eigenvalues and theirrelation with graphs. Adiabatic computing!Interior Point Method and Maximum Flow Problem.Equivalent to solving system of linear equations that canbe reduced to restricted Laplacian Systems .11

Resistor Networks. Measurement of effective resistancebetween two vertices.

Partial Differential Equations. Laplacian Matrix of path graph Vibration of a string. FEM to solve Laplaces equationsusing a triangulation with no obtuse angle. 12




49/51

Conclusion


50/51

. . . . . .

There is a connection!

Koutis, Miller and Peng 13 made tremendous progress on thisproblem, they produces ultra-sparsiers that lead to analgorithm for solving linear systems in Laplacian that takestime O (m log2 n (log log n )2 log 1).This is much faster than any algorithm known to date!!

13 http://arxiv.org/abs/1003.2958v1

Conclusion
http://arxiv.org/abs/1003.2958v1


51/51

. . . . . .

There is a connection!

Koutis, Miller and Peng 13 made tremendous progress on thisproblem, they produces ultra-sparsiers that lead to analgorithm for solving linear systems in Laplacian that takestime O (m log2 n (log log n )2 log 1).This is much faster than any algorithm known to date!!

To Do : Solve electrical networks using interior pointmethods. Plug in Laplacian.

13 http://arxiv.org/abs/1003.2958v1
http://arxiv.org/abs/1003.2958v1

Date post:	09-Apr-2018
Category:	Documents
Upload:	dilawar-singh
View:	220 times
Download:	0 times

My Seminar on Linear Systems

Documents