Spectral Methods and Algorithms: applications in...

transcript

UCSB 2011

Welcome

Spectral Methods and Algorithms: applications inneuroscience

Jonathan J. Crofts

Department of Physics & MathematicsNottingham Trent University

August 5, 2011

jonathan.crofts@ntu.ac.uk

J. J. Crofts () UCSB 2011 August 5, 2011 1 / 51

UCSB 2011

Introduction and Motivation

Outline1 Introduction and Motivation2 Algebraic Graph Theory3 Network Reorderings

Directed Hierarchies4 Matrix Functions and Walks

Approximate Bipartite SubstructuresWeighted Networks

Strokes Vs Controls5 Connecting it all Together

UCSB 2011

Background

Joel E. CohenMathematics is biology’s next microscope, only better

Quantative/computational work in biology may be data driven or mayarise through modelling

ModelQuantative, simplified description of a natural system

Useful for

testing/comparing hypothesesmaking predictions

This talk will focus on networks: extracting useful information andmodelling

UCSB 2011

Background

Joel E. CohenMathematics is biology’s next microscope, only better

Quantative/computational work in biology may be data driven or mayarise through modelling

ModelQuantative, simplified description of a natural system

Useful for

testing/comparing hypothesesmaking predictions

This talk will focus on networks: extracting useful information andmodelling

UCSB 2011

Typical Tasks

Data Driven:

find well-connected clustersfind specific connectivity substructuresfind ‘important’ nodes or linkscompare the properties of one network with another

Modelling Arguments:

summarize a network in terms of a few parametersexplain how the connectivity has arisendiscover missing or spurious linksmake predictions concerning future growth of the network

UCSB 2011

Network Science: connections are important

Complex networks are the structural skeletons of complexsystems

UCSB 2011

Algebraic Graph Theory

UCSB 2011

Graph Spectra

Spectral methods:matrix representation of the networkstudy the spectra of the resulting matrix, i.e., eigenvalues andeigenvectors

Importantly:this allows us to compute graph invariants using basic linearalgebra; andto implement data-mining tools to study networks, i.e., determinepatterns and features

UCSB 2011

Graph Spectra

Spectral methods:matrix representation of the networkstudy the spectra of the resulting matrix, i.e., eigenvalues andeigenvectors

Importantly:this allows us to compute graph invariants using basic linearalgebra; andto implement data-mining tools to study networks, i.e., determinepatterns and features

UCSB 2011

Graph Spectra: adjacency matrix

0 0 0 1 0 1 0 00 0 0 0 0 1 0 10 0 0 1 0 0 0 01 0 1 0 1 0 1 10 0 0 1 0 0 0 01 1 0 0 0 0 1 00 0 0 1 0 1 0 00 1 0 1 0 0 0 0

{1, i ∼ j0, otherwise

here ∼ denotes that vertices i and j are adjacent

UCSB 2011

Other Possibilities Include . . .

2 0 0 −1 0 −1 0 00 2 0 0 0 −1 0 −10 0 1 −1 0 0 0 0

−1 0 −1 5 −1 0 −1 −10 0 0 −1 1 0 0 0

−1 −1 0 0 0 3 −1 00 0 0 −1 0 −1 2 00 −1 0 −1 0 0 0 2

Graph Laplacian

{di , i = j−aij , otherwise

here di denotes the degree of node i

UCSB 2011

Other Possibilities Include . . .

Normalised Laplacian:

1, i = j− aij√

di dj, otherwise

Signless Laplacian:

{di , i = jaij , otherwise

less well studiedcan determine bipartite structures (Kirkland and Paul 2011)

Line Graph:used to detect community structure (Evans & Lambiotte 2010)

UCSB 2011

An Example: neuronal network of C. elegans

C. elegans are tiny (1mm long),transparent, round worms

Model organism in biology

Connectome consists of some302 neurons linked by over 7000synaptic connections

UCSB 2011

Motivation

UCSB 2011

Local C. elegans Neuronal Network

0 50 100

Adjacency Matrix

0 50 100

Reordered Adjacency Matrix

Reordering the frontal neurons of C. elegans using eigenvectorsof the signless Laplacian Q reveals bipartite substructures

UCSB 2011

C. elegans Example Ctd.

0 10 20

Bipartite Substructure

UCSB 2011

Properties of the Different Graph Spectra

Important: cospectral graphs are not necessarily isomorphic

Example:

χG = λ5 − 4λ3

UCSB 2011

Important: cospectral graphs are not necessarily isomorphic

Example:

χG = λ5 − 4λ3

UCSB 2011

Properties the spectrum (eigenvalues) can and cannot distinguish:

Matrix # edges bipartite # components # bipartitecomponents

A Yes Yes No NoL Yes No Yes NoL No Yes Yes YesQ Yes No No Yes

UCSB 2011

Which graphs are determined by their spectrum? (Van Dam &Haemers (2003))

For almost all graphs this is an open questionNumerical simulations suggest that ‘almost all graphs are’

A combination of eigenvalues and eigenvectors does the trick,e.g., subgraph centrality

CS (i) =n∑

eλk x[k ]2

1.69051.69053.76221.69051.6905

2.38112.38111.00002.38112.3811

UCSB 2011

Which graphs are determined by their spectrum? (Van Dam &Haemers (2003))

For almost all graphs this is an open questionNumerical simulations suggest that ‘almost all graphs are’

A combination of eigenvalues and eigenvectors does the trick,e.g., subgraph centrality

CS (i) =n∑

eλk x[k ]2

1.69051.69053.76221.69051.6905

2.38112.38111.00002.38112.3811

UCSB 2011

Network Reorderings

UCSB 2011

Network Reorderings

An Example: short-range structure

The network reordering problem:

0 50 100 150 200

nz = 2804

0 50 100 150 200

nz = 2804

A(p,p)

• Solve

min∑{p∈P}

(pi − pj)2aij

P denotes the set of permutationsof the integers {1, . . . ,n}

• An approximate solution isgiven by the first, non-zero eigen--vector of L = D − A

UCSB 2011

Network Reorderings

0 50 100 150 200

nz = 2804

0 50 100 150 200

nz = 2804

A(p,p)

• Solve

min∑{p∈P}

(pi − pj)2aij

UCSB 2011

Network Reorderings

0 50 100 150 200

nz = 2804

0 50 100 150 200

nz = 2804

A(p,p)

• Solve

min∑{p∈P}

(pi − pj)2aij

UCSB 2011

Network Reorderings

Directed Hierarchies

UCSB 2011

Network Reorderings

0 1 0 0 0 0 0 00 0 1 0 0 1 0 10 0 0 0 0 0 0 00 1 0 0 1 0 0 00 0 0 0 0 1 0 00 0 0 0 0 0 0 00 0 0 0 1 0 0 10 0 0 0 0 0 0 0

0 0 0 1 0 1 0 00 0 0 0 0 1 0 10 0 0 1 0 0 0 00 0 0 0 1 0 1 10 0 0 0 0 0 0 00 0 0 0 0 0 1 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0

In general it is possible to find such an ordering iff we have a DAG

UCSB 2011

Network Reorderings

0 1 0 0 0 0 0 00 0 1 0 0 1 0 10 0 0 0 0 0 0 00 1 0 0 1 0 0 00 0 0 0 0 1 0 00 0 0 0 0 0 0 00 0 0 0 1 0 0 10 0 0 0 0 0 0 0

0 0 0 1 0 1 0 00 0 0 0 0 1 0 10 0 0 1 0 0 0 00 0 0 0 1 0 1 10 0 0 0 0 0 0 00 0 0 0 0 0 1 00 0 0 0 0 0 0 00 0 0 0 0 0 0 0

In general it is possible to find such an ordering iff we have a DAG

UCSB 2011

Network Reorderings

Out Minus In Degree

One-Sum Optimisation Problem

minp∈P

∑i,j

(pi − pj)aij

∑i,j

(pj − pi)aij =∑

pi · degouti −

pj · deginj

pi · (degouti − degin

UCSB 2011

Network Reorderings

Out Minus In Degree

One-Sum Optimisation Problem

minp∈P

∑i,j

(pi − pj)aij

∑i,j

(pj − pi)aij =∑

pi · degouti −

pj · deginj

pi · (degouti − degin

UCSB 2011

Network Reorderings

Synthetic Network Example

0 20 40 60 80 100 120

nz = 959

0 20 40 60 80 100 120

nz = 959

Sorted by in minus out degree

Hierarchical structure is uncovered in RHS using out-in degree

UCSB 2011

Matrix Functions and Walks

UCSB 2011

Paths Vs Walks

Path Walk

Shortest Path

UCSB 2011

Counting Walks

From the following identity

(Ak )ij =n∑

n∑i2=1

· · ·n∑

ik−1=1

ai,i1ai1,i2 · · · aik−1,j ,

we see that (Ak )ij counts the number of different walks of length kbetween nodes i and j ; moreover, the quantity

Fij = (c0I + c1A + c2A2 + c3A3 · · · )ij

with the ck constant, gives a measure of the total number of walksbetween nodes i and j

UCSB 2011

Counting Walks

From the following identity

(Ak )ij =n∑

n∑i2=1

· · ·n∑

ik−1=1

ai,i1ai1,i2 · · · aik−1,j ,

we see that (Ak )ij counts the number of different walks of length kbetween nodes i and j ; moreover, the quantity

Fij = (c0I + c1A + c2A2 + c3A3 · · · )ij

with the ck constant, gives a measure of the total number of walksbetween nodes i and j

UCSB 2011

Counting Walks: matrix functions

For suitable choices of {ck}k≥0 the series overpage convergesfor example, ck = {δk}k≥0, gives the matrix resolvent (I− δA)−1

different choices of ck allow for different scalings

In particular, if f is defined on the spectrum of A, we can define

F = f (A) = Pf (D)P−1

(here A = PDP−1 is the eigendecomposition)

Proof Let

f (A) = c0I +∞∑

substituting A = PDP−1 into the above

f (A) = f (PDP−1) = c0I +∞∑

ck (PDP−1)k = P

c0I +∞∑

UCSB 2011

Counting Walks: matrix functions

For suitable choices of {ck}k≥0 the series overpage convergesfor example, ck = {δk}k≥0, gives the matrix resolvent (I− δA)−1

different choices of ck allow for different scalings

In particular, if f is defined on the spectrum of A, we can define

F = f (A) = Pf (D)P−1

(here A = PDP−1 is the eigendecomposition)

Proof Let

f (A) = c0I +∞∑

substituting A = PDP−1 into the above

f (A) = f (PDP−1) = c0I +∞∑

ck (PDP−1)k = P

c0I +∞∑

UCSB 2011

Example: communicability

( 0 1 1 11 0 1 01 1 0 11 0 1 0

), A2 =

( 3 1 2 11 2 1 22 1 3 11 2 1 2

), · · ·

Communicability between distinct nodes i and j(I + A + A2/2! + A3/3! + · · ·

that is(eA)

Spectral form:(∑n

k=1 eλk x[k ](i)x[k ](j))

UCSB 2011

Example: communicability

0 50 100

Aexp(A)

Communicability applied to the frontal network of C. elegans

UCSB 2011

Approximate Bipartite Substructures

UCSB 2011

Approximate Directed Bipartite Community

Crofts, Estrada, Higham, Taylor Elec. Trans. Numer. Anal (2010)

Distinct subsets of nodes S1 and S2 such thatS1 has few internal linksS2 has few internal linksthere are many S1 → S2 linksfew other links involve S1 or S2

0 5 10 15 20 25 30

UCSB 2011

An alternating walk of length k from node i1 to node ik+1

is a list of nodesi1, i2, i3, . . . , ik+1

such that ais,is+1 6= 0 for s odd, and ais+1,is 6= 0 for s even

Loosely, an alternating walk is a traversal that successively followslinks in the forward and reverse directions

UCSB 2011

This Motivates . . .

f (A) = I − A +AAT

2!− AAT A

AAT AAT

4!− · · ·

Overall idea: f (A) + f (AT ) haspositive values representing inter-community S1 ↔ S1and S2 ↔ S2 relationships, andnegative values representing extra-community S1 ↔ S2relationships

Also, f (A) + f (AT ) is a symmetric matrix, soamenable to standard clustering techniques

Note: f (A) defined above is not a matrix functionJ. J. Crofts () UCSB 2011 August 5, 2011 33 / 51

UCSB 2011

Synthetic Example

30nz = 252

30282624222018161412108642

exp(A)

30282624222018161412108642

exp(−A)

30282624222018161412108642

f(A) + f(A T)

−500

UCSB 2011

C. elegans Neural Data

Automates the computations of Durbin, (PhD thesis, Cambridge, 1987)

nz = 964

Reordered f(A) + f(AT)nz = 96

Subnetwork of A

UCSB 2011

Weighted Networks

UCSB 2011

Weighted Networks

Communicability for a Weighted Network

W is symmetric with non-negative real weightsLet

di =n∑

wik & D := diag (di)

Normalisation:W 7→ D−1/2WD−1/2

for at least 2 reasonsavoid overflowto stifle promiscuous nodes

Communicability measure:

D−1/2WD−1/2)

UCSB 2011

Weighted Networks

Communicability for a Weighted Network

W is symmetric with non-negative real weightsLet

di =n∑

wik & D := diag (di)

Normalisation:W 7→ D−1/2WD−1/2

for at least 2 reasonsavoid overflowto stifle promiscuous nodes

Communicability measure:

D−1/2WD−1/2)

UCSB 2011

Weighted Networks

But What Does it Actually Mean?

0 w12 w13 w14

w21 0 w23 w24w31 w32 0 w34w41 w42 w43 0

The k th powers of W provide ameasure of the total strengthcontained within walks of length kbetween nodes i and j :

(W 2)ij =∑

wikwkj , (W 3)ij = · · ·

UCSB 2011

Weighted Networks

Example: anatomical connectivity data

9 subjects - at least 6 monthsfollowing first, left-hemisphere,subcortical stroke; and 10 (18)age matched controlsDiffusion Tensor Imagingcomputes all connectionsbetween all voxelsConnectivity network based onthe Harvard-Oxford corticaland subcortical structuralatlas: 48 cortical regions and 8subcortical regions

http://wwww.fmrib.ox.ac.uk/fsl/fslview/

atlas-descriptions.html

UCSB 2011

Weighted Networks

Unsupervised Clustering of Patients

0 5 10 15 20−0.5

−0.4

−0.3

−0.2

−0.1

0.4Raw

0 5 10 15 20−2

−1.5

−0.5

−3 Normalised

0 5 10 15 20−0.6

−0.5

−0.4

−0.3

−0.2

−0.1

0.4Communicability

Crofts & Higham, Roy. Soc. Interface (2009)

UCSB 2011

Weighted Networks

Communicability Adds Value to the Raw Data

10 20 30 40 50

Raw Data (Control)

10 20 30 40 50

Communicability (Control)

UCSB 2011

Weighted Networks

What Are we Actually Detecting?

QuestionExactly what enables us to differentiate between strokes & controls?

If it is merely the fact that many connections have been destroyedclose to the infarcted region this is not very interesting – an MRIscan can tell us this with no further analysis needed!

Why not rerun the tractographies including only those connectionswithin the non-stroke hemisphere?If we still distinguish between the two classes, this will be morerelevant from a bio point of view

UCSB 2011

Weighted Networks

What Are we Actually Detecting?

QuestionExactly what enables us to differentiate between strokes & controls?

If it is merely the fact that many connections have been destroyedclose to the infarcted region this is not very interesting – an MRIscan can tell us this with no further analysis needed!

Why not rerun the tractographies including only those connectionswithin the non-stroke hemisphere?If we still distinguish between the two classes, this will be morerelevant from a bio point of view

UCSB 2011

Weighted Networks

RHS Hemisphere Sorted by Brain Region

0 10 20 30−0.6

−0.4

−0.2

0.4raw − weighted degree

0 10 20 30−0.6

−0.4

−0.2

0.4comm over 56 nodes

0 10 20 30−1

−0.5

1.5x 10

−3 norm − weighted degree

0 10 20 30−0.06

−0.04

−0.02

0.04norm comm over 56 nodes

UCSB 2011

Weighted Networks

Left (Stroke Side) Vs Right

−0.12 −0.1 −0.08 −0.06 −0.04 −0.02 0 0.02 0.04−0.1

−0.08

−0.06

−0.04

−0.02

−0.1 −0.08 −0.06 −0.04 −0.02 0 0.02 0.04 0.06−0.06

−0.04

−0.02

UCSB 2011

Weighted Networks

Overlay of Stroke Overlap Volume

Highlighted regions are those where patients showed reducedcommunicability relative to controls

UCSB 2011

Weighted Networks

Crofts et al. NeuroImage (2011)

And increased communicability

UCSB 2011

Weighted Networks

Generalised singular value decomposition

A = UCX−1 and B = VSX−1

of a pair of matrices can be used to determine clusters that are‘good’ in one network and ‘poor’ in the other - and vice versa[Xiao et al. (2011)]

Another way to understand this is to note that, in the case whereA and B are invertible the GSVD is closely related to the SVD ofAB−1 and BA−1, hence

AB−1 = UCS−1V T and BA−1 = VSC−1UT

UCSB 2011

Weighted Networks

GSVD Example: anatomical connectivity data

10 20 30 40 50

Controls

10 20 30 40 50

Stroke

GSVD finds a group of brain regions that are much better connected incontrols than strokes

UCSB 2011

Connecting it all Together

UCSB 2011

The Take Home Message

Spectral methods provide a useful tool for determiningdifferent types of network architecture

scalablemost work has focused on the adjacency matrix

Walk based measures are attractive since:combinatorics are described in terms of basic linear algebrainformation does not necessarily flow along geodesicswalks are more tolerent to errors

GSVD and extensions:tensor decompositions (plus multiple tasks/modalities,time-dependent networks)NNMF (Lee et al (2010)), ICA (Smith et al (2005))

UCSB 2011

Thank you!

Colleagues at Strathclyde: Des Higham, Ernesto Estrada & AlanTaylor

Colleagues from Oxford: Heidi Johanson-Berg & Tim BehrensThe stroke data was supplied by Rose Bosnell

This work was supported by the Medical Research Council underproject no. MRC G0601353

Spectral Methods and Algorithms: applications in...

Documents