Date post: | 14-Apr-2017 |
Category: |
Data & Analytics |
Upload: | so-yeon-kim |
View: | 56 times |
Download: | 1 times |
Convolutional Neural Networks on Graphs
with Fast Localized Spectral FilteringDefferrard, Michaël, Xavier Bresson, and Pierre Van-
dergheynstNIPS 2016
Unstructured data as graphs• Majority of data is naturally unstructured, but can be
structured.• Irregular / non-Euclidean data can be structured with
graphs• Social networks: Facebook, Twitter.• Biological networks: genes, molecules, brain connectivity.• Infrastructure networks: energy, transportation, Internet, tele-
phony.
• Graphs can model heterogeneous pairwise relation-ships.• Graphs can encode complex geometric structures.
CNN architecture
• Convolution filter translation or fast Fourier transform (FFT).• Down-sampling pick one pixel out of n.
Generalizing CNNs to graphs• Challenges• Formulate convolution and down-sam-
pling on graphs• How to define localized graph filters?• Make them efficient
Generalizing CNNs to graphs1. The design of localized convolutional filters on graphs2. Graph coarsening procedure (sub-sampling)3. Graph pooling operation
• : undirected and con-nected graph
• Spectral graph theory• Graph Laplacians
• Normalized Laplacians
: set of vertices : set of edges : weighted adjacency matrix : diagonal degree matrix : identity matrix
Graph Fourier Transform
Graph Fourier Transform• Graph Fourier Transform• (Eigen value decomposition)• Graph Fourier basis • Graph frequencies = 1. Graph signal , 2. Transform
Spectral filtering of graph signals• Convolution on graphs
• filtered signal
• A non-parametric filter Non-localized in vertex domain Learning complexity in Computational complexity in
Polynomial parametrization for localized fil-ters•
order polynomials of the Laplacian -> -localized Learning complexity in Still, computational complexity in because of multiplication with Fourier
basis
• Filter localization on graph
Recursive formulation for fast filter-ing
• Chebyshev expansion • Filtered • multiplications by a sparse costs
Learning complexity in Computational complexity in
Graph coarsening and pooling
• Graph coarsening• To cluster similar vertices together, multilevel clustering algorithm is needed.• Pick an unmarked vertex and matching it with one of its unmarked neighbors that maxi-
mizes the local normalized cut • Pooling of graph signals
• Balanced binary tree structured coarsened graphs• ReLU activation with max pooling
• e.g.
level 0
level 1
level 2
Graph ConvNet (GCN) architecture
Experiments• MNIST
• CNNs on a Euclidean space• Comparable to classical CNN• Isotropic spectral filters
• edges in a general graph do not pos-sess an orientation
Experiments• 20NEWS
• structure documents with a feature graph
• 10,000 nodes, 132,834 edges
𝑂 (𝑛2)
𝑂 (𝑛)
Conclusion• Contributions• Spectral formulation of CNNs on graphs in GSP• Strictly localized spectral filters are proposed• Linear complexity of filters• Efficient pooling on graphs
• Limitation• Filters are not directly transferrable to a different graph
References• Deep Learning on Graphs, a lecture on A Network Tour of Data
Science (NTDS) 2016• Shuman, David I., et al. "The emerging field of signal processing
on graphs: Extending high-dimensional data analysis to networks and other irregular domains." IEEE Signal Processing Magazine 30.3 (2013): 83-98.• How powerful are Graph Convolutions? (
http://www.inference.vc/how-powerful-are-graph-convolutions-review-of-kipf-welling-2016-2/)• GRAPH CONVOLUTIONAL NETWORKS (
http://tkipf.github.io/graph-convolutional-networks/)