Big Data Analysis with Signal Processing on Graphs

transcript

Big Data Talks Nile University

Talk 8

Big Data Analysis with Signal Processing onGraphs

Introduction

Fundamentals of Graph Theory

DSP on Graphs

Graph Products

Applications

Mohamed Seif m.seif@nu.edu.eg 8-1

Contents

Introduction

DSP on Graphs

Graph Products

Applications

Big Data Analysis with Signal Processing on Graphs 8-2

Introduction

How to relate the Signal Processing

Definitions with Graphs?

Introduction

How to relate the Signal Processing

Definitions with Graphs?

Contents

Introduction

DSP on Graphs

Graph Products

Applications

What Is Graph?

In graph theory, the graph G is defined as a tuple G = (V, E) where

V = {v0, v1, . . . , vN−1} is the set of N nodes and

E = {eij ,∀(i, j) ∈ {0, 1, . . . , N − 1}} is the set containing all links

between the nodes.

Example:

V = {1, 2, 3, 4}

E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}}

What Is Graph?

between the nodes.

Example:

V = {1, 2, 3, 4}

E = {{1, 2}, {2, 3}, {3, 4}, {4, 1}}

What Is Graph? (cont’d)

between the nodes.

What Is Graph?

between the nodes.

Alternative Representation of Graphs

1. Adjacency matrix AN×N , defined as

Ai,j =

1 if vi & vj are connected

0 o.w.

2. Laplacian graph LN×N , defined as

L = D −A, where D is the degree matrix

Li,j =

deg(vi) if i = j

−1 if i 6= j & vi is adjacent to vj

0 o.w.

Contents

Introduction

DSP on Graphs

Graph Products

Applications

Graph Signals

Given the graph, the data set forms a graph signal, defined as a map

s : V → C, vn → sn (1)

It is convenient to write graph signals as vectors

s = [s0, s1, . . . , sN−1]T ∈ CN×1 (2)

Graph Shift

In DSP, a signal shift, implemented as a time delay

s = [s0, s1, . . . , ˜sN−1]T = Cs (3)

where C is the N ×N cyclic shift matrix.

DSP on Graphs extends the concept of shift to general graphs by

defining the graph shift as a local operation that replaces a signal value

sn at node vn by a linear combination of the values at neighbors of vn

weighted by their edge weights:

sn =∑

m∈Nn

An,msm (4)

Graph Shift (cont’d)

It can be interpreted as a first-order interpolation, weighted

averaging, or regression on graphs, which is a widely used operation

in graph regression, distributed consensus, telecommunications.

Then, the graph shift is written as

s = [s0, s1, . . . , ˜sN−1]T = As (5)

Graph Filters and Z-Transform

In signal processing, a filter is a system H(.) that takes an input

signal s and outputs a signal:

s = [s0, s1, . . . , ˜sN−1]T = H(s) (6)

Among the most widely used filters are linear shift-ivariant (LSI) ones.

The z-transform provides a convenient representation for signals and

filters in DSP. (In short)

An alternative representation for the output signal is given by

s = h(C)s (7)

where h(c) =∑N−1

n=0 hnCn (Resultant is a circulant matrix)

s = [s0, s1, . . . , ˜sN−1]T = H(s) (6)

s = h(C)s (7)

s = [s0, s1, . . . , ˜sN−1]T = H(s) (6)

s = h(C)s (7)

s = [s0, s1, . . . , ˜sN−1]T = H(s) (6)

s = h(C)s (7)

Graph Filters and Z-Transform (cont’d)

DSP on Graphs extends the concept of filters to general graphs.

Similarly to the extension of the time shift to the graph shift, filters

are generalized to graph filters as polynomials in the graph shift , and

all LSI graph filters have the form

h(A) =L−1∑l=0

hlAl (8)

In analogy with signal filters, the graph filter output is given by

s = h(A)s (9)

Graph Fourier Transform

Mathematically, a Fourier transform with respect to a set of operators

is the expansion of a signal into a basis of the operators eigen

functions.

Since in signal processing the operators of interest are filters, DSPG

defines the Fourier transform with respect to the graph filters.

s = [s0, s1, . . . , ˜sN−1]T = GFT{s} (10)

Graph Fourier Transform

Mathematically, a Fourier transform with respect to a set of operators

is the expansion of a signal into a basis of the operators eigen

functions.

Since in signal processing the operators of interest are filters, DSPG

defines the Fourier transform with respect to the graph filters.

s = [s0, s1, . . . , ˜sN−1]T = GFT{s} (10)

Graph Fourier Transform (cont’d)

For simplicity, assume that A is diagonalizable and its decomposition

A = V ΛV −1 (11)

where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N

are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are

eigenvalues of A

In general A can be diagonalized using Jordan decomposition.

For simplicity, assume that A is diagonalizable and its decomposition

A = V ΛV −1 (11)

where the columns vn of the matrix V = [v0 · · · vN−1] ∈ CN×N

are the eigenvectors of A and Λ = diag(λ0, . . . , λN−1) are

eigenvalues of A

In general A can be diagonalized using Jordan decomposition.

The eigenfunctions of graph filters h(A) are given by the eigenvectors

of the graph shift matrix A

Since the expansion into the eigenbasis is given by the multiplication

with the inverse eigenvector matrix, which always exists, the graph

Fourier transform is well defined and computed as

s = [s0, s1, . . . , ˆsN−1]T = V −1s (12)

= Fs (13)

where F = V −1 is the graph Fourier transform matrix.

s = [s0, s1, . . . , ˆsN−1]T = V −1s (12)

= Fs (13)

s = [s0, s1, . . . , ˆsN−1]T = V −1s (12)

= Fs (13)

The inverse graph Fourier transform reconstructs the graph signal

from is frequency content by combining graph frequency components

weighted by the coefficients of the signal’s graph Fourier transform:

s = s0v0 + s1v1 + · · ·+ ˆsN−1vN−1 (14)

= F−1s = V s (15)

Low and High Frequencies on Graphs

The values sn in (12) are the signal’s expansion in the eigenvector

basis and represent the frequency content of the signal s.

The eigenvalues λn of the shift matrix A represent graph frequency

content, and the eigenvectors vn represent the corresponding graph

frequency component.

To conclude, the higher λn, the higher frequency content and vice

versa.

Frequency Response of Graph Filters

In addition to expressing the frequency content of graph signals, the

graph Fourier transform also characterizes the effect of filters on the

frequency content of signals.

The filtering operation s = h(A)s can be written using

h(A) =∑L−1

l=0 hlAl and s = V −1s as follows

s = h(A)s = h(F−1AF )s = F−1h(Λ)Fs (16)

where h(Λ) is a diagonal matrix with values h(λn) =∑L−1

l=0 hlλln

As a result,

s = h(A)s⇔ Fs = h(Λ)s (17)Big Data Analysis with Signal Processing on Graphs 8-32

h(A) =∑L−1

s = h(A)s = h(F−1AF )s = F−1h(Λ)Fs (16)

l=0 hlλln

As a result,

h(A) =∑L−1

s = h(A)s = h(F−1AF )s = F−1h(Λ)Fs (16)

l=0 hlλln

As a result,

Frequency Response of Graph Filters (cont’d)

That is, the frequency content of a filtered signal is modified by

multiplying its frequency content element-wise by h(λn) . These

values represent the graph frequency response of the graph

filter.

The relation is a generalization of the classical convolution theorem

to graphs: filtering a graph signal in the graph domain is equivalent in

the frequency domain to multiplying the signal spectrum by the

frequency response of the graph filter. s = h(A)s⇔ Fs = h(Λ)s

Frequency Response of Graph Filters (cont’d)

That is, the frequency content of a filtered signal is modified by

multiplying its frequency content element-wise by h(λn) . These

values represent the graph frequency response of the graph

filter.

The relation is a generalization of the classical convolution theorem

to graphs: filtering a graph signal in the graph domain is equivalent in

the frequency domain to multiplying the signal spectrum by the

frequency response of the graph filter. s = h(A)s⇔ Fs = h(Λ)s

Contents

Introduction

DSP on Graphs

Graph Products

Applications

Product Graphs

Consider two graphs G1 = (V1, A1) and G2 = (V2, A2) with |V1| = N1

and |V2| = N2 nodes, respectively. The product graph, denoted by �, of

G1 and G2 is the graph

G = G1 �G2 = (V, A�) (18)

with |V| = N1N2 and dim(A�) = N1N2 ×N1N2.

Common Product Graphs Types

1. Kronecker product

A⊗ = A1 ⊗A2 (19)

Example:

If we have two matrices B ∈ CM×N and C ∈ CK×L, then the

Kronecker product is defined as follows

B ⊗ C =

· · ·...

· · ·

b1,M C

bM,M C

∈ CMK×NL (20)

Common Product Graphs Types

1. Cartesian product

A× = A1 ⊗ IN2 + IN1 ⊗A2 (21)

2. Strong product

A� = A1 ⊗A2 +A1 ⊗ IN2 + IN1 ⊗A2 (22)

Examples on Product Graphs

Notes on Product Graphs

Signal Processing on Product Graphs

The computation of filtering and Fourier transform on graphs and

improve algorithms, data storage, and memory access for large data sets

can modularized thanks to graph products. Such as

Filtering

Computation complexity: O(N2) =⇒ O(N(N1 + N2))

Fourier transform

Computation complexity: O(N3) =⇒ O(N31 + N3

Filtering

Fourier transform

Filtering

Fourier transform

Contents

Introduction

DSP on Graphs

Graph Products

Applications

Like-wise traditional DSP problems:

Data compression

Fourier transform or through wavelet expansions, or adaptive filter design

Detection of corrupted data

High pass filter

Applications

Data compression

High pass filter

Applications

Data compression

High pass filter

Challenges of Big Data

While there is no single, universally agreed upon set of properties that

define big data.

Some of the commonly mentioned ones are volume, velocity, and

variety of data.

While there is no single, universally agreed upon set of properties that

define big data.

variety of data.

First of all, the sheer volume of data to be processed requires efficient

distributed and scalable storage, access, and processing.

High velocity of new data arrival demands fast algorithms to prevent

bottlenecks and explosion of the data volume and to extract valuable

information from the data and incorporate it into the decision-making

process in real time. (FFT in DSP)

Finally, collected data sets contain information in all varieties and forms,

including numerical, textual, and visual data. To generalize data analysis

techniques to diverse data sets, we need a common representation

framework for data sets and their structure.

variety of data.

Thank You!

Big Data Analysis with Signal Processing on Graphs

Engineering