Collective Sensing: a Collective Sensing: a Fixed-Point Approach in Fixed-Point Approach in

the Metric Spacethe Metric Space11

Xin LiXin Li

LDCSEE, WVULDCSEE, WVU

1This work is partially supported by NSF ECCS-0968730

Unreasonable Effectiveness Unreasonable Effectiveness of Mathematics in of Mathematics in

EngineeringEngineering ““Unreasonable effectiveness of mathematics in Unreasonable effectiveness of mathematics in

natural sciences” Wigner’1960natural sciences” Wigner’1960 To understand how nature works, you need to grasp To understand how nature works, you need to grasp

the tool of mathematics firstthe tool of mathematics first The tension between mathematicians and The tension between mathematicians and

engineersengineers Wavelets vs. filter banksWavelets vs. filter banks ““the hype that would arise around wavelets caused the hype that would arise around wavelets caused

surprise and some understandable resentment in the surprise and some understandable resentment in the subband filtering community” in subband filtering community” in Where do wavelets Where do wavelets come from?come from? I. Daubechies’1996 I. Daubechies’1996

Compressed sensing is another example of how Compressed sensing is another example of how mathematicians have stolen the show from mathematicians have stolen the show from engineersengineers

Mathematical Structures Mathematical Structures are are

Double-Bladed SwordsDouble-Bladed SwordsHilbert-space: a completeInner-product space

Quantum mechanics

Fourier/waveletanalysis

Learning theory

PDE(e.g., Total-Variation)

Mathematical formalism(Hilbert, Ackermann, Von Neumann …)

Metric space: a set witha notion of distance

General relativity

Fixed-point theorems

Game theory

Dynamic systems

Mathematical constructivism(Poincare, Brouwer, Weyl …)

Criticism of Compressed Criticism of Compressed SensingSensing

Where does sparsity come from?Where does sparsity come from?

Nonlinear processing of wavelet coefficientsNonlinear processing of wavelet coefficients Nonlinear diffusion minimizing TVNonlinear diffusion minimizing TV

What is wrong?What is wrong? Over-emphasize the role of Over-emphasize the role of localitylocality (it does (it does

not hold in complex systems)not hold in complex systems) Inner-product is an Inner-product is an artificialartificial structure (it structure (it

carries little insight about how patterns form carries little insight about how patterns form in nature)in nature)

basisfunctions approximation of l0

12||||||||

2

1min 2

ll αΦαfα

signalof interest

A Physical View of SparsityA Physical View of Sparsity

How nature works? (e.g., variational How nature works? (e.g., variational principle)principle) Reaction-diffusion systems A. Turing’1952Reaction-diffusion systems A. Turing’1952 ““More is Different.” P.W. Anderson’1972More is Different.” P.W. Anderson’1972 Self-organizing systems I. Prigogine’1977Self-organizing systems I. Prigogine’1977 Fractals and Chaos Mandelbrot’1977Fractals and Chaos Mandelbrot’1977 Complex networks 1990s-Complex networks 1990s-

Implications into image processing Implications into image processing Hilbert space might not be a proper Hilbert space might not be a proper

mathematical framework for characterizing mathematical framework for characterizing the complexity of natural images? the complexity of natural images?

From Hilbert-space to From Hilbert-space to Metric-spaceMetric-space

Images are viewed as the fixed-Images are viewed as the fixed-points in the metric space points in the metric space ff==PPff

Nonlinear mapping Nonlinear mapping PP characterizes characterizes the organizational principle the organizational principle underlying imagesunderlying images

Example (nonlocal filter):Example (nonlocal filter):

Non-expansiveness of PNLF guarantees the existence of fixed-points

'')',';,(

'')','()',';,(

dydxyxyxw

dydxyxfyxyxwfNLFP

Bilateral , nonlocal mean and BM3D filters are special cases of PNLF

““Phase SpacePhase Space”” of Image of Image SignalsSignalsSA-DCT TV BM3D Nonlocal-TV

Local filters Nonlocal filters

Nonlocal Regularization Nonlocal Regularization MagicMagic

BM3D

Nonlocal-TV

Key Observation: As the temperature (regularization) parameter varies, nonlocalmodels can traverse different phases corresponding to coarse/fine structures

From Compressed Sensing From Compressed Sensing to Collective Sensingto Collective Sensing

Key messages:

1. From local to nonlocal regularization thanks to the fixed-point formulation in the metric space (PNLF depends on the clustering result or similarity matrix)

2. From convex to nonconvex optimization: deterministic annealing (also-called graduated nonconvexity) is the ``black magic” behind

Variational Variational InterpretationsInterpretations

12

||||)(||)( 2

llDJ ffgf

V

ffgfu

lwlw uDJ12|)(|||)(||)( 2

U

ffgfu uv

lvluw uDJ~

2

02|)(|||)(||)(

TV:

Nonlocal TV:

BM3D:

Application (I): Collective Application (I): Collective SensingSensing

l1-magicPSNR=68.53dB

OursPSNR=84.47dB

l1-magicPSNR=19.53dB

OursPSNR=40.97dB

Application (II): Lossy Application (II): Lossy CompressionCompression

House (256×256) Barbara (512×512)

JPEG-decoded

NL-enhancedNL-enhanced

SPIHT-decoded

MATLAB codes accompanying this work are available at my homepage:http://www.csee.wvu.edu/~xinl/ or Google “Xin Li WVU”

Image Comparison Image Comparison ResultsResults

JPEG-decoded at rate of 0.32bpp(PSNR=32.07dB)

NL-enhanced at rate of 0.32bpp(PSNR=33.22dB)

SPIHT-decoded at rate of 0.20bpp(PSNR=26.18dB)

NL-enhanced at rate of 0.20bpp(PSNR=27.33dB)

Maximum-Likelihood (ML) Decoding

Maximum a Posterior (MAP) Decoding

Application (III): Image Application (III): Image DeblurringDeblurring

ISNR(dB) comparison among competing deblurring schemes for cameraman image: uniform 9×9 blurring kernel and noise level of BSNR=40dB

Image Comparison Image Comparison ResultsResults

original degraded TVMM

OursISTSADCT

Unexpected ConnectionsUnexpected Connections

Spectral clusteringSpectral clustering Eigenvectors of graph Laplacian Eigenvectors of graph Laplacian

determine a provably optimal embeddingdetermine a provably optimal embedding Nonlinear dynamical systemsNonlinear dynamical systems

Regularization implemented by the joint Regularization implemented by the joint force of excitation and inhibition in a force of excitation and inhibition in a neuron networkneuron network

Statistical physicsStatistical physics Variational principle underlying Ising Variational principle underlying Ising

model, spin glass and Hopfield network model, spin glass and Hopfield network

Summary and Summary and ConclusionsConclusions

One way of competing with One way of competing with mathematicians is to think like mathematicians is to think like physicistsphysicists

Basis construction/pursuit is only one Basis construction/pursuit is only one (local and suboptimal) way of (local and suboptimal) way of understanding understanding sparsitysparsity

Nonlocal regularization can more Nonlocal regularization can more effectively handle effectively handle complexitycomplexity of natural of natural imagesimages

The distinction between signals and The distinction between signals and systems is artificial and a systems is artificial and a holisticholistic ((collectivecollective) view is preferred) view is preferred

Ongoing WorksOngoing Works

Duality between similarity and dissimilarityDuality between similarity and dissimilarity The implication of sensory inhibition into image The implication of sensory inhibition into image

processingprocessing From graphical models to complex From graphical models to complex

networksnetworks The role of complex network topology The role of complex network topology

Unification of signal reconstruction and Unification of signal reconstruction and object recognitionobject recognition To remove the artificial boundary between low-To remove the artificial boundary between low-

level and high-level visionlevel and high-level vision