Sparse Signal Processing Parcimonie en Traitement du Signal
Rémi Gribonval INRIA Rennes - Bretagne Atlantique, France
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Two inverse problems in audio processing
• Source localization ✓ S. Nam
• Audio inpainting✓ A. Adler, N. Bertin, V. Emiya,
M. Elad, C.Guichaoua, M. Jafari, M. Plumbley
2
echange.inria.fr
small-project.eu
lundi 12 novembre 12
November 8th 2012 - R. GRIBONVAL - Let’s Imagine the Future
Source localizationwith S. Nam
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Localization with few microphones
• Possible goals
✓ localize emitting sources✓ reconstruct emitted signals✓ extrapolate acoustic field
• Linear inverse problem
• Need a model
4
y = Mx
time-seriesrecorded
at sensors
(discretized) spatio-temporal acoustic field
2 Rm 2 RN
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Localization with few microphones
• Possible goals
✓ localize emitting sources✓ reconstruct emitted signals✓ extrapolate acoustic field
• Linear inverse problem
• Need a model
4
y = Mx
time-seriesrecorded
at sensors
(discretized) spatio-temporal acoustic field
2 Rm 2 RN
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Physics-driven design of model
• Pressure field
• Wave equation on a domain
• Boundary + initial conditions, e.g.
5
(�p� 1c2
�2
�t2 p)(�r, t) = s(�r, t), �r ⇥ D
�p
�n(⇥r, t) = 0, ⇥r � �D
p(�r, t)
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Physics-driven design of model
• Pressure field
• Wave equation on a domain
• Boundary + initial conditions, e.g.
5
(�p� 1c2
�2
�t2 p)(�r, t) = s(�r, t), �r ⇥ D
�p
�n(⇥r, t) = 0, ⇥r � �D
p(�r, t)
} �x = z
x
Discretization
sources& boundaries
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Group sparse source model
• Few non-moving sources = spatially sparse
6
space
time t
�r
z�r,t
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Group sparse regularization
• Inverse problem
• Regularization with mixed norm
✦ Convex optimization: efficient & provably convergent algorithms
✦ Promotes group sparsity, cf Kowalski & Torresani 2009, Eldar & Mishali 2009, Baraniuk & al 2010, Jenatton & al 2011
7
y = Mx
x = arg minx
12ky �Mxk2
2 + �k�xk1,2
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Setting✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location✓ 6 microphones, random location✓ known complex boundaries✓ ground truth generated with naive
discretization
• Results
Sparse Field Reconstruction
8
Ground truth
Sparse reconstruction
S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Setting✓ 2D+t vibrating plate 77x77 ✓ 2 sources, random location✓ 6 microphones, random location✓ known complex boundaries✓ ground truth generated with naive
discretization
• Results
Sparse Field Reconstruction
8
Ground truth
Sparse reconstruction
S. Nam and R. Gribonval. Physics-driven structured cosparse modeling for source localization, ICASSP 2012
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Localizing the source next door
• Domain, Source and Microphones
9
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Localizing the source next door
• Domain, Source and Microphones
• Sparse source localization
9
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Localizing the source next door
• Domain, Source and Microphones
• Sparse source localization
9
Reasons of success•sparsity of sources•known room shape•known boundaries
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Localizing the source next door
• Domain, Source and Microphones
• Sparse source localization
9
Reasons of success•sparsity of sources•known room shape•known boundaries
What if shape is unknown ?
lundi 12 novembre 12
November 8th 2012 - R. GRIBONVAL - Let’s Imagine the Future
Audio inpaintingwith A. Adler, V. Emiya, M. Elad, M. Jafari, M. Plumbley
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Declipping as a linear inverse problem
11
• Original (unknown) samples
• Clipped (observed) samples
• Subset of reliable samples
• Linear inverse problem
M
x
y
yreliable
yreliable =
x
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Sparse audio models
• Time domain • Time-frequency domain
(Black = zero)
12
Analysis
Synthesis
x ⇡ Dz
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Model ✓ sparsity in time-frequency dictionary
• Algorithm: ✓ find sparse coefficients such that
✦ (Orthonormal) Matching Pursuit (Mallat & Zhang 93)✓ + ensure compatibility with clipping constraint
✦ Convex optimization✓ estimate
A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012
Audio Declipping
13
0 0.01 0.02 0.03 0.04 0.05
−0.5
0
0.5
time (s)
Ampl
itude
x = Dz
y = MDzz
x = Dz
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Model ✓ sparsity in time-frequency dictionary
• Algorithm: ✓ find sparse coefficients such that
✦ (Orthonormal) Matching Pursuit (Mallat & Zhang 93)✓ + ensure compatibility with clipping constraint
✦ Convex optimization✓ estimate
A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012
Audio Declipping
13
0 0.01 0.02 0.03 0.04 0.05
−0.5
0
0.5
time (s)
Ampl
itude
x = Dz
y = MDzz
x = Dz
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Model ✓ sparsity in time-frequency dictionary
• Algorithm: ✓ find sparse coefficients such that
✦ (Orthonormal) Matching Pursuit (Mallat & Zhang 93)✓ + ensure compatibility with clipping constraint
✦ Convex optimization✓ estimate
A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012
Audio Declipping
13
0 0.01 0.02 0.03 0.04 0.05
−0.5
0
0.5
time (s)
Ampl
itude
x = Dz
y = MDzz
x = Dz
Clipped
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Model ✓ sparsity in time-frequency dictionary
• Algorithm: ✓ find sparse coefficients such that
✦ (Orthonormal) Matching Pursuit (Mallat & Zhang 93)✓ + ensure compatibility with clipping constraint
✦ Convex optimization✓ estimate
A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012
Audio Declipping
13
0 0.01 0.02 0.03 0.04 0.05
−0.5
0
0.5
time (s)
Ampl
itude
Declipped
x = Dz
y = MDzz
x = Dz
Clipped
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Model ✓ sparsity in time-frequency dictionary
• Algorithm: ✓ find sparse coefficients such that
✦ (Orthonormal) Matching Pursuit (Mallat & Zhang 93)✓ + ensure compatibility with clipping constraint
✦ Convex optimization✓ estimate
A. Adler, V. Emiya, M. Jafari, M. Elad, R. Gribonval and M. D. Plumbley, Audio Inpainting, IEEE Trans Audio Speech and Language Proc., 2012
Audio Declipping
13
0 0.01 0.02 0.03 0.04 0.05
−0.5
0
0.5
time (s)
Ampl
itude
Declipped
x = Dz
y = MDzz
x = Dz
Clipped Original
lundi 12 novembre 12
November 8th 2012 - R. GRIBONVAL - Let’s Imagine the Future
Summary & next challenges
lundi 12 novembre 12
R. GRIBONVAL - Let’s Imagine the Future November 8th 2012
Inverse problems ... and sparse models
15
Observation Domain
lundi 12 novembre 12
R. GRIBONVAL - Let’s Imagine the Future November 8th 2012
Inverse problems ... and sparse models
16
Observation Domain
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Choosing a model
17
• Expert knowledge (Fourier / wavelets)✓ Harmonic analysis / physics✓ Evolution of species
• Training from corpus ✓ Dictionary learning✓ Individual experience
• «Online» training / adaptivity ?✓ Blind Calibration & Deconvolution✓ Adaptation to new environment
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
Data Jungle
• New data beyond signals and images
18
✓HyperspectralSatellite imaging
✓Spherical geometryCosmology, HRTF (3D audio)
✓GraphsSocial networksBrain connectivity
✓Vector valued Diffusion tensor
Key problem
Versatile low-dimensional models
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
What’s next, please ?
• Unified efficient data processing ✦ Signal processing✦ Machine Learning
• Ground-breaking advances✦ Compressive acquisition and compressive learning✦ Sparse models beyond dictionaries
• Upcoming applications✦ Inpainting / super-resolution (image/video/audio)✦ Distributed video coding✦ Astronomical imaging (interferometry)✦ Low-dose biomedical imaging (CT & IRM) ✦ Audio recording @ high spatial resolution✦ Low-power compressive-sensors ✦ Dynamic high-resolution brain imaging ✦ ...
19
lundi 12 novembre 12
November 8th 2012R. GRIBONVAL - Let’s Imagine the Future
• Frédéric Bimbot• Nancy Bertin, Emmanuel Vincent• Current Docs & Postdocs:✓ Alexis Benichoux, Anthony Bourrier, Srdjan Kitic,
Lei Yu, Cagdas Bilen, ...
• Stéphanie Lemaile• Jules Espiau
22
SPECIAL TH NKS
lundi 12 novembre 12