Wavelets and Signal Processing:
A Match Made in Heaven
Martin Vetterli, EPFL, 29.9.2015with Y.Barbotin, T.Blu, P.Marziliano, H.Pan, R.Parhizkar
Outline
2
• A Tale of Two Communities
• The Golden Age
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions
Acknowledgments
3
• My friends from the two communities
• My collaborators
• H.G.Feichtinger and B.Torresani
4
Outline
• A Tale of Two Communities:
– Jean Morlet
– Signal processors
– Harmonic analysts
• The Golden Age
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions
5
Ondelettes et/and Wavelets
“Le découverte de Morlet n'a pas reçu un bonaccueil. Peu de temps auparavant, un malfaiteurbelge était arrivé à persuader l'entreprise que l'onpouvait « flairer le pétrole » à l'aide d'« avionsrenifleurs ».Passant de l'extrême crédulité à l'extrême méfiance,Elf-Aquitaine répondit à la découverte de JeanMorlet en le mettant en « préretraite » “*
The story of Jean Morlet, the inventor of wavelets
* M.Nowak, Y.Meyer, La Recherche, Feb. 2005
Jean Morlet, 2001
6
Ondelettes et/and Wavelets
Morlet proposed this inversion formula based on intuition and numerical evidence. The story goes that when he showed it to a mathematician for verification, he was told: “This formula, being so simple, would be known if it were correct...”
The continuous wavelet transform by Morlet (1984)
7
Speech Processing: Subband Coding
This looks quite boring…
8
Claude Galand at Work!
9
Time Frequency Methods in Signal Processing
• Crochiere, Esteban, Galand 1976
• Short-time Fourier transform
• Perfect reconstruction filter banks
• Transform coding and KLT
• Subband speech and image coding
10
Harmonic Analysis: Beautiful and…Esoteric ?
Weierstrass function (1872)
11
Harmonic Analysis: Haar and Fourier bases
• Heisenberg uncertainty, Gabor expansion
• Balian-Low
12
The Meeting of the Minds
• Compactly supported wavelets: Ingrid Daubechies
• Multiresolution analysis: Stéphane Mallat, Yves Meyer
• Many Contributors: Stromberg, Lemarié, Battle, Cohen, ….
• Local cosine bases: Coifman, Meyer, Malvar
13
The Meeting of the Minds
Strömberg wavelet (1983)
14
Algorithms
• Wavelets based on filter banks• Orthogonal • Biorthogonal • Multidimensional
• Wavelets packets• Adaptive bases
• Mallat’s algorithm
Roy Lichtenstein: Magnifying Glass, 1963
15
Outline
• A Tale of Two Communities
• The Golden Age:
– When Wavelets Were Going to Cure Cancer
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions
16
Wavelets are looking for applications…… but applications were not waiting for wavelets!
• Wavelets are beautiful• They don’t have to be necessary useful!
• Wavelets create a framework• Many disparate constructions have a common interpretation
• Wavelets and time-frequency-scale is a way of thinking about problems
17
A few stories
• Bell Laboratories Murray Hill• Is a Daubechies’ filter a filter?
• New York Times Science Section• Image compression will be improved, maybe a hundred fold!
• Wright Patterson Airforce Base, Ohio• Speech compression will be improved 10 times!
18
AFIT/AFOSR Wavelets Workshop participants
Greg Wornell, Stephane Mallat, Alexander Grossmann, Alan Oppenheim, Patrick Flandrin, Thomas Barnwell III, Leon Cohen, Gregory Beylkin, Jon Sjogren, Albert Cohen, Ingrid Daubechies, Martin Vetterli, P.P. Vaidyanathan, Robert Tenney, Ronald Coifman, Alan Willsky, Robert Ryan, Bruce Suter, Mark Oxley, John Benedetto, Greg Warhola.
19
Outline
• A Tale of Two Communities
• The Golden Age
• The Beauty and the Beast
– The Story of JPEG 2000
– Contributions of Wavelets
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions
20
JPEG vs JPEG 2000
JPEG• Based on KLT• Theory from 60’s• Block based• Fast DCT
JPEG 2000• Based on wavelets• Theory from 80’s• No blocks• Fast WT
21
And the Winner is…
• JPEG is used in 98% of the cases• Mobile phones, digital
cameras
• JPEG 2000• Used in frame based digital
cinema
• Improvement of 1-2 dB does not justify change of standards
• Patent situation is murky…
From wikipedia page of JPEG2000: « However, the JPEG committee has acknowledged that undeclared submarine patents may still present a hazard. »
22
An example patent…
• Goupillaud, Morlet and Grossman patent
23
Main Contributions of Wavelets to SP:
Use of more general norms (getting rid of the tyranny of SNR ;)
Simple and powerful non-linear approximation for piecewise smooth functions
Thus: a piecewise smooth signal expands as:
• phase changes randomize signs, but not decay• a singularity influence only L wavelets at each scale• wavelet coefficients decay fast
25
Applications: Denoising
original
wavelet
13.8 dB
noisy
countourlets
15.4 dB
26
Lessons learned for signal processing
There is life beyond SNR
• Other norms are key (l1, TV, Sobolev)
There are exotic spaces that are actually useful
• Besov spaces
Sparsity is a key principle
• It helps in more ways than we thought
• It is critical for inverse problems
• It helps regularization
Pseudo diagonalization
…
and all this with solid theory and efficient algorithms!
27
Outline
• A Tale of Two Communities
• The Golden Age
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
– Maximally compact sequences
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions
28
Time-Frequency Tilings
Fourier
Wavelets
29
Maximally compact sequences
Continuous time
• Time and frequency spreads: 2nd moments
• Heisenberg uncertainty bound
• Gaussians are maximally compact
Discrete time
• Time and frequency spreads?
• Uncertainty bound?
• What are the minimizers?
Localization for Discrete Sequences
• Option 1: Extension from analog signals
The signal is periodic in the freq. domain but the frequency definitions are not!
• Option 2: Use the first trigonometric moment (circular statistics)
Localization for Discrete Sequences
Localization for Discrete Sequences
• Option 1: Extension from analog signals
Uncertainty Principle for Discrete Sequences
Heisenberg: For sequences with :
Uncertainty Principle for Discrete Sequences
Maximally Compact Sequences
• The most compact sequence in time for a given frequency spread:
Theorem: For finding maximally compact sequences, solve the SDP:
The solution to above SDP is rank-1 and decomposed to .
Maximally Compact Sequences, Example
• Example:
Theorem: Fourier transform of maximally compact sequences are Mathieu functions.
New Uncertainty Bounds for Sequences
Maximally Compact Sequences: Gaussians?
There is a gap!
A New Benchmark
Uncertainty Principle for sequences
43
Outline
• A Tale of Two Communities
• The Golden Age
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
– Sampling 2.0
• A Community of Interest and an Interesting Community
• Conclusions
44
Given a class of objects, like a class of functions (e.g. bandlimited, SISS)
And given a sampling device, as usual to acquire the real world
– Smoothing kernel or low pass filter
– Regular, uniform sampling
Obvious question:
When is there a countable representation?
When does a minimum number of samples uniquely specify the function?
sampling kernel
An example from my garden...The sampling question:
45
From Analog to Digital… and back!
46
Shannon’s Theorem… a Bit of History
Whittaker
Nyquist
Kotelnikov
Whittaker
Raabe
Gabor
Shannon
1915
1928
1933
1935
1938
1946
1948
1949
Someya
47
Classic Case: Subspaces
Shannon bandlimited case
or 1/T degrees of freedom per unit time
But: a single discontinuity, and no more sampling theorem…
Are there other signals with finite number of degrees of freedom per unit of time that allow exact sampling results?
48
Examples of Non-bandlimited Signals
49
Classic Cases and Beyond…
Sampling theory beyond Shannon?
– Shannon: bandlimitedness is sufficient but not necessary
– Shannon bandwidth and shift-invariant subspaces: dimension of subspace
Is there a sampling theory beyond subspaces?
– Finite rate of innovation: Similar to Shannon rate of information
– Non-linear set up
– Position information is key!
Thus, develop a sampling theory for classes of non-bandlimited but sparse signals!
t
x(t)
t0x1
t1
x0…
xN-1
tN-1
Generic, continuous-time sparse signal
50
Signals with Finite Rate of Innovation
The set up:
For a sparse input, like a weighted sum of Diracs
– One-to-one map yn x(t)?
– Efficient algorithm?
– Stable reconstruction?
– Robustness to noise?
– Optimality of recovery?
sampling kernel
51
The sampling theorem (VMB02)
For the class of periodic FRI signals which includes
– Sequences of Diracs
– Non-uniform or free knot splines
– Piecewise polynomials
– …
There are sampling schemes with sampling at the rate of innovation with perfect recovery and polynomial complexity
Variations: finite length, 2D, local kernels etc
52
What’s Maybe Surprising….
Bandlimited
Manifold
53
Current challenges…
The Tyranny of the pixel!
The
Vio
lin, F
elix
Val
lott
on
54
The good old super-resolution problem….
Up-sampled image with mask regularizerUp-sampled image without mask regularizer
The irony of it all…
55
As a signal processor:• I went from l2(Z) to L2(R)Meanwhile:• Compressed sensing. • Beautiful theory of sparsity in RN!
56
Outline
• A Tale of Two Communities
• The Golden Age
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions
The Wavelet and SP crowd
57I.Daubechies, Y.Meyer, R.Coifman, D.Donoho, A.Cohen, S.Mallat, M.Unser, R.Malvar,M.Smith, P.P.Vaidyanathan, A.Aldroubi, H.G.Feichtinger, K.H.Groechenig and many others!
58
My Wavelet and SP gang
T. Blu, O. Rioul, C. Herley, J. Kovacevik, K. Ramchandran, T. Nguyen, A. Ortega, V. Goyal, R. Parhizkar, P. Marziliano, Y. Barbotin, H.Pan, and many more….
59
• Outline
• A Tale of Two Communities
• The Golden Age
• The Beauty and the Beast
• Time-Frequency-Scale as a Way of Life
• Signal Processing in the Age of Sparsity
• A Community of Interest and an Interesting Community
• Conclusions: Foundations of Signal Processing
60
It has changed my view of the (signal processing) world!
Wavelets brought a new understanding of
known methods
«Understanding is a lot like sex. It's got a
practical purpose, but that's not why people
do it normally». Frank Oppenheimer
Wavelets raised high expectations and
produced new methods and successes
«As new developments emerge in any field, its
important to let them mature without
generating unrealistic expectations so that the
beautiful and important aspects have time and
good soil in which to blossom”. Al Oppenheim
61
It has changed my view of the (signal processing) world!
62
63
“The math is right. It’s just in poor taste.’’© New Yorker
64
65