Perceptual declipping of audio signals through compressed sensing:

algorithm design and evaluation

Tussentijdse presentatie

Naim Mansour Promotor: Prof. dr. ir. Marc MoonenAssistent: Ir. Bruno Defraene

2

Overzicht• Onderwerp & doelstellingen (vermelding Steven) – 3 min.• Compressed sensing – 5 min.

– Wat?– Theoretisch– Declipping (don’t forget perfect reconstruction)

• CS & Declipping – 4 min.– Specifieke theorie– Eerder werk (INRIA, AxBe)– Kort: perceptuele component

• Toelichting gemaakte keuzes & motivatie (2 keuzes) – 3 min.– Don’t forget frame length (basically all details)

• Implementatie & resultaten (demo) – 5 min.• Planning, en plannen voor fase 2 – 2 min.• Dank & vragen

3

Overview• Subject• Compressed Sensing• CS & Declipping• Perceptual components• Extra: IRL1• Implementation

• Evaluation

4

Subject

• Declipping of audio signals

• Through compressed sensing

• Perceptual

• Algorithm design & evaluation

5

Compressed Sensing: general• Candès, Romberg, Tao – 2006• Recover sparse signal from sub-Nyquist rate sampled measurements• Consider the signal s, sparse in a fixed basis :

• Measurement basis selects reliable values from s according to ( is known as the sensing base):

• Reconstruction through constrained L0/L1 minimization:

𝒚𝑀× 1=𝜱𝑀×𝑁𝜳 𝑁×𝑁 𝒙𝑁 ×1=𝑨𝒙

𝒔=∑𝑖=1

𝑁

Ψ 𝑖𝑥 𝑖=𝜳 𝒙

6

• Solution equals translation of null(A)-plane by vector z• L0 & L1 lead to sparse solutions, L2 doesn’t

• L1 minimization is convex -> convex optimization,• L0 minimization non-convex -> greedy opt.

𝒙 ′=𝑎𝑟𝑔𝑚𝑖𝑛‖𝒛‖𝟎𝑠 .𝑡 .𝑨𝒛=𝒚𝒙 ′=𝑎𝑟𝑔𝑚𝑖𝑛‖𝒛‖𝟐𝑠 .𝑡 .… 𝒙 ′=𝑎𝑟𝑔𝑚𝑖𝑛‖𝒛‖𝟏𝑠 .𝑡 .…

Compressed Sensing: Choice of Lp

7

Compressed Sensing: AxBe model• Other possible model (Bölcskei & Studer, – 2011)

• In case of clipping, we consider to be measurement including clipped samples (). No explicit measurement matrices, and , to obtain sparse error base ().

• Recovery through projected Lp minimization:

𝒚𝑀× 1=𝜱𝑀×𝑁 𝑠𝜳 𝑁 𝑠×𝑁𝑠

𝒙𝑁𝑠× 1+𝜣𝑀×𝑁 𝑒𝜠𝑁 𝑒×𝑁𝑒

𝒆𝑁𝑒×1=𝑨𝒙+𝑩𝒆

𝒚=𝒔+𝒆 ,𝒆=𝒔𝒄−𝒔 , 𝒔𝒄=𝑐𝑙𝑖𝑝𝑝𝑒𝑑𝑣𝑒𝑟𝑠𝑖𝑜𝑛𝑜𝑓 𝒔

𝒚𝑀× 1=𝜳𝑀 ×𝑀 𝒙𝑀 ×1+𝜠𝑀 ×𝑀 𝒆𝑀 ×1=𝑨𝒙+ 𝑰𝒆

8

Compressed Sensing: Recovery• Certain theoretical bounds for perfect recovery of signal• Classical model (no noise assumption):

• AxBe model:

• Coherence of a basis: measure of decorrelation in analysis domain

• Fourier base: DCT base:

,

𝑛𝑥𝑛𝑒<( 1𝜇𝐴❑ )

2

,𝑛𝑒=‖𝑒‖0

𝜇𝐴❑= max

𝑘, 𝑙 ,𝑘≠𝑙|𝒂𝑘

𝐻𝒂𝑙|

9

CS & Declipping: recovery• Recovery ability dependent on coherence of sensing base• Classical CS: Usage of pseudorandom measurement matrices

(e.g. iid Gaussian sampling) leads to very low coherence• Declipping: reliable, “sampled” values in signal are unclipped ones

-> clearly not pseudorandom!• Coherence of combined Fourier/DCT base with clipping sensing base =

coherence Fourier/DCT• Recovery guarantees for DCT base ( reliable samples):

• Perfect recovery of real audio signals practically always impossible, since

M 900 800 700 600 500 400 300

nx (max) 8,6699 4,6459 3,3281 2,7202 2,2953 2,1393 1,89

10

CS & Declipping: • Missing samples will always lie beyond the clipping threshold • Lp minimization can be improved through introduction of additional

linear constraints

11

CS & Declipping: previous work• INRIA• Bölcskei

12

Perceptual components• Perceptual weighting matrix based on acoustic loudness perception

• Psychoacoustically optimized (adaptive) basis

13

Extra: IRL1• Iteratively reweighted L1 minimization (Candès, Wakin, Boyd – 2007)

14

Implementation: general• 2 main choices

– PCS through bounded L1 minimization, using perceptual weighting, Axy & AxBe models (further improvement through IRL1)

– PCS through bounded L0 minimization, using psychoacoustic wavelet basis, Axy & AxBe models

• Incremental design: implementation & evaluation with & without bounds, with & without perceptual components,…

15

16

Implementation: Clipping

17

Evaluation: general

18

Evaluation: SNR vs. PEAQ

19

• SNR no guarantee for audio quality!!

20

Planning & future prospects

21

Planning & future prospects

22

Planning & future prospects

• Semester 2– Execute psychoacoustic experiments– Finish algorithms – Write final texts

23

References

• http://people.ee.duke.edu/~willett/SSP//Tutorials/ssp07-cs-tutorial.pdf

• Recovery of Sparsely Corrupted Signals blablabla

24

?

25

Zalig Kerstfeest!