Concept of a Time Frequency Masking Model based on Gabor ...Peter Balazs (ARI) Multipliers & Masking...

Concept of a Time Frequency Masking Model based onGabor filters and 2D convolution

Peter Balazs

Acoustics Research Institute (ARI)Austrian Academy of Sciences

in cooperation with:NuHAG Vienna

LATP, CMI & LMA, CNRS Marseille

FYMA, UCL Louvain-La-Neuve

Amade workshop05.10.2006

Peter Balazs (ARI) Multipliers & Masking 1 / 24

Overview:

1 IntroductionMP3

2 Simultaneous Masking

3 Gabor Filters

4 Time Frequency Masking

5 Current Research and Conclusion

Overview:

1 IntroductionMP3

3 Gabor Filters

Overview:

1 IntroductionMP3

3 Gabor Filters

Overview:

1 IntroductionMP3

3 Gabor Filters

Overview:

1 IntroductionMP3

3 Gabor Filters

Application : MP3-Player

• MP3• encoding / decoding scheme• MPEG1/MPEG2 (Layer 3)• signal processing• psychoacoustical model

• Signal Processing• applications: mobile phone,

UMTS, xDSL or digitaltelevision

• engineering field• mathematical field

• Psychoacoustic Masking

Psychoacoustic Masking: introduction

Masking:presence of one stimulus, the masker, decreases the response to anotherstimulus, the target.

Irrelevance Filter: searches (and deletes) perceptional irrelevant data (incomplex signals) using a masking model, supposing that one audiocomponent masks the others.

Typical Application:

1 Sound / Data Compression

2 Sound Design

3 Background - Foreground Separation

4 Improvement of Speech or Music Recognition

2 Sound Design

Psychoacoustic Masking : existing algorithm I

Existing algorithm in : simple model, but effective algorithm!Original audio file

”LossyCoding”

jumpref.wavMedia File (audio/wav)

Psychoacoustic Masking : existing algorithm II

Existing algorithm in : simple model, but effective algorithm!Irrelevance Filter

Original audio file

”Lossy Coding”

jumpmask.wavMedia File (audio/wav)

jumpdiff.wavMedia File (audio/wav)

Psychoacoustic Masking : existing algorithm III

Existing algorithm in : Original audio file (Spectrum)

Psychoacoustic Masking : existing algorithm IV

Existing algorithm in : Masked signal (Spectrum)

Psychoacoustic Masking : existing algorithm V

models simultaneousfrequency masking and usessingle spectra.

It calculates an adaptive threshold function for the spectra, but works witha time-frequency analysis. This is an adaptive Gabor Filter withcoefficients in {0, 1}.With Gabor theory some properties are explained:

• perfect reconstruction,• time frequency concentration,• smoothness and• numerical efficiency.

Psychoacoustic Masking : existing algorithm V

models simultaneousfrequency masking and usessingle spectra.

It calculates an adaptive threshold function for the spectra, but works witha time-frequency analysis. This is an adaptive Gabor Filter withcoefficients in {0, 1}.With Gabor theory some properties are explained:

• perfect reconstruction,• time frequency concentration,• smoothness and• numerical efficiency.

Psychoacoustic Masking : psychoacoustical experiments

Condition Correct Score1 68.6 %2 58.5 %3 50.5 %4 49.4 %

Psychoacoustic Masking : temporal masking

Also temporal masking exists:

Temporal and frequency masking have been investigated a lot. But thereare only very few studies for time frequency behaviour.

Goal: Investigate a true time-frequency masking model based on GaborFilters. Develop an efficient irrelevance algorithm!

What is a Gabor Filter?

Example for Gabor Filter = Gabor Multiplier

Original audio file:

jump1.wavMedia File (audio/wav)

Symbol:

Result of Gabor Multiplier.

result_jump1.wavMedia File (audio/wav)

Time Frequency Masking

Psychoacoustic Masking : simultaneous masking I

Existing Model, using bark scale

Time Frequency Masking : using Gabor multipliers

Simple concept for a time frequency masking model and algorithm:

Convolution with Gabor transform ⇒ Symbol for Gabor Filter.Other, more complex 2D convolution kernels are possible!

Current Research : psychoacoustical experiments

Masking effect of time frequency atoms is important =⇒ Current research.

=⇒ Sophie

-5

0

5

10

15

20

25

30

35

40

-5

0

5

10

15

20

25

30

35

40

-5

0

5

10

15

20

25

30

35

40

Qui

et T

hres

hold

s (d

B S

PL)

Signal Frequency (Hz)

S1 S4

S2

S3

S5

S6

2521 3181 4000 5015 6275 2521 3181 4000 5015 6275

300-msec Sinusoid3.3-msec Gaussian

Current Research : psychoacoustical experiments

Masking effect of time frequency atoms is important =⇒ Current research.=⇒ Sophie

-5

0

5

10

15

20

25

30

35

40

-5

0

5

10

15

20

25

30

35

40

-5

0

5

10

15

20

25

30

35

40

Qui

et T

hres

hold

s (d

B S

PL)

Signal Frequency (Hz)

S1 S4

S2

S3

S5

S6

2521 3181 4000 5015 6275 2521 3181 4000 5015 6275

300-msec Sinusoid3.3-msec Gaussian

Current Research

• Investigate the mathematical and numerical background for a Gaborfilter further: (cooperation with NuHAG/LATP/UCL)

• Bessel and frame multipliers• irregular Gabor multipliers• numerics of discrete Gabor analysis

• Psychoacoustic experiment: (cooperation with LMA)• thresholds for Gaussian atoms• time frequency masking effects between two atoms• more complex sounds

• Algorithms: (cooperation with LMA)• efficient implementation (real time)• experimental testing of its validity for different applications

Current Research

Conclusion

• Gabor Filter Theory gives an insight on time-variantfiltering.

• A simple Time Frequency Masking model exists.Further investigation starts at the time frequencyatom level .

• A lot of interesting research, both applied andtheoretical, still has to be done.

Conclusion

Personal References:

P. Balazs, Regular and Irregular Gabor Multipliers with Application toPsychoacoustic Masking, PhD Thesis, Universität Wien (2005)

P. Balazs, Basic Definition and Properties of Bessel Multipliers, Journal ofMathematical Analysis and Applications (in press, available online)

P. Balazs, B. Laback, G. Eckel and W. A. Deutsch, Perceptional Sparsity bySimultaneous Masking, preprint

P. Balazs, J.-P. Antoine, Weighted and controlled frames, submitted

Thank you for your attention!

IntroductionMP3

Simultaneous MaskingGabor FiltersTime Frequency MaskingCurrent Research and ConclusionReferences

Date post:	03-Aug-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Concept of a Time Frequency Masking Model based on Gabor ...Peter Balazs (ARI) Multipliers & Masking...

Documents