INTELLIGENT CONTROL

APPROACH TO THE COCKTAIL PARTY

PROBLEM

MILA GOROBETS

ENEL 667 - INTELLIGENT CONTROL - COURSE PRESENTATION

APRIL 7, 2014

2

OUTLINE What is the cocktail party problem? Selected approach and Results

Blind Signal Separation

Signal Prediction

Cancellation of Contaminants

Other Applications Conclusions Future Work

3

COCKTAIL PARTY PROBLEM Most humans can select a single voice or sound from a

mixture Tuning into one sound Tuning out everything else

Works best with two ears [1] Related to localization

Can we get a computer to do this reliably?

4

APPROACH OVERVIEW Use 3 neural networks to separate and predict two different signals One signal is cancelled Assume stereo signals are available (but sources can be close)

Stereo Signal Mixing

Blind Signal Separation

Prediction of Signal 1

Prediction of Signal 2

Separated Signal 1

Separated Signal 2

Signal arriving at Ear 1

Signal arriving at Ear 2

Signal 1 at Ear 1+

+ Signal 1 at Ear 2

Signal 1

Signal 2 Neural Network 1

Neural Network 2

Neural Network 3

5

1. BLIND SIGNAL SEPARATION Two measurement sources (microphones) Recursive update [2] Performs decorrelation

+

+

Signal at Ear 1

Signal at Ear 2

Separated Signal 2

Separated Signal 11/(1-C11C21)

Z-11/(1-C11C21) Z-1 Z-1 Z-1

Z-11/(1-C11C21) Z-1 Z-1 Z-1

-C11

C21

1/(1-C11C21)

C11

C22

+

C23 C24 C2n

-C12 -C13 -C14 -C1m

-C22 -C23 -C24 -C2n

C21 +

C12 C13 C14 C1m

6

1. BLIND SIGNAL SEPARATION (CONT’D) Signals are separated with MSE

below 1e-3 Problems occur when:

Stereo deteriorates (-> mono) Signals are of similar frequencies

8.3 8.31 8.32 8.33 8.34 8.35 8.36 8.37 8.38 8.39 8.4

x 104

-0.5

0

0.5Desired Signal

Sample number

Am

plitu

de (

V)

Separated signal Desired Signal Error

8.3 8.31 8.32 8.33 8.34 8.35 8.36 8.37 8.38 8.39 8.4

x 104

-0.5

0

0.5Contaminating Signal

Sample number

Am

plitu

de (

V)

Separated signal Desired Signal Error

8.3 8.31 8.32 8.33 8.34 8.35 8.36 8.37 8.38 8.39 8.4

x 104

-0.5

0

0.5

1

7

1. BLIND SIGNAL SEPARATION (CONT’D) The separator acts as a filter that passes the desired signal

Simple Case: Voice + Alarm

9

2. SIGNAL PREDICTION Multilayer Perceptron

Input, hidden and output layers Hyperbolic tangent activation functions

Learning using Backpropagation [3] Hybrid [3,4]

Backpropagation for Outer Layer

Recursive Least Squares for Inner Layer

Control System to Enhance Output

10

2. SIGNAL PREDICTION STABILITY Hybrid updating

Lyapunov candidate function:

11

2. SIGNAL PREDICTION STABILITY Taylor Series Expansions, substitutions, etc give the following:

Leading to update equations:

12

2. SIGNAL PREDICTION STABILITY Substitute update equations in:

13

2. SIGNAL PREDICTION STABILITY But we can also see that:

In which case we can express the derivative as:

And then V is bounded by a decaying exponential:

The system is semi-globally (due to constrained initial values) UUB

P must remain invertible (creates problems)

14

2. SIGNAL PREDICTION (CONT’D)

Separated Signal

MLP

128-point Time

Domain Buffer

Predicted Signal

+

K

+

1/s

G

X

|| ||

+

tanh

F

A

+

16

2. SIGNAL PREDICTION (CONT’D) Control system offers adjustment to improve prediction

17

2. SIGNAL PREDICTION (CONT’D) Weight convergence (constant frequency sinusoids)

18

3. CANCEL THE CONTAMINANTS Depending on the desired signal, only one of the predictions is used C1 and C2 are found during the separation stage

Predicted Signal

C1

Signal at Ear 1

+Audible Signal

at Ear 1

Predicted Signal

C2

Signal at Ear 2

+Audible Signal

at Ear 2

+

+

Signal at Ear 1

Signal at Ear 2

Separated Signal 2

Separated Signal 11/(1-C11C21)

Z-11/(1-C11C21) Z-1 Z-1 Z-1

Z-11/(1-C11C21) Z-1 Z-1 Z-1

-C11

C21

1/(1-C11C21)

C11

C22

+

C23 C24 C2n

-C12 -C13 -C14 -C1m

-C22 -C23 -C24 -C2n

C21 +

C12 C13 C14 C1m

19

3. CANCEL THE CONTAMINANTS (CONT’D)

20

SUMMARY FOR TEST SIGNALS

Sin

e

Sum

of

sines

Sm

oke A

larm

Man's

Voic

e

Gir

l's

Voic

e

Mach

inery

Nois

y S

ines

Dri

ll0

10

20

30

40

50

60

70

Crosstalk reduction in select signals

Sine

Sum of Sines

Smoke Alarm

Man's Voice

Girl's Voice

Machinery

Noisy Sine

Drill

Contaminating Signal

Redu

ction

in C

ross

talk

(dB)

DESIRED SIGNAL:

24

REFERENCES [1] “Cocktail Party Effect,” Wikipedia. Available:

http://en.wikipedia.org/wiki/Cocktail_party_effect

[2] C. Jutten and J. Herault, “Blind Separation of Sources, Part I: An Adaptive Algorithm Based on Neuromimetic Architecture,” Signal Proc, vol. 21, no. 1, pp. 1-10, July 1991.

[3] G. W. Ng. Application of Neural Networks to Adaptive Control of Nonlinear Systems. Somerset: Research Studies Press, 1997, pp. 103-133.

[4] J. A. K. Suykens, J. P. L. Vandewalle, and B. L. R. De Moor. Artificial Neural Networks for Modelling and Control of Non-Linear Systems. Netherlands: Kluwer Academic Publishers, 1996, pp. 46-49.

[5] M. T. Pourazad et al, “Heart sound cancellation from lung sound recordings using time-frequency filtering,” Med Biol Eng Comput, no. 44, pp. 216-225, 2006.

[6] J. M. Diebele et al, “Dynamic Separation of pulmonary and cardiac changes in electrical impedance tomography,” Physiol Meas, vol. 29, no. 6, pp. 1-14, 2008.

25

26

27

28