Benjamin Blankertz- Acquisition and Analysis of Neuronal Data 2008 BCI: Lecture #13

8/3/2019 Benjamin Blankertz- Acquisition and Analysis of Neuronal Data 2008 BCI: Lecture #13

1/33

A c q u i s i t i o n a n d A n a l y s i s o f N e u r o n a l D a t a 2 0 0 8

B C I L e c t u r e # 1 3

B e n j a m i n B l a n k e r t z

M a c h i n e L e a r n i n g L a b o r a t o r y , B e r l i n I n s t i t u t e o f T e c h n o l o g y

F r a u n h o f e r F I R S T ( I D A )

b l a n k e r @ c s . t u - b e r l i n . d e
http://[email protected]/


2/33

1

T e c h n i c a l N o t e

C o n v e n t i o n s i n N o t a t i o n

x , N , ( i t a l i c s ) : s c a l a r

x, ( b o l d f a c e , l o w e r c a s e ) : c o l u m n v e c t o r

X, ( b o l d f a c e , u p p e r c a s e ) : m a t r i x

I d e n o t e s t h e i d e n t i t y m a t r i x


3/33

2

T o d a y ' s T o p i c a n d I n t r o

C l a s s i c a t i o n o f s p e c t r a l f e a t u r e s , n a m e l y m o d u l a t i o n s o f t h e a m p l i t u d e i n s p e c i c

f r e q u e n c y b a n d s .

C o m m o n S p a t i a l P a t t e r n ( C S P ) a n a l y s i s c a n b e u s e d t o c l a s s i f y d i e r e n t c o n d i t i o n s

t h a t a r e c h a r a c t e r i z e d b y a m o d u l a t i o n o f t h e a m p l i t u d e o f b r a i n r h y t h m s ( [ 1 , 2 ] ) .

T h e s e m o d u l a t i o n s c a n o f t e n b e o b s e r v e d , s i n c e a c t i v a t i o n o f a b r a i n a r e a l e a d s t o a n

E v e n t - R e l a t e d D e s y n c h r o n i z a t i o n ( E R D ) o f n e u r o n s ( [ 3 , 4 , 5 ] ) . I n s c a l p r e c o r d i n g s

( E E G ) a n a t t e n u a t i o n o f i d l e r h y t h m s c a n b e o b s e r v e d d u r i n g a c t i v a t i o n o f t h e

r e s p e c t i v e b r a i n r e g i o n .


4/33

3

R e m i n d e r : S p e c t r u m o f M a c r o s c o p i c B r a i n A c t i v i t y

10 20 30 40 50

Frequency [Hz]

Spec

tralPower[dB]

~ 1/f

T h e g u r e s h o w s a n i d e a l i z e d s p e c t r u m .

S o m e b r a i n r h y t h m s a r e n a m e d a c c o r d i n g t o t h e i r o r i g i n , e . g . , , , .


5/33

4

R e m i n d e r : M o d u l a t i o n o f B r a i n R h y t h m s

M o s t r h y t h m s a r e i d l e r h y t h m s , i . e . , t h e y a r e a t t e n u a t e d d u r i n g a c t i v a t i o n .

- r h y t h m ( a r o u n d 1 0 H z ) i n v i s u a l c o r t e x :

Oz

eyes closed eyes open

- r h y t h m ( a r o u n d 1 0 H z ) i n m o t o r a n d s e n s o r y c o r t e x :

C4

arm movesarm at rest


6/33

5

S e n s o r i M o t o r R h y t h m s ( S M R s )

H a n d m o v e m e n t s a n d t h e i r i m a g i n a t i o n a r e a c c o m p a n i e d b y a n e v e n t - r e l a t e d

d e s y n c h r o n i z a t i o n ( E R D ) :

a n a t t e n t u a t i o n o f s e n s o r i m o t o r r h y t h m s o f t h e r e s p e c t i v e a r e a .

10 20 30 40 50

SpectralPo

wer[dB]

~1/f

Frequency [Hz]

left hand right hand

highalpha

lowalpha

E R D m a p s : c l a s s w i s e a v e r a g e d b a n d - p o w e r


7/33

5

B C I w i t h M a c h i n e L e a r n i n g

feedb

ack

sess

ion

machinelearning

calibra

tion

session

trialslabeled

extraction

feature

signal

outputclassifier

extractionfeature

measurementsupervised

spontaneous

EEG

o i n e : c a l i b r a t i o n ( 1 0 2 0 m i n u t e s )

R R R L L L R L

c o l l e c t t r a i n i n g s a m p l e s

o n l i n e : f e e d b a c k ( u p t o 6 h o u r s )

c l a s s i c a t i o n o f s l i d i n g w i n d o w s ( 1 s )


8/33

6

E x p e r i m e n t a l D e s i g n

S u b j e c t s i t t i n g r e l a x e d i n a c h a i r w i t h a r m r e s t s .

V i s u a l c u e s ( a r r o w s ) i n d i c a t e w h i c h t y p e o f m o t o r i m a g e r y i s t o b e p e r f o r m e d : l e f t

h a n d , r i g h t h a n d , r i g h t f o o t .

A f t e r 1 5 t r i a l s , a b r e a k o f 1 5 s i s g i v e n .

I n t o t a l 7 5 t r i a l s o f e a c h m o t o r i m a g e r y c o n d i t i o n a r e r e c o r d e d .

2 sec 2 sec4 sec


9/33

7

N e u r o p h y s i o l o g i c a l F e a t u r e M R P

T h e m o v e m e n t o f a l i m b i s p r e c e d e d b y a n e g a t i v e s h i f t o f s c a l p p o t e n t i a l s i n t h e

c o r r e s p o n d i n g s e n s o r i m o t o r a r e a : B e r e i t s c h a f t s p o t e n t i a l o r r e a d i n e s s p o t e n t i a l .

A l s o , t h e i m a g i n a t i o n o f a m o v e m e n t o r a s e n s a t i o n i s a c c o m p a n i e d b y s u c h a n e g a t i v e

s h i f t .

0 1000 20004

3

2

1

0

1

2

3C3

[ms]

[V]

0 1000 20005

0

5C4

[ms]

[V]

[uV]

4 2 0 2 4

left right

T h i s f e a t u r e c a n b e c l a s s i e d a s d i s c u s s e d i n t h e l a s t l e c t u r e ( s p a t i o - t e m p o r a l f e a t u r e s ) .


10/33

8

N e u r o p h y s i o l o g i c a l F e a t u r e E R D

I m a g i n i n g a m o v e m e n t o r s e n s a t i o n i n a l i m b c a u s e s a l o c a l b l o c k i n g o f t h e

c o r r e s p o n d i n g s e n s o r i m o t o r r h y t h m : E v e n t - R e l a t e d D e s y n c h r o n i s a t i o n ( E R D ) .

10 20 30 40

20

2

4

6

8

10C3 lap

[Hz]

[dB]

10 20 30 40

2

4

6

8

10

12C4 lap

[Hz]

[dB]

0 2000 4000

0.6

0.4

0.2

0

0.2C3 lap

[ms]

[V]

0 2000 4000

0.5

0.4

0.3

0.2

0.1

0

0.1

C4 lap

[ms]

[V]

[dB]

1 0 1

left right


11/33

9

C a l c u l a t i o n o f E R D / E R S C u r v e s

1000 1500 2000 2500 3000 3500 4000 4500 [ms]

raw

bandpass

rectified

average

smoothing

...


12/33

1 0

C o m m o n S p a t i a l P a t t e r n ( C S P ) A n a l y s i s

G o a l : F i n d s p a t i a l l t e r s t h a t o p t i m a l l y c a p t u r e m o d u l a t i o n s o f b r a i n r h y t h m s

O b s e r v a t i o n : p o w e r o f a b r a i n r h y t h m v a r i a n c e o f b a n d - p a s s l t e r e d s i g n a l .

min variancefor

min variancefor

unknownsources

discriminativesignals

V1

left

right

EEGno classspecificinfluenceon variance

csp:L1,2,...

csp:R1,2,...

V

projection filter

forward

model model

backward

observedsignals


13/33

1 1

T h e G o a l o f C S P

2425 2430 2435 [s]

csp:R1

csp:R2

csp:L1

csp:L2

right left right


14/33

1 2

C S P T o y E x a m p l e

3 2 1 0 1 2 33

2

1

0

1

2

3

b after CSP filterin15 10 5 0 5 10 15

15

10

5

0

5

10

15

a before CSP filterin


15/33

1 3

T e c h n i c a l A p p r o a c h t o C S P

Xi RCT

: i- t h t r i a l o f b a n d - p a s s l t e r e d E E G w i t h

C b e i n g t h e n u m b e r o f c h a n n e l s a n d T t h e n u m b e r o f s a m p l e d t i m e p o i n t s

(c) RCC: a v g . c o v a r i a n c e m a t r i x o f t h e Xi ' s b e l o n g i n g t o c o n d i t i o n c {1, 2}

T h e n t h e C S P b a c k w a r d m o d e l V i s t h e s i m u l t a n e o u s d i a g o n a l i z e r o f (1)

a n d (2)

V

(1)V = (1), ( 1 )

V(2)V = (2), ( w i t h (i) d i a g o n a l )

w i t h a s c a l i n g s u c h t h a t (1) +(2) = I. T h i s i s e q u i v a l e n t t o s o l v i n g t h e g e n e r a l i z e d

e i g e n v a l u e p r o b l e m

V

(1)V

=D

&V

(

(1)

+

(2)

)V

=I

I n M a t l a b t h i s c a n b e d o n e b y

[ V , D ] = e i g ( S i g m a 1 , S i g m a 1 + S i g m a 2 ) .


16/33

1 4

C S P M o r e P r a c t i c a l

E E G - s i g n a l s d u r i n g m o t o r i m a g e r y , b a n d - p a s s l t e r e d ( h e r e 9 1 3 H z ) :

728 730 732 734 736 738

C3

C4

right left left

l e f t d a t a

L

:= XL

XL

R

:= XR

XR

XL

( T c h a n s ) ( T c h a n s ) X

R

r i g h t d a t a

V

L

V = D & V(L

+R

)V = I

c h o o s e e i g e n v e c t o r vi f r o m V t h a t h a s a l a r g e e i g e n v a l u e di w . r . t . L

.

728 730 732 734 736 738

CSP

left right right

var(XL vi) = dil a r g e

var(XR

vi) = 1di s m a l l


17/33

1 4

C S P M o r e P r a c t i c a l


728 730 732 734 736 738

C3

C4

right left left

l e f t d a t a

L

:= XL

XL

R

:= XR

XR

XL

( T c h a n s ) ( T c h a n s ) X

R

r i g h t d a t a

V

L

V = D & V(L

+R

)V = I

c h o o s e e i g e n v e c t o r vi f r o m V t h a t h a s a s m a l l e i g e n v a l u e di w . r . t . L

.

728 730 732 734 736 738

CSP

left right right

var(XL vi) = dis m a l l

var(XR

vi) = 1di l a r g e


18/33

1 5

C S P a t W o r k
http://csp_animation.avi/


19/33

1 6

T r a i n i n g C S P - b a s e d C l a s s i c a t i o n

D e t e r m i n e m o s t d i s c r i m i n a t i v e f r e q u e n c y b a n d ,

b a n d - p a s s l t e r E E G i n t h a t b a n d ,

e x t r a c t s i n g l e t r i a l s u s i n g t h e t i m e i n t e r v a l i n w h i c h E R D / E R S i s e x p e c t e d ,

c a l c u l a t e a n d s e l e c t C S P l t e r s ,

a n d a p p l y t h e m t o E E G s i n g l e t r i a l s ,

c a l c u l a t e t h e l o g v a r i a n c e w i t h i n t r i a l s .

T h i s r e s u l t s i n a l o w d i m e n s i o n a l f e a t u r e v e c t o r f o r e a c h t r i a l ( d i m e n s i o n a l i t y e q u a l s

n u m b e r o f s e l e c t e d C S P l t e r s ) .

T r a i n a l i n e a r c l a s s i e r l i k e F i s h e r ' s D i s c r i m i n a n t , L D A o r L S R o n t h e f e a t u r e s . ( S i n c e t h e s e f e a t u r e s a r e l o w - d i m e n s i o n a l , s h r i n k a g e i s t y p i c a l l y n o t n e c e s s a r y . )


20/33

1 7

T r a i n i n g C S P - b a s e d C l a s s i c a t i o n

( 1 )

5 10 15 20 25 30 35 40 45

2

4

6

8

10

12

[Hz]

[dB]

C4 lap

left

right

( 2 )

0 1000 2000 3000 4000 5000

0.4

0.3

0.2

0.1

0

0.1

0.2

0.3

0.4

[ms]

[V]

C4 lap

left

right

( 4 )

3 2 1 0 1

3

2

1

0

1

log(var(CSPL))

log(var(CSPR

))

leftright

( 3 )

csp1 [0.68] csp2 [0.64] csp3 [0.60] csp4 [0.71] csp5 [0.68] csp6 [0.61]

28.2 17.0 39.3 15.5


21/33

1 8

A p p l y i n g C S P - b a s e d C l a s s i c a t i o n

P r o j e c t b a n d - p a s s l t e r e d E E G w i t h s p a t i a l C S P l t e r s ,

c a l c u l a t e t h e v a r i a n c e i n s h o r t w i n d o w s ,

t a k e t h e l o g a r i t h m ,

a n d a p p l y t h e c l a s s i e r w e i g h t i n g .

I n s u m m a r y :

f(X; {v}Jj=1, {cj}Jj=0) =

J

j=1

cj logv

j XXvj

+ c0,

w h e r e X RCT i s a n e w s e g m e n t o f b a n d - p a s s l t e r e d E E G r e c o r d i n g t h a t w e w a n t t o c l a s s i f y , {vj}

Jj=1 a r e t h e s e l e c t e d e i g e n v e c t o r s ( C S P l t e r s ) , a n d {cj}

Jj=0 i s t h e

c l a s s i e r w e i g h t i n g .


22/33

1 9

V a l i d a t i n g a C l a s s i c a t i o n M e t h o d

C r u c i a l m e a s u r e : g e n e r a l i z a t i o n p e r f o r m a n c e , i . e . , t h e a c c u r a c y o b t a i n e d w h e n t h e

c l a s s i e r i s a p p l i e d t o n e w d a t a , w h i c h h a v e n o t b e e n s e e n b e f o r e .

k - f o l d c r o s s - v a l i d a t i o n i s o n e m e a n s t o e s t i m a t e t h e g e n e r a l i z a t i o n p e r f o r m a n c e :

S p l i t t h e d a t a s e t i n t o k s e t s o f ( a p p r o x i m a t e ) e q u a l s i z e .

I n t u r n , t a k e o n e s e t o u t , ( s e l e c t a l l p a r a m e t e r s a n d ) t r a i n t h e c l a s s i e r o n t h e r e m a i n i n g k 1 ` t r a i n i n g ' s e t s a n d a p p l y i t t o t h e l e f t o u t ` t e s t ' s e t .

C a l c u l a t e t h e a v e r a g e a c c u r a c y o b t a i n e d o n t h e k t e s t s e t s .

N o t e : W h e n a p r e p r o c e s s i n g s t e p ( l i k e C S P ) u s e s t h e c l a s s l a b e l s , i t n e e d s t o b e

p e r f o r m e d w i t h i n t h e c r o s s - v a l i d a t i o n o n t h e t r a i n i n g s e t s o n l y ! T a k e t h e s p a t i a l l t e r

o b t a i n e d b y C S P o n t h e t r a i n i n g d a t a a n d a p p l y i t t o t h e t e s t d a t a .


23/33

2 0

A p p e n d i x


24/33

2 1

C h a l l e n g e : N o n s t a t i o n a r i t y

o i n e : c a l i b r a t i o n ( 1 0 2 0 m i n u t e s )

c o l l e c t t r a i n i n g s a m p l e s

R R R L L L R L

o n l i n e : f e e d b a c k ( u p t o 6 h o u r s )

c l a s s i c a t i o n o f s l i d i n g w i n d o w s ( 1 s )

S u b j e c t c o n d i t i o n s m a y c h a n g e w i t h t i m e !

m o n o t o n e t y p i c a l l y v a r i a b l e

C h a l l e n g e : H o w d o w e c o p e w i t h t h e s e n o n s t a t s ?

i g n o r e & p r a y

a d a p t i v e f e a t u r e e x t r a c t i o n a n d c l a s s i c a t i o n

i n v a r i a n t f e a t u r e e x t r a c t i o n a n d c l a s s i c a t i o n [ N E W ! ]


25/33

2 2

O u r A p p r o a c h

O b s e r v a t i o n : C a l i b r a t i o n m e a s u r e m e n t i s s h o r t a n d m o n o t o n e .

P r o b l e m : S y s t e m m a y b e s u s c e p t i b l e t o n o n t a s k - r e l a t e d b r a i n a c t i v i t y t h a t w a s n o t

p r e s e n t i n t h e c a l i b r a t i o n .

I d e a : P e r f o r m s h o r t m e a s u r e m e n t o f p o t e n t i a l d i s t u r b a n c e s .


26/33

2 3

T o w a r d s I n v a r i a n t C S P ( i C S P )


728 730 732 734 736 738

C3

C4

right left left

d a t a : XR

r i g h t

R

:=

X

R

XR

l e f t

d a t a : XL

X

L

XL

L

:=

E E G - s i g n a l s d u r i n g d i s t u r b a n c e m e a s u r e m e n t , s a m e b a n d - p a s s

C3

C4

O1

O2

d i s t u r b a n c e d a t a : Y

:= YY


27/33

2 4

I n v a r i a n t C S P

S o l v e g e n e r a l i z e d E i g e n v a l u e p r o b l e m s

V

L

L

VL

= DL

&

V

L (L

+R

+ )VL

= Ia n d

V

R

R

VR

= DR

& V

R

(L

+R

+ )VR

= I

w h e r e VL

, VR

a r e m a t r i c e s o f e i g e n v e c t o r s , DL

a n d DR

d i a g o n a l m a t r i c e s o f

e i g e n v a l u e s .

C h o o s e e i g e n v e c t o r vL

f r o m VL

t h a t h a s t h e l a r g e s t e i g e n v a l u e dL

w . r . t . L

.

728 730 732 734 736 738

CSP

left right right

[s]

var(XL

vL

) = dL

l a r g e

var(X

R

vL

) 1d

L

s m a l l

CSP var(Y vL

) 1dL

s m a l l


28/33

2 4

I n v a r i a n t C S P


V

L

L

VL

= DL

&

V

L (L

+R

+ )VL

= Ia n d

V

R

R

VR

= DR

& V

R

(L

+R

+ )VR

= I

w h e r e VL

, VR


a n d DR



C h o o s e e i g e n v e c t o r vR

f r o m VR

t h a t h a s t h e l a r g e s t e i g e n v a l u e dR

w . r . t . R

.

728 730 732 734 736 738

CSP

left right right

[s]

var(XR

vR

) = dR

l a r g e

var(X

L

vR

) 1d

R

s m a l l

CSP var(Y vR

) 1dR

s m a l l


29/33

2 4

I n v a r i a n t C S P T r a d e - o P a r a m e t e r


V

L

L

VL

= DL

&

V

L ((1)(L

+R

) + )VL

= Ia n d

V

R

R

VR

= DR

& V

R

((1)(L

+R

) + )VR

= I

w h e r e VL

, VR


a n d DR




30/33

2 5

E x p e r i m e n t a l S e t u p

R L R R R L L L R L

R L R L R L R L R L R LCondition (2): low visual inputCondition (1): high visual input

Motor imagery measurements

test set

train iCSP

Fake Feedbackartifacts and variable mood


31/33

2 6

R e s u l t s : F i l t e r s a n d R o b u s t n e s s t o O c c i p i t a l A l p h a

T e s t s a m p l e s a r e s u p e r i m p o s e d w i t h 5 I C s o f o c c i p i t a l m u l t i p l i e d b y 0 , . 5 , 1 , 2 .

filter filter patternpattern

invariant CSPoriginal CSP

=0.0 =0.5 =1.0 =2.0

10.7% 11.4% 12.9% 37.9%

original CSP errors:

=0.0 =0.5 =1.0 =2.0

9.3% 10.0% 9.3% 11.4%

invariant CSP errors:


32/33

2 7

R e s u l t s : M o d e l s e l e c t i o n a n d C l a s s i c a t i o n

T o s e l e c t t h e m o d e l p a r a m e t e r a c r o s s - v a l i d a t i o n o n t h e t r a i n s e t i s p e r f o r m e d .

0 0.2 0.4 0.6 0.80

5

10

15

20

25

30

35

xi

error[%]

Subject cvtest

train

0 0.2 0.4 0.6 0.80

5

10

15

20

25

30

35

xi

error[%]

Subject zvtest

train

CSP iCSP0

5

10

15

20

25

error[%]

cv

zv

zk

zq

0 0.2 0.4 0.6 0.80

5

10

15

20

25

30

35

xi

error[%]

Subject zktest

train

0 0.2 0.4 0.6 0.80

5

10

15

20

25

30

35

xi

error[%]

Subject zqtest

train


33/33

2 8

R e f e r e n c e s

[ 1 ] B . B l a n k e r t z , R . T o m i o k a , S . L e m m , M . K a w a n a b e , a n d K . - R . M l l e r , O p t i m i z i n g S p a t i a l F i l t e r s f o r

R o b u s t E E G S i n g l e - T r i a l A n a l y s i s , I E E E S i g n a l P r o c M a g a z i n e

, 2 5 ( 1 ) : 4 1 5 6 , 2 0 0 8 , U R L

h t t p : / / d x . d o i . o r g / 1 0 . 1 1 0 9 / M S P . 2 0 0 8 . 4 4 0 8 4 4 1 .

[ 2 ] H . R a m o s e r , J . M l l e r - G e r k i n g , a n d G . P f u r t s c h e l l e r , O p t i m a l s p a t i a l l t e r i n g o f s i n g l e t r i a l E E G

d u r i n g i m a g i n e d h a n d m o v e m e n t , I E E E T r a n s R e h a b E n g

, 8 ( 4 ) : 4 4 1 4 4 6 , 2 0 0 0 .

[ 3 ] C . N e u p e r a n d W . K l i m e s c h , e d s . , E v e n t - r e l a t e d D y n a m i c s o f B r a i n O s c i l l a t i o n s

, E l s e v i e r , 2 0 0 6 .

[ 4 ] G . P f u r t s c h e l l e r , C . B r u n n e r , A . S c h l g l , a n d F . L . d a S i l v a , M u r h y t h m ( d e ) s y n c h r o n i z a t i o n a n d

E E G s i n g l e - t r i a l c l a s s i c a t i o n o f d i e r e n t m o t o r i m a g e r y t a s k s , N e u r o I m a g e

, 3 1 ( 1 ) : 1 5 3 1 5 9 , 2 0 0 6 .

[ 5 ] G . P f u r t s c h e l l e r a n d F . L o p e s d a S i l v a , E v e n t - r e l a t e d E E G / M E G s y n c h r o n i z a t i o n a n d

d e s y n c h r o n i z a t i o n : b a s i c p r i n c i p l e s , C l i n N e u r o p h y s i o l

, 1 1 0 ( 1 1 ) : 1 8 4 2 1 8 5 7 , 1 9 9 9 .
http://dx.doi.org/10.1109/MSP.2008.4408441

Date post:	06-Apr-2018
Category:	Documents
Upload:	noscript
View:	220 times
Download:	0 times

Benjamin Blankertz- Acquisition and Analysis of Neuronal Data 2008 BCI: Lecture #13

Documents