Proceedings of the Second APSIPA Annual Summit and Conference, pages 203–206,Biopolis, Singapore, 14-17 December 2010.

Time-Constrained Filter Bank Common SpatialPattern for Motor Imagery Brain-Computer

InterfacesDong Huang∗, Cuntai Guan, Kai Keng Ang, and Zheng Yang Chin

∗ Institute for Infocomm Research, Agency for Science, Technology and Research (A*STAR), Singapore 138632E-mail: dhuang@i2r.a-star.edu.sg

Abstract—One of most important tasks or key steps in thedesigning of an EEG-based BCI system is the optimization ofspatio-temporal filters for each subject due to the poor spatialresolution of the EEG recordings, as well as the topographicalarrangement and frequency specificity of brain activities. Ahighly popular technique for the optimization of spatial filtersis Common Spatial Pattern (CSP). To address the problem ofselecting spectral filtering bands, Filter Bank Common SpatialPattern (FBCSP) was recently proposed, which improves theperformance of CSP significantly. This paper provides a deepinsight to the CSP algorithm and proposes a novel methodtermed Time-Constrained Filter Bank Common Spatial Pattern(TFBCSP). TFBCSP eludes the problem of selecting subject-dependent frequency bands by adopting the filter bank approachas in FBCSP and exploits the short-duration nature of ERD/ERSby imposing a time constraint on the EEG samples whose vari-ance is maximized/minimized. Favorable results were obtainedby the proposed method on dataset IVa from BCI CompetitionIII.

I. INTRODUCTION

A Brain-Computer Interface (BCI) is an emerging technol-ogy that provides a direct communication pathway between ahuman and an external device. In James Cameron’s epic film‘Avatar’, a humanoid avatar is controlled by the mind of aparaplegic marine soldier. BCI systems can do amazing jobslike mind-reading, direct control of external devices by humanmind in science fictions and films. In real life, the modernBCI is far more crude, confronted with many challengingproblems. With the surge of interest among researchers fromneuroscience, engineering, and signal processing, there hasbeen fast development for BCI systems in terms of algorithmsand applications. Among the various BCI systems, one ofthe most important BCI systems is the electroencephalogram(EEG)-based motor imagery BCI. In EEG-based motor im-agery BCI, EEG signals measured from the scalp duringthe mental imagination of movement is translated into thecommand to an external device. The main advantages of EEGare non-invasiveness, and low-cost. However, the designingof an EEG-based BCI system in motor imagery problems isquite complex, given the high variability of the EEG signalsfor different subjects, target events, and mental states.

Multichannel EEG recordings give a rather blurred imageof brain activities because EEG electrodes are separated fromcurrent sources in the brain by cerebrospinal fluid (CSF), theskull, and the scalp. Neurophysiological research has shown

that macroscopic brain activities are often characterized by adesynchronization/synchronization in certain frequency bandslocated over various cortical areas, which are termed event-related desynchronization/synchronization (ERD/ERS) [8, 9].For example, motor activities, both actual and imagined, canoften cause an attenuation of the µ-rhythm. Because of thepoor spatial resolution of the EEG recordings, as well asthe topographical location and frequency specificity of brainactivities, one of most important tasks or key steps in thedesigning of an EEG-based BCI system is the optimiza-tion of spatio-temporal filters for each subject. Among themyriad of signal processing methods for EEG-based BCIs,a highly successful spatial filtering method is the CommonSpatial Pattern (CSP) algorithm [10], as evidenced by BCICompetition II and III [2, 7]. CSP finds spatial filters thatmaximize the variance of the spatially filtered signal under onecondition while minimizing it for the other condition. Insteadof being a black-box process, CSP on band-pass filtered EEGsignals corresponds to neurophysiological understandings ofERD/ERS. Since variance of band-pass filtered signals is equalto band-power, the maximization/minimization of variance isequal to maximization/minimization of band-power of thefiltered signal. CSP analysis performed on band-pass filteredEEG signals effectively discriminates the mental states corre-sponding to ERD/ERS. The success of CSP thereby greatlydepends on a proper selection of subject-dependent frequencybands and time periods for which CSP is applied on.

Various extensions of the original CSP have been proposedto enhance its performance. Instead of manual selection offrequency bands for the successful application of CSP, simplefrequency filters are determined for each channel simultane-ously with the spatial filters in Common Spatio-Spectral Pat-tern (CSSP) [4]. A temporal FIR filter, which can be of highercomplexity than the simple filters in CSSP, is simultaneouslyoptimized with the spatial filters in Common Sparse SpectralSpatial Pattern (CSSSP) [3]. The Filter Bank Common SpatialPattern (FBCSP) [1] proposed the use of filter bank to avoidthe manual selection of frequency bands for CSP. To exploitthe spatial relationship between the electrodes, a spatiallyregularized CSP was proposed in [6]. An extension from two-class to multi-class by joint approximate diagonalization wasproposed in [5].

In this paper, a novel machine learning algorithm termed

203

10-0102030206©2010 APSIPA. All rights reserved.

Time-Constrained Filter Bank Common Spatial Pattern (TF-BCSP) is proposed for processing EEG signals in motorimagery BCI applications. TFBCSP is proposed to addressthe two important problems for CSP: appropriate selectionof frequency bands and time period. The proposed TFBCSPadopts the filter bank approach of FBCSP to avoid the manualselection of frequency bands and improses a time constrainton the EEG samples whose variance is maximized/minimized.In the following sections, the TFBCSP method and the clas-sification results are presented.

II. PROPOSED METHOD: TFBCSP

A. Problem definition

The task of EEG-based motor imagery BCI is to infer theimagined motions (motor imageries) based on the EEG datameasured from the scalp. As the EEG electrodes are separatedfrom current source in the brain by CSF, the skull, and thescalp, the relationship of the EEG data X ∈ RC×N (C is thenumber of channels and N is the number of time points in asingle trial) captured from the scalp and the source signalsS ∈ RC×N in the brain can be described as X = f(S)where f(·) is the transfer function of the head. Because thevolume conduction of the head can be approximated as a lineartransformation [11], f(·) is a linear function and

X = AS (1)

where A is the source mixing matrix of size C × C. Notethat although there are C sources (C rows) assumed in (1),the number of sources of interest, which is associated with themental activities of interest, is usually less than C and equalto the number of classes (mental tasks). The other sourcesare associated with background mental activities or noises inthe brain signal. It is beneficial to find a de-mixing matrixW ∈ RC×C such that sources related to the mental activitiesof interest can be estimated:

S = WX. (2)

Each row of W is referred to as a spatial filter, while eachcolumn of A is referred to as a spatial pattern. It is easy to seethat W = A−1. The determination of W by CSP on band-passfiltered EEG signals within a certain time period (post-stimulusperiod) corresponds to the neurophysiological understandingof ERD/ERS.

B. ERD/ERS, CSP, and FBCSP

Neurophysiological studies show that processing of motorcommands or somatosensory stimuli causes an changes inthe activity of local neurons, which in turn results in anattenuation or increase of rhythmic activity called ERD orERS respectively [8, 9]. Three important characteristics ofERD/ERS should be born in mind when we try to exploitERD/ERS for the classification of EEG signals: frequencybands, spatial location, and time period (or the post-stimulusperiod).

CSP finds spatial filters W that maximize the variance of thefiltered signals from one class while minimizing it for the other

class. Applied on band-pass filtered signal, CSP maximizesthe power ratio of the two classes within that particularfrequency band. This contrast in band power corresponds toneurophysiological understandings of ERD/ERS and could beused to discriminate the two classes.

Let Σ(1), Σ(2) ∈ RC×C be the estimates of the spatialcovariance matrices of the band-pass filtered EEG signal ofthe two classes (e.g., left hand imagination and right handimagination):

Σ(c) =1

|Dc|∑i∈Dc

Σi, where Σi =1

NXiX

Ti , c ∈ {1, 2},

(3)where Dc is the set of trials belonging to class c and |Dc| isthe size of the set. CSP finds the transformation matrix W thatsimultaneously diagonalizes the two covariances matrices:

WΣ(c)WT = Λ(c), c ∈ {1, 2}. (4)

This simultaneous diagonalization can be simply achieved bysolving the generalized eigenvalue problem:

Σ(1)w = λΣ(2)w. (5)

If the eigenvectors w are scaled to satisfy Λ(1) + Λ(2) = I (Iis the identity matrix), i.e., λ

(1)i + λ

(2)i = 1, for i = 1, ..., C,

a large value of λ(1)i close to one (which means λ

(2)i close

to zero) indicates that the corresponding spatial filter yieldssignals of class 1 having large power, while signals of class2 having small power. The correspondence of CSP withERD/ERS and also the success of CSP for BCI applicationsentails the employment of a spectral filter to extract informa-tion from the relevant frequency band and a spatial filter tocontrast the power of signals of the two classes.

The problem of selecting subject-dependent frequencybands is addressed by FBCSP, which uses a filter bank toextract components of different frequencies and then appliesCSP to find optimal spatial filters for each frequency band.The detail of FBCSP is given in [1].

C. Time-Constrained FBCSP

The problem of manual selection of frequency bands isaddressed by FBCSP. However, there still remains the problemof selection of time period. In the following, we first givea deep insight of CSP by a reformulation of its criterionfunction. Based on this insight, we imposes a time constrainton the EEG samples by a new definition of the spatial co-variance matrices of the two classes. With the time constraint,the spatial covariance matrices measure only the variation ofspatial patterns that are within a short-time interval, whichcorresponds to the short-duration of ERD/ERS.

The spatial covariance matrices Σ(c)i defined in (3) can be

204

transformed as:

Σ(c)i =

1

NXiX

Ti =

1

N

N∑j=1

xi(j)xi(j)T

=1

N

N∑j=1

(1

N

N∑k=1

xi(j)xi(j)T

)

=1

2N2

N∑

j=1

N∑k=1

xi(j)xi(j)T +

N∑j=1

N∑k=1

xi(k)xi(k)T

=

1

2N2

N∑

j=1

N∑k=1

xi(j)xi(j)T +

N∑j=1

N∑k=1

xi(k)xi(k)T

−N∑

j=1

N∑k=1

xi(j)xi(k)T −N∑

j=1

N∑k=1

xi(k)xi(j)T

=

1

2N2

N∑j=1

N∑k=1

(xi(j) − xi(k)) (xi(j) − xi(k))T

, (6)

where xi(j), denoting the jth column of the ith trial Xi, isthe EEG data recorded at time instance j in the ith trial.

The spatial covariance matrix for a trial shown in (6) is for-mulated in a pairwise way. This pairwise definition indicatesthat the spatial covariance matrix Σ

(c)i measures the mean

differences between all pairs of spatial patterns in a singletrial, i.e., xi(j) and xi(k) for j, k = 1, · · · , N . From thisinsight, we can get another way of interpreting CSP, that is, itmaximizes/minimizes the mean variation between all pairs ofspatial patterns. However, we need to only consider variationof nearby samples due to the short-duration of ERD/ERS.Variation between spatial patterns that are separated by a largetime interval may be mainly due to non-stationarity nature ofbrain signals.

The time-constrained spatial covariance matrices for eachtrial is defined as:

S(c)i =

1

2N2

N∑j=1

N∑k=1

η(j − k)Sj,k (7)

where Sj,k = (xi(j) − xi(k))(xi(j) − xi(k))T . The symbolS is used instead of Σ to denote the new time-constrainedcovariance matrix. η is the ‘time-constraint’ function thatlimits the calculation of the new covariance matrices S

(c)i to

pairs of spatial patterns that are within a certain time interval.The time-constraint function η decreases as the time intervalbetween the jth and kth spatial patterns increases.

Since we are interested in information within certain fre-quency bands, e.g., β band, variation of spatial patterns withinvery short time period can be left out because it is mainlydue to high frequency noise. Removing the symmetry ofxi(j)− xi(k) and xi(k)− xi(j) in (7), the covariance matrixS

(c)i in (7) is simplified as:

S(c)i =

1

N2

N−τ∑j=1

∑k=j+τ :τ :N

η(j − k)Sj,k (8)

where τ is the sampling interval for the calculation of S(c)i .

Variation of spatial patterns that are within the same samplinginterval is left out in (8).

In this paper, we adopt a Gaussian-like function for the ηfunction, defined as

η(△ t) =

{exp

(−△t2

2σ2

)if | △ t| < 3σ

0 otherwise(9)

where σ determines the length of the time constraint.Substituting (9) into (8), we have

S(c)i =

1

N2

N−τ∑t=1

d0∑d=1

η(dτ)St,t+dτ (10)

where d0 is the largest integer that satisfies dτ < 3σ andt + dτ < N .

The spatial covariance matrices of the two classes estimatedby TFBCSP are thus

Σ(c) =1

|Dc|∑i∈Dc

S(c)i , c ∈ {1, 2}, (11)

where S(c)i is defined in (10).

The spatial filters W of TFBCSP are optimized by simul-taneous diagonalization of the spatial covariance matrices ofthe two classes as defined in (11) and can be easily solved bythe generalized eigenvalue problem just as (5) of CSP. As forFBCSP in [1], a filter bank is used by TFBCSP to extractoptimal spatial filters for each frequency band. Instead ofdirectly selecting a subject-specific time-period, the proposedmethod addresses this problem by imposing a time constraintin the definition of the spatial covariance matrices.

III. EXPERIMENTS

The performance of the proposed TFBCSP for motor im-agery BCI applications is tested on EEG data set IVa fromBCI Competition III [2]. The dataset is collected from 5subjects (labeled ‘aa’, ‘al’, ‘av’, ‘aw’, and ‘ay’) who performedright hand and right foot motor imageries. The data for eachsubject comprises 280 trials of EEG measurements from 118electrodes. We took the 500-2500 msec interval after visualcue of each trial.

We implemented 5 different algorithms to investigate theeffectiveness of the proposed TFBCSP. The 5 algorithms are1)CSP with no frequency filtering, 2)CSP with wide-band fil-tering (8-30Hz), 3)Time-Constrained CSP without filter bank(denoted as TCSP), 4)FBCSP, and 5)TFBCSP. The purpose ofimplementing 1) is to show the importance of spectral filteringfor CSP. Algorithms 3),4) and 5) are implemented to show therespective effect of time constraint and filter bank for CSP.As in [1], a zero-phase Chebyshev Type II Infinite ImpulseResponse (IIR) filter is used in the filter bank. The filterbank extracts 9 components ranging from 4 to 40 Hz with abandwidth of 4Hz. The sampling interval τ and time constraintσ (or d0) are simply set as τ = 0.03 sec × 100 samples/sec =3 samples, and σ = 0.25 sec × 100 samples/sec = 25 samples,since the sampling frequency is 100Hz. A common parameter

205

that needs to be set for all the 5 algorithms is the number ofspatial filters selected from the set of spatial filters W . Weselected the first 2 and last 2 spatial filters. The classifier usedis Support Vector Machine (SVM).

The classification performance is obtained by 10 × 10-fold cross-validation. The 10-fold cross-validation procedurerandomly divides the dataset into ten equally sized distinctpartitions. Each partition is then used for testing, while otherpartitions are used for training. This results in 10 accuraciesor error rates, which are averaged to give the accuracy of 10-fold cross-validation. The 10-fold cross-validation procedureis repeated ten times and the accuracies of the ten runs areagain averaged.

Table I shows the test accuracies, estimated by 10 × 10-fold cross-validation procedure. Note that TCSP is applieddirectly on the raw EEG data without any frequency filtering.Comparing the performance of TCSP with CSP on band-passfiltered (8-30Hz) signals, we can observe that the accuraciesof TCSP are comparable to those of CSP on band-pass filteredEEG data. However, if we compare TCSP to CSP without anyband-pass filtering, the advantage of TCSP is obvious. The useof filter bank for extracting optimal spatial filters for differentfrequency components further improve the performance ofCSP and TCSP. Overall, TFBCSP achieves the best accuraciesfor subjects ‘al’,‘av’,‘aw’, and ‘ay’, as well as the averageaccuracy over the five subjects.

IV. CONCLUSIONS

This paper proposed a novel machine learning algorithmtermed Short-Term Filter Bank Common Spatial Pattern (TF-BCSP) for processing EEG data in motor imagery BCI appli-cations. Corresponding to the neurophysiological phenomenaof ERD/ERS, the proposed method eludes the problem ofselecting frequency bands by a filter bank and exploits theshort-duration nature by the formulation of time-constrainedspatial covariance matrices. Preliminary results on data fromBCI Competition III dataset IVa demonstrated the effective-ness of the proposed TFBCSP.

REFERENCES

[1] K. Ang, Z. Chin, H. Zhang, and C. Guan. Filterbank common spatial pattern (FBCSP) in brain-computer

TABLE IAVERAGE TEST ACCURACIES (%) OF 10× 10-FOLD CROSS-VALIDATION ON

BCI COMPETITION III DATASET IVA

SubjectsCSP without

band-passfiltering

CSP withwidebandfiltering

TCSP FBCSP TFBCSP

‘aa’ 51.2 82.4 82.9 87.1 87.0

‘al’ 62.0 99.1 98.6 98.8 99.0

‘av’ 51.3 71.2 71.0 74.1 76.6

‘aw’ 47.9 90.7 93.3 95.6 96.0

‘ay’ 50.8 95.0 95.5 94.1 94.4

Average 52.7 87.7 88.3 89.9 90.6

interface. In Proceedings of the IEEE InternationalJoint Conference on Neural Networks (IJCNN’08), pages2391–2398, Hong Kong, 2008.

[2] B. Blankertz, K. R. Muller, G. Curio, T. M. Vaughan,G. Schalk, J. R. Wolpaw, A. Schlogl, C. Neuper,G. Pfurtscheller, T. Hinterberger, M. Schroder, andN. Birbaumer. The BCI competition 2003: Progressand perspectives in detection and discrimination of EEGsingle trials. IEEE Trans. Biomed. Eng., 51(6):1044–1051, 2004.

[3] G. Dornhege, B. Blandertz, M. Krauledat, F. Losch,G. Curio, and K.-R. Muller. Optimizing spatio-temporalfilters for improving brain-computer interfacing. InAdvances in Neural Inf. Proc. Systems (NIPS 05), vol-ume 18, pages 315–322, Cambridge, MA, 2006. MITPress.

[4] S. Lemm, B. Blankertz, C. Curio, and K.-R. Muller.Spatial-spectral filters for improving classification ofsingle trial EEG. IEEE Trans. Biomed. Eng., 52(9):1541–1548, 2005.

[5] S. R. Liyanage, J. Xu, C. Guan, K. K. Ang, and T. H. Lee.EEG signal separation for multi-class motor imagery us-ing common spatial patterns based on joint approximatediagonalization. In IEEE World Congress on Computa-tional Intelligence & International Joint Conference onNeural Networks (IJCNN), pages 18–23, Barcelona, July2010.

[6] F. Lotte and C. Guan. Spatially regularized commonspatial patterns for EEG classification. In InternationalConference on Pattern Recognition (ICPR), pages 23–26,Istanbul, Turkey, August 2010.

[7] D. McFarland, C. Anderson, K.-R. Muller, A. Scholgl,and D. Krusienski. BCI meeting 2005 — workshop onBCI signal processing: Feature extraction and translation.IEEE Trans. Neural Syst. Rehab. Eng., 14(2):135–138,Jun 2006.

[8] G. Pfurtscheller and F. d. Silva. Event-related synchro-nization of mu rhythm in the EEG over the cortical handarea in man. Neurosci. Lett., 174:93–96, 1994.

[9] G. Pfurtscheller and F. d. Silva. Event-related EEG/MEGsynchronization and desynchronization: basic principles.Clinical Neurophysiology, 110:1842–1857, 1999.

[10] H. Ramoser, J. M uller-Gerking, and G. Pfurtscheller.Optimal spatial filtering of single trial EEG during imag-ined hand movement. IEEE Trans. Nueral Syst. Rehab.Eng., 8(4):441–6, Dec. 2000.

[11] R. Srinivasan. Methods to improve the spatial resolutionof EEG. International Journal of Bioelectromagnetism,1(1):102–111, 1999.

206