978-1-4244-9350-0/11/$26.00 ©2011 IEEE 583
2011 4th International Conference on Biomedical Engineering and Informatics (BMEI)
A Collaborative Brain-Computer Interface
Yijun Wang, Yu-Te Wang, Tzyy-Ping Jung*
Swartz Center for Computational Neuroscience University of California San Diego
San Diego, USA
Xiaorong Gao, Shangkai Gao Dept. Biomedical Engineering, School of Medicine
Tsinghua University Beijing, China
Abstract—Electroencephalogram (EEG) based brain-computer interfaces (BCI) have been studied for several decades since the 1970s. Current BCI research mainly aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. The BCI technology can also benefit normal healthy users; however, little progress has been made in real-world practices due to low BCI performance caused by technical limits of EEG. To overcome this bottleneck, this study uses a collaborative BCI to improve overall performance through integrating information from multiple users. A dataset involving 15 subjects participating in a Go/NoGo decision-making experiment was used to evaluate the collaborative method. Using collaborative computing techniques, the classification accuracy for predicting a Go/NoGo decision was enhanced substantially from 75.8% to 91.4%, 97.6%, and 99.1% as the number of subjects increased from 1 to 5, 10, and 15, respectively. These results suggest that a collaborative BCI can effectively fuse brain activities of a group of people to improve human behavior.
Keywords-Brain-computer interface (BCI); collaborative computing; Electroencephalogram (EEG); human performance; Go/NoGo decision making
I. INTRODUCTION
The human brain is the most complex system in the world. The functional brain imaging technologies such as functional magnetic resonance imaging (fMRI) and Electroencephalogram (EEG) give us an opportunity to observe brain activities related to thoughts, emotions, and behavior, and therefore, help us understand the relationship between the brain and behavior. Recently, a new technology known as brain-computer interface (BCI) or brain-machine interface (BMI) has made a significant progress in brain science [1]. The BCI study covers the three aspects in exploring the human brain: understanding the brain, protecting the brain, and creating the brain. During the past two decades, the BCI technology has become a hot research topic in the areas of neuroscience, neural engineering, medicine, and rehabilitation [2][3].
In essence, a BCI is a communication channel that bypasses the traditional pathway of peripheral nerves and muscles, and creates a direct link between the human brain and an output device [1]. Currently, the main focus of BCI research lies in the clinical use which aims to provide a new communication channel to patients with motor disabilities to improve their quality of life. In current BCI systems, commonly used neural recording technologies include EEG, Magnetoencephalogram (MEG), Electrocorticogram (ECoG), fMRI, near infrared spectroscopy (NIRS), and neuronal
recording. Among these methods, EEG is the most widely used modality in current BCI studies due to its advantages such as simple and inexpensive equipment, flexibility and mobility, and short time constants. In present-day BCIs, the following EEG signals have been paid much attention: visual evoked potential (VEP), sensorimotor mu/beta rhythms, P300 evoked potential, slow cortical potential (SCP), and movement-related cortical potential (MRCP) [1].
Although the EEG-based BCI technology has achieved great successes, moving a BCI system from a laboratory demonstration to a real-life application still poses severe challenges to the BCI community. Applications of the BCI technology are very limited due to bottleneck problems including high system cost, low communication speed, low recognition accuracy, and easy user fatigue [4]. To overcome these problems, a practical solution is to develop a multi-user collaborative BCI system, which can utilize collective intelligence from a group of users. Recently, we first proposed the framework for a collaborative BCI system and further investigated the feasibility and practicality of the system [5]. The development of group-synchronized neural recording systems and group collaborative cognitive computing methods will open a totally new direction for BCI research.
In this study, we propose to study the feasibility of using a collaborative BCI system to improve human decision making in a Go/NoGo decision-making task. In the Go/NoGo task, the N2 event-related potential (ERP) component, which reflects the processing of motor inhibition, will be used as a feature for identifying the NoGo condition. To evaluate the performance of the collaborative BCI, EEG-based prediction of a Go/NoGo decision will be executed using a single-trial classification paradigm and a collaborative classification paradigm respectively.
II. METHODS
A. System diagram
Figure 1 shows the system diagram of a collaborative BCI. Similar to a single-user BCI, a collaborative BCI consists of three major parts: a data acquisition module, a signal processing module, and a command translation module. Consequently, there are three major procedures in system operations:
1) Brain signals from a group of users are acquired by multiple EEG recording devices, and then are synchronized with common environmental events.
584
2)
3)
ComfromdatanewhavEEGpro
B.
sevdes
1)
2)
3)
IdeparparrecosignA cBCmay
1)
Integrated EEfeatures for d
Extracted fetranslated to to give senso
mpared to a sinm multiple usa recording an
w algorithms ve to be develoG analysis, wh
ocessing modul
Figure
System implem
The implemeveral specific signs:
Multiple EEindependently
Multiple-subjsynchronizedevents.
Multiple-subjprocedures ha
eally, the systeradigm similar radigm, EEG dorded, then thnal processing centralized para
CI system; howy be limited fo
Data transmisystems needsending/receirequires highand storage. standing and EEG systemsare more pre
EG and event ddecoding users’
atures from aoperation comry feedback to
ngle-user BCI, ers will lead t
nd signal-procefor implemen
oped to performhich plays the le.
1. System diagram
mentation
entation of a requirements
EG recordingy and simultan
ject data nd with respect
ject data recave to be perfo
em can be imto a conventio
data from multhrown into a and command
adigm is optimwever, practicar the following
ission: When wd to be conneciving in real th capacities of
Because userwalking are al
s, portable and ferable in natu
data are process’ intentions.
a group of usmmands, which
the users.
the complexityto technical chssing procedurnting collabor
m the proceduremost importan
m of a collaborative
collaborative for hardwar
g systems nneously.
need to be to the commo
cording and ormed in real tim
mplemented usional BCI (Figutiple subjects conventional
d translation usimal for designinality of systemg reasons:
wired EEG systcted to the datime. Thereforedata communirs’ natural belways limited wmobile EEG r
ural environme
sed for extracti
sers are direccan also be us
y of system inphallenges in bores. For examprative computie of collaboratint role in the da
e BCI.
BCI has posre and softwa
need to wo
received aon environmen
data processime.
ing a centralizure 2(a)). In thare received aBCI module fing a data serv
ng a collaboratim implementati
tems are used, ta server for dae, the data sevication, memo
ehaviors such when using wirrecording devicents. Then, a da
ing
ctly sed
put oth ple, ing ive ata
sed are
ork
and ntal
ing
zed his
and for
ver. ive ion
all ata ver ry, as
red ces ata
serwir
2) Comacurreqcolinvinccolcomper
3) Syscondevshoeveconperof
Figure 2collabora
To this stuimplem2(b), thsubsystBCI sucapabilparadigand theprocessbeen wparadigcollaboparadig
rver requires a reless infrastru
omputational cachine learningrrent BCI stuquire large amllaborative BCvolved, the crease. Becaullaborative BCmputation, the rformance CPU
stem robustnesnsists of muvices. To assurould have the aen when subsnnection loss)rformance shoua subsystem or
2. (a) A centralizeative BCI.
solve the probudy proposes
mentation of a he whole systtems and a simubsystem worklity in EEG dgm, the amoune data server, asing, are signif
well studied in pgm is a moreorative BCI. Tgm is that the o
low-latency, hucture, which m
cost: Advanceg techniques h
udies [6] [7]. mount of comCI where a lar
computationaluse real-time CI will leaddata server ha
Us and large am
ss: A collaboraultiple EEG rre the stability ability to keep
sets of the wh). In other wuld not be serior subsystems.
ed paradigm and
lems existing ia distributed collaborative
tem consists omplified data ks independentdata acquisitiont of data transas well as the ficantly reduceprevious studiee practical solThe only disadoverall costs of
high bandwidthmight be very c
ed signal prohave been wiThese approa
mputational resrge amount ofl cost will
data proced to a large as to be equippmounts of mem
ative BCI systerecording andof the system,
p the whole syshole system fawords, the ovously affected
(b) a distributed
in the centralizparadigm to BCI. As show
of multiple disserver. For eatly, each subsyon and processmitted betweencomputational
ed. The single-es. Therefore thlution for impdvantage of thf the system har
h, and reliable ostly.
ocessing and dely used in aches always sources. In a f subjects are
significantly essing in a
amount of ed with high-
mory.
em inevitably d processing , the software stem working ail (e.g., data verall system by the failure
paradigm for a
zed paradigm, facilitate the
wn in Figure stributed BCI ach subject, a ystem has its ssing. In this n subsystems cost for data user BCI has
his distributed plementing a he distributed rdware might
585
incrfor cansys
C.
persingexpbe phoimatargquiappsubwercodfor
D.
eacindremsegremThicon(SVeacesti
E.
colclasvotclasa w
whweiclasthe
A.
imamebetwthe shomig
rease due to theach user. In
n be integrated tem cost, and i
Go/NoGo dec
Following a rformed in altegle-photographperimental setu
found in [otographs werages and flasheget, subjects hickly and accupeared, subjectbject, 32-channre recorded todes at a 1000 Hoffline analysi
Single-trial EE
This study pech subject usingdependent commove eye-movegments in a prmoving the ERPird, the intercncatenated, andVM)-based clach subject, a imate the classi
Collaborative
Using the dislaborative datassifier [11], whting system. Insses are labeled
weighted voting
ere m is the nuight and y(i) ssifier was traitraining accur
Event-related
As shown in age onset, the dial frontal coween the Go aFz electrode)
owed a larger Nght reflect the
he employmenpractice, portinto the EEG
improve system
ision-making e
Go/NoGo parnation an "anh recognitionup and the ima9]. During te randomly med for 20 ms ohad to lift theurately as posss had to withh
nel EEG data oogether with stHz sampling rais, totally resul
EG classificati
erformed a sing a standard ma
mponent analyement and musredefined timeP baseline calccepted ERPs d then inputtedssifier to predi10x10-fold c
ification perfor
e classification
stributed systema analysis washich consists o
n the case of a bd as +1 and -1
g can be describ
m
i 1
sign
umber of subjeis the output ined as a sub-
racy was used a
III. R
potentials
Figure 3, durN2 ERP comp
ortex (MFC), sand NoGo cond. Compared toN2 componentmotor inhibit
nt of a data-protable data-procrecording dev
m practicality a
experiment
aradigm, 15 imal" categorin task. Dges used in thethe experimen
mixed with noon a computer eir finger fromsible. When noold their buttonof 10 blocks (timulus/responte and downsalting in 500 tria
ion
ngle-trial EEG achine-learning
ysis (ICA) wascle artifacts [1e window werculated within [
from all 32 d to a supportict the Go/NoGcross validatiormance.
m paradigm (Fs performed wof multiple subbinary classificrespectively, t
bed as follows
iyiw )()(
ects, w(i) is theof a sub-clas
classifier for eas the voting w
RESULTS
ing 180 ms - ponent, which showed a signiditions (paired
o the Go trials,t (-9.8 uV vs. tion process. A
ocessing platforcessing platformvice to reduce ts well [8].
human subjezation task andetails of te experiment cnt, target (G
on-target (NoGscreen. For ea
m the button on-target imagn press. For ea(100 trials eacnse related eveampled to 200 Hals per conditio
classification g paradigm. Firas employed 10]. Second, ERe extracted af[-100 ms - 0 melectrodes we
t vector machiGo decision. F
on was used
Figure 2(b)), twith an ensemb
-classifiers andcation where twthe procedure f:
(
e subject specissifier. An SVeach subject, a
weight.
250 ms afterlocated over t
ificant differend t-test, p<10-5,, the NoGo tri-4.5 uV), whi
A subsequent
rm ms the
cts d a the can Go) Go) ach
as ges ach h), ent Hz on.
on rst, to
RP fter
ms]. ere ine For
to
the ble d a wo for
(1)
ific VM and
an the nce , at als ich P3
componmedial patternconditiodecisio
Figure 3.condition
B. Sing
Thesignificprior tosubjectsubject75.8±6courseswas enh74.4±6ms to 2suggestby sing
Figure 4.line indic
nent also largefrontal and pa
s of N2 and Pons provide
on using EEG [
. Scalp ERP wavens at all electrode p
gle-trial classif
e single-trial cantly higher to mean responts (377±48 msts when using .7%, range: 64s of the N2 andhanced from 6.3% as the len
250 ms, 300 mted that a Go/N
gle-trial EEG cl
. Accuracy of singcates the chance lev
ely differed undarietal areas. TP3 componentthe basis fo
[12].
e forms and differpositions.
ification
EEG classifithan the channse time (RT) s). Figure 4 sthe time wind
4.4% - 85.2%)d P3 componen61.2±4.1% to 6ngth of time w
ms, and 350 ms,/NoGo decisionlassification.
gle-trial EEG classvel (50%).
der two conditThe distinct spats under the Gor predicting
rence waves unde
ication achievnce level (50%
of the Go triashows the accudow of [0 RT). Consistent wnts, the predict68.1±4.3%, 70window increas, respectively. n can be reliab
sification for all su
tions over the atio-temporal
Go and NoGo a Go/NoGo
er Go and NoGo
ved accuracy %) using data
als across all uracy for all
T] (mean±std: with the time tion accuracy .9±4.5%, and sed from 200 These results bly predicted
ubjects. The dash
586
C.
the 1, intetimaccsubnumrespaccacccollsubaro150decmedmo
Figuof thacrodevi
BCclasimpcollmuthiscoll
be softchacollcollof aplatrese
Collaborative
Figure 5 showlength of time5, 10, and 1
eraction betweme. Using the curacy for prebstantially frommber of subjepectively. Th
celeration of decuracy and thlaborative sys
bjects were incund 200 ms af
0 ms earlier thcoding the grodial frontal cotor inhibition.
ure 5. Classificatiohe window length.oss all subjects (37iation.
IV.
This study demCI to acceleratessification accprovement ovelaborative BCI
uch earlier thans study desiglaborative BCI
The prototypedirectly transfe
ftware requiremallenges that laborative BClaborative BCIan EEG recordtform. Becausearch are stil
e classification
ws the classifice windows use5 subjects ween the numbertime window dicting a Go/N
m 75.8% to 91ects increased
he results alsecision-makinghe number otem. As show
cluded, the Go/fter the stimuluhan the subjecoup ERP activortex, which ar
on accuracy of diff. The vertical line 7 ms). The dash li
CONCLUSION
monstrated an e decision-makcuracy of the r that of the sinI allowed the n his/her actuagned and deI technology to
e system demonerred to an onl
ments can be mhave to be
CI system canI needs multiplding system ane commercial l expensive, t
cation accuracyed for data anaere put togethr of subjects an
of [0 RT], tNoGo decisio.4%, 97.6%, a
d from 1 to so clearly shg depended on of subjects inwn in Figure /NoGo decisious onset, whicct’s actual movities arising mre related to th
ferent numbers of sindicates the mean
ines indicate mean
N AND DISCUSS
application of king in a Go/system show
ngle-user BCI. subject’s decil motor responemonstrated th improve huma
nstrated in the line system if t
met. Currently, resolved be
n become a le BCI platformnd a real-time s
EEG productthe total cost
y as a function alysis. Results fher to show tnd the predictithe classificatin was enhanc
and 99.1% as t5, 10, and 1
howed that tboth the desir
nvolved in t5, when all n could be mah was more th
otor response,mainly from the processing
subjects as a functn response time (R
n accuracy ± stand
SIONS
the collaborati/NoGo task. T
wed a significaFurthermore, t
ision to be manse. In summahe use of tan performance
current study cthe hardware athere are seve
efore an onlireality. First,
ms, which conssignal-processits used for EEt for building
of for the ion ion ced the 15, the red the 15
ade han by the of
tion RT) dard
ive The ant the ade ary, the e.
can and eral ine
a sist ing EG g a
collaborequirecommuplatformserver. the dataadvanctechnolcollabo
ThiBioscieArmy Cooperviews aof the aofficialResearcGovernfor Gonotation
Thethe EEG
[1] J. RVauClin
[2] M. pre
[3] N. Clin
[4] Y. basdes
[5] Y. impdoi
[6] D. KruFeaEng
[7] F. Lof cNeu
[8] Y. inte025
[9] A. Botnatu
[10] A. Dof ana
[11] DudYor
[12] K. DynNeu
orative BCI wils specific softw
unication betwms, and betwFurthermore,
a server has toes in biomedlogy, it will soorative BCI sys
is work is suence Inc. ReseResearch Lab
rative Agreemand the conclusauthors and shol policies, eithch Laboratorynment is authoovernment pun herein.
e authors woulG data.
R. Wolpaw, N. Biughan, “Brain-comn. Neurophysiol.,
A. Lebedev, M. esent and future,” T
Birbaumer, “Bran. Neurophysiol.,
Wang, X. Gao, Bsed on visual evsigns,” IEEE EMB
Wang, T. P. Junproving human pi:10.1371/journal.p
J. McFarland, Cusienski, “BCI Meature extraction ang., vol. 14, pp. 135
Lotte, M. Congedclassification algorural Eng., vol. 4, p
T. Wang, Y. Wanerface for commu5018, 2011.
Delorme, G. Routtom-up and Topural scenes,” Cog.
Delorme, S. Makesingle-trial EEG
alysis,” J. Neurosc
da RO, Hart PE, Srk: Wiley Interscie
Yamanaka, Y. Ynamics Associateurosci. vol. 22, no
ll be high. Secware developm
ween EEG systween the BCI
data processin be implement
dical electronicoon be possib
stem.
ACKNOWLEDG
upported by aearch was alsoboratory and
ment Number sions containedould not be inther expressed y or the U.S
orized to reprodurposes notwi
ld like to thank
REFERENC
irbaumer, D. J. Mmputer interfaces vol. 113, pp. 767-
A. L. Nicolelis, Trends Neurosci.,
ain-computer-intervol. 117, pp. 479-
B. Hong, C. Jia, Svoked potentials: B. Mag., vol. 27, p
ng, “A collaboratperformance,” PLopone.0020422, 201
C. W. Anderson, eeting 2005 - Wond translation,” IE5-138, 2005.
o, A. Lecuyer, F. rithms for EEG-bapp. R1-R13, 2007
ng, T. P. Jung, “Aunication in daily
usselet, M. Mace, p-down processing. Brain Res., vol. 1
eig, “EEGLAB: anG dynamics inci. Meth., vol. 134,
Stork DG, Pattern ence Press, 2000.
Yamamoto, "Singed with Voluntary. 4, pp. 714-727, 2
cond, a collaboment, which allotems and sign
subsystems ang in BCI subted in (near) recs and teleco
ble to impleme
GMENT
a gift fund fro sponsored in
was accompW911NF-10-
d in this documterpreted as rep
or implied, oS Governmenduce and distriithstanding an
k A. Delorme
CES McFarland, G. Pfur
for communicatio791, 2002.
“Brain-machine vol. 29, pp. 536-5
rface research: C483, 2006.
. Gao, “Brain-comfeasibility of p
pp. 64-71, 2008.
tive brain-computoS ONE, vol. 6, 11.
K. R. Muller, Aorkshop on BCI siEEE Trans. Neura
Lamarche, B. Arnased brain-comput.
A cell-phone basedy life,” J. Neura
M. Fabre-Thorpeg in the fast vis19, no. 2, pp. 103-
n open source toolcluding independ, pp. 9-21, 2004.
Classification (2n
gle-trial EEG Pory Response Inhib2010.
rative system ows seamless
nal-processing and the data bsystems and eal time. With ommunication ent an online
from Abraxis n part by the lished under 2-0022. The
ment are those presenting the of the Army nt. The U.S ibute reprints ny copyright
for providing
rtscheller, T. M. on and control,”
interfaces: past, 46, 2006.
oming of age,”
mputer interfaces practical system
ter interface for no. 5, e20422,
. Schlogl, D. J. gnal processing: al Syst. Rehabil.
naldi, “A review ter interfaces,” J.
d brain-computer al Eng., vol. 8,
e, “Interaction of sual analysis of 113, 2004.
lbox for analysis dent component
nd Edition). New
ower and Phase bition," J. Cog.