Department of Computer Science Development and Application of an EEG-based Brain- Computer Interface...

Department of Computer Science

Development and Application of an EEG-based Brain-

Computer Interface

By Aleksey Tentler

Committee:

Chair: Dr. Jessica Bayliss

Reader: Dr. Roger Gaborski

Observer: Dr. Ankur Teredesai


Overview

• Introduction

• Related Work

• System Development

• Experiments

• Problems Encountered

• Future Work

• Summary


Introduction

• There exist diseases of the nervous system that gradually cause the body’s motor neurons to degenerate– Example: Amyotropic Lateral Sclerosis (ALS)

• Eventually causes total paralysis

• The affected individual becomes trapped in his own body, unable to communicate


Introduction (continued)

• A Brain-Computer Interface (BCI) enables communication under such circumstances

• Using data recorded from the brain, the BCI processes it, interprets the intention of the user, and acts on it.


Thesis Work

• We develop and demonstrate a simple version of a BCI, capable of detecting an answer to a Yes/No question– The BCI has a robust and flexible design that can be

expanded in the future to encompass more complex communication schemes.

• We present a panel of experiments that attempt to determine what effect, if any, the methods of stimulus presentation have on communication accuracy


BCI defined

• A BCI generally consists of three main components:

1. A Signal-acquisition module

2. A Signal-processing module

3. A Control module


BCI defined (continued)

Signal-acquisition module

1. Records data from the brain

2. Does some low level filtering

3. Passes the data on to be interpreted


BCI defined (continued)

Signal-processing module

1.Receives data from the acquisition module

2.Detects whether a particular feature or a potential is present in the data


BCI Defined (Continued)

Control module

1. Anything from a hardware device to a software application on a computer screen

2. Gives feedback to the user

3. Can present him with choices of commands or actions at his disposal



Control module (continued)4. Can communicate with the signal acquisition and/or

signal processing to control when to start recording and when to expect a particular feature to occur

5. Can, in some cases, also perform some analysis and/or display of the data to increase the accuracy of detection– Functions in conjunction with or instead of the signal-

processing module.


Signal Acquisition

Signal Processing

Control Device

FeedbackData from Brain

Filtered Data

Detected Features

Control(Optional)

Control(Optional)

FilteredData

(optional)

A Model of a generalized BCI




BCI Types:– Dependent

– Uses the activity in the brain’s normal output pathways

– Independent– Not based on the brain’s normal output

pathways

– Based on cognitive brain activity

http://www.innovativeoandpofla.com/prosthetic_Arm_Pictures.htm



BCI Types:

– Invasive– Involves attaching electrodes directly

to the brain tissue

– The patient’s brain gradually adapts its signals to be sent through the electrodes

– Non-invasive• Involves putting electrodes on the scalp of the

patient, and taking readings

http://spiderman.sonypictures.com/downloads/

http://www.tcnj.edu/~leynes/lab/tour.html



Non-invasive EEG BCI

– Signal Types:– Signals requiring training by operant

conditioning– Slow cortical potentials, alpha, mu, and beta

rhythms

– Signals not requiring training– P3 evoked potentials


Non-invasive EEG P3 BCI

P3 potential types:

1. P3a– Evoked whenever a novel, unexpected, and

unrecognizable stimulus is inserted into a random sequence of frequent and infrequent stimuli

SeptemberFebruaryDecemberMarchJulyNovember

2. П3б– Наблюдается когда появляется что-то ожидаемое,

имеющее отношение к тому над чем человек думает

2. P3b

– Evoked when a task-relevant stimulus that has been expected occurs in a random sequence of frequent and infrequent stimuli


Non-invasive EEG P3b BCI

• P3b General Form

• A positive wave that peaks about 400 ms after a visual stimulus and about 300 ms after an auditory stimulus

• Might be less pronounced, or completely absent, if the user is not focused on the task

http://p300.scripps.edu/default.htm


Non-invasive EEG P3b BCI

Accuracy Issues– Software-dependent

– Affected by how well the software can classify a P3 potential when it’s present in the data

– User-dependent– Affected by whether the user is focused on task

– Habituation is possible


Related Work

• Wadsworth Brain-Computer Interface Center, Albany– Use signals such as mu or beta rhythms that require

training, which is often difficult and takes time.

– Developed an application called BCI2000

• University of South Florida– E. Donchin

– Using the BCI2000 software

– Working with the P3 evoked potentials


Related Work (continued)

• P. Kennedy, Atlanta and by J. Donoghue, Brown University– Invasive methods, requiring surgery– Allows the patient to directly interface with a

computer through implanted electrodes– Requires a long training period


Demand for Software

• Currently working with an ALS patient– Tried to get help from

Wadsworth, USF, and Atlanta

• It seems that the current available software does not meet all the necessities of flexibility, stability, and accuracy


Software Development

• Designed as a platform framework– Allows future designers and researchers to

easily add or replace modules as needed– Don’t have to be knowledgeable in the low-

level signal handling and transfer


BCI Structure

Acquisition Module(Server)

Signal Processing

Module(Client)

User Application

(Client)

Event Time

Event Epoch

Signal Detected(as a short)

Event Epoch

Signal Detected (as a short)

EEG Visualization

Module(Client)

Event Epoch


Acquisition Module

Acquisition GUI

Acquisition Thread

EEG Data Server and

Event Receiver

Listeners (over

network)

User Application

(over network)

Hardware

Circuit & Filter


Signal Processing Module

Data Receiver / Signal Sender

Client

Signal Processor

Signal Processing Console

Acquisition Module (over

network)


User Application Module

Application Thread

Event Sender / Signal Receiver

Client

Application GUI

Acquisition Module

(over network)

BCIGui

Specific App

handleEvent()handleData()

handleSignal()handleDecision()


Experiments

• Subjects

• Procedure

• Online Analysis

• Offline Analysis

• Results


Subjects

• Nine total subjects– 8 high-school to college-age males– 1 female ALS patient

• To increase the motivation of non-ALS subjects, upon the completion of the experiment they were given candy or the opportunity to play computer games in the AI Lab.


Procedure

• The background and goals of the experiment were explained to each subject.

• The subject was fitted with an electrode cap and face electrodes


Procedure (continued)

• In the experiment, the subject was asked to use the three application GUIs

• The order of use of each GUI was determined randomly.


GUIs

2-Button Application 1-Button Application



• For each GUI, seven trials were conducted consecutively

• Each trial was composed of 34 total flashes of the stimuli.

• The target response for each trial alternated between “Yes” and “No”.



• For each trial, the subject was told they can either:– Count the number of times that the target response

flashed

– Pick a question which had the target response as the answer, and visualize achieving the answer each time the target response flashed.

• Before the official trials started, the subject was given 4 sample runs to get used to the system


Procedure( continued)

• At the end of each trial, the system’s response was shown to the subject.

• At the end of the experiment, the subject was asked for his/her opinion on which of the three GUIs was the easiest or hardest to use and for what reason(s)


Online Analysis

• The system presented 34 total events to the user, keeping track of the average response epoch for each event type.

• Epochs in which the eye channel electrodes showed peaks over 80 v or below –80 v were not included in the average– Avoided false detections and misdetections due

to eye movements.


Online Analysis (continued)

• After all the events have been flashed, the system would analyze the two average event type epochs, and try to determine which one of them seemed closer to a P3 potential.

• Examined several measurements of the two epochs– Positive peak at about 400 ms after the stimulus

– Negative peak before the positive peak at about 300 ms

– Positive peak before the negative peak at about 100 ms



• Compare the maximums/minimums and means of the two average epochs for those areas

• Each comparison of a mean or a maximum/minimum counted as one “vote”.



• Compare the ratio of votes for each possible answer to make a decision

• If the numbers of votes for each epoch were equal, the system would announce that no answer was detected.


Offline Analysis

• Data was further analyzed offline using MATLAB

– Used several more analysis techniques:

• Peak-picking

• Peak aligning

– Attempt to get better support for the results and to analyze which, if any, analysis methods work better than others.


Results with the Max/Min Finding Algorithm

• Overall accuracy - 93.46 %• 90 % for the 1-Button GUI• 92.31 % for the 2-Button GUI• 98.04 % for the 2-Button Novelty GUI

• 93.62% for the first GUI• 90.74% for the second GUI• 96.15% for the third GUI.


Results


Overall Grand Average


GUI Averages by Type


GUI Averages by Order

Figure 15: Overall P3 Grand AverageFigure 16: 1-Button Grand AveragesFigure 17: 2-Button Grand AveragesFigure 18: 2-Button Novelty Grand AveragesFigure 19: First GUI Grand AveragesFigure 20: Second GUI Grand Averages


Peak-Picking Results

% Positive identified correctly= = True Positive / (True Positive + False Negative)

And% Negative identified correctly=

=True Negative / (True Negative + False Positive)


Discussion

• Objective Results were not statistically significant enough to prove difference in GUIs

• Subjects’ comments– 1-button application was harder to use than the 2-button one and

caused eye strain for 4 out of 5 subjects who felt a difference– For the 2-button GUI, once the participant knew which button was

“yes” and which button was “no,” the distinction could be made at the slightest flash of the stimulus detected out of the corner of the eye

– The 2-button GUI with the color change in between, was, according to the participants, almost the same as the regular 2-button GUI

• For some seemed to boost their attention (2 out of 5)• For others distracting. (2 out of 5)


Problems Encountered

• Acquisition– With USB 1.0 connection, the system could not have

access to streaming data, making it necessary to get the data 1 second at a time.

• Each data request sent to the acquisition board resulted in a minimal delay.

– The computer system that was used had the Windows 2000 operating system on it

• Windows is not designed to be a real-time processing system and therefore, the absence of OS-generated delays could not be guaranteed.



• Signal-processing– The signal detection method depended on the P3

potentials being in the right place at the right time.

– The method can fail in cases where for certain reasons the P3 potential is delayed.

• Could have occurred toward the end of each experiment as the participant was getting fatigued

• Also with certain types of GUIs, such as the 1-Button GUI, which required the participants to read the stimulus text before making a decision



• User application– Timing problems caused by delays that can be

expected to be present on a Windows 2000 system

– The flash of the stimulus often occurred later than the time that the command to flash was added to the system event queue.

– The delay was variable and inconsistent, offsetting the detection algorithm.


Peak-Aligning

• An attempt to correct for the variable delay– Align peaks using the

Visual Evoked Potential (VEP)

– VEP occurs at around 100 ms after the stimulus


Peak-Aligning Results


Overall Grand Average


GUI Averages by Type


GUI Averages by Order



• Other Issues– No easy way to tell the cause of an incorrect

result• Algorithm Error• User Inattention

– Visually analyzed epochs to improve algorithm– Custom fitted offline algorithm for each

subject to avoid mistakes due to individual differences


Future Work• Acquisition

– Can be made to allow easy pluggability of modules that interface with different types of hardware

– Use a photocell to avoid timing errors

• Signal Processing– A signal-processing algorithms should be created that can

achieve a decision with as few stimulus flashes as possible– The module should be enhanced to allow the user to

calibrate it to his/her individual brain waves and P3 potentials, making the module learn from past data and modify its detection routines accordingly


Future Work (continued)

• User Application– Better and more efficient User Applications

should be developed– Displaying the choices to minimize the complexity

of the task– Turning BCI On/Off

– Use a graphics package instead of windows forms to get better control of when stimuli flash



– Run more experiments with larger numbers of subjects

– Other variable differences should be tested for• Size of text and stimuli boxes

• Effect of one color vs. another

• Effect of flashing text vs. background

– Determine what is the minimum number of flashes needed to achieve a recognizable P3



– Applications• We are currently working with the couple that

contacted us, to create a viable BCI for a patient with advanced ALS. We are also currently trying to make contact with other organizations that might be able to put us in touch with patients who would agree to participate in our experiments.


Conclusion

• We have created a robust, modular, BCI platform• We have tested it by running a panel of

experiments, using an application that was developed based on this platform.

• We have also set up a useable EEG/BCI Lab in the AI Lab, which, in the future, can be used both for continued development of BCI applications, and for other EEG/EMG research at RIT


Contact: Aleksey [email protected]

Вопросы??שאלות

Fragen?Preguntas?

Domande?

Questions?

Date post:	11-Jan-2016
Category:	Documents
Upload:	peregrine-patterson
View:	212 times
Download:	0 times

Department of Computer Science Development and Application of an EEG-based Brain- Computer Interface...

Documents