Brain Interface DesignBrain Interface Designfor Asynchronous Controlfor Asynchronous Control

ASIMO made by HONDAASIMO made by HONDA

What is Neil Squire SocietyWhat is Neil Squire Society

• The long-term objective of the brain interface project is to create a multi-position, brain-controlled switch that is activated by brain signals measured directly from the scalp of an individual.

• They believe that such a switch will allow an individual with a severe disability to have effective control of devices such as assistive appliances, computers

Project Background

The Neil Squire Society is the organization in Canada that for the past 20 years has used technology, knowledge and passion to empower Canadians with physical disabilities

Introduction(i)Introduction(i)

• Within the context of general brain interface (BI) technology research, The Nail Squire Brain lab has focused on BI system design specifically for self-paced design specifically for self-paced or asynchronous control environmentsor asynchronous control environments.

• This asynchronous mode of device is more naturalmore natural than the more commonly studied synchronized control mode whereby the system dictates the control of the user

Introduction(ii)Introduction(ii)

• Schemes of Presentation

– Presentation about an overview of asynchronous control.

– Summary of Neil Squire Society`s effort to develop asynchronous BI system.

– Discussion of several major issues that have arisen from asynchronous BI development.

Which feature is used in BCI systems?Which feature is used in BCI systems?

• In typical BCI based on EEG, the operator generates a “control signal” by consciously changing his cognitive state when he wants to control device.

• The change in cognitive state is measured as specific temporal patterns,temporal patterns, andand signal power level signal power level in the operator`s EEG activity

Temporal patterns

signal power

time

AsynchronousAsynchronousVs.Vs.

Synchronous Synchronous

Synchronous Vs. AsynchronousSynchronous Vs. Asynchronous

What is the Synchronous mode?What is the Synchronous mode?

• Synchronous mode

– In this system, the system initiates the period of control, not the user, and user is expected to be consciously controlling the interface during the control periods likely fig.

What is the Synchronous mode?What is the Synchronous mode?

• Drawbacks of the synchronous

– BI transducer will make a control signal regardless of whether

the person is actually intending control.

– User cannot make control signal until system polling period to

occur in order to engage BI transducer.

What is the Asynchronous mode?What is the Asynchronous mode?

• Asynchronous mode

– In this system, which we will call asynchronous control applications, are characterized by alternating periods of attentive alternating periods of attentive idleness and active control idleness and active control as illustrated in Fig.

– In this system, there are applications that require constant users require constant users attention and irregularattention and irregular, user-initiated control. This system is usually not communications applications but control applications.

Asynchronous Control (i)Asynchronous Control (i)• Asynchronous control refer to the type of control where output

signals are changed or commands are issued only when control is when control is intended.intended.

• During the no control(NC) state, one would expect the system output to remain neutral or unchanged.remain neutral or unchanged.

EEG SignalEEG Signal

Intentional controlIntentional control

No ControlNo Control

When user is intended

When user is talking, daydreaming, and thinking

Asynchronous Control (ii)Asynchronous Control (ii)

• Examples of asynchronous control– turning on lights, changing television channels, and interacting with a computer.– When you remove your hand from your computer mouse, you enter an NC state and

the mouse output remains stable and unchanged-that is, the mouse pointer does not continue to move on the computer screen. In other word, The mouse is then available for control simply by replacing your hand.

• In short, asynchronous control allows the user to define when things asynchronous control allows the user to define when things happen.happen.

• According to these points, asynchronous control is more characteristic of most real-world control applications than synchronous control.

Asynchronous Control (iii)Asynchronous Control (iii)

• System idling : The neutral or unchanging system output response desired during periods of NC

– For effective system, we should use the idling system to be realized.

– This system is analogous to car engine.

– Using FP(false positive) rate measures how well BI transducers idle for more complete

measure of asynchronous control.

Classification of EEG DeviceClassification of EEG Device

• Idle support indicates if the interface device will support idling

• Available define when the interface device allows user control

Classification of EEG DeviceClassification of EEG Device

• Idle support : If control paradigms not support the idling, the system produce an unintended action if and when the user enters the NC state. (Midas touch problem)

Control ParadigmControl Paradigm

AvailabilityAvailability

Idle supportIdle support

No idle support Idle support

Periodically Synchronous System-paced

Continuously Constantly engaged Asynchronous

• Constantly engaged mode is impractical operation where the user is continuously controlling the interface without a break and any NC activity will cause an error.

EEG-BasedEEG-BasedAsynchronous Brain-SwitchesAsynchronous Brain-Switches

Obtaining the EEG signal related movementObtaining the EEG signal related movement

• Obtain the EEG signals when spinal cord injury (SCI) people and able-bodied people move their finger.

– The only difference between the people with (SCI) and those without are the actual physical movements that may or may not occur during their attempted finger movement.

• 10-20 System of Electrode Placement

• The international 10-20 system of electrode

placement is the most widely used method to

describe the placement of electrodes at specific

intervals along the head.

• Even numbers refer to the right hemisphere and

odd numbers refer to the left hemisphere

Electrode placement for LF-ASD

Region of the motor cortex

Electrode placement for Experiments Electrode placement for Experiments

Motor cortex Motor cortex

: The anatomical region of the brain

known as Area 4 was given the name

primary motor cortex (symbol: M1) after

Penfield showed that focal stimulations in

this region elicited highly localized muscle

contractions at various locations in the

body.

Electrode placement for Experiments Electrode placement for Experiments

Electrode placement for LF-ASD

They chose to limit the features to the top

six primarily because these features were

the minimal set that provided uniform provided uniform

coverage of the motor areas (SMA, MI) coverage of the motor areas (SMA, MI)

of the cortexof the cortex.

The strongest discriminatory features

were found in autocorrelations within six

electrode pairs F1-FC1, Fz-FCz, F2-FC2, F1-FC1, Fz-FCz, F2-FC2,

FC1-C1, FCz-Cz, and FC2-C2 FC1-C1, FCz-Cz, and FC2-C2 on the 10-

20 system for electrode placement.

Region of the motor cortex

Asynchronous Brain-SwitchesAsynchronous Brain-Switches

• For asynchronous control application

* Idle supportIdle support

* Low FP rates* Low FP ratesSpecific signal Specific signal

processing algorithmprocessing algorithm

Classify NC state and IC state Classify NC state and IC state

• Problems in State Classification.

i) This needs an indication of intent in order to process the data.

ii) It is not case for synchronous applications because user intent is

assumed during the control periods.

Classify NC state and IC state Classify NC state and IC state

• Method 1.– Subjects to self-report intent during the data recording

DisadvantageDisadvantage

i) Self-report complicates the signal analysis because one does not know exactly when the movement was made.

ii) It is hard to provide the user with any form of feedback during

these sessions.

Classify NC state and IC state Classify NC state and IC state

• What is the Outlier Processing Method?

- The outlier processing method (OPM) outlier processing method (OPM) is the only BCI technique that has been designed specifically to differentiate idle from active EEG in an asynchronous control applications

- They use the outlier processing method (OPM) outlier processing method (OPM) for extracting single-trial voluntary movement-related potentials(VMRPs) from EEG related to finger movement.

Classify NC state and IC state Classify NC state and IC state

• Method 2.– Separate the VMRP state from NC state via OPM in the obtained signal.

• They find the feature that VMRPs has higher relative power than NC state in VMRPs has higher relative power than NC state in

1-4Hz bandwidth1-4Hz bandwidth via time-frequency analysis of EEG pattern.

• Using this feature, they has developed the low-frequency asynchronous low-frequency asynchronous

switch design(LF-ASD)switch design(LF-ASD)

Feature SpaceFeature Spacet

power

Classify NC state and IC stateClassify NC state and IC state

• The BI experiments results by SCI people and able-bodied people SCI people and able-bodied people are sameare same. In other words, results by actual physical movements and imagery of movements are same.

• LF-ASD demonstrated TP rates of 30-78 percent during IC states in combination with low FP rates of 0.5-2 percent during NC. The

result is independent subjects.

• When the FP rate is high, the LF-ASD system is frustrating to use. Thus, they intentionally operated the system in this case.

Thank You.

Feature Extraction Methodology• First, signals prefiltered between 1-4Hz and the compound feature described by (1)