doc.: IEEE 802.15-19-0421-02-0dep

Submission

Project: IEEE P802.15 Working Group for Wireless Personal Area Networks (WPANs)Submission Title: [WNG Brain-Machine Interface based on Electrocorticography using high speed UWB wireless body area network]Date Submitted: [18 July 2019]Source: [Takafumi Suzuki1, Masayuki Hirata2] [1; Center for Information and Neural Networks(CiNet), National Institute of Information and Communication Technology(NICT), 2; Dept. of Neurological Diagnosis and Restoration, Graduate School of Medicine, Osaka University] Address [1; 1-4 Yamadaoka, Suita, Osaka, 565-0871

2; 2-2 Yamadaoka, Suita, Osaka, 565-0871]Voice:[1; +81-80-9098-3264, 2;+81-6-6210-8429], FAX: [+81-6-6210-8430], Email:[1; t.suzuki@nict.go.jp, 2;mhirata@ndr.med.osaka-u.ac.jp] Re: []Abstract: [A important use case of dependable body area network(WBAN) for implanted devices is introduced to perform reliable and massive data for ECoG-based Brain machine interface to require amendment for IEEE802.15.6 wireless medical body area network.]Purpose: [information]Notice: This document has been prepared to assist the IEEE P802.15. It is offered as a basis for discussion and is not binding on the contributing individual(s) or organization(s). The material in this document is subject to change in form and content after further study. The contributor(s) reserve(s) the right to add, amend or withdraw material contained herein.Release: The contributor acknowledges and accepts that this contribution becomes the property of IEEE and may be made publicly available by P802.15.

Slide 1 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Brain-Machine Interface based on Electrocorticography using high speed

UWB wireless body area network

Slide 2

September 2019

Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

Takafumi SuzukiCenter for Information and Neural Networks, NICT

Masayuki HirataDepartment of Neurological Diagnosis and Restoration,

Osaka University Graduate School of Medicine

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 3 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

Outline■ECoG-BMI system

■1st Genaration 128ch system： Clinical ECoG-BMI system→ Clinical test in 2020

■2nd Generation 4096ch system → Next generation system・Flexible electrode technology ・UWB wireless technology

■BMI (system evaluation)・Real-time decoding・Robotic arm control and cortical adaptation flexible ECoG electrode-mesh 8:1 Multiplxer and logic

15 mm

20.7 mm

14 m

m 19 mm50μmPt Black

UWB-T(FPGA)

Controller

ZigBee

Qi

Battery

UWB transmitter andOperation control block

Wireless power supply

Outside body

128 Mbps(wireless)

UWB receiver andPC connection block

6.4Mbps(8 x 800kbps)

x81~64ch

512ch base unit No.8

Recording

No.165~128ch

Recording

No.1

3585~4096ch

Recording

No.64

MUX-A

(FPGA)

1~64ch

512ch base unit No.1

Recording

No.165~128ch

Recording

No.1

449~512ch

Recording

No.8

MUX-A

(FPGA)800kbps

MUX-B

(FPGA)

Multi-channel recording and digital multiplexing block

51.2 Mbps(64 x 800kbps)

UWB-R(FPGA)

LED(time)

ZigBee

USBPCZigBee

Inside body

4.9 mm2

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 4 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

grasp open

Real time control of external devices just by thinkingA promising therapy for severely disabled patients

communication robot control

ALS, spinal cord injury, amputated limb, stroke, ……..

Thank you !

Robot arm

BMI Project (Osaka U & NICT)

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 5 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

ECoG： ElectrocorticogramGood balance (Information rate, invasiveness, long-term stability)

④Spike or LFP

×High Invasive◎Info:high×Short-term

② EEG

◎No invasive△Info : low（On/Off（1bit）

selection slowly）○Long-term

③ECoG(Electrocorticogram)

△Low invasive○Info: middle（Robotic hand

control in RT）○Long-term

①fMRI, MEG

◎No invasive◎Info: high×Huge system

Recording methods for clinical BMI

EEG

ECoG

x10info

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 6 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

ECoG signalWaldart, et al, J.Physiology, 2009

SUA:  Single Unit ActivityMUA: Multi Unit Activity

ECoG(50～180Hz)sync with motion= informative

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 7 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

Clinical Research (using wired ECoG-BMI system)

Grasp（Yanagisawa, Ann Neurol)

ALS patient (1st Clinical Research)（NHK news）

(by Osaka Univ.)

Implantable system is necessary for daily life support by BMI system ⇒ “Wireless” is the key

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 8 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

1st generation ECoG BMIsystem

- 128ch (# of electrodes)- ISM band (2.4GHz)- Clinical test in 2020- Communication

device for ALS- Robot arm control

for Paralysis

doc.: IEEE 802.15-19-0421-02-0dep

Submission

A fully implantable wireless BMI system

Prototype 64/128ch systemsix months in a monkey (2013)

4.9m

m

4.9mm

32ch

LN

A

32ch

LN

AADC x2

IF

64ch ECoG Electrode

titanium head casing

Wi-Fi transmitter andwireless power supply unit (epoxy embedded)

wireless data / power control board

coil

rechargeable Li polymer battery

64ch neural recording LSI

Pre-clinical test in 2017-⇒ Clinical test in 2020

September 2019

Slide 9 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

1st Generation 128ch system (Improvement for clinical use)■Abdomen unit

→ Integrated into head unitSmaller system can decrease various risks.

Current version: Casing, non-touch energy supply

・LSI improvement → Lower noise, safety, etc.

TSMC CMOS0.25μm（7.1mm×7.3mm）

September 2019

Slide 10Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

Prototype 64/128ch systemsix months in a monkey (2013)

4.9m

m

4.9mm

32ch

LN

A

32ch

LN

AADC x2

IF

64ch ECoG Electrode

titanium head casing

Wi-Fi transmitter andwireless power supply unit (epoxy embedded)

wireless data / power control board

coil

rechargeable Li polymer battery

64ch neural recording LSI

doc.: IEEE 802.15-19-0421-02-0dep

Submission

1st Generation 128ch system (Improvement for clinical use)

TSMC CMOS 0.25μm (7.1mm×7.3mm)-32ch x 4chips-Noise (input) 3μVpp-Capable of High-γ

band recording

Inside

Receiver Coil forenergy supply

Surface electrodearray withbio-compatiblesilicone

Wireless transmitter (2.4 GHz ISM Band) ～1.9Mbps - GLP test (bio-compatiblilty )

- Implant test (animal)

September 2019

Slide 11Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission Slide 12 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

2nd generation ECoG BMIsystem

- 4096ch～ (# of electrodes)- UWB band (7.9GHz)- Clinical test in 2030?- Robot arm control

for Paralysiswith individual finger control

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Next generation multi-channel BMI system more than 4,000 channels

Issue of multi-channel system: volume of dataex. ECoG, 1kS/s, 12bit-ADC⇒ ~1Mbps@100ch, ~100Mbps@10,000ch

Our targetimplantable, distributed, and wireless

For more accurate estimation of movement intentions a large number of recording channels recording at several regions simultaneously

September 2019

Slide 13 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Improvement for next generation (128ch-> 4096ch) Issues to be solved(1): High density electrode array

Silicone + Pt array Parylene-C + Pt (or Au)

(2): LSI (amplifier + ADC)32ch x 4 chips 64ch x 64chips

(3): Wireless transmittingISM (1.9Mbps) UWB(128Mbps)

September 2019

Slide 14Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

2nd Generation(4096ch ECoG-BMI system)

Flexible electrode technology

September 2019

Slide 15Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Electrode Array

3D-shapeIndividutl MRI Data1st Generation

Up: High Density(IED: 2.5mm)

Down:Clinical

(Safety, High density, Stability)

(Hirata M, IEICE Trans Commun, 2011)

3D-double surfacefor intra-sulcus

Flexible electrode array・Parylene-C、 High Density: IED

50μm・Relationship between intracortical

and ECOG signal

2nd Generation

September 2019

Slide 16Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Flexible Electrode array for ECoG

・50μm x 50μm・Platinum black

1mm

Toda, Neuroimage(2011)

GoldParylene-CSilicon Wafer

Aluminum Mask

Oxygen plasmaetching

Platinum black

Parylene-C

(a)

(b)

(c)

(d)Monkey128ch

- > 2.5 years- into sulcus

(Hasegawa Lab, Niigata Univ.)

September 2019

Slide 17 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

2nd Generation(4096ch ECoG-BMI system)

Wireless technology⇒UWB(Ultra-Wide Band)

September 2019

Slide 18Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Super multi-channel system using UWB(4096ch～)

Electrode array+ LSI(64ch x 64units)

BMI- Robotic hand control- Feedback

Large-scaleRecording

Real-timeDecoding

Wireless transmitting

(UWB)

1st Generation (128ch)-ISM band (2.4GHz) [1.9Mbps]12bit x 1kHz x 128ch = 1.5 Mbps

2nd Generation (>4000ch)12bit x 1kHz x 4096ch = 49Mbps UWB(Ultra Wide Band)

+ Distributed system

September 2019

Slide 19Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

UWB system (4096ch) -Prototype-

waterproof package

UWB transmitter

UWB receiver

flexible ECoG electrode-meshNeural recording LSI board(64ch-LNA, MUX, 12b-ADC)

8:1 Multiplexer and logic

15 mm

20.7 mm

14 m

m

19 mm

UWB-T(FPGA)

Controller

ZigBee

Qi

Battery

UWB transmitter andOperation control block

Wireless power supply

Outside body

128 Mbps(wireless)

UWB receiver andPC connection block

UWB@7.9 GHz ±0.625 GHz

6.4Mbps(8 x 800kbps)

x8

1~64ch

512ch base unit No.8

RecordingNo.1

65~128ch Recording

No.1

3585~4096ch

RecordingNo.64

MUX-A(FPGA)

1~64ch

512ch base unit No.1

RecordingNo.1

65~128ch Recording

No.1

449~512ch

RecordingNo.8

MUX-A(FPGA)

800kbps

MUX-B(FPGA)

Multi-channel recording and digital multiplexing block

51.2 Mbps(64 x 800kbps)

UWB-R(FPGA)

LED(time)

ZigBee

USB

PCZigBee

Inside body

4.9 mm2

128 Mbps(wireless)

September 2019

Slide 20 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

UWB transmitter and receiver

RF board

back view

front view BPF SMT for Antenna

base board

batteryconnectorAC/DCconnector

Magnetic powerswitch connector

Zigbee board

antenna

rechargeable battery(430mAh, 3.7V)

back view

front view Zigbee boardsampling timeLED [ms]

base boardRF board

debug port

~100mm

USB port

Including UWB transmitter, BPF, antenna, Zigbee, Li-ion polymer battery

-Including UWB receiver, antenna, Zigbee unit-Connected to PC by USB2.0-Real-time Graphical view

helical antenna

September 2019

Slide 21 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

UWB system (Specification)Number of channels 64~4096 ch

Supply voltage 3.3 V

Total power of implant devices 2.03 W @4096ch (version 2014)

UWB bandwidth 7.275~8.525 GHz for internationally usage

UWB data rate 128 Mbps

MUX-A and MUX-B (common hardware)

Xilinx Spartan6 XC6SLX16, FPC connector (11pin)

UWB transmitter (water proof casing)

RF, BPF and ZigBee board

LVDS receiver (51.2Mbps), Magnetic Power SW (20mm range)and rechargeable Li polymer battery (400mAh)

UWB receiver

RF board, base board (Xilinx vertex4, USB2.0) and Zigbee board

PC

Core i7 3820 Win7 (USB2.0, GUI application)

September 2019

Slide 22 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Evaluation of UWB in human equivalent liquid phantom

Measured characteristics of received signal strength.

(a) Diagram and (b) photograph of the measurement setup.

Packet error rate with respect to the phantom depth.

OK

NG

too near

too far

128 Mbps UWB wireless communication is available below 20 mm between inside to outside body.

20 dB dropin body

September 2019

Slide 23 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)

doc.: IEEE 802.15-19-0421-02-0dep

Submission

Summary

Slide 24 Takafumi Suzuki(NICT), Masayuki Hirata(Osaka University)

September 2019

Our collaborative project 1st and 2nd generation ofECoG BMI are introduced.

Medical applications will needhigh speed (100Mbps～) and securewireless communication.

doc.: IEEE 802.15-19-0421-02-0dep

Submission

September 2019

Thank you for your attention

Slide 25 Takafumi Suzuki(NICT), Masayuki Hirata(University of Osaka)