Soutenance Lorraine Perronnet preview - InriaSoutenance Lorraine Perronnet preview Created Date:...

PhDdefense ofLorrainePerronnetSeptember 7th,2017

1

Directors:AnatoleLécuyer ChristianBarillot

Advisors:MaureenClercandFabienLotte

Combining EEG fMRIforNeurofeedback

Neurofeedback (NF)

2

Introduction>

Motor rehabilitation ofstrokepatients

Definition: “Neurofeedback is a type of biofeedback in which neural activity is measured,and a visual, an auditory or another representation of this activity is presented to theparticipant in real time to facilitate self-regulation of the putative neural substrates thatunderlie a specific behaviour or pathology” [Sitaram et al. 2016]

Neurofeedback (NF)

3

AcquisitionBrainactivity ismonitored

Pre-processingSignalis cleanedfrom nonneuronal

components

Feature extractionAfeature ofinterest

is extracted

FeedbacktranslationThefeature is fed backtothesubject viaavisual,

auditory ortactilefeedback

Subject self-regulationSubject perceives feedback

andadapt his mentalstrategy tocontrolit

REAL-TIMECLOSEDLOOP

Introduction>

NFmodalities

4

EEG

MEGfNIRS

fMRI

Temporalresolution (s)

Spatialresolution(m

m)

111111

0.01 0.1 1 10

10

1EEG+fMRI

Highspatial(mm)andhightemporal(ms)resolution

Introduction>

Problem andmotivation

Limitedefficiency/efficacy ofunimodal NFapproaches

Designnovel NFapproaches combining EEGandfMRI that couldbe moreeffectivethan unimodal approaches

5

Introduction>

5)Feedback

Challengesofcombined EEG/fMRI forNF

6

EEG

fMRI

1)Neurovascularcoupling

6)Real-timeprocessing

4)Dataintegration

2)Experimentaldesign

3)fMRI featureselection

3)EEGfeatureselection

Introduction>

Thesis objectives

1. Identify critical methodological aspectsthat differ between EEG-nfandfMRI-nf (Related works>EEG-nf vsfMRI-nf)

2. ExplorehowtocombineEEG andfMRI forNF(Related works>Contribution1)

3. Develop anexperimental EEG/fMRI NFplatform (Contribution2)4. Evaluate added valueofbimodalEEG-fMRI-nf overunimodal NF

(Contribution3)5. Proposeandevaluate strategies torepresent EEG andfMRI

simultaneously (Contribution4)

7

Introduction>

Outline

• Related works• Contribution1(methodo.):Taxonomy ofEEG/fMRI NFstudies

• Contribution2(techno.):EEG/fMRI NFplatform• Contribution3(study):Unimodal vsbimodalNF• Contribution4(methodo.+study):Towards integrated feedback• Conclusion• Perspectives

8

EEG-nf vsfMRI-nf

EEG-nf fMRI-nf

NFsignal • Amplitudeofspecificfrequencybandsatone,twoelectrodesites

• Slowcorticalpotentials[Rockstroh etal.1990]

• Z-scoreNF[Thatcher etal.,1998]• Source-based(Loreta-NF,BSS-NF)

[Cannonet al.2009,Whiteetal.2014]

• AveragepercentsignalchangeinROI• Differentialsignalbetweentworegions• MVPA, Effectiveconnectivity[Sulzer etal.,2013]

Task design Block,continuous/self-paced BlockTask duration Flexible: usually2-5minutes,fewseconds

forMI, tensofminutesfordeepstateNF15- 45seconds

Nbofsessions 20- 40 5- 10

9

Related works >

Cross-modalevaluation /validation

• fMRI before/after EEG-nf• Plasticity induced byasinglealphadownEEG-NFsession[Rosetal.,2012]• After 30minutesofNF,increase ofconnectivity within

regions ofthesalience networkinvolved inintrinsicalertness (dACC)

• Passive fMRI during EEG-nf• fMRI signatureofMI-based EEG-nf [Zich etal.,2015]

• EEGandBOLDcontralateral activity is correlated• EEGandBOLDlateralization patternsarenotalways correlated

• Passive EEG during fMRI-nf• Correlation between amygdala BOLDactivity and

frontalEEGasymmetry during fMRI-nf inMDDpatients [Zotev etal.,2016]• Average frontalalphaasymmetry changessignificantly

correlated with theamygdala BOLDlaterality

11

RealBCIilliterates

PseudoBCIilliterates

Related works >

fMRI-informed EEG-nfEEG finger-print (EFP){electrode,frequency}offMRI deep regional activation[Meir-Hasson etal.,2014],[Linetal.2017]:time-frequencydecomposition ofEEG,ridge regression

13

CommonEFPmodel(valid across subjects andsessions)[Meir-Hasson etal.,2016]:oneclassclassification,hierarchicalclustering algorithm applied totheestimated EFPmodels’coefficients

Related works >

EEG-fMRI-nf[Zotev etal.,2013]

• Methods• Participants:6healthy subjects• Task:emotional self-regulation• EEG feature:frontalhigh-beta(21-30Hz)asymmetry• fMRI feature:left amygdala

• Authors hypothesized that:EEG-fMRI-nf >EEG-nf |fMRI-nf• Limitations

• 2separate feedbackgauges• Noevaluation against unimodal NF

15

Related works >

Outline



17

Systemdescription(1)

• Goal• Develop aplatform abletodosimultaneous acquisitionandreal-timeprocessing ofEEG andfMRI toprovide unimodal andbimodalNF

• Challenges• Multimodal• Real-timeperformance• Artifacts (gradient,pulse,helium pump,ventilation)• Novel approach,nocomprehensive solutionavailable

18

Contribution2:EEG/fMRI NFplatform >

Systemdescription(2)

19

NFBUnit(InhouseMatlab/C/C++/Javacode)

• Real-timeparallel EEG andfMRI processing(pre-processing,feature extraction)

• NFcalculation• Communicationwith subject (feedback,

cues andinstructions)• Experiment/protocol control• Timingcontrolandsynchronization

EEGsubsystem

fMRI subsystem

64ch.MR-safe BrainProducts EEGcap

3TSiemensVerio

Fiber delay (~80ms)+displayrefresh (1-17ms)

EEGupdate<200ms

fMRI update≤250ms

Subject


Published in:MMano,ALécuyer,EBannier, LPerronnet,SNoorzadeh,CBarillot (2017). HowtobuildahybridneurofeedbackplatformcombiningEEGandfMRI. FrontiersinNeuroscience,11, 140.

My role

• State-of-the-artandspecifications• Issuedetection andresolution• Recruiting volunteers• Runningtheexperiments andanalyzing thedata

20


Outline



21

• Goal: evaluate theadded valueofEEG-fMRI-nf compared tounimodal EEG-nf andfMRI-nf• Participants:10healthy subjects(28+/- 5.7y,2females)• Design:within-subject• Collected data:EEG +fMRI• Task:kinesthetic motor-imagery (kMI)oftherighthandunder unimodal/bimodalNFconditions• Evaluationcriteria:

• EEG andfMRI activationlevels• fMRI activationmaps• Questionnaires

Goalandmethods

22

Contribution3:Unimodal vsbimodalNF>

Experimental protocol

23

1session

MotorlocalizerRight-handclenching

NF1Right-handkMI

NF2Right-handkMI

NF3Right-handkMI

ConditionsA,BandCarepseudo-randomly ordered foreach subject

Rest Task(10×)

MI_preRight-handkMI

MI_postRight-handkMI

fMRI ROI

20s 20s


Features

24

EEG feature:

• Electrodes:C1andC2• Frequency band:µ(8-12Hz)• Baseline:from previous rest block• NFrate:8Hz

fMRI feature:

• ROI:9×9×3boxoverleft andrightM1[Chiew etal.,2012]

• Baseline:from previous rest block• NFrate:0.5Hz(=TR)

Features:laterality indicesbetween left andrightmotor area

M1


Experimental conditions

25

+

ConditionA:EEG-NFEEG

fMRI

+

ConditionB:fMRI-NFEEG

fMRI

+

ConditionC:EEG-fMRI-NFEEG

fMRI


Unimodal Bimodal

Hypotheses

HypothesesH1:Generalized NFeffectH2:DirectNFeffectH3:Compromiseeffect

26

˃˃

Level ofNF-related fMRI activityLevel ofNF-related EEGactivity

≤≤ ≤≤(H3)

(H3)

(H2)(H2)

(H2)

(H2)

˃˃

˃˃

˃˃

>>0 (H1)

>>0(H1) >>0(H1)

>>0 (H1)


>>0(H1)>>0(H1)

Demo

2727


Unimodal Bimodal

Results >BOLDactivationmaps

28

• Stronger,bigger andmorewidespread activationsduring EEG-fMRI-NF=>higher level ofengagementorhigher level ofself-regulation ?

fMRI-NF EEG-fMRI-NFEEG-NF

28


Unimodal Bimodal

(TASK>REST;p>0.05FWEcorrected;k>10voxels)

Results >NFperformance(online)

29

• EEG laterality significant inNF2,fMRI laterality significant inNF1• Highinterandintrasubject variability• Loss ofperformanceononlinefMRI laterality between NF1andNF3• Nosignificant difference between NFconditions• Laterality indicesmight havebeentoo hardtoregulate inonesession


EEG laterality (byNFcond.) EEG laterality (byrun order) fMRI laterality (byrun order)fMRI laterality (byNFcond.)

A:EEG-nfB:fMRI-nfC:EEG-fMRI-nf

Results >NFperformance(posthoc)

30

• EEG andBOLD activity significantly higher during NFthan during MI_pre =>H1• BOLD activity significantly higher during EEG-fMRI-nf than during EEG-nf =>H2• Nosignificant difference onEEG between NFconditions


EEG ERDonCSPfiltereddata(byNFcond.)

EEG ERDonCSPfiltereddata(byrun order)

fMRI PSCinposthoc ROI(byrun order)

fMRI PSCinposthoc ROI(byNFcond.)

A:EEG-nfB:fMRI-nfC:EEG-fMRI-nf

Results >Questionnaire

31

fMRI was easier tocontrolthan EEG 6/10EEG was easier tocontrolthan fMRI 3/10EEG andfMRI were asdifficult tocontrol 1/10Same attentiongiven toboth feedbackdimensions 8/10Moreattentiongiven tothe dimensionthat washardertocontrol

2/10

During EEG-fMRI-NF:


+

EEG

fMRI

Discussion

• Need further studies toreinforce our results andevaluate therest ofthehypotheses• Oppositetendency ofonlineEEG andfMRI features• Onemodality can be regulated attheexpense oftheother

32


Summary

• We conducted astudy that compared forthefirsttimeEEG-fMRI-nf toEEG-nf andfMRI-nf• Mainresults• Participantsareabletoregulate hemodynamic andelectrophysiologicalactivity simultaneously during unimodal andbimodalMI-based NF

• BOLDactivity higher during EEG-fMRI-nf than during EEG-nf

33


Published in:LPerronnet,ALécuyer,MMano,FLotte,MClerc,CBarillot (2017).UnimodalversusbimodalEEG-fMRIneurofeedbackofamotorimagerytask. FrontiersinHumanNeuroscience.

Outline

• Related works• Contribution1(methodo):Taxonomy ofEEG/fMRI NFstudies

• Contribution2(techno):EEG/fMRI NFplatform• Contribution3(study):Unimodal vsbimodalNF• Contribution4(methodo +study):Towards integrated feedback• Conclusion• Perspectives

34

FeedbackdesignforEEG-fMRI-nf

35

• InEEG-fMRI-nf,greater amount ofinformationwith nontrivialrelationship=>Howtorepresent theEEGandfMRI features simultaneously?

• Problem ofseparate feedbacks• 2feedbacks,2targets ~2concurrentregulation tasks• Highcognitiveload• Does notallow todefine aNFtarget characterized bythepairoffeatures

• Concept:we proposetointegrate theEEGandfMRI features inasinglefeedback

Contribution4:Towards integrated feedback>

[Zotev etal.,2013]

ToappearUnderreview

36

Contribution4:Towards integrated feedback>

Outline



47

Conclusion• Goal:designnovel NFapproaches combining EEGandfMRI

• Contribution1(methodo.):Taxonomy ofEEG/fMRI NFstudies• Thetaxonomy showsthere aremany ways ofcombining EEGandfMRI forNFpurpose• We havefocused onEEG-fMRI-nf:simultaneous onlineuseofEEGandfMRI asNFsignal• Thereis still roomleft forimprovements andforthedevelopment ofnewapproaches

• Contribution2(techno.):EEG/fMRI NFplatform• We havedeveloped anefficientplatform that allowed ustotestandevaluate methods forbimodalNF• Itwill continuetobe improved andused forexperiments

• Contribution3(study):Unimodal vsbimodalNF• We havedemonstrated that during anMItask bimodalEEG-fMRI-nf triggersstronger BOLDactivationsthan unimodal EEG-nf

• Contribution4(methodo.+study):Towards integrated feedback• We haveintroduced theconceptofintegrated feedbackforEEG-fMRI-nf (onefeedback/onetarget)• We haveproposed two integrated feedbackstrategies,a2Danda1D• The1Dfeedbackis easier tocontrolonasinglesession• The2DfeedbacktriggersmoreactivationintherightSPLandencouragessubjects toexplorementalstrategies

48

Conclusion

Perspectives

• Experimental design• Mixedprotocols• Investigate other modality couples(EEG+fNIRS ?)

• Feedback• Investigate other integrated feedbackparadigms• Multi-sensory bimodalfeedback

• Applications• Upcoming clinical tests(depression,stroke)

49

Perspectives

Publications• Journal

• L Perronnet, A Lécuyer, M Mano, F Lotte, M Clerc, C Barillot (2017). Learning 2-in-1: towards integrated EEG-fMRI-NF. [Review in progress].• L Perronnet, A Lécuyer, M Mano, F Lotte, M Clerc, C Barillot (2017). Unimodal versus bimodal EEG-fMRI neurofeedback of a motor imagery

task. Frontiers in Human Neuroscience.• L Perronnet, A Lécuyer, F Lotte, M Clerc, C Barillot (2016). Entraîner son cerveau avec le neurofeedback / Brain training with neurofeedback. Les

interfaces cerveau-ordinateur 1 : Fondements et méthods / Brain-Computer Interfaces 1: Foundations and Methods. pp. 277-292, (Wiley-ISTE).• M Mano, A Lécuyer, E Bannier, L Perronnet, S Noorzadeh, C Barillot (2017). How to build a hybrid neurofeedback platform combining EEG and

fMRI. Frontiers in Neuroscience, 11, 140.

• Conferences• L Perronnet, A Lécuyer, F Lotte, M Clerc, C Barillot. Neurofeedback unimodal ou bimodal ? Intérêt de l’EEG et de l’IRMf. 2ème journée nationale

sur le neurofeedback, ESPCI Paris, France, January 2017. [Invited talk]• L Perronnet, A Lécuyer, M Mano, E Bannier, F Lotte, M Clerc, C Barillot. EEG-fMRI neurofeedback of a motor imagery task. 22nd Annual Meeting

of the Organization for Human Brain Mapping (OHBM 2016), Palexpo, Geneva, Switzerland, June 2016. [Poster]• M Mano, E Bannier, L Perronnet, A Lécuyer, C Barillot. Design of an Experimental Platform for Hybrid EEG-fMRI Neurofeedback Studies. 22nd

Annual Meeting of the Organization for Human Brain Mapping (OHBM 2016), Geneva Palexpo, Switzerland, June 2016. [Poster]• L Perronnet, Anatole Lécuyer, Marsel Mano, Elise Bannier, Fabien Lotte, Maureen Clerc, & Christian Barillot. HEMISFER: Hybrid EegMrI and

Simultaneous neuro-FEedback for brain Rehabilitation. 1ère journée nationale sur le neurofeedback, ICM, Paris, France, January 2016. [Poster]• E Bannier, M Mano, S Robert, I Corouge, L Perronnet, J Lindgren, A Lécuyer, C Barillot (2015). On the feasibility and specificity of simultaneous

EEG and ASL MRI at 3T. Proceedings of ISMRM. [Abstract]

50

51

• AnatoleandChristian• Allthemembers ofthejury• Volunteers• Marsel• Elise,Isabelle• Angélique,Armelle,Nathalie• VisagesandHybrid members• TheMRtechnicians• Doctors• Family• Friends• Shiatsuteacher• Theperson present byhis absence• Andallofyou !

SpecialTHANKSto

https://lowpe.github.io/lorraineperronnet/

Date post:	28-May-2020
Category:	Documents
Upload:	others
View:	12 times
Download:	0 times

Soutenance Lorraine Perronnet preview - InriaSoutenance Lorraine Perronnet preview Created Date:...

Documents