+ All Categories
Home > Documents > Real-time fMRI brain computer interfaces: Self-regulation ... et al (2014)_rtfMRI... · Brain...

Real-time fMRI brain computer interfaces: Self-regulation ... et al (2014)_rtfMRI... · Brain...

Date post: 11-May-2020
Category:
Upload: others
View: 6 times
Download: 0 times
Share this document with a friend
17
Biological Psychology 95 (2014) 4–20 Contents lists available at ScienceDirect Biological Psychology jo ur n al hom epa ge: www.elsevier.com/locate/biopsycho Real-time fMRI brain computer interfaces: Self-regulation of single brain regions to networks Sergio Ruiz a,b,, Korhan Buyukturkoglu a,c , Mohit Rana a,c , Niels Birbaumer a,d , Ranganatha Sitaram a,e,f,∗∗ a Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Silcherstrasse 5, D-72076 Tuebingen, Germany b Departamento de Psiquiatría, Escuela de Medicina, Centro Interdisciplinario de Neurociencias, Pontificia Universidad Católica de Chile, Avda. Libertador Bernardo O’Higgins N 340, 32349 Santiago, Chile c Graduate School of Neural & Behavioural Sciences, International Max Planck Research School, Österbergstrasse 3, D-72074 Tübingen, Germany d Ospedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), 30126 Venezia, Italy e Department of Biomedical Engineering, University of Florida, P.O. Box 116131, Gainesville, FL 32611-6131, USA f Sri Chitra Tirunal Institute of Medical Sciences and Technology, Thiruvananthapuram 695 011, Kerala, India a r t i c l e i n f o Article history: Received 31 August 2012 Accepted 18 April 2013 Available online 1 May 2013 Keywords: Real-time fMRI Brain computer interfaces Neurofeedback Brain connectivity a b s t r a c t With the advent of brain computer interfaces based on real-time fMRI (rtfMRI-BCI), the possibility of performing neurofeedback based on brain hemodynamics has become a reality. In the early stage of the development of this field, studies have focused on the volitional control of activity in circumscribed brain regions. However, based on the understanding that the brain functions by coordinated activity of spatially distributed regions, there have recently been further developments to incorporate real-time feedback of functional connectivity and spatio-temporal patterns of brain activity. The present article reviews the principles of rtfMRI neurofeedback, its applications, benefits and limitations. A special emphasis is given to the discussion of novel developments that have enabled the use of this methodology to achieve self- regulation of the functional connectivity between different brain areas and of distributed brain networks, anticipating new and exciting applications for cognitive neuroscience and for the potential alleviation of neuropsychiatric disorders. © 2013 Elsevier B.V. All rights reserved. Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.1. The development of a BCI based on fMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2. General design of a rtfMRI-BCI system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2. Applications of rtfMRI for the regulation of circumscribed brain regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1. Studies on healthy participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1. Self-regulation of emotional brain areas and emotional processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.2. Self-regulation of other brain regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2. Studies on clinical populations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.3. Progress and limitations of single-ROI rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3. Brain connectivity enhancement with rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.1. Direct self-modulation of brain functional connectivity with rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.2. Brain network feedback with rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4. fMRI-BCI limitations and future directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Corresponding author at: Departamento de Psiquiatría, Escuela de Medicina, Pontificia Universidad Católica de Chile, Marcoleta 391, 2 piso, 32349 Santiago, Chile. Tel.: +56 2 354 3808; fax: +56 2 207 0583. ∗∗ Corresponding author at: Institute of Medical Psychology and Behavioural Neurobiology, Eberhard-Karls-University of Tübingen, Silcherstrasse 5, D-72076 Tuebingen, Germany. Tel.: +49 7071 29 73263; fax: +49 7071 29 5956. E-mail addresses: [email protected], [email protected] (S. Ruiz), [email protected] (K. Buyukturkoglu), [email protected] (M. Rana), [email protected] (N. Birbaumer), [email protected] (R. Sitaram). 0301-0511/$ see front matter © 2013 Elsevier B.V. All rights reserved. http://dx.doi.org/10.1016/j.biopsycho.2013.04.010
Transcript

Rb

SRa

b

Bc

d

e

f

a

ARAA

KRBNB

C

T

G

n

0h

Biological Psychology 95 (2014) 4– 20

Contents lists available at ScienceDirect

Biological Psychology

jo ur n al hom epa ge: www.elsev ier .com/ locate /b iopsycho

eal-time fMRI brain computer interfaces: Self-regulation of singlerain regions to networks

ergio Ruiza,b,∗, Korhan Buyukturkoglua,c, Mohit Ranaa,c, Niels Birbaumera,d,anganatha Sitarama,e,f,∗∗

Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Silcherstrasse 5, D-72076 Tuebingen, GermanyDepartamento de Psiquiatría, Escuela de Medicina, Centro Interdisciplinario de Neurociencias, Pontificia Universidad Católica de Chile, Avda. Libertadorernardo O’Higgins N◦340, 32349 Santiago, ChileGraduate School of Neural & Behavioural Sciences, International Max Planck Research School, Österbergstrasse 3, D-72074 Tübingen, GermanyOspedale San Camillo, Istituto di Ricovero e Cura a Carattere Scientifico (IRCCS), 30126 Venezia, ItalyDepartment of Biomedical Engineering, University of Florida, P.O. Box 116131, Gainesville, FL 32611-6131, USASri Chitra Tirunal Institute of Medical Sciences and Technology, Thiruvananthapuram 695 011, Kerala, India

r t i c l e i n f o

rticle history:eceived 31 August 2012ccepted 18 April 2013vailable online 1 May 2013

eywords:

a b s t r a c t

With the advent of brain computer interfaces based on real-time fMRI (rtfMRI-BCI), the possibility ofperforming neurofeedback based on brain hemodynamics has become a reality. In the early stage of thedevelopment of this field, studies have focused on the volitional control of activity in circumscribed brainregions. However, based on the understanding that the brain functions by coordinated activity of spatiallydistributed regions, there have recently been further developments to incorporate real-time feedback of

eal-time fMRIrain computer interfaceseurofeedbackrain connectivity

functional connectivity and spatio-temporal patterns of brain activity. The present article reviews theprinciples of rtfMRI neurofeedback, its applications, benefits and limitations. A special emphasis is givento the discussion of novel developments that have enabled the use of this methodology to achieve self-regulation of the functional connectivity between different brain areas and of distributed brain networks,anticipating new and exciting applications for cognitive neuroscience and for the potential alleviation of

neuropsychiatric disorders.

© 2013 Elsevier B.V. All rights reserved.

ontents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.1. The development of a BCI based on fMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2. General design of a rtfMRI-BCI system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2. Applications of rtfMRI for the regulation of circumscribed brain regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1. Studies on healthy participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.1.1. Self-regulation of emotional brain areas and emotional processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1.2. Self-regulation of other brain regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2. Studies on clinical populations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3. Progress and limitations of single-ROI rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3. Brain connectivity enhancement with rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.1. Direct self-modulation of brain functional connectivity with rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143.2. Brain network feedback with rtfMRI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4. fMRI-BCI limitations and future directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

∗ Corresponding author at: Departamento de Psiquiatría, Escuela de Medicina, Pontificel.: +56 2 354 3808; fax: +56 2 207 0583.∗∗ Corresponding author at: Institute of Medical Psychology and Behavioural Neurobiolermany. Tel.: +49 7071 29 73263; fax: +49 7071 29 5956.

E-mail addresses: [email protected], [email protected] (S. Ruiz), [email protected] (N. Birbaumer), [email protected]

301-0511/$ – see front matter © 2013 Elsevier B.V. All rights reserved.ttp://dx.doi.org/10.1016/j.biopsycho.2013.04.010

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

ia Universidad Católica de Chile, Marcoleta 391, 2 piso, 32349 Santiago, Chile.

ogy, Eberhard-Karls-University of Tübingen, Silcherstrasse 5, D-72076 Tuebingen,

[email protected] (K. Buyukturkoglu), [email protected] (M. Rana),e (R. Sitaram).

l Psyc

1

ffdphcn

otdecif

fttbifd

1

rr(usacsii

bctnsstd2cdrwbrotbt

dwbpt

S. Ruiz et al. / Biologica

. Introduction

Nearly a decade has already passed since the first success-ul application of brain computer interfaces based on real-timeMRI (rtfMRI-BCI). The technical aspects of this methodology haveeveloped rapidly thereafter, with a constantly growing number ofublications and teams around the world devoted to this field. But,ow far have we gone? What are the scientific and clinical appli-ations of rtfMRI-BCI? What are its limitations and challenges thateed to be worked on in the coming years?

The aim of this article is two-fold. Firstly, we will give anverview of rtfMRI-BCI, focused on its use for neurofeedback andhe self-control of circumscribed brain activity. Secondly, we willescribe an exciting trend in rtfMRI literature, i.e., the recentmergence of approaches that move from self-regulation of singleircumscribed brain areas, to self-regulation of brain connectiv-ty and network activation pertaining specific neurophysiologicalunctions.

First, we will introduce the concept of rtfMRI-BCI and its basiceatures. We will then review the previous studies that have usedhis methodology in healthy and clinical populations, focusing onhose that investigated modulation of single brain regions, theirehavioral effects and their use as therapeutic tools in clinical sett-

ngs. Finally, we will examine the latest developments in rtfMRIor the self-regulation of brain connectivity and brain circuits, andiscuss the future directions for this field.

.1. The development of a BCI based on fMRI

A BCI is a system that measures the activity of the central neu-al system (CNS) and converts it into artificial output that replaces,estores, enhances, supplements, or improves its natural outputsWolpaw & Wolpaw, 2012). In the last decades, many studiessing these techniques have been done in motor rehabilitation introke and restoring communication in the completely paralyzednd locked-in patients (Birbaumer & Cohen, 2007). However, BCIsannot only modify the CNS outputs but also can transform brainignals into available sensory inputs that can in turn modify behav-or. This novel variation of BCI has recently attracted more interest:.e. BCI-neurofeedback.

Since the 1960s, several studies have shown that subjects cane trained by non-invasive BCI-neurofeedback to gain voluntaryontrol of different components of the EEG spectrum by operantraining using the feedback of specific components of the EEG sig-al as reward. These studies have demonstrated that learned brainelf-regulation gained through neurofeedback training can lead topecific behavioral modifications, and therefore a BCI could poten-ially be used as a therapeutic tool for neuronal and psychiatricisorders (Birbaumer, 2006; Kotchoubey et al., 2001; Strehl et al.,006). However, EEG-BCI neurofeedback is limited by its inherentonstraints, i.e. low spatial resolution and the inability to accesseeper brain structures. In this sense, modern techniques of neu-oimaging that permit a non-invasive assessment of brain functionith high spatial resolution offer an alternative for neurofeed-

ack. One of these techniques in particular, functional magneticesonance imaging (fMRI), which measures changes in the blood-xygen level-dependent (BOLD) signal, has become an invaluableool for research since its introduction in the clinic in the 1980s. Butefore fMRI could be used as a BCI neurofeedback system, manyechnical and methodological advances were needed.

For fMRI research, experiments usually follow a serial proce-ure in which brain fMRI images are acquired from the participants

hile performing a particular task under investigation, followed

y an offline analysis of signal preprocessing and statistical map-ing. This entire procedure might take several hours or days dueo the large data generated, and due to the subsequent high

hology 95 (2014) 4– 20 5

computational cost. In an effort to monitor data quality whileimages are being acquired, to reduce imaging and post-processingtimings, and to create “interactive experimental paradigms”, Coxand colleagues developed the first online adaptation of processingalgorithms (Cox, Jesmanowicz, & Hyde, 1995). Further research hasrefined online image acquisition and processing in the followingaspects: quality, speed and contrast-to-noise ratio; removal of mag-netic susceptibility artifacts; implementation of head-movementcorrections; and optimization of the imaging preprocessing steps(Cox & Jesmanowicz, 1999; Gembris et al., 2000; LaConte, Peltier, &Hu, 2007; Mathiak & Posse, 2001; Posse et al., 1999, 2001; Sitaramet al., 2011b; Smyser, Grabowski, Frank, Haller, & Bolinger, 2001;Voyvodic, 1999; Weiskopf, Klose, Birbaumer, & Mathiak, 2005; Yoo,Guttmann, Zhao, & Panych, 1999). Thanks to these technical andmethodological advances, the development of fMRI-BCI becamefeasible (for more information regarding the technical advancesthat enabled the implementation of rtfMRI, a detailed descriptioncan be found in Sitaram, Lee, Ruiz, and Birbaumer (2011a)).

1.2. General design of a rtfMRI-BCI system

A rtfMRI-BCI works as a closed-loop system that extracts infor-mation from BOLD signals in real-time, so that this information canbe provided to the subjects as contingent feedback to enable thecontrol of brain activity. In general, the rtfMRI-BCI system includes:(1) the subject, (2) signal acquisition, (3) preprocessing, (4) signalanalysis, and (5) feedback.

To illustrate the design of an fMRI-BCI system, we will describethe system built at the Institute of Medical Psychology andBehavioral Neurobiology, University of Tubingen, the general char-acteristics of which are common to most current fMRI-BCI systems.Signal acquisition: experiments are conducted using a 3.0 T wholebody scanner with a standard head coil. Whole brain images areacquired using an echo planar imaging (EPI) sequence (Bandettini,Wong, Hinks, Tikofsky, & Hyde, 1992) (for further technical detailsregarding signal acquisition and analysis in rtfMRI, please see (Cariaet al., 2007; Caria, Sitaram, & Birbaumer, 2012). Images are recon-structed, distortion corrected, and averaged on the MRS computer.Signal analysis: the signal analysis component is implementedin our studies using Turbo-Brain Voyager (Brian Innovations,Maastricht, The Netherlands) (Goebel, 2012). The signal analy-sis component retrieves reconstructed images, and performs dataprocessing (including 3D motion correction) and statistical analy-sis. The time series of selected regions of interest (ROIs) are thenexported to the custom built visualization software that providesfeedback to the subjects using either a video projection or MRI com-patible goggles, in “real-time”. For ROI selection, real-time fMRIallows accessing the brain with high spatial resolution. Feedback:Current rtfMRI neurofeedback studies have often used a modality of“continuous feedback”, although “intermittent feedback” has alsobeen utilized (Johnson et al., 2012; Yoo & Jolesz, 2002). The delayin which the feedback is presented depends on the time involvedfor acquiring and processing the images. Current rtfMRI systemsare able to provide updated feedback information every repetitiontime (TR; 1.5 s) of the EPI pulse sequence. However, the hemody-namic feedback is inherently delayed: the BOLD signal is typicallyobserved to start in 1 s and to peak in 6 s after the stimulus onset.Furthermore, the BOLD response is an indirect measure of brainactivity. Despite these limitations of the BOLD feedback, evidencefor a strong correlation between the BOLD signal and the underlyingelectrical neural activity (Logothetis, 2008), and recent demonstra-tion of control of the BOLD signal with real-time fMRI has garnered

growing research interest in this approach.

Participants of a rtfMRI neurofeedback experiment learn, withoperant training, to activate circumscribed brain areas. Contin-gent feedback of the BOLD signal from a ROI represents a reward

6S.

Ruiz

et al.

/ Biological

Psychology 95 (2014) 4– 20

Table 1RtfMRI-BCI studies for self-regulation of circumscribed brain areas.

Study ROI Participants Control groups Strategies suggestedfor self-regulation/regulationstrategies used

Feedbackmodality/delay

Preservedself-regulationwithout feedback(transfer)

Behavioral task Main findings

Healthy subjects1. Emotion brain areasPosse et al. (2003) Amygdale 6 healthy (2 male, 4

female)– Instructions for

performing aself-mood inductiontask/

Auditory/60 sapprox.

– Subjective ratings ofself-mood after fMRIsessions (in thescanner)

Successfulself-regulation ofROI

Weiskopf et al.(2003)

ACC 1 healthy male – Notprovided/imagery oflandscapes, sportsand socialinteraction

Visual/2 s – Subjective ratings ofaffective states aftereach fMRI session(in the scanner)

Successfulself-regulation ofROI

Caria et al. (2007) Right anterior insula 15 healthy (9female, 6 male) (9experimental, 6control)

1- Trained with shamfMRI feedback

Suggessted:recalling personaland affectivelyrelevant events

Visual/1.5 s Yes – Successfulself-regulation ofROI

2- No fMRIinformation provided(only emotionalimagery)

Used: positive andnegative affectivemental strategies

Caria et al. (2010) Left anterior insula 27 healthy (9 exp, 18cont.)

1- Trained with shamfMRI feedback

Suggested and used:free emotionalimagery or “anyother useful mentalstrategy forfeedback regulation”

Visual/1.5 s - Ratings of valanceand arousal ofemotional pictures(IAPS) followingblocks ofself-regulation (inthe scanner)

Increased negativevalence ratings ofthe aversive stimulifollowing insulaself-regulation

2- No fMRIinformation provided(only emotionalimagery

Veit et al. (2012) Left anterior insula 11 healthy (8female, 3 male)

– Suggested and used:to imaginethemselves beinginvolved and todistance themselvesin the situationdepicted byemotional picturesfor up and downregulation,respectively

Visual/1.5 s – Self ratings ofemotionalregulation in thescanner after eachfMRI session

Successfulself-regulation ofROI

Zotev et al. (2011) Left amygdale 28 healthy male (14exp, 14 cont.)

One group trainedwith sham fMRIfeedback

Suggested: happyautobiographicalmemories

Visual/2 s Yes – Successfulself-regulation ofROI

Used: happymemories of closefamily members orjoyful events

S. R

uiz et

al. /

Biological Psychology

95 (2014) 4– 207

Table 1 (Continued)

Study ROI Participants Control groups Strategies suggestedfor self-regulation/regulationstrategies used

Feedbackmodality/delay

Preservedself-regulationwithout feedback(transfer)

Behavioral task Main findings

Hamilton et al.(2011)

sACC 17 healthy female (8exp, 9 cont.)

One group trainedwith sham fMRIfeedback

Suggested: positiveaffect strategies

Visual/around 2 s No – Successfulself-regulation ofROI

Used: beingoutdoors, food, sex,parties,warm-relaxingbubble bath, playingsports, etc.

2. Motor areasYoo et al. (2002) Hand motor and

somato-sensoryareas

5 healthy (2 male, 3female)

– Overt handmovementstrategies/handmovement(increasing muscularrecruitment,increasing tappingfrequency, etc.)

Visual/60 s – – Successfulmodulation andenhanced brainactivations of theROI through theadjustments ofmovements

deCharms et al.(2004)

Somato-motorcortex

9 healthy (6 exp, 3cont)

Trained with shamfMRI feedback

Suggested and used:imagery ofdominant right handfinger tapping,flexion–extension,pronation–supination,abduction–adduction,and opposition ofdigits

Visual/<2 s Yes – Successfulself-regulation ofROI

Yoo et al. (2004) Several distributedbrain areas relatedto particularcognitive strategies

3 healthy male – Different cognitivestrategies (mentalcalculation, mentalspeech, right and lefthand motorimagery) thatelicited a particularbrain activation,were used to spatialnavigation through amaze

Bold signal wasclassified and usedto “navigate”through a maze

– – Successfulnavigation through amaze by properspecific brainactivity

Lee et al. (2009) Primary motor areas 3 healthy (1 female,2 male)

– Instructed and used:imagining clenchingof the left and righthandsindependently at arate of 2 Hz

Visual (observingthe robotic armmovement itself)

– – Successfulself-regulation ofROI

A robotic arm wascontrolled by theregulation ofregional corticalactivations in theprimary motor areas

Yoo et al. (2008) Hand motor area 24 healthy (12 exp,12 cont.)

Trained with shamfMRI feedback

Instructed and used:imagining squeezingthe right fist

Visual/1–2 s Yes – Successfulself-regulation ofROI, learning thatpersisted after atwo-weekself-practice period

8S.

Ruiz

et al.

/ Biological

Psychology 95 (2014) 4– 20

Johnson et al. (2012) Premotor cortex 13 healthy (6 male, 7female)

The same subjectparticipated inconditions ofintermittent orcontinuous feedback,and with either realfeedback or shamfeedback

Suggested and used:imagery of righthand movementssuch as writing,playing a musicalinstrument orcompleting asports-relatedmovement

Visual/continuous(updated everyscanned brainvolume) orintermittent

– – Intermittentpresentation offeedback was moreeffective thancontinuouspresentation inpromotingself-modulation ofbrain activity

Chiew et al. (2012) Primary motorcortex

18 healthy (13 exp, 5contr.)

Trained with shamfMRI feedback

Suggested:kinesthetic motorimagery (kMI) of theexecution andfeeling of handmovements

Visual – Button pressreaction time task(in the scanner)

6 of 13 subjects ofthe experimentalgroup successfullyincreased a“laterality indexvalue” (thedifference betweenthe contralateral ROIrelative to the handinvolved in kMI). Nobehavioralmodificationdetected due totraining

Used: Imaginedhand clenching andany other kMIstrategy restricted tosingle handmovements

Berman et al. (2012) Primary motorcortex

15 healthy The same subjectsparticipated indifferent conditions:overmovement + feedback,and motorimagery + feedback

Suggested and usedfor motor imagery:mental imagery ofhand or fingermovement

Visual/1 s – – Successfulregulation only withover movement andfeedback

Lack of consistentROI (M1) activationwith motor imageryand feedback

Sulzer et al. (2012) SN/VTA 25 healthy (12 exp,13 contr.)

On group trained withinverse feedback

Suggested:“rewards” such asfood, romantic orsexual imagery, timewith family andfriends, personalachievements

Visual No – Successfulself-regulation ofROI with positivefeedback, which wascorrelated withincreased SCR(emotional arousal)

Used: imaginingpleasant scenes

3. Auditory cortexYoo et al. (2006) Left auditory cortex 22 healthy (8

female, 14 male) (11exp, 11 contr.)

No fMRI informationprovided

Suggested: tomodulate attentiontoward frequencymodulated andamplitude-modulatedsounds

Auditory/at end ofeach fMRI session

– – Successfulself-regulation ofROI

S. R

uiz et

al. /

Biological Psychology

95 (2014) 4– 209

Table 1 (Continued)

Study ROI Participants Control groups Strategies suggestedfor self-regulation/regulationstrategies used

Feedbackmodality/delay

Preservedself-regulationwithout feedback(transfer)

Behavioral task Main findings

Used: counting thenumber of pitches ora mental generationof similar soundsalong with thepresented sound,attending to thepitch/tone of thesound componentitself, hearing soundas though the soundlevel is played in“low volume” orhearing from adistance

4. Language related areasRota et al. (2009) rIFG 12 healthy (7 exp, 5

cont.)Trained with shamfMRI feedback

Used: strategiesconnected to speech,such as imaginationof lecturing before aclass of students,arguing scenes anddebates. Otherstrategies includedimagined singing,imagined recitationof poems, andrecalling oldconversations withfriends

Visual/1.5 s – Identification ofemotional prosody,and speededgrammaticalityjudgments(syntacticprocessing) (in thescanner) followingblocks ofself-regulation

Successfulself-regulation ofROI. A significantimprovement ofaccuracy in theidentification ofemotional prosodicintonations

Clinical populationsStudy ROI Participants Control groups Strategies suggested

for self-regulation/regulationstrategies used

Feedbackmodality/delay

Preservedself-regulationwithout feedback(transfer)

Behavioral task Main findings

1. Neurological disordersSubramanian et al.

(2011)SMA 10 Parkinson’s

patient (6 male, 4female) (5 exp, 5cont.)

Trained with shamfMRI feedback

Suggested: any kindof motor imagery

Visual/2 s – UPDRS,finger-tapping test(before/aftertraining) (out thescanner)

Successfulself-regulation ofROI

Used: imagery ofcomplex motortasks, includingsports or manualtasks from work.

Improvement inmotor speed (fingertapping) and clinicalratings of motorsymptoms (assessedwith UPDRS)

Buyukturkoglu et al.(2012)

SMA 1 Parkinson’s patient - Suggested and used:any kind of handmotor imagery

Visual/1.5 s – Sequential buttonpress task followingblocks ofself-regulation

Successfulself-regulation ofROI

3 healthy subjects Decreased handmotor performancefollowingself-regulation

10S.

Ruiz

et al.

/ Biological

Psychology 95 (2014) 4– 20

deCharms et al.(2005)

rACC 36 healthy (20 m, 16f) and 12 chronicpain patients (8 m, 4f)

One group trainedwithout fMRIinformation using thesame mentalstrategies that theexperimental group

Suggested and used:attending towardthe painful stimulusvs. away from it

Visual/1–2 s – Pain sensitivityratings (out of thescanner)

Successfulself-regulation ofROI

One group trainedwithout fMRIinformation andattentional strategies

Attempt to perceivethe stimulus as aneutral sensoryexperience vs. atissue-damaging,frightening, oroverwhelmingexperience. Attemptto perceive thestimulus as eitherlow or high intensity

Decreasedperception of themagnitude of painamong the patients

One group trainedwith a different ROI forfeedback

Attempt to controlthe painfulexperience, or allowthe stimulus tocontrol the percept

One group trainedwith non- contingentrtfMRI information

A patient grouptrained withautonomicbiofeedback

Sitaram et al. (2012) PMv 6 subjects (2 withhemiparesis, 4healthy)

– Suggested: differentimageries to findone that resulted inthe biggest feedbackoutput

Visual/1.5 s Yes ICI and facilitationusing TMS in apaired pulseparadigm

Successfulself-regulation ofROI

Visuomotor taskusing a pinch-forcetransducer (out ofthe scanner)

Decreasedintracorticalinhibition associatedwith the trainingEnhancement in theperformance of apinch-force task

Haller et al. (2010) Primary auditorycortex

6 patients withchronic tinnitus (3male, 3 female)

– Suggested and used:mental strategiesthat helps to reducethe intensity of thetinnitus noise intheir daily life

Visual/1.5 s – Follow-up clinicalquestionnaireapproximately 2weeks after training(out of the scanner)

Successfulself-regulation ofROI. Improvement intinnitus symptomsin at least two out ofsix patients

2. Psychiatric disordersRuiz et al. (2013) Insula 9 schizophrenia

patients (5 female, 4male)

– Recallingemotionally relevantexperiences

Visual/1.5 s No Recognition ofemotional facesfollowingself-regulation (inthe scanner)

Successfulregulation of ROI

PANAS (before/afterthe training) (out ofthe scanner)

Enhanced accuracyfor the recognitionof disgust faces andreduced accuracy forrecognizing happyfaces, followingself-regulation

S. Ruiz et al. / Biological Psyc

Tabl

e

1

(Con

tinu

ed

)

Stu

dy

RO

I

Part

icip

ants

Con

trol

grou

ps

Stra

tegi

es

sugg

este

dfo

r

self

-re

gula

tion

/reg

ula

tion

stra

tegi

es

use

d

Feed

back

mod

alit

y/d

elay

Pres

erve

dse

lf-r

egu

lati

onw

ith

out

feed

back

(tra

nsf

er)

Beh

avio

ral t

ask

Mai

n

fin

din

gs

Li

et

al. (

2012

)A

CC

/MPF

C

10

hea

lth

y

ciga

rett

esm

oker

sTh

e

sam

e

subj

ects

par

tici

pat

ed

intw

o

dif

fere

nt

con

dit

ion

s:re

du

cin

g

the

crav

ing

+

dec

reas

ing

AC

C

acti

vity

, an

d

resi

st

the

crav

ing

+

incr

easi

ng

mPF

Cac

tivi

ty

Red

uci

ng

the

crav

ing

+

resi

stin

g

tocr

avin

g,

for

each

ofth

e

two

con

dit

ion

s

Vis

ual

Subj

ecti

ve

rati

ng

ofcr

avin

g

ind

uce

d

byvi

sual

cues

foll

owin

gbl

ocks

ofse

lf-r

egu

lati

on

(in

the

scan

ner

)

Succ

essf

ul d

ecre

ased

AC

C

acti

vity

usi

ng

a“r

edu

cin

g

crav

ing”

men

tal s

trat

egy

Lin

den

et

al. (

2012

)V

LPFC

, In

sula

,D

LPFC

, OFC

16

subj

ects

wit

hM

ajor

dep

ress

ion

(8ex

p, 8

con

t.)

No

fMR

I in

form

atio

n

pro

vid

ed.

Sam

e

men

tal s

trat

egy

than

the

exp

erim

enta

l gro

up

outs

ide

ofth

e

scan

ner

Sugg

este

d:

reca

llin

gp

osit

ive

mem

orie

s,im

ager

y

of

the

pos

itiv

e

scen

es

Vis

ual

/2

s

HD

RS

Succ

essf

ul

self

-reg

ula

tion

ofR

OIs

Use

d:

stra

tegi

esin

clu

din

g

hol

iday

s,th

ough

ts

abou

tfa

mil

y,

bein

g

hap

py

and

imag

ery

ofbe

auti

ful s

cen

esfr

om

nat

ure

POM

S Im

pro

vem

ent

incl

inic

al

sym

pto

ms

asse

ssed

wit

h

the

HD

RS

PAN

AS

(ou

t

of

the

scan

ner

)

RO

I:

regi

on

of

inte

rest

;

AC

C:

ante

rior

cin

gula

te

cort

ex;

DLP

FC:

dor

sola

tera

l

pre

fron

tal

cort

ex;

HD

RS:

Ham

ilto

n

Dep

ress

ion

Rat

ing

Scal

e;

IAPS

:

inte

rnat

ion

al

affe

ctiv

e

pic

ture

s

syst

em;

ICI:

intr

acor

tica

l

inh

ibit

ion

;

M1:

pri

mar

ym

otor

cort

ex;

MPF

C:

med

ial p

refr

onta

l cor

tex;

OFC

:

orbi

tofr

onta

l cor

tex;

PAN

AS:

pos

itiv

e

and

neg

ativ

e

affe

ct

sch

edu

le;

PMv:

ven

tral

pre

mot

or

cort

ex;

POM

S:

pro

file

of

moo

d

stat

es;

rAC

C:

rost

ral a

nte

rior

cin

gula

te

cort

ex;

rIFG

:ri

ght

infe

rior

fron

tal g

yru

s;

sAC

C:

subg

enu

al

ante

rior

cin

gula

te

cort

ex;

SCR

:

skin

con

du

ctan

ce

resp

onse

;

SMA

:

sup

ple

men

tary

mot

or

area

;

SN/V

TA:

subs

tan

tia

nig

ra/v

entr

al

tegm

enta

l are

a

com

ple

x;

UPD

RS:

un

ified

Park

inso

n’s

dis

ease

rati

ng

scal

e;

and

VLP

FC:

ven

trol

ater

al

pre

fron

tal c

orte

x.

hology 95 (2014) 4– 20 11

or reinforcement. Many modalities of feedback can be employed,including verbal, auditory, tactile, and a combination of these.Monetary reward could also act as a form of reinforcement. Forvisual feedback, time-series graphs directly coming from BOLD acti-vation have been used (deCharms et al., 2004; Weiskopf et al.,2003). Sitaram and colleagues introduced a “thermometer” type offeedback that shows snap shots of brain activity as changes in ther-mometer levels (bars going up and down) (Caria et al., 2007; Caria,Sitaram, Veit, Begliomini, & Birbaumer, 2010; Ruiz et al., 2013).Even more spectacular approaches such as virtual reality anima-tions (Sitaram et al., 2005) and video-based feedback (Sitaram et al.,2012) have also been explored.

Real-time fMRI has been used for diverse purposes, includ-ing the assessment of the quality of fMRI data and neurosurgicalapplications. Used as a neurofeedback system, rtfMRI-BCI has twoapplications. In the first place, it offers the possibility to study therelation between behavior and localized brain activation. Unliketraditional fMRI studies in which brain activations are analyzedafter they are elicited by a particular task, rtfMRI-based neuro-feedback enables the manipulation of the brain activity as anindependent variable to study its influence on behavior as thedependent variable. Secondly, the behavioral modification inducedby the operant training of localized brain areas could represent anovel therapeutic approach for brain disorders.

2. Applications of rtfMRI for the regulation ofcircumscribed brain regions

2.1. Studies on healthy participants

Majority of studies to-date have used rtfMRI for training healthyindividuals to modulate specific brain areas presumably related tobrain functions of movement, cognition, perception and emotion.In order to illustrate the characteristics of rtfMRI experiments, herewe highlight the studies that targeted emotional brain regions,as emotional processing has been of special interest in fMRI-BCIresearch (for a complete summary of rtfMRI-BCI studies for singleROI regulation, please see Table 1).

2.1.1. Self-regulation of emotional brain areas and emotionalprocessing

The first attempt to explore the use of this methodology to mod-ulate the activations of a brain area related to emotional processingwas conducted by Posse et al. (2003). In this study, participantsperformed a mood self-induction paradigm (aided by the presen-tation of emotional faces) of neutral and sad affective states whilein the scanner. After each trial of self-mood induction (30 s), partic-ipants were provided with a delayed verbal feedback of amygdalaactivation during the previous self-induction block. All partici-pants successfully achieved sad induction and, interestingly, theirself-mood ratings were correlated with the level of activity in theamygdala. However, as the self-induction task was continually per-formed through the experiment, it was not possible to ascertainwhether amygdala activation was actually achieved due to the pre-sentation of emotional faces, or due to the BCI feedback, leaving atleast a possibility that both factors could have worked together.

In a subsequent experiment, Weiskopf et al. (2003) appliedrtfMRI neurofeedback to achieve self-regulation of BOLD signalin the anterior cingulate cortex (ACC). A single participant wasinstructed to up-regulate and bring to baseline ACC activations,aided by a continually updated contingent feedback (delay < 2 s.),

introducing the concept of “immediate feedback” in rtfMRI-BCI. Results showed that the participant successfully increasedthe BOLD signal of ACC in the up-regulation blocks. However,the study did not correlate the BOLD signal change with any

1 l Psych

bsBu

tit(wBbimtaitit(t(fsogerw

tHniinaerfvTlm

tttgoaoeujws

wuUdors

Buyukturkoglu et al. (2012) showed that behavioral results asso-ciated with feedback training are not always consistent among

2 S. Ruiz et al. / Biologica

ehavioral measurement. Although designed as a single-subjecttudy, this work was one of the first convincing evidence thatOLD signal of circumscribed brain regions can be self-regulatedsing “immediate” rtfMRI neurofeedback.

Caria and colleagues performed a series of experiments inves-igating the use of rtfMRI-BCI for the regulation of anteriornsular activity, emphasizing the potential behavioral modifica-ion induced by learned brain self-regulation. In an initial studyCaria et al., 2007), nine participants of the experimental groupere trained with continually updated visual feedback from theOLD signal coming from right anterior insula (delay 1.5 s), andy recalling autobiographical emotional events. Successful train-

ng (observed in all the participants) resulted in a significantlyonotonic increased activation cluster in the anterior portion of

he right insula across fMRI sessions, indicating training effectsnd learning. Immediately after the training, participants werenstructed to achieve insula self-regulation, but this time withouthe presence of feedback. This “transfer” session showed a signif-cant increase in BOLD signal in insula compared with the initialraining session, suggesting that learned self-regulation can persistat least in the short-term) after rtfMRI-BCI training. Importantly,wo control groups were added, one trained with “sham feedback”non-specific fMRI information), and a second one in which noMRI information was provided, but were instructed to use theame mental strategy used by the experimental group. As nonef these groups achieved insula self-regulation, this was the firstroup study that convincingly showed that volitional control ofmotionally relevant brain areas can be attained by rtfMRI neu-ofeedback training, and that this learning is specifically achievedith contingent fMRI feedback.

In a subsequent study, Caria et al. (2010) explored the rela-ionship between brain self-regulation and emotional behavior.ealthy participants underwent four rtfMRI neurofeedback scan-ing sessions to modulate the BOLD response in the left anterior

nsula guided by visual feedback, as in the previous study. Partic-pants were presented with either an emotionally negative or aeutral picture following each self-regulation and baseline blocksnd asked to rate the emotional valence of the picture. As hypoth-sized, participants learned to increase and decrease the BOLDesponse significantly in the anterior insula guided by contingenteedback, and behavioral data showed a significant difference ofalence ratings of the aversive pictures in the last training session.hese results demonstrated that rtfMRI neurofeedback manipu-ation of paralimbic regions such as insula is possible and can

odulate a specific emotional response.Zotev et al. (2011) explored the use of rtfMRI with amygdala as

he region of interest. Unlike the experiment by Posse et al. (2003),his study included a large experimental group trained with con-ingent feedback (BOLD signal from left amygdala) and a controlroup trained with sham feedback. Their results confirmed previ-us studies in insula cortex, showing a progressive increase in themygdala activation in the experimental group, and a persistencef the learning effect in a subsequent transfer session. The studyxplored the relationship between the capability for self-regulationsing rtfMRI feedback and a few trait psychological measures: sub-

ects with greater difficulty in identifying feelings, and the onesith higher sensitivity to others peoples’ anger, were less able to

elf-regulate during the training.Hamilton, Glover, Hsu, Johnson, and Gotlib (2011) tested

hether individuals can down-regulate the activity of the subgen-al anterior cingulate (sACC) cortex with rtfMRI neurofeedback.sing positive affect strategies, healthy women were able toown-regulate the BOLD signal from sACC, unlike a control groupf participants trained with sham feedback. The learned down-

egulation, however, did not persist in a subsequent session whereubjects were not provided with feedback.

ology 95 (2014) 4– 20

2.1.2. Self-regulation of other brain regionsBesides brain areas related to emotional processing, the second

most commonly studied brain regions in rtfMRI research corre-sponds to somatomotor areas (Berman, Horovitz, Venkataraman,& Hallett, 2012; Chiew, LaConte, & Graham, 2012; deCharms et al.,2004; Johnson et al., 2012; Yoo & Jolesz, 2002; Yoo, Lee, O’Leary,Panych, & Jolesz, 2008). Successful self-regulation of these regionshas been used to move an external mechanical arm and com-puter devices (Lee, Ryu, Jolesz, Cho, & Yoo, 2009; Yoo et al., 2004),therefore suggesting a potential use in neurorehabilitation. Thesestudies have shown that healthy subjects can successfully achieveself-regulation of somatomotor areas with motor imagery andrtfMRI feedback. However, at least one study presented contrastingresults, as participants were not able to display a consistent con-trol of M1 using motor imagery and rtfMRI feedback (Berman et al.,2012).

Other studies in healthy subjects have also targeted auditoryregions (Yoo et al., 2006), language areas (Rota et al., 2009) anddeep brain structures, i.e. basal ganglia (Sulzer et al., 2012). Forfurther details, please see Table 1.

2.2. Studies on clinical populations

The observed behavioral modifications induced by operanttraining of single brain regions in healthy subjects, naturally led toefforts for implementing this methodology in clinical populations.

In the first full-group, controlled rtfMRI neurofeedback studyleading to a resultant impact on behavior or disease symptoms,deCharms et al. (2005) focused on the use of self-regulation of ros-tral ACC to investigate the modulation of pain perception. Theyshowed that is it possible to gain deliberate control of rACC aidedby contingent fMRI feedback. It was also demonstrated that similarself-regulation can be achieved by a group of chronic pain patientswho reported decrease in the level of ongoing pain after the rtfMRIneurofeedback training.

Based on current views of the neural basis of tinnitus, Haller,Birbaumer, and Veit (2010) executed a pilot experiment in whicha small number of patients with chronic tinnitus were trained toreduce the activations of the auditory cortex (down-regulation).After a single day of training, most of the patients learned todown-regulate their activations in the ROI using visual contingentfeedback. Interestingly, a decrease of the subjective report of tinn-itus was observed in two out of six participants, suggesting thatfMRI-BCI could potentially produce beneficial effects for the treat-ment of this disorder.

Sitaram et al. (2012) successfully used rtfMRI neurofeedback totrain healthy individuals and stroke patients to regulate the BOLDsignal of ventral premotor cortex (PMv). To measure the effects ofself-regulation, paired pulses of transcranial magnetic stimulation(TMS) were used to induce intracortical inhibition and facilitation,while simultaneously measuring motor evoked potential (MEP) onthe participant’s finger. Results showed evidence for a reduction inintracortical inhibition after feedback training, indicating the ben-eficial effect of self-regulation training on motor cortical outputs.

Subramanian et al. (2011) trained a small group of patients suf-fering form Parkinson’s disease to achieve self-regulation of thesupplementary motor complex over two fMRI sessions, using motorimagery. After successful training, patients showed an improve-ment in motor speed (finger tapping) and clinical ratings of motorsymptomatology, effect that was not produced in an equal sizecontrol group of patients trained with motor imagery but notfMRI feedback information. However, in a recent experiment,

studies. In fact, in a pilot study with one Parkinson’s patient and3 healthy subjects, all the participants learned to up-regulate the

S. Ruiz et al. / Biological Psychology 95 (2014) 4– 20 13

Fig. 1. Activation clusters and BOLD time series in insula cortex of a representative schizophrenia patient through rtfMRI-BCI training. The figure shows the cluster activationsfor the contrast self -regulation vs. baseline during (A) an early (first day of training) and (B) a late session (last day) of rtfMRI-BCI training (P < 0.05 (FWE), coordinate z = 0).Bold-times series for right anterior insula are also shown in the respective sessions (blue and red bars represent the experimental conditions baseline and self-regulation,r r insu

S

BMatsfslsea

eiptotf

nbcasidr

espectively). An enhanced activation and self-control of the BOLD signal of anterio

ource: Adapted with permission from Ruiz et al. (2013).

OLD signal in the SMA using motor imagery and rtfMRI feedback.ore importantly, in order to explore behavioral modifications

ssociated with learning, participants performed a sequential but-on press task following up-regulation and baseline blocks in thecanner. Contrary to expectations, after the training the motor per-ormance (measured by the number of button presses in correctequence) was better following baseline blocks (in which no regu-ation of SMA was performed) than following up-regulation blocks,uggesting that operant learning of SMA might interfere with motorxecution, and that the relation between neural activation of SMAnd motor performance may need further exploration.

In the first application of rtfMRI-BCI on addictive disorders, Lit al. (2012) trained nicotine addictive patients to reduce the crav-ng response toward smoking cues. In a “reduce craving paradigm”,articipants successfully managed to reduce their craving responseoward craving-inducing pictures, while decreasing the activationf the anterior cingulate cortex. A significant correlation betweenhe induced changes in ACC activation and the corresponding dif-erence in cue craving ratings was also found.

Ruiz et al. (2013) aimed to evaluate if patients with schizophre-ia can learn self-control of the BOLD signal in the anterior insulay rtfMRI neurofeedback. Their results showed that patients withhronic schizophrenia are able to learn volitional control of thectivity of anterior insular cortex (Fig. 1). Following blocks of insula

elf-regulation, patients detected significantly more disgust facesn a face emotion recognition task, in line with the extensive evi-ence of the role of insula in face disgust recognition. However, foreasons that need more exploration, patients detected less happy

la can be seen in the late session of training.

faces following self-regulation. Notably, negative symptoms andthe duration of the illness were negatively correlated with the suc-cess of self-regulation.

In another study on a psychiatric population, Linden et al. (2012)applied rtfMRI neurofeedback to a group of patients suffering formunipolar depression. Patients were instructed to up-regulate sev-eral ROIs that included several brain regions strongly implicated inthe control of emotions (ventrolateral prefrontal cortex (VLPFC),insula, dorsolateral prefrontal cortex (DLPFC), medial temporallobe or the orbitofrontal cortex), while engaging in positive moodimagery. Successful regulation was observed in the experimentalgroup, accompanied by a significant clinical improvement at theend of the training.

2.3. Progress and limitations of single-ROI rtfMRI

As can be seen for the studies summarized in the previoussection (and in Table 1), fMRI-BCI experiments have rapidly gainedin complexity and ambition. In fact, in a few years, researchers havetargeted diverse brain areas as ROIs including amygdala, anteriorinsula, inferior frontal gyrus (IFG), several subdivisions of the ACC,superior temporal gyrus (STG), and several sensorimotor areas andother emotion-related areas. The results of most of these studieshave shown that if self-regulation learning occurs, this effect is spe-

cific for the target ROI and not due to general arousal or artifactualconfounds. Technical advances in imaging acquisition and process-ing have reduced the feedback delay from more than a minute tonear 1 s. Furthermore, recent experiments frequently incorporate

1 l Psych

loBbjiaatto

aoln

giloti

Rfa

ttdopKW

nqlo

3

3r

ctows(srns2nbc

bvt

4 S. Ruiz et al. / Biologica

arger groups, and include one or more control groups (usuallyne of them trained with sham feedback), in order to prove thatCI training is the specific factor that results in self-regulation andehavioral modifications. These behavioral changes, in healthy sub-

ects, have included modulation of affective states, modificationsn the evaluation of emotional visual stimuli, auditory attentionnd linguistic processing. In patients, behavioral modulationsssociated with rtfMRI-BCI training have included the reduction ofinnitus, modulation of the accuracy in recognizing emotional faces,emporal reductions of nicotine craving, reduction in the severityf symptoms of depression, and motor performance modulation.

All previously discussed studies on rtfMRI trained subjects tochieve self-regulation of single brain regions, taking advantagef one of the hallmark features of fMRI, i.e., its high spatial reso-ution and its capability to assess circumscribed brain regions foreurofeedback.

However, this very same feature leads to potential issues thato beyond theoretical aspects as they could have important clinicalmplications. Complex cognitive processes are not considered to beimited to the activation of a single, unique brain area. In fact, mostf the processes so far examined with rtfMRI studies (e.g., emo-ion processing, motor response, language, pain perception, etc.)nclude the coordinated activity of several brain regions.

Hence, an observed behavioral modification induced by single-OI regulation would certainly increase our knowledge about the

unctionality of the specifically trained brain region, but most prob-bly it will not be reflecting the natural functioning of the brain.

Similarly, considering that normal brain functioning involveshe concerted action of multiple brain networks, it is not surprisinghat abnormal brain states, as in several neuronal and psychiatricisorders, are thought to arise from the uncoordinated activationf distributed brain regions, or from their impaired functional cou-ling (Friston & Frith, 1995; Honey et al., 2005; Just, Cherkassky,eller, Kana, & Minshew, 2007; Noonan, Haist, & Muller, 2009;ang et al., 2007; Zhang et al., 2010).Could rtfMRI-BCI be used to enhance or modulate neural con-

ectivity, and not only the activity of single ROIs? To answer thatuestion, we will discuss new developments for the direct modu-

ation of the functional neural connectivity, and for the modulationf brain networks.

. Brain connectivity enhancement with rtfMRI

.1. Direct self-modulation of brain functional connectivity withtfMRI

FMRI analysis can reveal information about the degree by whichomponents of large-scale neural systems are functionally coupledogether during a specific task or resting state. Several method-logical approaches exist for the evaluation of neural couplingith fMRI signals, categorized as “data driven” versus “hypothe-

is driven”, or based on the simple correlation of the BOLD signal“functional connectivity”) versus others that include the analy-is of the causation and direction of the influence between brainegions (“effective connectivity”) (for reviews on the different tech-iques for the assessment of functional brain connectivity, pleaseee: (Li, Guo, Nie, Li, & Liu, 2009; Rogers, Morgan, Newton, & Gore,007). In traditional fMRI studies, the analysis of neuronal con-ectivity is performed “offline”. However, if this information coulde processed and provided in real-time, the rtfMRI-BCI for “neuralonnectivity modulation” could become a reality.

Interestingly, the idea that rtfMRI could be used to enhancerain connectivity does not come only from a theoretical point ofiew. A few previous rtfMRI-BCI works explored whether the con-rol acquired over a particular single brain area is associated with

ology 95 (2014) 4– 20

changes in brain network connectivity. Post hoc analysis of theseexperiments suggest that the successful modulation of a single ROI(insula cortex, Lee et al., 2011; Veit et al., 2012; amygdala, Zotevet al., 2011; and ACC, Hamilton et al., 2011) is accompanied withmodulations in the connectivity of brain networks.

Furthermore, one study explored this phenomenon in a clini-cal population, i.e. schizophrenia patients (Ruiz et al., 2013), usingGranger causality modeling (GCM), methodology that examines“effective connectivity” using temporal information in one or moretime-series of signals from a certain brain region to predict signaltime courses in another (Abler et al., 2006; Seth, 2005, 2010). Afterlearned self-regulation of insula cortex, the effective connectivityof the emotional network was enhanced. This change was reflectedin the larger causal density (CD: the fraction of interactions amongROIs in a network that are causally significant) of the network inthe strongest session of regulation (of the last day of training) com-pared with the weakest session of regulation (of the first day oftraining) (Fig. 2a and b).

These studies suggest that rtfMRI could be used to build a moreefficient neural pathway and enhance brain connectivity. However,it is important to point out that these enhancements of brain con-nections were achieved as a by-product of single ROI self-regulationtraining (and not by a “direct” training of the brain connectivity),and therefore the particular regions of the brain involved could notbe determined a priori.

To address this problem, we performed the first study thataimed to “directly” train subjects to achieve self-control of the func-tional connectivity of two brain areas brain regions with rtfMRI-BCI(Ruiz et al., 2011). A group of healthy participants were trainedto increase the functional connectivity between fronto-temporalcortex during self-regulation blocks in a few scanning sessions ofrtfMRI, by contingent visual feedback of the correlation coefficientbetween inferior frontal gyrus (IFG) and superior temporal gyrus(STG).

The equation to compute the magnitude of feedback signal was:

Magnitude of feedback

= (TOT BOLDregulation − TOT BOLDbaseline) × (1 + EC)

where TOT BOLD = (BOLD in ROI1 + BOLD in ROI2) is the total BOLDin the two ROIs, ROI1 is the left IFG, ROI2 is the left STG, and EC isthe correlation coefficient derived from the BOLD time-series in thetwo ROIs computed from a sliding window of current and past timepoints. Correlation-methods perform with high sensitivity estimat-ing the presence of a network connection with fMRI signals (Smithet al., 2011) and have been extensively documented for the off-line analysis of brain network connectivity (e.g., Meyer-Lindenberget al., 2005; Satterthwaite et al., 2010). A number of other pos-sibilities for estimating brain connectivity exist (e.g., computingbivariate Granger causality measures), but these measures requiremore samples of data to be reasonably accurate for their use withrtfMRI-BCI.

Results of this experiment showed that participants trained withcontingent feedback were able to learn self-regulation of the con-nectivity between IFG and STG after a few sessions of training.Secondly, we explored the behavioral modification induced by thetraining in an automatic semantic priming task performed before(pre-test) and after (post-test) the rtfMRI training sessions, as it hasbeen postulated that the neural bases of this phenomenon includethe coupling of fronto-temporal areas (Sass, Krach, Sachs, & Kircher,2009a; Sass, Sachs, Krach, & Kircher, 2009b). As hypothesized, a

noticeable “enhancement of the priming effect” was observed fol-lowing blocks of learned IFG-STG connectivity.

In summary, we showed for the first time that it is possibleto train subjects to enhance the functional connectivity of two

S. Ruiz et al. / Biological Psychology 95 (2014) 4– 20 15

Fig. 2. Group analysis of effective connectivity changes due to fMRI-BCI training using granger causality modeling (GCM). (A) Directed influences of the emotional networkduring self-regulation, in the weakest (in the first day of training) and the strongest regulation session (last day of training). Red arrows indicate bidirectional influencesbetween ROIs. (B) Number of outgoing (outflow) and incoming (inflow) connections of each ROI of the emotional network during self-regulation (the graph depicts theinformation contained in figure (A) MidFG: left middle frontal gyrus, MPFC: left medial prefrontal cortex, ACC: left anterior cingulate cortex, L Insula: left anterior insula, RI

S

caca

3

gvasicbatte

iint

nsula: right anterior insula, SMG: left supra marginal gyrus).

ource: Adapted with permission from Ruiz et al. (2013).

ircumscribed brain areas using contingent rtfMRI feedback. Thepplication of this new “functional connectivity enhancement”,ould offer a novel non-invasive method to modulate abnormallyctivated networks in brain disorders.

.2. Brain network feedback with rtfMRI

Previous sections have discussed the methodological back-round and applications of real-time fMRI, focusing on the topics ofolitional control of hemodynamic activity in circumscribed brainreas and functional connectivity between brain regions. Althoughuch approaches have been used in many studies, they are limitedn terms of their ability to enable modulation of the entire neuralircuitry involved in any brain function. After all, and as mentionedefore, brain’s involvement in perception, cognition, emotion andction is known to encompass specific networks of spatially dis-ributed brain regions. An answer to such a challenge has becomehe target of further developments in this field, which will be cov-red in this final section.

Decoding brain states requires knowledge of how information

s encoded in spatially distributed regions and in temporally vary-ng patterns. Conventional neuroimaging works by first acquiringeuroelectric or hemodynamic activity from multiple regions ofhe brain repeatedly while the subject performs different mental

tasks or is presented different stimuli, followed by a statistical com-parison of the activity in each region separately. This approach,called “univariate analysis”, ignores dynamic interactions betweenbrain regions. In contrast, recent developments (Cox & Savoy,2003; Haxby et al., 2001; Haynes & Rees, 2005; Sidtis, Strother, &Rottenberg, 2003) have demonstrated that the sensitivity to decod-ing brain activity is greatly enhanced when activity in spatiallydistributed regions of the brain and their temporal variations areincluded in the analysis by employing “pattern recognition” tech-niques.

Several pattern recognition techniques have already beenapplied to fMRI data, including linear discriminant analy-sis (LaConte et al., 2003), naïve Bayes (Pereira, Mitchell, &Botvinick, 2009), support vector machine (SVM) (LaConte, Strother,Cherkassky, Anderson, & Hu, 2005), neural networks (Hanson,Matsuka, & Haxby, 2004), canonical variates analysis (Mourao-Miranda, Reynaud, McGlone, Calvert, & Brammer, 2006) and fisherlinear discriminant (Shaw et al., 2003). SVM is probably themost widely used approach to predict brain states from fMRIsignals. Studies have reported the superior performance of SVM

in comparison to other existing methods of pattern classifica-tion (LaConte et al., 2003, 2005; Martinez-Ramon, Koltchinskii,Heileman, & Posse, 2006; Shaw et al., 2003; Strother et al.,2004).

16 S. Ruiz et al. / Biological Psychology 95 (2014) 4– 20

Fig. 3. A real-time fMRI brain state classification system. The real-time fMRI brain state classification system comprises of the following subsystems: (1) image acquisitionsubsystem, which is a modified version of the standard echo-planar imaging (EPI) sequence written in C and executed on the scanner host computer and (2) fMRI-BCIsubsystem, which performs image preprocessing, brain state classification and visual feedback, implemented in C and Matlab scripts (Mathworks, Natwick, MA) and executedo ed im

S

spfbafdmbhiTfitbdsfippapta&

assdi

n a 64-bit Windows desktop. A Perl-script on the scanner host transfers the acquir

ource: Adapted with permission from Sitaram et al. (2011b).

How does an SVM classify brain signals? In a binary SVM clas-ifier, where two brain states need to be classified (e.g., brain stateertaining to left hand versus right hand movements), the first taskor the scientist is to represent the fMRI signals from all brain voxelseing measured, corresponding to each instance of a brain state, as

pair of corresponding input vectors; the first vector is called theeature vector and the second, the labels vector. From this inputata, each instance of a brain state is represented as a point in aulti-dimensional state space. Typically, fMRI signals from whole

rain images will have more than 60,000 voxels, resulting in aigh-dimensional feature space. SVM classification is performed

n two distinct steps: 1, classifier training and 2, classifier testing.he objective of the training procedure is for the SVM algorithm tond a high-dimensional plane, called the “hyperplane”, to separatehe points in the feature space corresponding to the two distinctrain states (Schölkopf & Smola, 2001). During the testing proce-ure, new fMRI signals are presented to the SVM classifier withoutpecifying the brain state the fMRI signals correspond to. The classi-er then determines the most probable brain state pertaining to theresented data based on which side of the hyperplane the featureoint is located. Accuracy, precision and other performance metricsre then computed based on the number of correct and incorrectredictions from several test data. For more details on the SVM pat-ern classification one can refer to a number of excellent tutorialsnd articles on the topic (Schölkopf & Smola, 2001; Shawe-Taylor

Cristianini, 2004; Steinwart & Christmann, 2008).Pattern analysis has been used for understanding the spatial

nd temporal neural patterns of different brain functions and their

tates. Recent studies that used this approach are: automated clas-ification of sleep stages (Tagliazucchi et al., 2012), spontaneousecisions and hidden intentions (Lages & Jaworska, 2012), track-

ng the unconscious generation of free decision (Bode et al., 2011),

ages after every scan (at an interval of 1.5 s) to the fMRI-BCI computer.

memory recall (Polyn, Natu, Cohen, & Norman, 2005), lie detection(Davatzikos et al., 2005), visual perception (Xu, Jiang, Ma, Yang, &Weng, 2012), task related intentions (Haynes et al., 2007), fear per-ception (Pessoa & Padmala, 2005), and emotion detection (Sitaramet al., 2011a), to name a few.

How can pattern classification be incorporated in real-time fMRIstudies? LaConte et al. (2007) were the first to demonstrate the real-time implementation of pattern classification of brain states fromfMRI signals. Their study demonstrated binary decoding and feed-back of motor or cognitive states of the brain. An additional use ofpattern analysis in real-time fMRI studies would be to determinewhich brain areas are particularly influential in volitional regula-tion and the behavior that is being targeted. Lee and colleaguesattempted to address the above question by applying their novelEffect Mapping method to investigate the effect of neurofeedbacktraining on functional reorganization of the emotional network. InfMRI signals, the Effect Mapping algorithm measures the effect ofeach brain voxel on the classifier output by considering the mutualinformation between the BOLD activations at each voxel and theoutput, and the weight value of the voxel in the estimated SVMweight vector. This methodology has demonstrated high classi-fication accuracy decoding mental states, and therefore could bean alternative method in the multivariate analysis of fMRI databy considering both discriminability and data distribution (Lee,Halder, Kubler, Birbaumer, & Sitaram, 2010).

The authors found that subjects who learn to volitionally con-trol left anterior insula with real-time feedback develop greaternumber of functional connections from left insula to other emotion-

related brain regions, particularly, right insula, amygdala, medialprefrontal cortex, anterior cingulate cortex, and posterior pari-etal regions. Sitaram et al. (2011b) further incorporated the EffectMapping technique into their real-time fMRI feedback system to

S. Ruiz et al. / Biological Psyc

Fig. 4. Effect maps and classification accuracy of emotional brain states. (A, b,c) Effect maps generated from single-subject whole brain SVM classification. Thefigures show the most discriminating voxels for: (a) happy vs. disgust classifica-tion (representative participant), (b) happy vs. sad classification and (c) disgust vs.sad classification (representative participant). Second level threshold applied was0.5, slices shown are in the range, Z (MNI coordinates) = −11 to +10 in steps of 3units. Brain regions: oMFC—orbital medial frontal cortex; arMFC—anterior rostralMFC; OFC—orbitofrontal cortex; ACC—anterior cingulate cortex; and PCC—posteriorcingulate cortex. (d) Classification accuracy of 24 regions of interest (ROIs) in asingle-subject ROI classification from 12 participants performing happy and dis-gust motor imagery. ROIs: Subgenual—subgenual cingulate (BA25); Nac—nucleusaccumbens; PVC—primary visual cortex; DLPFC—dorsolateral prefrontal cortex;LiG—lingual gyrus; VLPFC—ventrolateral prefrontal cortex; STG—superior tempo-rf

S

eisettpMd

tS

al gyrus; MiTG—middle temporal gyrus; MFG—medial frontal gyrus; SFG—superiorrontal gyrus; and MiFG—middle frontal gyrus.

ource: Adapted with permission from Sitaram et al. (2011b).

nhance online classification. They showed for the first time thatt is possible to classify multiple emotional states (i.e., happiness,adness and disgust) in real-time, and used the technique fornhancing volitional control of the emotion circuitry. Fig. 3 showshe general design of the real-time brain state classification sys-em used in this study (the general design is common to otherattern-classification feedback studies). Fig. 4 displays the Effectaps and classification accuracy results for the classification of

ifferent emotional states.In what could be the first most compelling use of real-

ime fMRI pattern classification for neuroscience investigation,hibata, Watanabe, Sasaki, and Kawato (2011) addressed a very

hology 95 (2014) 4– 20 17

controversial question of whether adult early visual cortex is suf-ficiently plastic to allow visual perceptual learning. To this end,authors implemented an online decoder to identify voxels in earlyvisual cortex (V1/V2) corresponding to three distinct Gabor patchgratings differing by 60◦ orientation from each other. The feed-back signal represented the likelihood of these voxels predictingthe perception of one of the patches, without instructing theparticipants about what the feedback signal represented. Duringneurofeedback training participants were rewarded to self-inducethe target spatial distribution, by the principle of operant learning.The study showed that following neurofeedback training, partic-ipants improve perceptual sensitivity to the target grating to theexception of the other two gratings. This inventive study underlinesthe potential of the tool in studying cause-and-effect relationships.As the authors note: It <such a technique> can “incept” a personto acquire new learning, skills, or memory, or possibly to restoreskills or knowledge that has been damaged through accident, dis-ease, or aging, without a person’s awareness of what is learned ormemorized.

Despite these exciting developments in scientific application ofreal-time fMRI pattern feedback, we are still far away from bringingsuch a technology to clinical use. One major technical hurdle is thatexisting approaches to online pattern classifier are built specificallyfor an individual based on his fMRI signals alone. In other words, thecurrent breed of online classifiers is incapable of generalizing theirdecoding capabilities to other subjects; they are highly “subject-dependent”. This state-of-art leads to two adverse implications: (1)the need to acquire large amount of preliminary functional data totrain a pattern classifier, and more importantly and (2) an inherentconceptual error in aiming to train a patient with a pattern classifierwhich is first built with the subject’s own abnormal brain signals.To overcome the above difficulties, we have recently attemptedto develop a subject-independent pattern classifier (SIC) from alarger set of group data from many individuals representing a cer-tain demographic population (e.g., healthy individuals of similarage, educational background, and specific cognitive or emotionalabilities). With such a pre-developed SIC classifier model, one canin principle, train new individuals or patients to help them bring tonormalcy a specific brain function that they are disadvantaged with.

In our most recent work (Rana et al., 2012b) to implementonline subject-independent classification, we have adopted func-tional preprocessing steps of co-registration and normalization tobe able to function in a real-time mode. By doing so, the fMRI sig-nals arriving to the classifier are automatically transformed into astandard space (Montreal Neurological Institute, MNI). This allowsfor reducing inter-subject differences in brain size and shape, thusfacilitating the classifier to generalize its predictions much better.

In a proof-of-principle study, we developed a SIC-classifiermodel, built on fMRI signals from 12 subjects who performed voli-tional regulation of emotion of happy and disgust (Rana, Ruiz,Gupta, Niels, & Sitaram, 2012a). We then applied this classifier toprovide real-time feedback to a number of new subjects, to trainthem to learn volitional regulation of happy and disgust emotions,aided by feedback from specific “emotion circuitries” pertaining tothe two states. Our results show that new subjects can learn togradually activate the right spatial pattern. As a consequence offeedback training, their emotional regulation improves with moretraining runs, and volitional control of a particular emotion (e.g.,happiness) has a facilitatory effect on a corresponding emotionalpriming task.

4. fMRI-BCI limitations and future directions

The previous studies serve to illustrate the exciting moment forrtfMRI-BCI. However, for both single ROI and brain connectivity

1 l Psych

fl

rtlegbnutatmrthfdi

estdto

qohfd

wld2a

mia

tFecans

momwodah

cooh

8 S. Ruiz et al. / Biologica

eedback (functional connectivity and network) with rtfMRI, someimitations and unsolved questions need to be mentioned.

For most of these experiments, the number of participantsemains low, probably due to the complicated, and often highlyime-consuming setting needed for a rtfMRI-BCI training (particu-arly if compared with “simple” fMRI experiments). And althoughfforts have been made to include control conditions and controlroups, there is no clear consensus regarding which of these is theest methodology for providing a reliable control condition (e.g.,o feedback, sham feedback, feedback from a control brain region,p-regulation versus down-regulation, etc.). On the other hand,he underling mechanisms of operant learning by rtfMRI trainingre not yet clearly understood. This phenomenon is aggravated byhe fact that not every study has included detailed reports of the

ental strategies used by the participants to achieve brain self-egulation, and no deep exploration has been made on the effecthat the initial instructions and guidance by the researchers canave in the learning process. Similarly, the influence on learning of

actors like motivation, attention, or the psychological and neuralifferences between “learners” and “non-learners” has been rarely

nvestigated.Furthermore, with few exceptions (deCharms et al., 2005; Haller

t al., 2010; Hamilton et al., 2011; Veit et al., 2012), most of thesetudies have focused in the “up-regulation” of brain signals, andherefore the question of whether subjects can voluntarily learn toecrease the activation of a particular brain area or network (factorhat can be of importance for brain disorders in which an abnormalver-activation underlies the psychopathology), remains unsolved.

Regarding the studies on clinical populations, many crucialuestions need to be answered before the clinical implementationf this methodology becomes a reality. In fact, very few studiesave explored whether brain self-regulation persists immediately

ollowing training if no feedback is provided, and if at all, withissimilar results (Ruiz et al., 2013; Sitaram et al., 2012).

Along the same lines, few preliminary studies have assessedhether the learned capability to self-regulate or behavioral modu-

ations and/or symptom alleviation will persist in the followingays after the rtfMRI-BCI training (Haller et al., 2010; Yoo et al.,008). So far, no study has reported the analysis of these aspects in

clinically significant long-term.Future studies should evaluate as to which of the three feedback

ethods, namely, single-ROI, functional connectivity and network,s appropriate for the given type of study i.e., neuroscience researchnd clinical treatment.

We expect single ROI neurofeedback to continue as a toolo determine links between specific brain regions and behavior.or therapeutic purposes, if large controlled studies confirm itsfficacy, it could offer a non-invasive approach for modulating cir-umscribed dysfunctional brain areas for intractable neurologicalnd psychiatric disorders, similar to new (although invasive) tech-iques for the stimulation of specific brain areas, i.e. deep braintimulation.

As stated before, network connectivity feedback for its partay better represent brain physiology, having also the advantage

f covering the whole brain. For clinical applications, it could beore suitable than single-ROI feedback for diseases of “brain net-ork connectivity” (i.e., schizophrenia, autism disorder, among

thers). Similarly, pattern classification of whole brain data and theevelopment of subject-independent pattern classifiers could offer

revolutionary approach to aid patients to achieve a “desirableealthy brain state”.

The combined use of these techniques in the same experiment

ould provide new insights into brain functioning, and shed lightn their distinct uses in clinical applications. Further developmentsf both single-ROI and brain circuitry feedback will inform us as toow these approaches can complement each other, thus opening

ology 95 (2014) 4– 20

exciting possibilities for scientific investigations in systems neuro-science, neuropsychology, psychiatry and neurology. It remains tobe seen how further progresses in these fields fulfill these promises.

Acknowledgments

This work was supported by the Deutschen Forschungsgemein-schaft (BI 195/64-1, BI 195/59-1, BI 195/65-1 and BI 195/56-1),the Centre for Integrative Neuroscience (CIN), Tübingen, Germany(Pool-Project 2011-08), and Comisión Nacional de InvestigaciónCientífica y Tecnológica de Chile (Conycit) through Fondo Nacionalde Desarrollo Científico y Tecnológico Fondecyt (Fondecyt: projectno. 11121153).

References

Abler, B., Roebroeck, A., Goebel, R., Hose, A., Schonfeldt-Lecuona, C., Hole, G., & Wal-ter, H. (2006). Investigating directed influences between activated brain areasin a motor-response task using fMRI. Magnetic Resonance Imaging, 24, 181–185.

Bandettini, P. A., Wong, E. C., Hinks, R. S., Tikofsky, R. S., & Hyde, J. S. (1992). Timecourse EPI of human brain function during task activation. Magnetic Resonance inMedicine: Official Journal of the Society of Magnetic Resonance in Medicine/Societyof Magnetic Resonance in Medicine, 25, 390–397.

Berman, B. D., Horovitz, S. G., Venkataraman, G., & Hallett, M. (2012). Self-modulation of primary motor cortex activity with motor and motor imagerytasks using real-time fMRI-based neurofeedback. NeuroImage, 59, 917–925.

Birbaumer, N. (2006). Breaking the silence: Brain–computer interfaces (BCI) forcommunication and motor control. Psychophysiology, 43, 517–532.

Birbaumer, N., & Cohen, L. G. (2007). Brain–computer interfaces: Communicationand restoration of movement in paralysis. J Physiol, 579, 621–636.

Bode, S., He, A. H., Soon, C. S., Trampel, R., Turner, R., & Haynes, J. D. (2011). Trackingthe unconscious generation of free decisions using ultra-high field fMRI. PLoSONE, 6, e21612.

Buyukturkoglu, K., Ruiz, S., Rana, M., Wächter, T., Soekadar, S. R., Hackley, S. A.,Birbaumer, N., & Sitaram, R. (2012). Real-time fMRI-brain computer interfacesfor rehabilitation of Parkinson’s disease patients. New Orleans, USA: Society forNeuroscience.

Caria, A., Sitaram, R., & Birbaumer, N. (2012). Real-time fMRI: A tool for local brainregulation. Neuroscientist, 18, 487–501.

Caria, A., Sitaram, R., Veit, R., Begliomini, C., & Birbaumer, N. (2010). Volitional controlof anterior insula activity modulates the response to aversive stimuli. A real-timefunctional magnetic resonance imaging study. Biological Psychiatry, 68, 425–432.

Caria, A., Veit, R., Sitaram, R., Lotze, M., Weiskopf, N., Grodd, W., & Birbaumer, N.(2007). Regulation of anterior insular cortex activity using real-time fMRI. Neu-roImage, 35, 1238–1246.

Chiew, M., LaConte, S. M., & Graham, S. J. (2012). Investigation of fMRI neurofeedbackof differential primary motor cortex activity using kinesthetic motor imagery.NeuroImage, 61, 21–31.

Cox, D. D., & Savoy, R. L. (2003). Functional magnetic resonance imaging (fMRI)“brain reading”: Detecting and classifying distributed patterns of fMRI activityin human visual cortex. NeuroImage, 19, 261–270.

Cox, R. W., & Jesmanowicz, A. (1999). Real-time 3D image registration for func-tional MRI. Magnetic Resonance in Medicine: Official Journal of the Society ofMagnetic Resonance in Medicine/Society of Magnetic Resonance in Medicine, 42,1014–1018.

Cox, R. W., Jesmanowicz, A., & Hyde, J. S. (1995). Real-time functional magneticresonance imaging. Magnetic Resonance in Medicine: Official Journal of the Societyof Magnetic Resonance in Medicine/Society of Magnetic Resonance in Medicine, 33,230–236.

Davatzikos, C., Ruparel, K., Fan, Y., Shen, D. G., Acharyya, M., Loughead, J. W., Gur,R. C., & Langleben, D. D. (2005). Classifying spatial patterns of brain activitywith machine learning methods: Application to lie detection. NeuroImage, 28,663–668.

deCharms, R. C., Christoff, K., Glover, G. H., Pauly, J. M., Whitfield, S., & Gabrieli, J. D.(2004). Learned regulation of spatially localized brain activation using real-timefMRI. NeuroImage, 21, 436–443.

deCharms, R. C., Maeda, F., Glover, G. H., Ludlow, D., Pauly, J. M., Soneji, D., Gabrieli,J. D., & Mackey, S. C. (2005). Control over brain activation and pain learned byusing real-time functional MRI. Proceedings of the National Academy of Sciencesof the United States of America, 102, 18626–18631.

Friston, K. J., & Frith, C. D. (1995). Schizophrenia: A disconnection syndrome? ClinicalNeuroscience, 3, 89–97.

Gembris, D., Taylor, J. G., Schor, S., Frings, W., Suter, D., & Posse, S. (2000). Functionalmagnetic resonance imaging in real time (FIRE): Sliding-window correlationanalysis and reference-vector optimization. Magnetic Resonance in Medicine:

Official Journal of the Society of Magnetic Resonance in Medicine/Society of MagneticResonance in Medicine, 43, 259–268.

Goebel, R. (2012). BrainVoyager—past, present, future. NeuroImage, 62, 748–756.Haller, S., Birbaumer, N., & Veit, R. (2010). Real-time fMRI feedback training may

improve chronic tinnitus. European Radiology, 20, 696–703.

l Psyc

H

H

H

H

H

H

J

J

K

L

L

L

L

L

L

L

L

L

L

L

M

M

M

M

N

P

S. Ruiz et al. / Biologica

amilton, J. P., Glover, G. H., Hsu, J. J., Johnson, R. F., & Gotlib, I. H. (2011). Modulationof subgenual anterior cingulate cortex activity with real-time neurofeedback.Human brain mapping, 32, 22–31.

anson, S. J., Matsuka, T., & Haxby, J. V. (2004). Combinatorial codes in ventraltemporal lobe for object recognition: Haxby (2001) revisited: Is there a “face”area? NeuroImage, 23, 156–166.

axby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001).Distributed and overlapping representations of faces and objects in ventral

temporal cortex. Science, 293, 2425–2430.aynes, J. D., & Rees, G. (2005). Predicting the stream of consciousness from activity

in human visual cortex. Current Biology, 15, 1301–1307.aynes, J. D., Sakai, K., Rees, G., Gilbert, S., Frith, C., & Passingham, R. E. (2007).

Reading hidden intentions in the human brain. Current Biology, 17, 323–328.oney, G. D., Pomarol-Clotet, E., Corlett, P. R., Honey, R. A., McKenna, P. J., Bullmore,

E. T., & Fletcher, P. C. (2005). Functional dysconnectivity in schizophre-nia associated with attentional modulation of motor function. Brain, 128,2597–2611.

ohnson, K. A., Hartwell, K., LeMatty, T., Borckardt, J., Morgan, P. S., Govindarajan,K., Brady, K., & George, M. S. (2012). Intermittent “real-time” fMRI feedbackis superior to continuous presentation for a motor imagery task: A pilot study.Journal of Neuroimaging: Official Journal of the American Society of Neuroimaging,22, 58–66.

ust, M. A., Cherkassky, V. L., Keller, T. A., Kana, R. K., & Minshew, N. J. (2007). Func-tional and anatomical cortical underconnectivity in autism: Evidence from anFMRI study of an executive function task and corpus callosum morphometry.Cerebral cortex, 17, 951–961.

otchoubey, B., Strehl, U., Uhlmann, C., Holzapfel, S., Konig, M., Froscher, W.,Blankenhorn, V., & Birbaumer, N. (2001). Modification of slow cortical poten-tials in patients with refractory epilepsy: A controlled outcome study. Epilepsia,42, 406–416.

aConte, S., Anderson, J., Muley, S., Ashe, J., Frutiger, S., Rehm, K., Hansen, L. K.,Yacoub, E., Hu, X., Rottenberg, D., & Strother, S. (2003). The evaluation of prepro-cessing choices in single-subject BOLD fMRI using NPAIRS performance metrics.NeuroImage, 18, 10–27.

aConte, S., Strother, S., Cherkassky, V., Anderson, J., & Hu, X. (2005). Support vectormachines for temporal classification of block design fMRI data. NeuroImage, 26,317–329.

aConte, S. M., Peltier, S. J., & Hu, X. P. (2007). Real-time fMRI using brain-stateclassification. Human brain mapping, 28, 1033–1044.

ages, M., & Jaworska, K. (2012). How predictable are “Spontaneous Decisions”and “Hidden Intentions”?, comparing classification results based on previousresponses with multivariate pattern analysis of fMRI BOLD signals. Frontiers inPsychology, 3, 56.

ee, J. H., Ryu, J., Jolesz, F. A., Cho, Z. H., & Yoo, S. S. (2009). Brain–machine inter-face via real-time fMRI: Preliminary study on thought-controlled robotic arm.Neuroscience Letters, 450, 1–6.

ee, S., Halder, S., Kubler, A., Birbaumer, N., & Sitaram, R. (2010). Effective functionalmapping of fMRI data with support-vector machines. Human Brain Mapping, 31,1502–1511.

ee, S., Ruiz, S., Caria, A., Veit, R., Birbaumer, N., & Sitaram, R. (2011). Detection ofcerebral reorganization induced by real-time fMRI feedback training of insulaactivation: A multivariate investigation. Neurorehabilitation and Neural Repair,25, 259–267.

i, K., Guo, L., Nie, J., Li, G., & Liu, T. (2009). Review of methods for functionalbrain connectivity detection using fMRI. Computerized Medical Imaging andGraphics: The Official Journal of the Computerized Medical Imaging Society, 33,131–139.

i, X., Hartwell, K. J., Borckardt, J., Prisciandaro, J. J., Saladin, M. E., Morgan, P.S., Johnson, K. A., Lematty, T., Brady, K. T., & George, M. S. (2012). Volitionalreduction of anterior cingulate cortex activity produces decreased cue crav-ing in smoking cessation: A preliminary real-time fMRI study. Addict Biology,http://dx.doi.org/10.1111/j.1369-1600.2012.00449.x [Epub ahead of print]

inden, D. E., Habes, I., Johnston, S. J., Linden, S., Tatineni, R., Subramanian, L., Sorger,B., Healy, D., & Goebel, R. (2012). Real-time self-regulation of emotion networksin patients with depression. PLoS ONE, 7, e38115.

ogothetis, N. K. (2008). What we can do and what we cannot do with fMRI. Nature,453, 869–878.

artinez-Ramon, M., Koltchinskii, V., Heileman, G. L., & Posse, S. (2006). fMRI patternclassification using neuroanatomically constrained boosting. NeuroImage, 31,1129–1141.

athiak, K., & Posse, S. (2001). Evaluation of motion and realignment for functionalmagnetic resonance imaging in real time. Magnetic Resonance in Medicine: Offi-cial Journal of the Society of Magnetic Resonance in Medicine/Society of MagneticResonance in Medicine, 45, 167–171.

eyer-Lindenberg, A. S., Olsen, R. K., Kohn, P. D., Brown, T., Egan, M. F., Weinberger,D. R., & Berman, K. F. (2005). Regionally specific disturbance of dorsolateralprefrontal-hippocampal functional connectivity in schizophrenia. Archives ofGeneral Psychiatry, 62, 379–386.

ourao-Miranda, J., Reynaud, E., McGlone, F., Calvert, G., & Brammer, M. (2006).The impact of temporal compression and space selection on SVM analysis ofsingle-subject and multi-subject fMRI data. NeuroImage, 33, 1055–1065.

oonan, S. K., Haist, F., & Muller, R. A. (2009). Aberrant functional connectivity inautism: Evidence from low-frequency BOLD signal fluctuations. Brain research,1262, 48–63.

ereira, F., Mitchell, T., & Botvinick, M. (2009). Machine learning classifiers and fMRI:A tutorial overview. NeuroImage, 45, S199–S209.

hology 95 (2014) 4– 20 19

Pessoa, L., & Padmala, S. (2005). Quantitative prediction of perceptual decisions dur-ing near-threshold fear detection. Proceedings of the National Academy of Sciencesof the United States of America, 102, 5612–5617.

Polyn, S. M., Natu, V. S., Cohen, J. D., & Norman, K. A. (2005). Category-specific corticalactivity precedes retrieval during memory search. Science, 310, 1963–1966.

Posse, S., Binkofski, F., Schneider, F., Gembris, D., Frings, W., Habel, U., Salloum, J. B.,Mathiak, K., Wiese, S., Kiselev, V., Graf, T., Elghahwagi, B., Grosse-Ruyken, M. L.,& Eickermann, T. (2001). A new approach to measure single-event related brainactivity using real-time fMRI: Feasibility of sensory, motor, and higher cognitivetasks. Human brain mapping, 12, 25–41.

Posse, S., Fitzgerald, D., Gao, K., Habel, U., Rosenberg, D., Moore, G. J., & Schneider, F.(2003). Real-time fMRI of temporolimbic regions detects amygdala activationduring single-trial self-induced sadness. NeuroImage, 18, 760–768.

Posse, S., Wiese, S., Gembris, D., Mathiak, K., Kessler, C., Grosse-Ruyken, M. L., Elghah-wagi, B., Richards, T., Dager, S. R., & Kiselev, V. G. (1999). Enhancement ofBOLD-contrast sensitivity by single-shot multi-echo functional MR imaging.Magnetic Resonance in Medicine: Official Journal of the Society of Magnetic Res-onance in Medicine/Society of Magnetic Resonance in Medicine, 42, 87–97.

Rana, M., Ruiz, S., Gupta, N., Niels, B., & Sitaram, R. (2012). Population based patternclassification of emotional brain states using fMRI signal. In 1st Swiss real-timefMRI Neurofeedback Conference Zurich, Switzerland.

Rana, M., Ruiz, S., Gupta, N., Suslow, T., Birbaumer, N., & Sitaram, R. (2012). Popula-tion based pattern classification of emotional brain states using fMRI signals. InAnnual Meeting of the Organization for Human Brain Mapping Beijing, China.

Rogers, B. P., Morgan, V. L., Newton, A. T., & Gore, J. C. (2007). Assessing func-tional connectivity in the human brain by fMRI. Magnetic Resonance Imaging,25, 1347–1357.

Rota, G., Sitaram, R., Veit, R., Erb, M., Weiskopf, N., Dogil, G., & Birbaumer, N.(2009). Self-regulation of regional cortical activity using real-time fMRI: Theright inferior frontal gyrus and linguistic processing. Human brain mapping, 30,1605–1614.

Ruiz, S., Lee, S., Soekadar, S. R., Caria, A., Veit, R., Kircher, T., Birbaumer, N., & Sitaram,R. (2013). Acquired self-control of insula cortex modulates emotion recogni-tion and brain network connectivity in schizophrenia. Human Brain Mapping,34, 200–212.

Ruiz, S., Rana, M., Sass, K., Kircher, T., Birbaumer, N., & Sitaram, R. (2011). Brainnetwork connectivity and behaviour enhancement: A fMRI-BCI study. In 17thAnnual Meeting of the Organization for Human Brain Mapping Québec City, Canada.

Sass, K., Krach, S., Sachs, O., & Kircher, T. (2009). Lion–tiger–stripes: Neural corre-lates of indirect semantic priming across processing modalities. NeuroImage, 45,224–236.

Sass, K., Sachs, O., Krach, S., & Kircher, T. (2009). Taxonomic and thematic categories:Neural correlates of categorization in an auditory-to-visual priming task usingfMRI. Brain Research, 1270, 78–87.

Satterthwaite, T. D., Wolf, D. H., Loughead, J., Ruparel, K., Valdez, J. N., Siegel, S. J.,Kohler, C. G., Gur, R. E., & Gur, R. C. (2010). Association of enhanced limbicresponse to threat with decreased cortical facial recognition memory responsein schizophrenia. American Journal of Psychiatry, 167, 418–426.

Schölkopf, B., & Smola, A. J. (2001). Learning with kernels. Support vector machines,regularization, optimization, and beyond. Cambridge, MA: The MIT Press.

Seth, A. K. (2005). Causal connectivity of evolved neural networks during behavior.Network, 16, 35–54.

Seth, A. K. (2010). A MATLAB toolbox for Granger causal connectivity analy-sis. Journal of Neuroscience Methods, 182, 262–273. http://dx.doi.org/10.1016/j.jneumeth.2009.11.020 [Epub 2009 Dec 2]

Shaw, M. E., Strother, S. C., Gavrilescu, M., Podzebenko, K., Waites, A., Watson, J.,Anderson, J., Jackson, G., & Egan, G. (2003). Evaluating subject specific prepro-cessing choices in multisubject fMRI data sets using data-driven performancemetrics. NeuroImage, 19, 988–1001.

Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. NewYork: Cambridge University Press.

Shibata, K., Watanabe, T., Sasaki, Y., & Kawato, M. (2011). Perceptual learningincepted by decoded fMRI neurofeedback without stimulus presentation. Sci-ence, 334, 1413–1415.

Sidtis, J. J., Strother, S. C., & Rottenberg, D. A. (2003). Predicting performance fromfunctional imaging data: Methods matter. NeuroImage, 20, 615–624.

Sitaram, R., Caria, A., Veit, R., Gaber, T., Kuebler, A., & Birbaumner, N. (2005).Real-time fMRI based brain–computer interface enhanced by interactive virtualworlds. In Proceedings of the 45th Annual Meeting Society for PsychophysiologicalResearch Lisbon, Portugal.

Sitaram, R., Lee, S., Ruiz, S., & Birbaumer, N. (2011). Real-time regulation and detec-tion of brain states from fMRI signals. In R. Coben, & J. Evans (Eds.), Neurofeedbackand neuromodulation techniques and applications (pp. 227–253). London: ElsevierInc.

Sitaram, R., Lee, S., Ruiz, S., Rana, M., Veit, R., & Birbaumer, N. (2011). Real-timesupport vector classification and feedback of multiple emotional brain states.NeuroImage, 56, 753–765.

Sitaram, R., Veit, R., Stevens, B., Caria, A., Gerloff, C., Birbaumer, N., & Hummel,F. (2012). Acquired control of ventral premotor cortex activity by feedbacktraining: An exploratory real-time FMRI and TMS study. Neurorehabilitation andNeural Repair, 26, 256–265.

Smith, S. M., Miller, K. L., Salimi-Khorshidi, G., Webster, M., Beckmann, C. F., Nichols,T. E., Ramsey, J. D., & Woolrich, M. W. (2011). Network modelling methods forFMRI. NeuroImage, 54, 875–891.

Smyser, C., Grabowski, T. J., Frank, R. J., Haller, J. W., & Bolinger, L. (2001). Real-time multiple linear regression for fMRI supported by time-aware acquisition

2 l Psych

S

S

S

S

S

T

V

V

W

0 S. Ruiz et al. / Biologica

and processing. Magnetic Resonance in Medicine: Official Journal of the Society ofMagnetic Resonance in Medicine/Society of Magnetic Resonance in Medicine, 45,289–298.

teinwart, I., & Christmann, A. (2008). Support vector machines. New York: Springer-Verlag.

trehl, U., Leins, U., Goth, G., Klinger, C., Hinterberger, T., & Birbaumer, N. (2006).Self-regulation of slow cortical potentials: A new treatment for children withattention-deficit/hyperactivity disorder. Pediatrics, 118, e1530–e1540.

trother, S., La Conte, S., Kai Hansen, L., Anderson, J., Zhang, J., Pulapura, S., & Rotten-berg, D. (2004). Optimizing the fMRI data-processing pipeline using predictionand reproducibility performance metrics: I. A preliminary group analysis. Neu-roImage, 23(Suppl. 1), S196–S207.

ubramanian, L., Hindle, J. V., Johnston, S., Roberts, M. V., Husain, M., Goebel, R.,& Linden, D. (2011). Real-time functional magnetic resonance imaging neu-rofeedback for treatment of Parkinson’s disease. Journal of Neuroscience, 31,16309–16317.

ulzer, J., Sitaram, R., Blefari, M. L., Birbaumer, N., Kollias, S., Stephan, K. E., & Luft,K. E. A. R. G. (2012). Neurofeedback-mediated self-regulation of the dopami-nergic midbrain. In Annual Meeting of the Society for Neurosciences New Orleans,Louisiana.

agliazucchi, E., von Wegner, F., Morzelewski, A., Borisov, S., Jahnke, K., & Laufs,H. (2012). Automatic sleep staging using fMRI functional connectivity data.NeuroImage, 63, 63–72.

eit, R., Singh, V., Sitaram, R., Caria, A., Rauss, K., & Birbaumer, N. (2012). Usingreal-time fMRI to learn voluntary regulation of the anterior insula in the pres-ence of threat-related stimuli. Social Cognitive and Affective Neuroscience, 7,623–634.

oyvodic, J. T. (1999). Real-time fMRI paradigm control, physiology, and behaviorcombined with near real-time statistical analysis. NeuroImage, 10, 91–106.

ang, K., Liang, M., Wang, L., Tian, L., Zhang, X., Li, K., & Jiang, T. (2007). Alteredfunctional connectivity in early Alzheimer’s disease: A resting-state fMRI study.Human Brain Mapping, 28, 967–978.

ology 95 (2014) 4– 20

Weiskopf, N., Klose, U., Birbaumer, N., & Mathiak, K. (2005). Single-shot compensa-tion of image distortions and BOLD contrast optimization using multi-echo EPIfor real-time fMRI. NeuroImage, 24, 1068–1079.

Weiskopf, N., Veit, R., Erb, M., Mathiak, K., Grodd, W., Goebel, R., & Birbaumer, N.(2003). Physiological self-regulation of regional brain activity using real-timefunctional magnetic resonance imaging (fMRI): Methodology and exemplarydata. NeuroImage, 19, 577–586.

Wolpaw, J., & Wolpaw, E. (2012). Brain-Computer Interfaces: Principles and practice(1st ed., pp. ). New York, USA: Oxford University Press.

Xu, G., Jiang, Y., Ma, L., Yang, Z., & Weng, X. (2012). Similar spatial patterns of neu-ral coding of category selectivity in FFA and VWFA under different attentionconditions. Neuropsychologia, 50, 862–868.

Yoo, S. S., Fairneny, T., Chen, N. K., Choo, S. E., Panych, L. P., Park, H., Lee, S. Y., &Jolesz, F. A. (2004). Brain–computer interface using fMRI: Spatial navigation bythoughts. Neuroreport, 15, 1591–1595.

Yoo, S. S., Guttmann, C. R., Zhao, L., & Panych, L. P. (1999). Real-time adaptivefunctional MRI. NeuroImage, 10, 596–606.

Yoo, S. S., & Jolesz, F. A. (2002). Functional MRI for neurofeedback: Feasibility studyon a hand motor task. Neuroreport, 13, 1377–1381.

Yoo, S. S., Lee, J. H., O’Leary, H., Panych, L. P., & Jolesz, F. A. (2008). Neurofeed-back fMRI-mediated learning and consolidation of regional brain activationduring motor imagery. International Journal of Imaging Systems and Technology,18, 69–78.

Yoo, S. S., O’Leary, H. M., Fairneny, T., Chen, N. K., Panych, L. P., Park, H., & Jolesz, F.A. (2006). Increasing cortical activity in auditory areas through neurofeedbackfunctional magnetic resonance imaging. Neuroreport, 17, 1273–1278.

Zhang, H. Y., Wang, S. J., Liu, B., Ma, Z. L., Yang, M., Zhang, Z. J., & Teng, G. J. (2010).

Resting brain connectivity: Changes during the progress of Alzheimer disease.Radiology, 256, 598–606.

Zotev, V., Krueger, F., Phillips, R., Alvarez, R. P., Simmons, W. K., Bellgowan, P., Drevets,W. C., & Bodurka, J. (2011). Self-regulation of amygdala activation using real-time FMRI neurofeedback. PLoS ONE, 6, e24522.


Recommended