Object-based attention is multisensory: co-activation of anobject’s representations in ignored sensory modalities

Sophie Molholm,1,2 Antigona Martinez,1 Marina Shpaner1,2 and John J. Foxe1,2

1The Cognitive Neurophysiology Laboratory, Program in Cognitive Neuroscience and Schizophrenia, Nathan Kline Institute forPsychiatric Research, 140 Old Orangeburg Road, Orangeburg, New York 10962, USA2Program in Cognitive Neuroscience, Department of Psychology, The City College of the City University of New York, 138th Streetand Convent Avenue, New York, NY 10031, USA

Keywords: auditory, ERPs, human, visual

Abstract

Within the visual modality, it has been shown that attention to a single visual feature of an object such as speed of motion, results inan automatic transfer of attention to other task-irrelevant features (e.g. colour). An extension of this logic might lead one to predictthat such mechanisms also operate across sensory systems. But, connectivity patterns between feature modules across sensorysystems are thought to be sparser to those within a given sensory system, where interareal connectivity is extensive. It is not clearthat transfer of attention between sensory systems will operate as it does within a sensory system. Using high-density electricalmapping of the event-related potential (ERP) in humans, we tested whether attending to objects in one sensory modality resulted inthe preferential processing of that object’s features within another task-irrelevant sensory modality. Clear evidence for cross-sensoryattention effects was seen, such that for multisensory stimuli responses to ignored task-irrelevant information in the auditory andvisual domains were selectively enhanced when they were features of the explicitly attended object presented in the attendedsensory modality. We conclude that attending to an object within one sensory modality results in coactivation of that object’srepresentations in ignored sensory modalities. The data further suggest that transfer of attention from visual-to-auditory featuresoperates in a fundamentally different manner than transfer from auditory-to-visual features, and indicate that visual-objectrepresentations have a greater influence on their auditory counterparts than vice-versa. These data are discussed in terms of‘priming’ vs. ‘spreading’ accounts of attentional transfer.

Introduction

Attention to a single visual feature of an object such as speed ofmotion, results in an automatic transfer of attention to other task-irrelevant features within the visual sensory modality (e.g. colour;see for example Schoenfeld et al., 2003). The question remainshowever, whether object-based selective attention extends across thesensory systems, such that an object’s features in other sensorymodalities are also automatically coactivated. The coactivation offeatures within a sensory system (e.g. colour and motion) is clearlypredicated upon extensive anatomical connectivity between therespective processing modules within the visual system (i.e. V4and MT; Felleman & VanEssen, 1991), whereas similar levels ofconnectivity between processing units across sensory systems maynot exist and have only recently begun to be studied (e.g. Falchieret al., 2002). Further, a number of intersensory attention studies, thatis studies where attention is directed toward one of two sensorymodalities, have shown a diminution of sensory processing forstimuli presented in the ‘ignored’ sensory modality (Foxe et al.,2005; Johnson & Zatorre, 2005).

There are reasons to expect that object-based attention will extendacross sensory modalities, however. Spatially directed attention ismultisensory (e.g. Spence & Driver, 1996; Eimer & Driver, 2001). Theco-occurrence of visual and auditory elements of an attended objectresult in enhanced visual selective attention processes (Molholm et al.,2004), indicating a connection between visual and auditory objectrepresentations, and a central tenet of the influential biased-compe-tition model of visual attention (see Duncan, 2006) is that when anobject’s representation is primed in one part of a representationalnetwork (e.g. its colour), its representations in other parts of thenetwork will be as well (e.g. its shape), even when these relatedrepresentations are irrelevant to the task at hand (e.g. O’Craven et al.,1999; Schoenfeld et al., 2003). An extension of this logic is that suchmechanisms will also operate across sensory systems in much thesame way as they do within vision.In a pair of studies by Woldorff and colleagues (Busse et al., 2005;

Talsma et al., 2007) in which simple and unrelated auditory and visualstimuli were presented, attention was shown to ‘spread’ to anunattended auditory stimulus when it was paired with an attendedvisual stimulus. In the present study we were interested morespecifically in how attention operates on familiar objects with well-known multisensory attributes, a situation far more typical of everydayexperience (searching for the dog, tracking a nearby moving car,etc....). We expected that this might function differently than attention

Correspondence: Dr Sophie Molholm, The Cognitive Neurophysiology Laboratory, asabove.E-mail: molholm@nki.rfmh.org

Received 17 February 2007, revised 8 May 2007, accepted 30 May 2007

European Journal of Neuroscience, Vol. 26, pp. 499–509, 2007 doi:10.1111/j.1460-9568.2007.05668.x

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing Ltd

to simple and arbitrarily paired multisensory stimuli. That is, theneural representations of features of common objects are likely boundtogether, so that in highlighting an object representation in one sensorysystem via voluntarily directed selective attention, representations ofthat object in other sensory systems would be likewise highlighted.For example, will selectively attending to the image of a dog result

in enhanced processing of a corresponding dog bark? Here, attentionwas explicitly directed at a specific object within the visual or auditorysensory modality, and attended and unattended objects were presentedto the same location. High-density scalp-recorded event-relatedpotentials (ERPs) were recorded and the presence of responses knownto index visual and auditory selective attention processes was assessedfor features of attended objects that were presented in the ignoredsensory modality. To presage our results, the data provide unambig-uous evidence for the multisensory nature of object-based attention.Further, asymmetries in the data are consistent with an objectrecognition system in which visual representations tend to dominate.

Materials and methods

Subjects

Twelve neurologically normal, paid volunteers participated (mean age26 ± 5.5 years; seven female; three left-handed). All reported normalhearing and normal or corrected-to-normal vision. The InstitutionalReview Board of the Nathan Kline Institute for Psychiatric Researchapproved the experimental procedures. Each subject provided writteninformed consent in line with the Declaration of Helsinki.

Stimuli

There were three stimulus types; sounds alone, images alone, andpaired images and sounds belonging to the same object.

Images

Three line drawings were presented; a guitar, a dog, and a hammer.These came from the Snodgrass and Vanderwart set (Snodgrass &Vanderwart, 1980) and were standardized on familiarity and com-plexity. They were presented on a 21-inch computer monitor located143 cm in front of the subject, and were black on a grey background.The pictures subtended an average of 4.8� of visual angel in thevertical plane and 4.4� of visual angle in the horizontal plane. Thesewere presented for 400 ms.

Sounds

Three complementary sounds, adapted from Fabiani et al. (1996),were presented; the strum of a guitar, the bark of a dog, and the bangof a hammer. These were 400 ms in duration, and were presented at acomfortable listening level of approximately 75 dB SPL over two JBLspeakers placed at either side of the monitor.

Procedure

Participants were seated in a comfortable chair in a dimly lit andelectrically shielded (Braden Shielding Systems) room and asked tokeep head and eye movements to a minimum, while maintainingcentral fixation. Eye position was monitored with horizontal andvertical electro-oculogram (EOG) recordings. The auditory, visual,and auditory-visual stimuli were presented equiprobably and inpseudo-random order. Stimulus onset asynchrony varied randomlybetween 800 and 1100 ms. A total of 596 stimuli were presented

within a block of approximately 10 min, which was further dividedinto three subblocks to allow for frequent breaks to maintain highconcentration and prevent fatigue.There were six blocked attention conditions; attend visual stimuli,

either to the dog image or to the guitar image in different blocks;attend auditory stimuli, either to the dog sound or to the guitar soundin different blocks; and attend both auditory and visual stimuli, eitherto the dog image and sound or the guitar image and sound in differentblocks. Subjects were instructed to make a button press response withtheir right index finger to consecutive repetitions of the attended object(dog or guitar) within the attended sensory modality (auditory, visual,or both). The probability of a target was 6% in each of the auditoryand visual attention conditions.In the attend visual condition, when dog was the attended object,

the target was the second of two dog-images presented in a row; in theattend auditory condition, when dog was the attended object, the targetwas the second of two dog-barks presented in a row; and in the attendboth auditory and visual condition, when dog was the attended target,the target could be the second of two dog-barks presented in a row, thesecond of two dog-images presented in a row, a dog-image thatfollowed a dog-bark, or a dog-bark that followed a dog-image. Overthe course of the experiment the dog and the guitar each served as thetarget object in each of the three sensory attention conditions at leasttwice (for a total of 12 blocks), and at most three times (for a total of18 blocks). The order of blocks of sensory-modality attended andobject attended (dog or guitar; nested within sensory-modalityattended) was counterbalanced across subjects. The hammer stimuliwere never attended, serving only as neutral-fillers to prevent anexcess of object-repetition trials (i.e. targets). Only data from thevisual and auditory attention conditions are considered here, as theauditory-visual attention condition is not germane to the currentquestion. See Fig. 1A for a schematic of the stimulus paradigm duringvisual (top) and auditory (bottom) attention conditions, with guitarserving as the target.

Data acquisition and analysis

High-density continuous EEG recordings were obtained from theBioSemi ActiveTwo 168 channel system. Recordings were initiallyreferenced online relative to a common mode active electrode anddigitally sampled at 512 Hz. The continuous EEG was divided intoepochs ()100 ms pre-stimulus to 500 ms post-stimulus onset) andbaseline corrected over the full 600 ms for the purpose of artifactrejection. Trials with blinks and eye movements were automaticallyrejected off-line on the basis of horizontal electro-oculogram placed onthe left and right canthi. An artifact criterion of ± 60 lV was used atall other scalp sites to reject trials with excessive EMG or other noisetransients. EEG epochs were sorted according to stimulus andattention condition, and averaged from each subject to compute theERP. The baseline for the ERP was defined as the epoch from)100 ms to stimulus onset. Only non-target responses were analysed,due to the temporally and spatially overlapping target and responserelated componentry typically associated with target responses. For allelectrophysiological analyses, the responses to the two objects (dogand guitar) were collapsed, to increase the signal-to-noise ratio. Theaverage number of accepted sweeps per stimulus-type within anattention condition was 157 (± 25; collapsed across objects). For theanalysis of the data, the waveforms were algebraically re-referenced tothe nasion. Separate group-averaged ERPs for each of the stimulustypes in each of the attention conditions were calculated for displaypurposes and for identification of the short-latency sensory-evoked

500 S. Molholm et al.

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

components (e.g. P1, N1, P2), and the well-characterized selectiveattention components the visual ‘selection negativity’ (SN) and theauditory ‘negative difference’ wave (Nd; e.g. Hansen & Hillyard,1980). Button press responses to the target trials were acquired duringthe recording of the EEG and processed offline. Responses falling

between 250 and 950 ms post-stimulus onset were considered valid.This window was used so that a response could only be associatedwith a single stimulus presentation.

Statistical analyses

Behaviour

To determine if attention was directed as intended, performance duringboth auditory and visual attention conditions was examined. Perform-ance between auditory and visual attention conditions was comparedto assess if there were significant differences in the successfulallocation of attention. Error patterns were also examined, to assesswhether subjects effectively ignored stimuli presented in the unatten-ded sensory modality.For individual subjects, accuracy and average reaction time (RT)

were calculated for each of the objects (dog and guitar), for each of thetarget types (auditory, visual, or visual-auditory) and for each of theunisensory-attention conditions (attend auditory and attend visual).Target type was determined by the second of the repeating stimuluspair. S1 of the target pair was counterbalanced such that an equalnumber of auditory, visual, and visual-auditory stimuli preceded S2.For the auditory attention condition targets could be auditory (A) orvisual-auditory (VA; when the attended auditory element was pairedwith its irrelevant visual counterpart); for the visual attention conditiontargets could be visual (V) or visual-auditory (VA; when the attendedvisual object was paired with its irrelevant auditory counterpart).Separate three-way anovas with factors of Object (dog or guitar),Target type (unisensory or multisensory), and Unisensory-attentioncondition (visual or auditory) were performed on the reaction-time andaccuracy data.

Electrophysiology

Baseline unisensory selective attention effects

Characterization of selective attention effects elicited under typicalunisensory conditions provided a baseline against which to compareselective attention effects for the attended objects when presented inignored sensory modalities.Visual selective attention effects were examined by comparing the

response to the visual objects when attended to the response tothe same when unattended (e.g. during visual attention, the response tothe dog image when dog was attended compared to the response to thedog image when guitar was attended). For individual subjects,amplitude data were averaged over a 100-ms latency window centredon the peak of the grand-averaged SN. This was performed for thethree electrodes over each of right and left occipital scalp wherethe SN was largest in the grand-mean difference waveform, and overthe central occipital scalp. The SN was expected to be mostprominent over bilateral occipital scalp sites, and to peak 250–350 msfollowing stimulus onset, in line with the extant literature (e.g. Hillyard &Anllo-Vento, 1998; Molholm et al., 2004).Auditory selective attention effects were examined by comparing

the response to the auditory objects when attended to the response tothe same when unattended (e.g. during auditory attention, the responseto the dog bark when dog was attended compared to the response tothe dog bark when guitar was attended). For individual subjects,amplitude data was averaged over a 100-ms latency window that wascentred on the peak of the grand-averaged Nd. This was performed fornine electrodes over fronto-central scalp where the Nd was largest inthe grand-mean difference waveform, three from left, three from right,

Fig. 1. (A) Schematic of experimental paradigm. A sequence of stimuli (‘–’,task-irrelevant object; ‘+’, task relevant object) when visual is attended (top)and when auditory is attended (bottom), where a repetition of ‘guitar’ in theattended sensory modality comprises a target trial (‘target’). The hammerstimuli (bang sounds and hammer images) were neutral filler stimuli, whichwere never attended. (B) Back (left) and top (right) views of the 168 channelrecording montage. Electrode sites used in the statistical analysis of the visualselective attention effects (back view; filled circles) and the auditory selectiveattention effects (top view; filled circles). Data were analysed from electrodesrepresenting activity over left (three), central (three), and right (three) scalpregions where the baseline selective attention effects were greatest.

Multisensory transfer of object-based attention 501

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

and three from central regions. The Nd was expected to be mostprominent over fronto-central scalp, and to peak 150–250 msfollowing stimulus onset, in line with the extant literature (e.g.Hansen & Hillyard, 1980).

Cross-sensory object-based selective attention effects

Two approaches were taken in assessing whether object-basedselective attention is multisensory. The first examined evidence inthe responses to the unisensory stimuli and the second did so in theresponses to the multisensory stimuli. These selective attention effectsare referred to as ‘uni cross-sensory’ SN or Nd and ‘multicross-sensory’ SN or Nd, respectively.For the unisensory visual stimuli, the response to the image when its

auditory counterpart was attended (e.g. guitar image when the guitarstrum was attended) was compared to the response to the image whenits auditory counterpart was not attended (e.g. guitar image when thedog bark was attended). In this case the presence of the SN wouldsignify that, even though presented in an ignored sensory modality, thevisual element of the attended object was preferentially processed.Likewise, the response to the sound when its visual counterpart wasattended was compared to the response to the sound when its visualcounterpart was not attended and the Nd was assessed.For the multisensory stimuli the same logic was applied. However

for the multisensory comparison we predicted the occurrence of bothvisual and auditory selective attention components (the SN and theNd), one reflecting selective attention to the attended object in theattended sensory modality and the other selective attention to itscounterpart in the ignored sensory modality. As the goal was toexamine the latter, we first subtracted out the response to theunisensory stimuli corresponding to the attended sensory modality.For example, in the visual attention condition for which the questionof interest is whether visual selective attention processes also affectprocessing of auditory features of the same object (as reflected bythe presence of the Nd), the response to the attended unisensoryvisual stimuli was subtracted from the response to the attended visual-auditory stimuli, and the response to the unattended unisensoryvisual stimuli was subtracted from the response to the unattendedvisual-auditory stimuli (see Fig. 2 for illustration). This left theauditory response along with any possible auditory cross-sensoryselective attention effects (any additional multisensory interactions inthe AV response would also be contained in these responses; if thesediffered for the attended vs. unattended conditions they would bereflected in the resulting comparison), and ensured that observedelectrical activity over the region of interest was not due to volumeconduction of the ‘primary’ selective attention effect.

The dependent measure used to assess selective attention effects forstimuli presented in the unattended sensory modality was the meanamplitude over a 100-ms latency window centred on the peak of thegrand-mean selective attention component of interest (the SN inthe case of the attend-auditory condition or the Nd in the case of theattend-visual condition). Data were taken from the same electrodes asused to test baseline unisensory selective attention effects (asdescribed above).

Statistical tests

Presence of selective attention effects. Omnibus four-way anovasfor each of the visual and auditory selective attention effects wereconducted. These each had factors of Condition [three levels –baseline attention effect (baseline); multisensory object attentioneffect in unisensory stimuli (uni cross-sensory); multisensory objectattention effect in multisensory stimuli (multi cross-sensory)],Attention (two levels – attended, unattended), Region (three levels– right, centre, and left), and Electrode (3). A significant interactionbetween Condition and Attention was followed-up with three two-way anovas with factors of Attention and Region, to assess thepresence of selective attention effects individually for each of theconditions. Where appropriate, Greenhouse-Geisser corrections weremade.

Onset and offset of selective attention effects. Running t-tests (two-tailed) were used to assess the onset latency and duration of the visual(SN) and auditory (Nd) selective attention effects for each of the threecomparison conditions. For each of these, the amplitude of theattended condition was compared to the amplitude of the correspond-ing unattended condition. The onset of the SN or Nd was defined asthe first significant time-point of at least ten consecutive significanttime-points, in data from a single electrode. Offset was defined as thelast of the following significant time-points. These tests wereperformed on each data-point (�2 ms steps) from 150 to 500 mspost-stimulus onset (150 ms being the lower limit at which the object-based selective attention effects might onset). Performing tests onmultiple time-points increases the probability of a false positive. Toreduce the chances of a false positive, the tests were restricted to thesubset of nine electrodes for which the selective attention componentwas originally tested (as defined above), and a criterion of tenconsecutive time points was used as the likelihood of getting ten false-positives in a row is considerably low (Guthrie & Buchwald, 1991).To provide a fuller description of the selective-attention effects, clusterplots of t-tests on data from all electrodes across the examined epoch()100 to 500 ms) were also inspected. With the potential to reveal

Fig. 2. Derivation of the auditory response (A¢) from the auditory-visual response. Responses are from a left fronto-central scalp site during the visual attentioncondition, to attended (dashed trace) and unattended (solid trace) objects. Positive is plotted up.

502 S. Molholm et al.

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

Fig. 3. Voltage maps and waveforms of the selective attention effects. Voltage maps of the visual (A) and auditory (B) selective attention effects for the baselineand two cross-sensory conditions, and the corresponding waveforms. Waveforms to the neutral stimuli are also included (these are to the hammer image in A and tothe hammer bang in B). Note that the waveforms for the multi cross-sensory conditions are derived, as illustrated in Fig. 2 (equivalent to V¢ for the multicross-sensory SN and A¢ for the multi cross-sensory Nd). Waveforms are from a left lateral occipital site for the visual selective attention effects, and from a leftfronto-central site for the auditory selective attention effects. Positive is plotted up. The scalp site from which the displayed waveforms were recorded is indicated bya pink dot.

Multisensory transfer of object-based attention 503

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

unpredicted effects, these cluster plots can serve as a hypothesisgeneration tool for future work.

Results

Behaviour

Mean reaction-time tended to be longer to auditory compared to visualtargets, with mean RTs of 559 ms and 523 ms, respectively. Thelonger RTs to the auditory targets may reflect that a minimal segmentof the dynamic auditory signal is necessary before a target decisioncan be made, whereas for the visual targets the entire signal is presentat target onset. This difference approached significance (main effect ofunisensory-attention condition F1,11 ¼ 4.2, P ¼ 0.07). RTs to unisen-sory targets were slightly faster than RTs to multisensory targets (536vs. 545 ms), and this difference was significant (F1,11 ¼ 8.4,P ¼ 0.01). There was no significant effect of Object on RTs(F1,11 ¼ 0.06, P ¼ 0.80), and there were no significant interactions.The analysis of the per cent hit data did not reveal any significant

main effects or interactions. Performance when attention was directedat the auditory stimuli was 85% and performance when attention wasdirected at the visual stimuli was 87% (F1,11 ¼ 1.34, P ¼ 0.27). Therewas no significant effect of Target type (i.e. whether the stimuli were

unisensory or multisensory, F1,11 ¼ 2.87, P ¼ 0.12) or of Object (i.e.whether the target object was guitar or dog, F1,11 ¼ 0.97, P ¼ 0.34).

Analysis of incorrect responses

In the auditory attention condition 7% of responses were false alarmsand in the visual attention condition 6% of responses were falsealarms, false alarms being instances where subjects responded to astimulus that was not a target. False alarms did not differ significantlybetween these conditions (t1,11 ¼ 0.83, P ¼ 0.424). The majority offalse alarms were due to correct responses that fell just out of the setresponse window (66%, thus classified if the response fell in theresponse window of the following stimulus). Another 18% of falsealarms occurred in response to a repetition of the attended object in theattended sensory modality where there was a single intervening task-irrelevant stimulus (i.e. these responses would have been correct ifthere wasn’t an intervening stimulus). 15% of the false alarms wereclassified as ‘other’, not following a clear pattern. Finally, and perhapsmost importantly, across all the subjects only one false alarm was to anobject repetition where one of the two was in the ignored sensorymodality (e.g. a guitar strum followed by guitar image, during auditoryattention; 0.7% of false alarms), and there were no false alarms torepetitions of the relevant object presented in the ignored sensorymodality. The absence of a meaningful number of these two latter

Fig. 4. Maps of the significant t-values from the comparison of the ERP to the ‘attended object’ vs. the ERP to the ‘unattended object’, across the 168 recordingchannels (ordinate), from 0 to 500 ms post-stimulus onset (abscissa). The electrodes are arranged from most anterior (top of plot) to most posterior (bottom of plot),sectioned by scalp region (FP, fronto-polar; F, frontal; FC, fronto-central; C, central; P, parietal; PO, parieto-occipital; O, occipital). Significant values are onlydisplayed when there are at least ten consecutive significant data-points. Maps are for visual (A, upper panels) and auditory (B, lower panels) stimuli in each of thethree comparison conditions (baseline, uni cross-sensory, and multi cross-sensory).

504 S. Molholm et al.

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

types of false alarms suggests that subjects effectively ignored stimuliin the unattended sensory modality, as instructed.

Electrophysiology

Visual selective attention effects

A clear selection negativity (SN), our measure of visual selectiveattention processes, was observed in the grand-mean data for thebaseline condition, in which responses to attended and unattendedimages (i.e. unisensory visual stimuli) in the visual attention conditionwere compared. The difference between these responses peaked at270 ms and was of maximal amplitude bilaterally over the lateraloccipital scalp (Fig. 3A). To test for the presence of object-basedselective attention effects in the unattended sensory modality, theresponses to the image that was a feature of the attended auditoryobject was compared to the response to the image that was not afeature of the attended auditory object. This revealed visual selectiveattention effects for both the cross-sensory conditions (see Fig. 3A),with scalp topographical distributions very similar to the baselinecondition. The SN peaked at �270 ms for the multi cross-sensorycondition, and somewhat later at 310 ms for the uni cross-sensory con-dition. The placement of the electrodes used in the statistical analysesis displayed in Fig. 1B.

To test the reliability of the observed visual selective attentioneffects, a four-way anova with factors of Condition (baseline, unicross-sensory, multi cross-sensory), Attention (attended object,unattended object), Region (RH, LH, midline), and Electrode (3, ineach region) was conducted. There was a main effect of Attention(F1,11 ¼ 56.10, P < 0.000). A significant Condition by Attentioninteraction (F2,11 ¼ 6.41, P ¼ 0.01) indicated that the magnitude ofthe SN differed across the conditions. There were no significanthigher-level interactions. Follow-up anovas revealed significantattention effects for the baseline SN and the multi cross-sensory SN(main effect of Attention for the two conditions, respectively,F1,11 ¼ 29.00, P < 0.001; F1,11 ¼ 27.96, P < 0.001). The unicross-sensory SN was not significant (F1,11 ¼ 2.61, P ¼ 0.14). There wereno significant higher-level interactions.

To determine the onset and offset of the visual selective attentioneffects, statistical cluster-plots were generated (Fig. 4). Theserevealed an onset of 196 ms and an offset of 346 ms for thebaseline SN and a later onset and offset of 237 ms and 366 ms forthe multi cross-sensory SN. In both cases onsets were slightly earlierat the sites over left occipital scalp. For the uni cross-sensorycondition there was no corresponding significant activity over thetested electrode sites.

Auditory selective attention effects

A clear negative difference wave (Nd), our measure of auditoryselective attention processes, was observed in the grand-mean data forthe baseline condition, in which responses to attended and unattendedsounds (i.e. unisensory auditory stimuli) in the auditory attentioncondition were compared. The difference between these responsespeaked at 220 ms, and was focused over the fronto-central scalpregion, with a slightly leftward distribution (Fig. 3B). To test for thepresence of object-based selective attention effects in the unattendedsensory modality, the response to the sound that was a feature of theattended visual object was compared to the response to the sound thatwas not a feature of the attended visual object. This revealed auditoryselective attention effects, for both the multi cross-sensory and unicross-sensory comparisons (see Fig. 3B). The Nd peaked at 220 ms

for the multi cross-sensory condition, and somewhat later at 300 msfor the uni cross-sensory condition. The scalp distributions of both thecross-sensory Nds were slightly more frontal and leftward comparedto the baseline Nd. The placement of the electrodes used in thestatistical analyses is displayed in Fig. 1B.A four-way anova with factors of Condition (baseline, unicross-

sensory, multi cross-sensory), Attention (attended object, unattendedobject), Region (RH, LH, midline), and Electrode was conducted totest the reliability of the Nd across the three conditions. There was asignificant main effect of Attention (F1,11 ¼ 40.97, P < 0.000). Thefactors Attention and Condition did not interact (F2,22 ¼ 1.03,P ¼ 0.37), indicating that the amplitude of the Nd did not significantlydiffer across the three conditions. There were no significant higher-level interactions.To determine the onset and offset latencies of the auditory selective

attention effects, statistical cluster-plots were generated (Fig. 4). Theserevealed an onset of 176 ms for the baseline Nd, with the differentialactivity remaining significant throughout the analysed epoch (out to500 ms). The multi cross-sensory Nd onset at approximately the samelatency, of 180 ms, but offset much earlier at approximately 255 ms.The uni cross-sensory Nd onset somewhat later at approximately190 ms and offset at approximately 423 ms.

Comparison of the attention effects with a neutral condition

A posthoc analysis was performed in which the amplitude of the‘attended’ and ‘unattended’ responses were each compared to a neutralcondition. This allowed us to consider whether the attention effectsbetter fit a model in which there was ‘suppressed’ processing of theunattended stimuli, or a model in which there was ‘enhanced’processing of the attended stimuli. The responses to the hammer-images and bang-sounds were considered neutral as throughout theexperimental session they were never attended and had no associatedtask relevance. While these stimuli were not exactly matched forsensory stimulation with the comparator attended and unattendedstimuli, the sensory responses were quite similar (see Fig. 3). At laterlatencies (�200 ms onward) the waveform followed the responses ofthe unattended conditions (except in the multi cross-sensory Ndcondition, where it was more positive than both the attended andunattended responses; see Fig. 3).We first tested whether there were effects of stimulus-type when the

neutral stimulus and the attended stimuli were all compared, and whenthe neutral stimulus and the unattended stimuli were all compared.A total of four one-way anovas were performed. In one theunattended auditory responses and the neutral auditory response wereall compared. This resulted in four levels of the stimulus-conditionfactor; baseline, uni cross-sensory, multi cross-sensory, and neutral. Inanother, the attended auditory responses and the neutral auditoryresponse were all compared. This resulted in the same four levels ofstimulus-condition. In a third the unattended visual responses and theneutral visual response were compared. This resulted in three levels ofthe stimulus-condition factor; baseline, multi cross-sensory, andneutral, and in a fourth, the ‘unattended’ visual responses and theneutral visual response were compared. This resulted in the same threelevels of stimulus-condition.Data were only included from conditions where there were

significant attention effects. To maximize the sensitivity of the tests,data were pooled across the three electrode sites over the lefthemisphere, where the attention effects had appeared greatest inamplitude (when considering the right, left, and central sites fromwhich data were taken for the original analyses). For the comparisons,the dependent variable was the mean amplitude over a 100-ms

Multisensory transfer of object-based attention 505

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

window that was centred on the peak of the selective attention effectsfor the attended and unattended responses (the same as used in theplanned tests of the selective attention effects), and over the samelatency window for the neutral stimulus conditions (170–270 ms forthe auditory neutral stimulus and 220–320 ms for the visual neutralstimulus).There were no significant effects of stimulus-condition when the

responses to the unattended and neutral stimuli were compared, foreither the visual or the auditory data (F2,22 ¼ 0.51, P ¼ 0.61 andF3,33 ¼ 1.09, P ¼ 0.37, respectively). There were, in contrast,significant effects of stimulus-condition when the responses to theattended and neutral stimuli were compared, and this was the case forboth the visual and the auditory data (F2,22 ¼ 14.52, P ¼ < 0.001)and (F3,33 ¼ 5.65, P ¼ 0.003, respectively). These significant effectswere followed-up by paired samples t-tests to determine which of theattended stimulus conditions differed significantly from the neutralcondition (three comparisons for the auditory stimuli and twocomparisons for the visual stimuli). The responses to the attendedstimuli were all significantly more negative going than the respectiveneutral conditions, at P £ 0.01. These data are consistent with an‘enhancement’ of attended responses explanation of the attentioneffects.

Discussion

The present study provides evidence that object-based attention ismultisensory. Selectively attending to an object in one sensorymodality resulted in facilitated processing of its features in the ignoredsensory modality. This was indexed by the presence of the visual andauditory selective attention ERP responses, the selection negativityand the negative difference wave, respectively (e.g. (Hansen &Hillyard, 1980; Hillyard & Anllo-Vento, 1998). A key question iswhat are the mechanisms underlying such multisensory object-basedattention effects?

Underlying mechanisms

Selective attention to a stimulus feature can result in the amplificationof the sensory driven response of the corresponding feature specificsensory processing network (e.g. Corbetta et al., 1990), a process thathas been conceptualized as ‘priming’ (e.g. Hillyard et al., 1998). Theneural representations of features of objects are likely strongly boundtogether (e.g. Beauchamp et al., 2004; Amedi et al., 2005) such that insetting an object representation in one sensory system at a lowerthreshold via selective attention, the thresholds of representations ofthat object in other sensory systems are likewise altered. Such a cross-sensory object priming account provides an excellent fit to theauditory cross-sensory attention data of the present study. Specifically,we observed auditory cross-sensory attention effects even in theabsence of the visual stimulus that was being attended at the time.Further, the onset of the auditory cross-sensory attention effectsapproximated that of the baseline comparison condition (180–190 msduring visual attention vs. 176 ms during auditory attention). Thispattern of data suggests that the sensory processing network wasprimed for this input. Consistent with this explanation, cross-sensorypriming has been demonstrated using behavioural measures, forauditory-visual as well as tactile-visual objects (Easton et al., 1997;Greene et al., 2001). Also supporting a representational system inwhich the multisensory features of an object are coactivated is aneuroimaging study showing tactile to visual priming for novel visualobjects studied by touch (James et al., 2002).

The auditory cross-sensory attention effects of the current studystrongly suggest that the automatic coactivation of a visual object’sunattended features, proposed in the highly influential biased-compe-tition model of visual attention (Desimone & Duncan, 1995; Duncan,2006) and supported by experimental data (e.g. Schoenfeld et al.,2003), also applies to a visual object’s representations in otherunisensory systems.The priming explanation doesn’t account as well, if at all, for the

visual cross-sensory attention effects. For one, a significant visualcross-sensory attention effect was only seen when the ignored visualstimulus was actually presented in conjunction with the attendedauditory stimulus. Thus, the presence of the explicitly attendedauditory stimulus was necessary for visual cross-sensory atten-tion effects to occur. What’s more, the observed cross-sensoryattention effect developed substantially later than that for the baselinecomparison condition. That is, the visual selective attention effectonset at 196 ms during visual attention (the baseline condition),whereas it didn’t onset until 237 ms during auditory attention. Onepossibility is that the auditory-to-visual transfer effects were justweaker than the visual-to-auditory transfer effects, and hence theyfailed to reach significance in one case and only reached significanceat a relatively late latency in the other, an issue of signal-to-noise.Consistent with this view, in Fig. 3 there is some suggestion of aselection negativity for the uni cross-sensory condition (Fig. 3A,middle panel), though this does not reach statistical significance in theplanned anova, and the running t-test significance map for the visualuni cross-sensory SN shows significant differences in just twoposterior electrodes in the timeframe of the SN (Fig. 4, top middlepanel). Weaker auditory to visual transfer of attention effects could bedue to the ‘dominance’ of visual object representations, a notion that isdiscussed below.However, an alternative explanation also fits the present visual

cross-sensory attention data, and indeed was suggested as the processunderlying the cross-sensory attention effect observed for simple andunrelated visual and auditory stimuli by the Woldorff group (Busseet al., 2005; Talsma et al., 2007). This is a spread of attention account,in which attention spreads from attended to unattended aspects of anobject. Such spreading of attention was originally proposed to accountfor the finding of better performance on targets presented to anunattended location that fell within the same object boundaries as anattended location (Egly et al., 1994; Davis et al., 2000). The spread ofspatial-attention within an object has received further support fromERP and fMRI neuroimaging studies, with attention effects present forstimuli presented to unattended locations that fell within the attendedobject (e.g. Martinez et al., 2006). In a similar vein, attention mayspread within an object across sensory systems, such that when anattended object is presented, its constituent parts also receivepreferential processing, even though presented in ignored sensorymodalities. By this explanation the so-called spread of attention shoulddepend upon the presence of the attended object, and the onset ofselective attention effects should onset somewhat later than that ofstandard within-modality selective attention effects, simply allowingfor the time for effects to transmit between visual and auditory corticalregions. In fact, this is precisely what was observed here for visualcross-sensory attention.There is also the question of the mediating neural structures and

pathways of cross-sensory attentional transfer. One possibility is thatdirect connections among an object’s feature-representations in thedifferent unisensory systems mediate the effects. There is feasibilityfor such a pathway insofar as non-human primate anatomical tracerstudies reveal direct connections between auditory and visual sensorycortices (Falchier et al., 2002; Rockland & Ojima, 2003; Cappe &

506 S. Molholm et al.

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

Barone, 2005), and human and non-human primate neurophysiologyhave shown that multisensory processing can occur early in time inwhat are considered unisensory cortical regions (Giard & Peronnet,1999; Molholm et al., 2002); for a discussion of these data see Foxe &Schroeder (2005) and Schroeder & Foxe (2005).

Equally plausible is that higher-order cortical regions mediate cross-sensory attentional transfer. In this case a multisensory or supramodalcortical region higher in the information processing hierarchy (e.g.polysensory superior temporal sulcus) might send signals to theunisensory cortices to modulate processing of the features of aparticular object, even for features that are not explicitly attended. Thiscould be initiated by the presentation of the attended stimulus, withinputs from unisensory cortex to higher-order regions triggering thetop-down activation of the object’s features in unattended sensorymodalities (though this explanation is limited to instances where thepresence of the attended stimulus is necessary for cross-sensoryattention effects to occur). Alternatively, it could be initiated in a top-down manner following task instructions. This general mechanism, inwhich higher-order cortical regions are responsible for cross-sensoryattentional effects in unisensory cortex, bears some resemblance to theway in which higher-order parietal and frontal regions of cortexpresumably mediate so-called supramodal spatial attention effects inunisensory cortices (Eimer & Driver, 2002).

Attentional transfer between simple and unrelated auditoryand visual features

A pair of studies from the Woldorff laboratory addresses attentionalspread across unrelated simple multisensory stimuli, during spatial(Busse et al., 2005) and intersensory (Talsma et al., 2007) attention.Using event-related potentials (ERPs) and simple and unrelated visualand auditory stimuli, Busse et al. (2005) showed an auditory selectiveattention effect for centrally presented task-irrelevant auditory tonalstimuli that were presented while subjects selectively attended tolateralized visual checkerboard stimuli. Under their design, visualstimuli were randomly presented to the right or left of fixation andsubjects were required to detect infrequent targets in either the right orleft stimulus stream in a given block. On 50% of trials the visualstimuli, half attended and half unattended, were accompanied by acentrally presented task-irrelevant tone. Although the tone waspresented to central auditory space, the co-occurrence of the visualstimulus resulted in the so-called ventriloquist illusion, a powerfulillusion in which the perceived location of a sound is shifted toward aconcurrently presented visual stimulus (Bertelson & Radeau, 1981).Auditory selective attention effects were then assessed by comparing‘attended’ vs. ‘unattended’ trials. The authors argue that because theauditory stimulus was presented to a different location than the visualstimulus, the presence of the observed auditory attention effect couldnot be attributed to well-established cross-sensory spatial attentionmechanisms (Hillyard et al., 1984; Eimer & Driver, 2001), but ratherwas due to the spreading of attention within the object acrossmodalities and space. This effect is intriguing and consistent with themultisensory spread of attention across features of an object.

Another ERP study from the same group provides further evidence(Talsma et al., 2007) for the multisensory spread of attention within anobject. Again, simple and unrelated auditory and visual stimuli werepresented alone or together, but now in an intersensory attentionparadigm in which subjects detected targets in the visual or the auditorysensory modality in separate blocks. ERP recordings revealed anunpredicted late-onset negativity in response to the auditory-visualstimulus for the visual attention condition. This had a topography

characteristic of the Nd, a response associated with selective processingof auditory information. What’s more, during visual attention thisresponse was only observed when the auditory stimulus was presentedwith the attended visual stimulus, and not when it was presented alone.These data are thus consistent with attention automatically spreadingacross sensory modalities, from attended to unattended dimensions ofthe arbitrarily paired stimuli. Of note, the latency of theNd response wasvery late (only significant by 420 ms) with respect to that seen inresponse to auditory stimuli during auditory attention conditions(significant by 280 ms). Also, such multisensory attention appeared tobe unidirectional, with no report of selective processing of the visualstimulus during auditory attention. Nevertheless, attentionwas shown tospread across sensory modalities, from the attended to the unattended ofarbitrarily paired simple visual and auditory stimuli.In these studies attention was directed at a location (Busse et al.,

2005) or at one of two sensory modalities (Talsma et al., 2007), ratherthan at specific objects. When attention was directed in this manner,the presence of the attended object was necessary for the occurrence ofcross-sensory attentional effects, whereas this was not the case in thepresent study for transfer of attention within an object from the visualto the auditory sensory modality. This strongly suggests that the cross-sensory object priming that we observe may be invoked specificallywhen attention is directed at objects.The topography of the auditory selective attention effects that we

observed in the cross-sensory conditions was similar in its fronto-central distribution to the cross-sensory auditory selective attentioneffects in both Busse et al. (2005) and Talsma et al. (2007). In thepresent study however, there was a tendency to a leftward distributionnot seen in Busse et al. (2005) or Talsma et al. (2007). This slightlateralization, seen for all three auditory selective attention effectsincluding the baseline condition, is very likely a function of thestimuli. Brain imaging studies using electroencephalography (Hauket al., 2006), magnetoencephalography (Pulvermuller et al., 2005) andfMRI (Pulvermuller & Hauk, 2006) have revealed that processing ofobject sounds and action words engages an object ⁄ event-specificrepresentational network relatively early in time (< 200 ms post-stimulus onset). Future studies will be needed to assess the similarityof the underlying neuronal generators as a function of how attention isdirected (e.g. toward objects, locations, or a sensory modality), as wellas assess how the latency of the effect may be related to the specificattentional processes that are engaged.

Visual dominance in multisensory object recognition?

The present data would also seem to argue that visual-objectrepresentations have a greater influence on their auditory counterpartsthan vice-versa. That is, when attention was directed at visual stimuli,there were greater cross-sensory attention effects than when attentionwas directed at auditory stimuli. This asymmetry was seen both in thecurrent study for well-known multisensory objects, and Talsma et al.(2007) for simple and unrelated auditory and visual stimuli. Consistentwith this asymmetry, in a behavioural cross-sensory priming study,visual to auditory priming was readily apparent while there was noevidence for auditory to visual priming (Greene et al., 2001). Theauthors attributed this to the lack of specificity of auditory objects. Butthis explanation would not apply to the current design, in which thestimulus set was small and subjects knew with certainty the object asound belonged to. As an alternative, we propose that visualrepresentations play a leading role in the representation of certaintypes of objects (see also, e.g. James et al., 2002; Molholm et al.,2004; Amedi et al., 2005; Gomez-Ramirez et al., 2007), perhaps due

Multisensory transfer of object-based attention 507

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

to their generally greater specificity (see e.g. Ernst & Bulthoff, 2004),as an image usually exactly specifies the object while a sound oftendoes not. This specificity of visual representations may have, throughphylogenetic or developmental means, led to the dominance of visualobject recognition areas in object processing.One possible issue is that, due to our delimited stimulus set,

associative-links between the auditory and visual stimuli may haveformed over the course of the experiment, and that these transientassociations might account for the cross-sensory selective attentioneffects seen here. This possibility does not cohere well withdifferences seen between our findings and those of Talsma et al.(2007). That is, one would expect such associative links to similarlyoccur for the repeatedly presented but previously unrelated auditoryand visual stimuli in Talsma et al. (2007). However, Talsma et al.(2007) only found cross-sensory attention effects for repeatedlypresented unrelated auditory and visual stimuli when (i) they werepresented together, and (ii) during visual attention. What’s more, thisauditory cross-sensory attention response was substantially delayed intime with respect to a baseline condition whereas the effects seen herewere not.

Conclusions

Perceptual and cognitive processes have traditionally been studied in thecontext of unisensory stimulation. This has served the important purposeof reducing the number of variables and allowing the experimenter toattribute specific effects to a single factor. Of course this approachhugely simplifies how perceptual and cognitive processes truly operatein a multisensory world. That is, practically all our experiences aremultisensory, with multisensory information about objects and eventshighly related (often complimentary, or even redundant). It is clear thatour nervous systemhas evolved to take advantage of such amultisensoryenvironment (Stein &Meredith, 1993). The current work contributes toa growing exploration of the multisensory nature of perception,establishing the interconnectedness of multisensory object representa-tions during selective attention processes.

Acknowledgements

This work supported by grants from the National Institute of Mental Health toSM (MH68174) and JJF (MH65350). We would also like to express our sincerethanks to Dr Simon Kelly for comments on an earlier version of thismanuscript, and to Jeannette Mahoney for her technical assistance in collectingthese data.

Abbreviations

ERP, event-related potential; Nd, negative difference wave; RT, reaction time;SN, selection negativity wave.

References

Amedi, A., von, K.K., van Atteveldt, N.M., Beauchamp, M.S. & Naumer, M.J.(2005) Functional imaging of human crossmodal identification and objectrecognition. Exp. Brain Res., 166, 559–571.

Beauchamp, M.S., Lee, K.E., Argall, B.D. & Martin, A. (2004) Integration ofauditory and visual information about objects in superior temporal sulcus.Neuron, 41, 809–823.

Bertelson, P. & Radeau, M. (1981) Cross-modal bias and perceptual fusionwith auditory-visual spatial discordance. Percept. Psychophys., 29, 578–584.

Busse, L., Roberts, K.C., Crist, R.E., Weissman, D.H. & Woldorff, M.G. (2005)The spread of attention across modalities and space in a multisensory object.Proc. Natl Acad. Sci. USA, 102, 18751–18756.

Cappe, C. & Barone, P. (2005) Heteromodal connections supporting multisen-sory integration at low levels of cortical processing in the monkey. Eur. J.Neurosci., 22, 2886–2902.

Corbetta, M., Miezin, F.M., Dobmeyer, S., Shulman, G.L. & Petersen, S.E.(1990) Attentional modulation of neural processing of shape, color, andvelocity in humans. Science, 248, 1556–1559.

Davis, G., Driver, J., Pavani, F. & Shepherd, A. (2000) Reappraising theapparent costs of attending to two separate visual objects. Vis. Res., 40,1323–1332.

Desimone, R. & Duncan, J. (1995) Neural mechanisms of selective visualattention. Annu. Rev. Neurosci., 18, 193–222.

Duncan, J. (2006) EPS Mid-Career Award 2004: brain mechanisms ofattention. Q. J. Exp. Psychol. (Colchester), 59, 2–27.

Easton, R.D., Srinivas, K. & Greene, A.J. (1997) Do vision and haptics sharecommon representations? Implicit and explicit memory within and betweenmodalities. J. Exp. Psychol. Learn. Mem. Cogn., 23, 153–163.

Egly, R., Driver, J. & Rafal, R.D. (1994) Shifting visual attention betweenobjects and locations. evidence from normal and parietal lesion subjects.J. Exp. Psychol. General, 123, 161–177.

Eimer, M. & Driver, J. (2001) Crossmodal links in endogenous and exogenousspatial attention: evidence from event-related brain potential studies.Neurosci. Biobehav. Rev., 25, 497–511.

Eimer, M., van, V.J. & Driver, J. (2002) Cross–modal interactions betweenaudition, touch, and vision in endogenous spatial attention: ERP evidence onpreparatory states and sensory modulations. J. Cogn. Neurosci., 14, 254–271.

Ernst, M.O. & Bulthoff, H.H. (2004) Merging the senses into a robust percept.Trends Cogn. Sci., 8, 162–169.

Fabiani, M., Kazmerski, V.A., Cycowicz, Y.M. & Friedman, D. (1996) Namingnorms for brief environmental sounds: effects of age and dementia.Psychophysiology, 33, 462–475.

Falchier, A., Clavagnier, S., Barone, P. & Kennedy, H. (2002) Anatomicalevidence of multimodal integration in primate striate cortex. J. Neurosci., 22,5749–5759.

Felleman, D.J. & VanEssen, D.C. (1991) Distributed hierarchical processing inthe primate cerebral cortex. Cereb. Cortex, 1, 1–47.

Foxe, J.J. & Schroeder, C.E. (2005) The case for feedforward multisensoryconvergence during early cortical processing. Neuroreport, 16, 419–423.

Foxe, J.J., Simpson, G.V., Ahlfors, S.P. & Saron, C.D. (2005) Biasing thebrain’s attentional set. I. Cue Driven Deployments of Intersensory SelectiveAttention. Exp. Brain Res., 166, 370–392.

Giard, M.H. & Peronnet, F. (1999) Auditory-visual integration duringmultimodal object recognition in humans: a behavioral and electrophysio-logical study. J. Cogn. Neurosci., 11, 473–490.

Gomez-Ramirez, M., Higgins, B.A., Rycroft, J.A., Owen, G.N., Mahoney, J.,Shpaner, M. & Foxe, J.J. (2007) The deployment of intersensory selectiveattention: a high-density electrical mapping study of the effects of theanine.Clin. Neuropharmacol., 30, 25–38.

Greene, A.J., Easton, R.D. & LaShell, L.S. (2001) Visual-auditory events:cross-modal perceptual priming and recognition memory. Conscious Cogn.,10, 425–435.

Guthrie, D. & Buchwald, J.S. (1991) Significance testing of differencepotentials. Psychophysiology, 28, 240–244.

Hansen, J.C. & Hillyard, S.A. (1980) Endogenous brain potentials associatedwith selective auditory attention. Electroencephalogr. Clin. Neurophysiol.,49, 277–290.

Hauk, O., Shtyrov, Y. & Pulvermuller, F. (2006) The sound of actions asreflected by mismatch negativity: rapid activation of cortical sensory-motornetworks by sounds associated with finger and tongue movements. Eur. J.Neurosci., 23, 811–821.

Hillyard, S.A. & Anllo-Vento, L. (1998) Event-related brain potentials in thestudy of visual selective attention. Proc. Natl Acad. Sci. USA, 95, 781–787.

Hillyard, S.A., Simpson, G.V., Woods, D.L., Van Voorhis, S.T. & Munte, T.F.(1984) Event-related brain potentials and selective attention to differentmodalities. In Reinoso-Suarez, F., (Ed), Cortical Integration. Ravel Press,New York, pp. 395–414.

Hillyard, S.A., Vogel, E.K. & Luck, S.J. (1998) Sensory gain control(amplification) as a mechanism of selective attention: electrophysiologicaland neuroimaging evidence. Philos. Trans. R. Soc. Lond. B Biol. Sci., 353,1257–1270.

James, T.W., Humphrey, G.K., Gati, J.S., Servos, P., Menon, R.S. & Goodale,M.A. (2002) Haptic study of three-dimensional objects activates extrastriatevisual areas. Neuropsychologia, 40, 1706–1714.

508 S. Molholm et al.

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509

Johnson, J.A. & Zatorre, R.J. (2005) Attention to simultaneous unrelatedauditory and visual events. behavioral and neural correlates. Cereb. Cortex,15, 1609–1620.

Martinez, A., Teder-Salejarvi, W., Vazquez, M., Molholm, S., Foxe, J.J.,Di Javitt, D.C.R.F., Worden, M.S. & Hillyard, S.A. (2006) Objectsare highlighted by spatial attention. J. Cogn. Neurosci., 18, 298–310.

Molholm, S., Ritter, W., Javitt, D.C. & Foxe, J.J. (2004) Multisensory visual-auditory object recognition in humans: a high-density electrical mappingstudy. Cereb. Cortex, 14, 452–465.

Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E. & Foxe,J.J. (2002) Multisensory auditory–visual interactions during early sensoryprocessing in humans: a high-density electrical mapping study. Brain Res.Cogn. Brain Res., 14, 115–128.

O’Craven, K.M., Downing, P.E. & Kanwisher, N. (1999) fMRI evidence forobjects as the units of attentional selection. Nature, 401, 584–587.

Pulvermuller, F. & Hauk, O. (2006) Category-specific conceptual processing ofcolor and form in left fronto-temporal cortex. Cereb. Cortex, 16, 1193–1201.

Pulvermuller, F., Shtyrov, Y. & Ilmoniemi, R. (2005) Brain signatures ofmeaning access in action word recognition. J. Cogn. Neurosci., 17, 884–892.

Rockland, K.S. & Ojima, H. (2003) Multisensory convergence incalcarine visual areas in macaque monkey. Int. J. Psychophysiol., 50,19–26.

Schoenfeld, M.A., Tempelmann, C., Martinez, A., Hopf, J.M., Sattler, C.,Heinze, H.J. & Hillyard, S.A. (2003) Dynamics of feature bindingduring object-selective attention. Proc. Natl Acad. Sci. USA, 100,11806–11811.

Schroeder, C.E. & Foxe, J. (2005) Multisensory contributions to low-level,‘unisensory’ processing. Curr. Opin. Neurobiol., 15, 454–458.

Snodgrass, J.G. & Vanderwart, M. (1980) A standardized set of 260 pictures:norms for name agreement, image agreement, familiarity, and visualcomplexity. J. Exp. Psychol. [Hum. Learn.], 6, 174–215.

Spence, C. & Driver, J. (1996) Audiovisual links in endogenous covert spatialattention. J. Exp. Psychol. Hum. Percept. Perform., 22, 1005–1030.

Stein, B.E. & Meredith, M.A. (1993) The Merging of the Senses. The MITPress, Cambrige, MA.

Talsma, D., Doty, T.J. & Woldorff, M.G. (2007) Selective attention andaudiovisual integration: is attending to both modalities a prerequisite forearly integration? Cereb. Cortex, 17, 679–690.

Multisensory transfer of object-based attention 509

ª The Authors (2007). Journal Compilation ª Federation of European Neuroscience Societies and Blackwell Publishing LtdEuropean Journal of Neuroscience, 26, 499–509