The interaction of spatial reference frames and ...€¦ · 1987, 1993; Pouget & Driver, 2000;...

Cognitive Affective amp Behavioral Neuroscience2001 1 (4) 307-329

Hemispatial or unilateral neglect is a visuospatial def-icit typically acquired after brain damage to the right pos-terior parietal lobe in which patients fail to perceive or acton information that appears on the side of space oppositethe lesion Because neglect occurs more frequently andwith greater severity after right-hemisphere than afterleft-hemisphere lesions we refer to neglect as left-sidedin this paper Although patients with left-sided neglect havenormal intellectual abilities and intact primary motor andsensory functions they may not notice objects on the leftmay leave food untouched on the left side of the plate andmay not shave or bathe the left side of the body (for re-cent reviews see Bisiach amp Vallar 2000 McGlinchey-Berroth 1997 Vallar 1998) Neglect is thought to occurbecause the neurons in one hemisphere have predomi-nant although not exclusive representation of the con-

tralateral side of space removal of neurons therefore im-pairs spatial representations for contralateral positions toa greater extent than for ipsilateral positions (Pouget ampDriver 2000 Rizzolatti Berti amp Gallese 2000) The directconsequence of damaging these neurons is that informa-tion appearing on the relative left is poorly activated incomparison with more rightward information and henceis neglected (Cate amp Behrmann 2001 Kinsbourne1987 1993 Pouget amp Driver 2000 Smania et al 1998)

One of the best examples of neglect comes from the per-formance of patients on copying or drawing tasks As isshown in Figure 1 during copying patients routinely omitfeatures on the left while incorporating the correspondingfeatures on the right This phenomenon is so typical thattasks like this are frequently used to diagnose the pres-ence of neglect and are considered to be especially sen-sitive to the deficit (Black et al 1994) In this paper wemake use of one such standard task that of copying daisiesto explore the mechanisms that give rise to hemispatialneglect

A key issue to be addressed in understanding neglectis to specify what constitutes ldquoleftrdquomdash that is with respectto what frame of reference is spatial position defined sothat information to the left of the midline is neglected Pos-sible reference frames include those whose origin and axes

307 Copyright 2001 Psychonomic Society Inc

This work was supported by Grants MH54766 and MH54246 fromNIH a Weston Visiting Professorship at the Weizmann Institute of Sci-ence and a James McKeen Cattell sabbatical award to MB and by aFulbright Fellowship to DCP We thank Jeffrey Beng-Hee Ho for hiscontribution to this work Correspondence concerning this article shouldbe addressed to either M Behrmann or D C Plaut Carnegie MellonUniversity Mellon Institute 115-CNBC 4400 Fifth Avenue PittsburghPA 15213-2683 (e-mail behrmanncondorcnbccmuedu or plautcmuedu)

The interaction of spatial reference frames and hierarchical object representations

Evidence from figure copying in hemispatial neglect

MARLENE BEHRMANNCarnegie Mellon University Pittsburgh Pennsylvania

and

DAVID C PLAUTCarnegie Mellon University Pittsburgh Pennsylvania

and Center for the Neural Basis of Cognition Pittsburgh Pennsylvania

In copying or drawing a figure patients with hemispatial neglect following right parietal lobe lesionstypically produce an adequate representation of parts on the right of the figure while omitting the cor-responding features on the left The neglect of information occupying contralateral locations is influencedby multiple spatial reference frames and by the hierarchical structure of the object(s) in the figure Thepresent work presents a computational characterization of the interaction among these influences toaccount for the way in which neglect manifests in copying Empirical data are initially collected frombrain-damaged and normal control subjects during two figure-copying tasks in which the hierarchicalcomplexity and orientation of the displays to be copied are manipulated In the context of the modelneglect is simulated by a ldquolesionrdquo (monotonic drop-off along gradient from right to left) that can affectperformance in both object- and viewer-centered reference frames The effect of neglect in both theseframes coupled with the hierarchical representation of the object(s) provides a coherent account ofthe copying behavior of the patients and may be extended to account for the copying performance ofother patients across a range of objects and scenes

308 BEHRMANN AND PLAUT

are defined with respect to the midline of the viewer (egaligned with the gaze head orientation or trunk of theviewer) the environment (eg based on landmarks such asthe walls of a room or defined gravitationally) or the object(eg determined by the intrinsic characteristics of objectssuch as principal axes of elongation or symmetry) Undermost viewing conditions these frames are all aligned sothere is no way to evaluate which reference frame deter-mines the spatial coding of stimuli and the subsequentneglect behavior Recent evidence however obtainedfrom a host of neuropsychological studies suggests thatneglect behavior is sensitive to spatial information de-fined with respect to multiple distinct reference framesIn light of this evidence it becomes important to understandhow information coded in these different referenceframes is integrated to yield coherent behavior

In this paper we examine specifically how informationappearing on the left defined by multiple reference framesmanifests in patientsrsquo figure-copying performance Wefocus on two major reference frames one defined ego-centrically by the midline of the viewer and the other de-fined allocentrically by the midline of an object In ad-dition we explore how spatial coding is affected by thecomplexity of the object being copied We start off byreviewing current findings that support the influence ofthese two forms of reference frame in neglect Then inthe first experiment we present empirical data from a taskin which the neglect patients copy a single daisy presentedin differing orientations To account for these data we for-mulate a computational account of the way in which theactivation of spatial information defined in multiple ref-erence frames may be synthesized to subserve behaviorand we present data obtained from such a computationaldemonstration We extend the account in the second ex-

periment by including empirical data on a more complexfigure-copying task and by showing that the critical as-sumptions underlying the computational model are suf-ficiently general to account for neglect performance underthese more challenging conditions

Object-Centered NeglectEvidence for neglect that is defined with respect to the

midline of the viewer is well established and not partic-ularly controversial For example there is a general con-sensus that early visual information is encoded with re-spect to retinal location and gaze direction (AndersenEssick amp Siegel 1985 Colby amp Goldberg 1999) andthere now exist numerous studies reporting neglect forspatial information appearing to the left of the retinalhead andor trunk midline (eg Bartolomeo amp Chokron1999 Behrmann Ghiselli-Crippa Sweeney Di Matteoamp Kass 2002 Beschin Cubelli Della Sala amp Spinaz-zola 1997 Bisiach Capitani amp Porta 1985 Chokron ampImbert 1995 Hillis amp Rapp 1998 Karnath Schenkel ampFischer 1991 Kooistra amp Heilman 1989 VuilleumierValenza Mayer Perrig amp Landis 1999)

The more controversial question concerns the role of areference frame centered on the midline of an individualobject so that spatial position is located with respect to arepresentation that depends on the objectrsquos extent shapeor motion Within such a representation the relationshipof object parts is defined with respect to each other allo-centrically and independently of the viewerrsquos position Manyrecent studies have examined whether neglect occurs forinformation appearing to the left of a midline defined byan individual object The result of many although not allsuch studies (Farah Brunn Wong Wallace amp Carpenter1990 however see Hillis amp Rapp 1998 for a reanalysis ofthese data) is that patients fail to report information ap-pearing to the left of the object midline even when thisinformation is located to the right of the midline of theviewer andor the environment (Behrmann amp Mosco-vitch 1994 Behrmann amp Tipper 1994 Driver amp Halli-gan 1991 Humphreys amp Riddoch 1995 PavlovskayaGlass Soroker Blum amp Groswasser 1997 YoungHellawell amp Welch 1992)

One of the earliest documented examples of object-based neglect is from Patient NG who had right-sidedneglect and who failed to read the rightmost letters of aword This was true when the word was presented verti-cally in mirror-reversed format and even when she wasrequired to spell words backwards (Caramazza amp Hillis1990a 1990b Hillis Rapp Benzing amp Caramazza 1998)Arguin and Bub (1993a) also showed that their patientrsquosinability to report a target letter in a horizontal array of fourelements depended on the object-relative position of theletter and not on the viewer-relative position In a morerecent series of studies Humphreys Riddoch and theircolleagues have explored several aspects of object-basedneglect and have shown that patients neglect letters posi-tioned to the left of individual words (Humphreys amp Rid-doch 1994 1995 Riddoch Humphreys Luckhurst

Figure 1 Representative examples of left-sided neglect duringa figure-copying task by a neglect patient

SPATIAL REFERENCE FRAMES AND HIERARCHICAL OBJECT REPRESENTATIONS 309

Burroughs amp Bateman 1995) Interestingly some of thesesame patients show neglect for entire words on the rightin multiple-stimulus displays simultaneous with the ob-ject-based effects providing support for accounts thatposit the involvement of multiple spatial frames and cod-ing between as well as within objects (see Haywood ampColtheart 2000 for further discussion of neglect dys-lexia and other examples of object-based findings)

Although most of the studies cited above use letters orwords as stimuli object-based neglect has also been re-ported in studies that use other types of stimuli For exam-ple Young and colleagues (Young et al 1992 YoungNewcombe de Haan Newcombe amp Hay 1990) reportedthat their patient performed poorly at identifying the lefthalf of chimeric faces even when the faces were pre-sented upside down and the relative left chimera occu-pied a position on the right side of space again suggest-ing that the left of the object is disadvantaged even whenit appears on the right of the viewer (also Walker Find-lay Young amp Lincoln 1996) The studies of Pavlovskayaet al (1997) and Grabowecky Robertson and Treisman(1993) used geometric shapes and showed that informa-tion falling to the left of the center of mass of an objectwas less well detected than information appearing to theright These data presuppose a computation of a center ofmass that is specific to the object the subsequent deter-mination of the object midline and the neglect of infor-mation to the left of this midline (see also Driver BaylisGoodrich amp Rafal 1994 Driver amp Halligan 1991) Con-sistent with this using a barbell stimulus with differentlycolored ends Behrmann and Tipper (1994 1999 Tipperamp Behrmann 1996) reported that the left of the barbellwas poorly processed even when it appeared on the rightof the viewer or of the environment A final finding thatis consistent with object-centered coding is that de-pending on the region to be searched in a visual searchtask patients show neglect defined by the borders of therelevant region (Karnath amp Niemeier in press) Whensubjects searched a large array subtending 180ordm theyshowed significant neglect for the left side of the arrayWhen subjects searched only a subset of this large arrayconstituting a 40ordm extent on the patientrsquos right side anddemarcated by having items in the relevant region dis-played in a particular color patients neglected the left ofthis small right-sided segment even though this area waswell searched initially

The findings of leftndashright coding with respect to the ob-ject midline are also consistent with data from studies con-ducted with nonhuman primates Both single-neuronrecording studies and lesion studies indicate a neural se-lectivity for one side of an object (also see Reuter-LorenzDrain amp Hardy-Morais 1996 for related data from nor-mal subjects) For example neural recordings obtainedwhen monkeys saccade to the relative left or right side ofan object show directional selectivity that is independentof the retinal position of the object or of the orbital di-rection of the saccade (Olson 2001 Olson amp Gettner1995 1996 Olson Gettner amp Tremblay 1999) Instead

this selectivity suggests that the neural coding is for aparticular side of the stimulus as coded intrinsically bythe object (see Deneve amp Pouget 1998 and Sabes Brez-nen amp Andersen 2002 for a somewhat different view)

Our Approach Copying Objects With Hierarchical Representations

The goal of this paper is to explore the contribution ofan object-centered spatial representation in neglect andto examine how this might coexist with the well-establishedviewer-based neglect We do so by combining empiricaland computational approaches in the context of a figure-copying task in an attempt to determine which parts areincluded and which neglected by various patients Copyinghas been used previously to characterize object-centeredneglect although the findings from these studies havenot been without criticism For example Gainotti Mes-serli and Tissot (1972) have published illustrations depict-ing neglect of the left side of several objects that werepresent in a scene For example the patient omitted theleft of a house while copying the right of a tree that waslocated further to the left of the house Marshall and Hal-ligan (1993) have also used a figure-copying task to showhow neglect may manifest in viewer- andor object-basedcoordinates and we will consider their findings in detaillater

Although the presence of object-based neglect underthe conditions of figure copying is provocative this con-clusion may not be entirely warranted (Driver amp Halligan1991) Because drawing is a sequential task with eachobject being the sole focus of attention for some period oftime the section being drawn becomes the entire environ-ment and so neglect may be determined by environment-rather than object-centered coordinates under these con-ditions It is difficult then to determine the contributionof an object-based reference frame under conditions offree copying and free viewing To circumvent this prob-lem we asked neglect patients to copy a daisy presentedin four different orientations as shown in Figure 2 so asto disambiguate the left and right of the object from the leftand right of other coordinate frames It has also been sug-gested that under conditions of misorientation it is cru-cial to disambiguate the intrinsic left and right to main-tain an objectrsquos identity (eg differentiating between theshape as a square or a diamond Attneave 1971) and anobject frame may be invoked under these conditions specif-ically to achieve this end

Like most natural objects the single daisy we employas a target has a hierarchical structure so that parts of theobject are in themselves objects at a smaller spatial scaleand these then decompose further into their own parts atan even smaller scale (Marr amp Nishihara 1978 Palmer1977) During the copying of a hierarchical figure likethis then a reference frame aligned with the midline ofa subpart of the object serves as the context frame for lo-cating and drawing its subparts Thus the object-centeredframe is not fixed throughout the task rather objects arerecursively decomposed and dynamically assigned to


roles as objects and parts depending on the current rel-evant level of the hierarchy (Hinton 1990) Accountingfor the copying performance of neglect patients (and ofnormal subjects) is complicated therefore because at onepoint in time the context frame may represent the spa-tial coordinates for copying a particular part whereas ata second point in time this same part may itself definethe context frame for the copying of its own subpartsImportantly it is commonly assumed that long-term hi-erarchical object representations are used to structuredrawing and copying (Lee 1989 Taylor amp Tversky 1992)and that these representations are the same as those thatmediate perception (Kosslyn 1987 Van Sommers 1989)In the case of the daisy we assume that the hierarchicalrepresentation is composed of three major parts ( parents)each of which can be broken down into their subparts(children see Figure 3) These children are decomposedfurther For example the central stem decomposes intothe oblique stems which break down further to encom-pass the leaves The representation used in this study hasin total four levels as is illustrated in Figure 3 We didnot break down simple geometric forms into individualstrokes (such as the pot or the daisy head) since we as-sumed that principles of perceptual organization wouldbe sufficiently strong to maintain the grouping and clo-

sure of primitive elements and resist neglect (Vuilleumieramp Sagiv 2001Vuilleumier Valenza amp Landis 2001)

To verify that this hierarchical object representationadequately captures normal copying performance wehad 20 normal subjects generate three copies of the tar-get daisies presented in each of the four orientations (up-right 90ordm rotation to the left or right and inverted n =240) and we tracked the order of the strokes Copyingperformance was considered to obey the hierarchicalrepresentation if the order in which the components weredrawn followed a depth-first traversal order through thehierarchy (ignoring the order among subparts) In otherwords once a stroke within a particular subtree is drawnall of its components and subcomponents must be drawnbefore a stroke within another subtree at the same levelis drawn Any stroke that did not adhere to this rule wascounted as a violation of the hierarchy In an analysis ofvariance (ANOVA) with daisy orientation as a variableand mean violations per subject as the dependent mea-sure the mean number of hierarchy violations was 13(SD 084) and was not significantly affected by the ori-entation of the daisy (F lt 1) We compared this numberof violations against that obtained from 120 randomlygenerated stroke sequences (mean 172 SD 26) in aone-way ANOVA and obtained a highly reliable differ-

Figure 3 A daisy and its hierarchical representation so that each part (child ) of an object (parent ) can be considered an object in its own right

Figure 2 Targets of copying Single daisy at four different orientations


ence between the distributions [F(1238) = 3953 p lt001] This difference suggests that the normal perfor-mance is not random and instead is orderly and basedon traversing a hierarchical representation such as theone shown in Figure 3 As such this supports our assump-tions about the internal structure of the daisy and we usethis hierarchy in the algorithm we adopt

EXPERIMENT 1 Neglect for Misoriented Single Daisies and

Computational Implementation

In this first experiment we present copying data frompatients with neglect using the same upright and mis-oriented daisies as those used for the normal subjectsWe then attempt to explain the neglect by implementingthe copying performance via a conventional tree traver-sal algorithm over a hierarchical data structure repre-senting the daisy (as in Figure 3) We do so by imposing aspatially defined lesion analogous to the deficit hypoth-esized to underlie the attentional impairment in patientswith right-parietal damage and then evaluating the per-formance of the model and its fit to the empirical data

MethodWe first describe the individual subjects and the methods we used

to obtain the empirical data Following this we describe the meth-ods employed for the computational simulations and then presentthe human and computational results together

Subjects Two neglect patients participated in this experiment The presence of neglect was initially defined by performance on abedside battery consisting of line bisection target cancellation drawing and copying (Black et al 1994) Performance on this bat-tery is measured in relation to boundaries established by controlsubjects Where performance deviates from the norm points areawarded and then on the basis of the final aggregated score sever-ity of neglect is determined The total is 100 points and the normalcutoff is 5 points

JM a 52-year-old right-handed male suffered an extensive rightmiddle cerebral artery infarction in June 1992 affecting the rightparietal cortex as well as the anterior portion of the thalamus Al-though he exhibited a left homonymous hemianopia initially thishad resolved by the time of this testing JM is also mildly hemi-paretic on the left although he walks unassisted He was self-employedas an engineer until the time of his stroke but has not returned towork He has participated in several other experiments (BehrmannGhiselli-Crippa amp Di Matteo 2002 Behrmann Ghiselli-Crippa Sweeney Di Matteo amp Kass 2002 Philbeck Behrmann Black ampEbert 2000) and the reader is referred to those papers for addi-tional biographical and lesion details JM obtained a neglect scoreof 69100 indicative of neglect in the moderate to severe range

GS is a 65-year-old right-handed male who was admitted to ahospital in early January 1996 following a history of hypertensio nand an incident of left upper extremity weakness and nausea A follow-up CT scan 10 days later indicated a resolving hemorrhagiclesion of the right parietal lobe with mass effect and decreased at-tenuation extending anteriorly to the frontal lobe consistent withedema He exhibited moderate hemineglect (41100) on bedsidetesting 2 months later as part of this study Although he had a lefttemporal f ield cut initially this had resolved by the time of testingand he was not hemiparetic

Procedure for human subjects The target picture of an indi-vidual daisy centered on a sheet of paper and a blank sheet ofpaper were placed in front of the subject with the latter in closer

proximity to the subject The center of the page was initially alignedwith the midline of the subjectrsquos head eyes and trunk and of thetable although the midlines likely shifted during the copying taskas the subject moved his eyes head or trunk The subject was in-structed to copy the daisy by using the dominant (right) hand totake as long as necessary to do so and to indicate when the taskwas complete There were four targets each containing a singledaisy in a different orientation (upright 90ordm left rotation inverted90ordm right rotation) and each picture was presented twice for a totalof eight pictures per subject

Procedure for computational implementation We instanti-ated the copying task in a computational simulation in order to ex-plore the implications of a spatial impairment in object- and viewer-centered reference frames We adopted the hierarchical representatio ndepicted in Figure 3 and supported by the data from the normal sub-jects and implemented it as a conventional tree data structure in whicheach node in the tree corresponded to a particular part of the daisyThe node for a part contained information on its location in the object-centered frame defined by its parent Specifically the object-centere dframe for a part was oriented and centered on its parent with a scaledefined by the horizontal extent of the parent (with x-coordinatesranging between +1 and -1) The viewer-centered frame was alwaysupright centered on the page and scaled by the horizontal extent ofthe daisy Thus for instance the rightmost petal in the upright daisyhas a viewer-centered x-coordinate of about 05 (ie the horizontalposition of its center is about halfway between the midline of the daisyand the tip of the right leaf) and an object-centered x-coordinate ofabout 20 (ie its horizontal distance from the center of its parent thecircle about twice the radius of the circle) For a misoriented daisythe viewer-centered positions of parts changed accordingly but theirobject-centered positions remained the same

For a particular orientation of the daisy the probability that a partwould be included and drawn in a particular frame was assumed tobe a monotonically increasing function of its horizontal position inthe frame (Figure 4) The specific (exponential) form of this func-tion is not critical since it influences only quantitative aspects ofthe results slightly different functions have similar consequences and the actual function probably differs from patient to patient inany event (Mozer in press Niemeier amp Karnath 2002) Impor-tantly the assumption of a leftndashright gradient is consistent withviews of neglect in which there is a weak-to-strong representatio nfrom left to right This gradient not only fits with existing views ofneglect (Kinsbourne 1977 1994) and its neural underpinning s(Pouget amp Driver 2000) but also has been successfully adopted inmany computational models of neglect (Monaghan amp Shillcock1998 Mozer amp Behrmann 1990 Pouget amp Driver 2000) Note thatwith the function we have adopted the probability of drawing a partis near 10 on the right side of the frame about 9 at the midline anddrops off sharply toward the left of the frame The overall likeli-hood that a part is drawn was assumed to be a weighted average ofits separate probabilities in the viewer-centered frame and in the object-centered frame (the effects of different relative weightingsare explored below) This assumption emerges from the finding thatneglect in different reference frames appears to be additive ratherthan multiplicative (Behrmann amp Tipper 1999) Furthermore thereare now several reports of clear dissociations between object- andviewer-based effects attesting to the apparent distinctiveness ofthese spatial representations (Humphreys amp Heinke 1998) All elsebeing equal in this implementation the effect of neglect is gener-ally stronger in the object-center ed frame than in the viewer-centered frame because the former is defined more locally (ieparts typically fall outside the +1-1 frame defined by the horizon-tal extent of their parents)

A depth-first tree traversal algorithm was used to determine theneglect pattern At every node the probability that the correspond -ing part is drawn is calculated on the basis of its viewer-centere d(assumed to remain fixed) and object-centered (defined relative toits parent) coordinates We assumed that if a part is not drawn none


of its subparts would be drawn Thus the probability of a partrsquosbeing drawn is the product of the probability of its parentrsquos beingdrawn and its own local probability based on its relative positionsin the viewer- and object-centered frames The order of traversalamong children of the same parent was irrelevant The outcome ofthe tree traversal was that every part was assigned a probability ofbeing drawn that was based on the orientation of the daisy and theparticular weightings of the viewer- and object-centered framesOnce the probabilities are calculated the program generates a coarse(piecewise linear) graphical rendition of the daisy and superim-poses the probabilities on it We present these graphical renditions In addition to evaluate the fit to the patient data we establish a thresh-old so that those parts whose probability falls below the thresholdare omitted in the final rendition We can then compare the actualldquodrawingsrdquo of the model with those of the patient and analyticallyevaluate the goodness of fit

Results and DiscussionTo understand the boundary conditions of the imple-

mentation we first explored the individual contribution ofthe viewer- and object-centered frame To do so we calcu-

lated the probability of each partrsquos being drawn for daisiesin all four orientationsmdashup left down and rightmdashandinitially the weighting of either the viewer- or the object-centered effect was set at 1 whereas the other effect wasset at 0 Because the misoriented but not upright daisyallows for the decoupling of the viewer- and the object-centered effects Figure 5 illustrates the independent con-tributions of viewer-centered neglect and object-centeredneglect in a left-facing daisy The numbers superim-posed on the daisy indicate the probability of each fea-turersquos being drawn calculated according to the algorithmdescribed above It is important to recognize that theprobability of a partrsquos being drawn is contingent on theprobability of its parentrsquos being drawnmdashif the parent orcontaining objects is omitted so is the child The proba-bilities for the subparts such as the petals and leavestherefore reflect the conditional probability of parent andchildrsquos both being drawn and are subsequently always lowerthan the probability of the parentrsquos being drawn alone

Figure 5 The probabilities that the parts of a left-facing daisy are drawn when neglect operates (A) solely inthe viewer-centered frame and (B) solely in the object-centered frame(s)

Figure 4 Function depicting the probability of drawing a part as a function of its horizontalposition The function is applied to both the viewer- and the object-based reference frames


As is evident from this figure when the viewer-centeredinfluence is 10 with no object-centered influence (Fig-ure 5A) information on the viewer-centered left has afairly low probability of being drawn with the probabil-ity of the daisy head at 75 and that of the petal that occu-pies the leftmost position at 38 It is interesting to notethat while the daisy head has a 75 probability the petalsto the relative right of the daisy head defined by the viewerhave a lower probability (62 and 63) because their prob-abilities are contingent on the daisy head Thus evenwhen the gradient is imposed purely egocentrically thereis still some influence of the object structure on perfor-mance The effect of inheritance is even more dramaticallyobserved in the right panel When the viewer-centeredeffect is set to have no influence and neglect arises solelywithin the object-centered frame (Figure 5B) informa-tion to the right of the canonical midline of the daisy hasa high probability of being drawn (approximately 94)whereas the petals and leaf on the left of the intrinsic axishave a very low probability of being drawn (approxi-mately 24) The leaf on the canonical left stem has aprobability of 06 both because it is conditional on itsparent stemrsquos being drawn and because it occupies themost extreme left position in the object-centered frameOf note then is that the neglect is more marked in theobject-centered than in the viewer-centered case Wenow explore the implications of these effects for humanperformance and determine whether these referenceframes and combinations thereof can provide an accountof the individual patientrsquos copying

Both patients showed neglect in their copying of the up-right daisy Note that because the standard copying taskconfounds the influences of reference frames centered onthe viewer the environment and the object we cannot de-termine the individual contribution of these different ref-erence frames to performance The critical data then comefrom the performance of the patients on the misorienteddaisies We discuss JMrsquos data first followed by those ofGS

Figure 6A presents examples of JMrsquos copy of one ofeach target daisy In order to account for his performancewe selected coefficients that would best reproduce thefindings the relative weightings of viewer- and object-centered neglect selected were 6 and 4 respectively Theresultant numerical values for each part are shown inFigure 6B and in Figure 6C we display the output of themodel when a threshold of 57 is applied to the data toreflect which features would be neglected Note that wedepict the targets with rounded leaves as in Figures 2and 3 and the output of the model with more rectangu-lar leaves as in Figure 6 in order to differentiate betweenthe two

As can be seen from Figure 6 JMrsquos data are reason-ably well captured by this mixture of object- and viewer-centered neglect The upright daisy produced by the modelis a close match to his copy with the exception of the leftstemleaf Of more interest are the misoriented daisies Theleft-facing daisy reflects a combination of the viewer- and

the object-based neglect since petals to the viewer-leftand object-left are omitted Oddly the daisy does notcontain one of the object-right petals (Figure 6A secondfrom left) As it turns out JM initially drew this petal andthen erased it removing a small part of the circle alongwith it The output of the model is a reasonable matchshowing the omission of petals to the left in both framesalthough again JM includes the leaf on the object-leftbut the model does not The match between model andpatient on the inverted daisy is good aside from the dis-crepant object-left leaf again and reflects very little ne-glect when the left of the object appears on the right ofthe viewer and vice versa the decrement for the left of theobject is balanced by the strength of the right of the viewerand there is apparent compensation for the neglect Thispattern arises again from a combination of object- andviewer-centered neglect and is consistent with the factthat patients are better able to detect a probe on the leftof the object when the object is located on the right of theviewer than when it is located on the left of the viewer(Behrmann amp Tipper 1994)

Thus far the output of the model does a fairly goodjob of accounting for JMrsquos performance with the ex-ception of the leaf on the object-left an issue that we re-turn to later A discrepancy between the model and the pa-tient however is observed on the right-facing daisy(Figure 6A extreme right) JM omits petals on the leftof the daisy head defined by the viewer frame but therest of the daisy is included The model on the other handomits the left stemleaf as above but retains all thepetals A possible explanation for this discrepancy concernsthe order of drawing JM drew the daisy head first andbecause the daisy head in isolation is symmetrical andhas no intrinsic axis the orientation of the daisy headpresented alone is ambiguous Note that under this con-dition there is no other information on the page such asthe stem or the pot to constrain the reference frameGiven the absence of constraints the petals on the left ofthe daisy head may be def ined initially as object-leftandor viewer-left and neglected Once the patient moveson to copy the remaining features of the daisy the orien-tation is anchored and the stem and the pot can then con-tribute to defining the coordinates Although this inter-pretation is speculative at present and we do not accountfor the temporal order and ambiguity effects in our pres-ent implementation we show below that this pattern israther commonly observed when patients draw the daisyhead first As we also show it is less common when thedaisy head is not drawn first lending support to this par-ticular interpretation

To quantify the goodness of fit between the model andthe patient we computed a contingency coefficient thatreflects the degree to which the model draws or omits thesame figure elements as JM collapsing across the fourdaisy orientations For comparison we also computedequivalent coefficients for three alternative models onewith solely viewer-centered neglect one with solely object-centered neglect and one with randomly distributed omis-


sions (with the same rate at each orientation as JM) Inthe first two cases drawing thresholds were defined toapproximate the number of omissions produced by theoriginal model Across the four orientations JM neglects10 out of a total of 60 figure elements (15 at each of fourorientations) The original model based on a combina-tion of 6 viewer and 4 object neglect omitted 11 ele-ments yielding a contingency coefficient of 33 (45 hits5 correct rejections 3 false positives 7 misses) By con-trast the other models all produced lower contingencycoefficients pure viewer-centered neglect with a thresh-old of 65 yields 10 omissions and a coefficient of 21pure object-centered neglect with a threshold of 24yields 12 omissions and a coefficient of 14 and finally10 randomly distributed omissions gives rise to a coeffi-cient of 07

Having established that the original model produces asimilar reproduction of JMrsquos performance and that thefit of the model to the data is reasonable we go on to ex-amine whether a different set of parameters in the samemodel can account for the behavior of the second patientGS Figures 7A(i) and 7A(ii) contain respectively thefirst and second of GSrsquos copies of each of the daisies inthe different orientations (these were drawn in counter-balanced order) Figure 7B presents the display depict-ing the probabilities associated with drawing individualparts and Figure 7C shows the rendition of the modelusing a threshold value For GS we use a 25 and 75weighting of the viewer frame and object frame and thethreshold for the final rendition is 55 (close to 57 for JM)

In his copies of the upright daisy GS demonstratesmarked neglect Interestingly in some but not all of the

Figure 6 (A) Copy of daisies by JM (B) Probability of drawing each part as function of algorithm produced by a mixture of 6viewer-centered neglect and 4 object-centered neglect (C) Output of model assuming a threshold probability of drawing a part of 57


Figure 7 A(i and ii) copy of daisies by GS (B) Probability of drawing each part as function of algorithm produced by a mixtureof 75 viewer-centered neglect and 25 object-centered neglect (C) Output of model assuming a threshold probability of drawing apart of 55


copies (compare Figure 7A upright and left-facing) heshows contrapositioning of the left branch and leaf Thisinclusion of a part on the incorrect side is not uncom-mon in neglect and is thought to reflect correct activationof object structure but with imprecise spatial position-ing (di Pellegrino 1995 Halligan Marshall amp Wade1992a 1992b Vallar Rusconi amp Bisiach 1994) Themodel reproduces the upright figure quite well althoughit is not equipped to deal with this variability in includ-ing or transposing the left stem and leaf GS exhibitsstrong object neglect as is evident in the left-facingdaisy In addition to omission of object-left petals in onecase the canonical left of the pot is excluded and in theother the lip of the pot is not connected to the base on thecanonical left The model captures the strong object-based influence reasonably well again with the excep-tion of how it deals with the variable transposition of theleaf Also as was mentioned previously we have notmade allowance for fragmentation of the simple ele-ments such as the pot itself into its components andhence we cannot reproduce the neglect of the line on theleft of the pot or the incomplete lipndashbase connection (al-though this limitation would be straightforward to rem-edy by increasing the depth of the hierarchical tree to in-clude line features)

In both copies of the inverted daisy and one of theright-facing daisy ( panel A[i] ) GS drew the daisy headfirst and petals on the left of the daisy head are omittedAs was discussed above the absence of a constrainingframe for the symmetrical daisy head might have givenrise to the neglect of these petals but under these condi-tions it is not possible to determine the separate influ-ence of the viewer or the object reference frame We havesuggested above that when there is a frame that constrainsperformance initially and the daisy head is not drawnfirst the neglect for the petals should not be as evidentInterestingly on GSrsquos copy of the second right-facingdaisy (panel A[ii] ) he did not draw the daisy head firstbut drew the pot first followed by the stem In direct com-parison with the same right-facing daisy in Figure 7A(i)he now shows only mild if any neglect of petals fromthe left of the daisy head including six (rather than four)petals here (with perhaps some contrapositioning or al-

lowance for positioning of the stem as is also the casewith the petals on the inverted daisy in panel A[ii] )

The discrepancies we have described above in termsof both transposition and temporal order of the daisy headpredictably manifest in a lower contingency coefficientfor the goodness of fit between the modelrsquos performanceand that of GS Over the two versions of each daisy GSneglects 165 out of 60 elements on average The modelomits 19 elements and yields a contingency coefficientof 20 (cf 33 for JM) Although this fit is substantiallybetter than that derived from randomly distributed omis-sions (coefficient of 05) it is only marginally better thanthe fit for pure object-centered neglect (19) and in fact isworse than that for pure viewer-centered neglect (28)The advantage of the last model directly reflects GSrsquosinitial viewer-centered behavior when he draws the daisyhead first In Experiment 2 we explore this model furtherwhen GS draws a different stimulus and show that whenthe transposition and temporal order issues are not in-volved there is a clear improvement of the goodness offit of the model to GSrsquos drawings

We also had the opportunity to obtain partial data froma third patient VD who was not well enough to com-plete the entire experiment and we include the availablecopies (note the two right-handed daisies in which thedaisy head was drawn first) in Figure 8 VD suffered a rightmiddle cerebral artery infarction at age 70 and scored37100 (mild to moderate neglect) on the bedside batteryInterestingly VD has strong viewer-centered neglect asis manifest in her omission of the entire pot and the un-usual pot completion in the two right-handed daisieswhen the pot is on the viewer-left She also omits the leftof the pot and the daisies on the viewer-left in the in-verted case Her pattern might be accounted for by astrong perhaps even sole contribution of viewer-centeredneglect (see Figure 5 for 100 viewer-centered neglect)with the constraint of temporal order of daisy head firstUnfortunately we do not have the full complement of herdata to evaluate the exact fit of the model to all the data

As is evident from the above discussion both PatientsJM and GS show the simultaneous effect of viewer-and object-based neglect when copying upright and mis-oriented daisies and the implemented algorithm with

Figure 8 Copy of some daisies by Patient VD Note that there are two instances of right-facing daisiesa consequence of the counterbalancing of the experiment


differential weighting of these two frames succeeds forthe most part in accounting for their performance Whenthe algorithm fails it does so in similar ways for the twopatients (and for VD too) and the failures are instructiveFor both patients the model does not adequately cope withthe left stem andor leaf The model tends to omit the leafwhereas JM tends to preserve it as does GS either bydrawing it in on the appropriate side or by contraposition-ing it This discrepancy between the model and the pa-tients suggests that there is something unusual about theleft stemleaf One possibility is that because of the rel-ative length of the stem and because of the symmetry ofthe two leaves the stemleaf becomes somewhat resis-tant to neglect The possible benefit afforded by percep-tual organization in offsetting the impact of neglect isalso relevant with regard to other components of the hi-erarchy For example we have not made allowance for theneglect of strokes that make up the pot or that make upthe petal or the leaf Omission of these strokes howeveris not very common in neglect Across all copies we seean instance of this fragmentation in the left-facing daisyfor GS in Figure 7A(i) and the incomplete daisy pot inFigure 7A(ii) as well as in the inverted pot for VD (Fig-ure 8) We suggest that the unity of the elements mayprotect against the neglectmdashhence the rather low fre-quency of this pattern We take up further the issue of ne-glect and the benefits conferred by perceptual organiza-tion in the General Discussion section

The second discrepancy between model and patientsis that of the omission of petals to the left of the daisyhead when the head was drawn first It appears that con-trary to our assumption about the absence of ordering ef-fects the temporal order may be relevant especiallywhen the subpart to be drawn is ambiguous in orientationand when left and right remain unconstrained When thedaisy head was drawn first JM GS and a third patientVD all omitted the petals on the left When other subpartswere drawn first these same petals were not as stronglyneglected A clear prediction then is that provided that

the subparts have a well-defined orientation or other sub-parts are drawn first this pattern of neglect will not beobtained Aside from these limitations that show ways inwhich the patients and model diverge the algorithm andassumptions provide a reasonable account of the mixtureof viewer- and object-centered effects in the copying per-formance of two patients with hemispatial neglect andreflect the combined influence of spatial position de-fined in an object- and a viewer-centered reference frame

EXPERIMENT 2 Neglect for Hierarchically Complex Objects and


The findings reported thus far indicating combinedeffects of viewer- and object-based neglect were achievedthrough the patientsrsquo copying of a single daisy that wasmisoriented to allow for the disambiguation of the dif-ferent reference frames In this second experiment wealso demonstrate how the combination of the differentreference frames can determine the outcome of a figure-copying task In this case however we use a more com-plex object as the target in order to extend the accountThe critical display is a double connected daisy that hasa more complicated hierarchical structure and by virtueof this allows us to observe the relative contribution ofthe viewer-based and object-based effects even when thestimulus remains upright Figure 9A shows the two sin-gle unconnected daisies whereas Figure 9B shows thedouble or connected daisy made of the two single daisiesThese displays are adapted from those used by Marshalland Halligan (1993) and their data and findings are re-ported below Ignoring the left daisy in both the uncon-nected and the connected displays would be indicative ofpure viewer-based neglect In contrast omitting the lefthalf of each daisy in the unconnected case and the entireleft daisy in the connected case (and possibly the petalson the left of the right daisy depending on the hierarchy)would be consistent with object-based neglect Of course

Figure 9 Targets of unconnected and connected daisy displays


various mixtures of these different influences might alsobe observed and we explore these different patterns bothempirically and computationally

As was mentioned above use of the more complex dis-play allows us to examine the influence of object repre-sentations with richer hierarchical structure on neglect In-deed in the f irst experiment some evidence for theimportance of the object hierarchy was obtained despitethe simplicity of the single daisy In that case both JMand GS omitted petals on the left of the daisy head (alsoVD in Figure 8) when the head was drawn first This sug-gests that the head itself although a child in the tree struc-ture may be considered an object or parent initially and itsleft (or the left of the head in viewer coordinates) neglectedbefore other subparts are drawn and that it can serve to an-chor for a particular reference frame In this experimentthen we explore the impact of object complexity on thepatientsrsquo and the modelrsquos performance As before we pre-sent the methods for the patients first followed by a de-scription of the algorithm and its implementation Follow-ing this we report the empirical and computationalfindings in an interleaved fashion along with goodness-of-fit data between the model and the patient output

MethodSubject GS who participated in the f irst experiment also

completed this study JM was unfortunatel y unavailable for test-ing in Experiment 2 We also present published data from 2 patientswith neglect described by Marshall and Halligan (1993)

Procedure for patient To produce a more complex object weused the same daisy as that in Experiment 1 In one condition the un-connected display we included two of these daisies located adjacentto each other centered on the same page with a 5-cm space be-tween them Each of these is an object in itself and so we mightthink of this display as reflecting two objects in a scene Given theprevious comment that we cannot reach definitive conclusions fromscene copying because of the sequential nature of the approach weadopted the design of Marshall and Halligan (1993) who connectedthe two daisies to form a single hierarchically more complex dis-play The daisy heads are of the same size in the two displays andthe connected display is simply formed via the connecting stem andpot as is shown in Figure 9B GS completed two copies of each ofthese two displays Note that the single daisies do not have pots here

Procedure for computational implementation The methodused here is identical to that in Experiment 1 except for the fol-lowing The object hierarchy for the connected daisy is a simplecombination of two single-daisy hierarchies and there is again notemporal order constraining which single daisy is drawn first Thealgorithm is depth f irst so that a single daisy must be completed inits entirety before the second daisy (or any other part) is begunUsing this representation and the same horizontal gradient as weused previously we attempted to simulate the performance of GSon these displays We also adopted the same mixture of weightingsin the two reference frames as that converged on for him in Exper-iment 1 (25 and 75 viewer and object weighting) and also kept thethreshold identical (at 55)

Results and DiscussionTo understand the boundary conditions of the implemen-

tation as before we first explored the individual contri-bution of the viewer- and object-centered frame withthese displays To do so we calculated the probability of

each partrsquos being drawn for the unconnected and theconnected displays Initially the weighting of either theviewer- or the object-centered effect was set at 1 and theother effect was set at 0 Figure 10A shows the effect ofthe viewer reference frame without any influence of an object-centered frame and Figure 10B shows the converse

Let us consider the unconnected case first An impor-tant difference between the two different referenceframes concerns the probabilities associated with thepetals and stemleaf complex on the left of the right daisyThese petals and stemleaf complex occupy a relativeright position in viewer-centered coordinates and thushave a high probability ( petals 93ndash94 stem 94 leaf 88)of being drawn when performance is calculated with a100 viewer frame In contrast when the object-centeredcoordinates determines performance these same petalshave a low probability of being drawn (25ndash36) and theleaf has an even lower probability (06) given that it iscontingent on the stemrsquos (25) being drawn It is alsoworth noting that in the 100 object-centered conditionthe probability of the left petals and leaf rsquos being drawnis equivalent for the daisies on the left and on the rightsince performance is determined only with respect to thedaisy itself and does not take pageviewer position intoaccount In contrast in the 100 viewer-centered casethe contribution of spatial position to the probabilitiesassociated with each part depends solely on the leftndashrightposition with respect to the viewer Thus the further lefta part is located the more the probability is lowered sothat the petals on the left of the left daisy have only a36ndash42 probability of being drawn

One further consideration in both the unconnected andthe connected displays is that in the 100 viewer-centeredcase the probability of drawing the central circle of thedaisy head (97) is higher than the probability of draw-ing the petals to the right of it (95) Indeed it might ap-pear counterintuitive for positions appearing furtherrightward to receive lower probabilities than parts thatappear to their left when probability is purely determinedby the viewer position This effect results from the as-sumption that a child (petal) will be drawn only if theparent (central circle) is drawn and this assumptionbased on the representation of the object and the hierar-chy applies independently of the reference frame Thusa petal will always inherit the probability of its parentdaisy head and will have lower probability because ofthis contingency This apparent discrepancy between petaland daisy head is remedied in the object-centered casein which petals that appear to the right of the daisy mid-line (in both displays and for both petals) have higherprobability than the corresponding daisy center by virtueof their rightward position in object-centered space Thissomewhat higher probability compensates for the lowerprobability associated with hierarchical inheritance

The contrasts between solely viewer-centered and solelyobject-centered effects become even more interestingwhen we compare directly the output of the algorithm onthe connected daisy to that of the unconnected condition


As is evident from the lower left panel of Figure 10 inwhich the viewer-centered frame operates alone at 100the probabilities for the daisy head for the connecteddaisy are identical to those for the unconnected daisiesThis occurs because it is the absolute position of the partsrelative to the viewer that determines the probabilitywhereas the position relative to the object itself has noeffect In the lower right panel we see the effect of the100 object-centered frame on the connected daisy andwe consider each of the two component daisies in turnThe probability of drawing the right daisy head and itsright petals are roughly equivalent to the probability inthe viewer-centered case In contrast the petals on theleft of this right daisy have a low probability of beingdrawn (24ndash35 vs 93ndash94) as compared with the viewer-centered condition and are closer to those in the 100object-centered unconnected case (25ndash36)

An even more interesting contrast comes from examin-ing the fate of the left daisy in the connected 100 object-centered case Here the right petals and stemleaf havea lower probability (68ndash69) than the two single-daisycase (99ndash100) since they occupy relative left positions

in an object frame defined by the entire connected daisyThey do however have a higher probability of being drawnthan the corresponding petals and stemleaf in the 100viewer-centered connected daisy (44ndash55) since theyare on the relative right of the frame defined by the rightdaisy head and are immune to the fact that they are left-ward in a viewer-def ined frame Needless to say thepetals (17ndash25) and stemleaf (0417) on the left of theleftward daisy in the 100 object-centered connecteddisplay have the lowest probabilities of all falling to theleft of the entire connected display as well as to the leftof the left daisy head These probabilities are even lower thanthose in the unconnected case (Figure 10 upper rightpanel) since the petals and stemleaf inherit their prob-ability from their parent the left daisy head which alreadyhas a leftward position in the object-centered frame de-fined by the entire connected daisy and its own reducedprobability of 69 These data show how the connecteddaisy in the 100 object-centered case reflects the posi-tion of the part in the object-centered frame and how thehierarchical representation also affects the probabilitiesby virtue of inheritance They contrast with the simpler

Figure 10 The probabilities that the parts of the single unconnected daisies and the connected daisy are drawnwhen neglect operates (A) solely in the viewer-centered frame (100) and (B) solely in the object-centered frame(100)


case of the viewer-centered effect where performance ismore straightforwardly determined by leftndashright positionin viewer-centered coordinates and where only a smallinfluence of the hierarchy is observed

Having laid out the extreme conditions with the soleinfluence of one of the coordinate systems we can now

evaluate whether the copying performance of patients canbe accounted for within this framework Figure 11 showsthe performance of Patient GS on the two types of dis-plays along with the numerical probabilities of the partsrsquobeing drawn by the model and the thresholded graphicalversions using the same weightings (75 viewer 25 ob-

Figure 11 (A) GSrsquos copy of unconnected and connected daisy display (B) Probability of drawing each part asa function of algorithm produced by a mixture of 75 viewer-centered neglect and 25 object-centered neglect(C) Output of model assuming a threshold probability of drawing a part of 55


ject) and threshold (55) as in Experiment 1 If we considerthe unconnected condition first the model does a rea-sonably good job of capturing his performance showingneglect of the left petals on both daisies The variabilityassociated with the probabilities for the left stemleafwhich gave rise to one of the discrepancies between themodelrsquos and the patientrsquos performance in Experiment 1 isalso seen here The left stemleaf is included on the leftdaisy but surprisingly is omitted on the daisy to its rightPerformance on the connected daisy is also well accountedfor by the model with neglect of the left petals on bothdaisies As in the unconnected case the left stemleaf isvariable in the patientrsquos performance in that it is in-cluded on the left daisy and contrapositioned on the rightWe revisit the issue of the left stemleaf in the final dis-cussion

Had we only had GSrsquos performance on the unconnecteddisplay we would be unable to determine whether theleft neglect is defined by the object or the environmen-tal position given that drawing proceeds sequentiallyUsing the connected conf iguration however we cannow verify not only that the probability of including con-tralesional parts is determined by their viewer-centeredposition but that there is a considerable contribution ofthe object-relative position In fact GS appears to showpredominantly object-centered effects manifesting atmultiple hierarchical levels When a single daisy is the ob-ject its left is neglected and when a connected daisy isthe object the left at multiple hierarchical levels is af-fected with even lower probabilities found further downthe hierarchy by virtue of inheriting the reduced proba-bilities of the parents

Using the same method to compute contingency coef-ficients as that in Experiment 1 we explored the goodnessof fit of our model a pure viewer-centered model (thresh-old of 77 as in Experiment 1) a pure object-centeredmodel (3 as in Experiment 1) and a model based on ran-dom distribution of omissions for the unconnected andconnected daisies On the unconnected daisies GS omit-ted 10 out of a possible 26 parts (counting transpositionand partial omission as omissions) yielding a good coef-ficient of 55 By contrast the viewer-centered and ran-dom models produced weaker coefficients of 14 and 0respectively The pure object model performs equiva-lently to our model and gives a coefficient of 55 On theconnected daisies GS omitted 10 out of a possible 31elements yielding a coefficient of 57 The pure viewer-centered model produces a coefficient of 002 whereasrandomly distributing the 10 omissions yields a coeffi-cient of 16 The pure object model like our model givesa coefficient of 57 Given that our model and the pureobject model yield equivalent coefficients why do wecontinue to favor our model over the object model Theanswer requires that we take the data from Experiment 1and Experiment 2 into consideration Although the mod-els seems equal in accounting for the data from Experi-ment 2 the pure object model cannot account for the datafrom the misoriented daisies in Experiment 1 In addition

the pure viewer model may do a better job of accountingfor the data from Experiment 1 (largely because of thetemporal order influence) but does poorly in accountingfor the data from Experiment 2 Our model which includesboth a viewer- and an object-centered contribution doesthe best job of accounting for both data sets and more-over when it fails (as in Experiment 1) it does so forpredictable and understandable reasons leading us tofavor our combined model over more pure models

The computational results from Experiment 2 havedovetailed rather well overall with the empirical find-ings On the basis of this we would suggest that one candiscover the coefficients that determine the patientrsquos copy-ing performance for both simpler and more complex ob-jects as a function of the spatial position of the parts ofthe display defined in multiple reference frames We wouldalso suggest that the approach we have adopted is gen-eral and can be extended to account for the performanceof other patients both on these kinds of tasks and on oth-ers To explore the generalizability of the approach a lit-tle further we have also determined the coefficients thatreplicate the performance of Marshall and Halliganrsquos(1993) two patients on both the connected and the dis-connected displays and the graphical output of the algo-rithm (shown in Figure 12) and in addition we have com-puted goodness-of-fit data

Marshall and Halligan (1993) originally introducedthe unconnected and connected daisy displays as an ele-gant way of examining the presence of object-centeredneglect and its coexistence with viewer-centered neglectOf relevance they documented the performance of twodifferent patients copying these displays and the outputof the two patientsrsquo performance is shown in Figure 12Their Patient 1 was considered to have 100 viewer-centered neglect according to their analysis since theentire unconnected left daisy is ignored as is the entireleft daisy of the connected display If we adopt a thresh-old of 56 (again very close to that used thus far on ourpatients) on the output of the 100 viewer-centered algo-rithm shown in Figure 10 we obtain a good fit to the data(see Figure 12 upper panel) Note that here the patientomits the left stemleaf in both displays whereas withthis threshold the left stemleaf survives in the model Itis the case however that if we adopted a much more con-servative threshold of 89 we would eliminate the leftstemleaf from the model mirroring the patientrsquos perfor-mance perfectly

The bottom panel in Figure 12 shows the outcome ofthe algorithm for the second patient of Marshall and Hal-ligan (1993) who on their analysis showed a combinedobject viewer neglect pattern This patientrsquos performanceis best captured when the weightings used are 75 object-centered and 25 viewer-centered as was also the casefor GS A more conservative threshold of 75 than thatused for GS however yields a very good reproductionof the data In the unconnected daisy case the left ofeach single daisy is neglected by the patient and themodel The patient includes the left stemleaf of the right


daisy but not of the left daisy but the model neglectsboth In the connected display both the patient and themodel neglect the left daisy entirely and in addition ne-glect the petals to the left of the right daisy The patientrsquoscontrapositioning of the left stemleaf on the right daisyis not reproduced by the model

The goodness of fit of our model to the Marshall andHalligan (1993) data is high Coefficients of 61 and 63 are

obtained for Patient 1 for the unconnected and the con-nected daisies respectively and these values for Patient 2are 64 and 68

The findings from this experiment illustrate how thebasic approach in which empirical performance is sim-ulated in a simple computational simulation outlined inExperiment 1 may be extended when a more complicateddisplay is used The same threshold and weightings used

Figure 12 Copy of (A) unconnected and (B) connected daisies with output of algorithm for two patients re-ported by Marshall and Halligan (1993)


for one patient in Experiment 1 work well to reproducehis data in Experiment 2 testifying to the robustness ofthe results from the first experiment In addition the al-gorithm is able to account for the performance of the twopatients reported by Marshall and Halligan (1993) in oneof the paradigmatic examples of a figure-copying taskand the model produces a very close fit to the patientsrsquodata for both connected and unconnected displays

GENERAL DISCUSSION

The goal of this paper has been to explore how the figure-copying performance of patients with hemispatialneglect might be accounted for by a simple algorithm inwhich the relative probability of informationrsquos being ne-glected or preserved is determined by spatial position Spa-tial position was defined with respect to two differentreference frames one viewer-centered and one object- centered and we examined how these different influ-ences operating alone or in combination give rise topatterns of performance in a figure-copying task In ad-dition to investigating the effects of position in differentreference frames we also manipulated the hierarchicalcomplexity of the objects to be copied and explored theimpact of object complexity on neglect

In the first experiment we required 2 patients to copya single daisy which could appear in one of four orien-tations We had previously verified the hierarchical rep-resentation of this single daisy by tracking the temporalorder of the strokes used by normal subjects in produc-ing such an object and showed that the daisy consisted ofthree children with each of those having children We thenexplored whether a computational algorithm that calcu-lates the probability of a partrsquos being included in a draw-ing on the basis of the spatial position of the part in thetwo reference frames (with the results combined addi-tively) over this hierarchical representation could repro-duce the pattern of data The match between the outputof the algorithm and the patient data was reasonablygood overall and by varying the weighting of the tworeference frames (and by applying a binary threshold)the model was able to produce very similar output to thatof the patients In the one case viewer- and object-centeredweightings of 6 and 4 were successfully used and in theother weightings of 25 and 75 were successful We alsopresented partial data from a 3rd patient in support ofour arguments

In the second experiment we used more complex dis-plays involving two unconnected daisies and a singleconnected daisy made by joining the two single daisies(Marshall amp Halligan 1993) By holding constant theweightings of one of the patients from the first experimentwe were able to reproduce his performance on thesemore complex displays and demonstrate a good fit of ourmodel to the data That we were able to show generaliza-tion of the weightings established initially to a set of noveldisplays suggests that the general approach we adoptedand the specific weightings in his case are robust Through

the dynamic reassignment of elements to object or partsroles this same model can account for neglect of objectson the left of a multiobject scene neglect on the left of asingle object and neglect for features on the left of a partof a single object (for a similar view on within- and between-objects coding see Humphreys amp Riddoch 19931995) We also showed that we could produce a goodrendition of the data (and good quantitative fit) from twopatients copying analogous displays reported by Mar-shall and Halligan (1993)

Strengths and Weaknesses of the AccountAlthough the performance of the model was reason-

ably good overall it failed consistently in some regardsand these instances are in themselves instructive Perhapsthe most noticeable failure concerns the left stemleafNote however that the inclusion or exclusion of theseparts is inconsistent even within a single patient In Ex-periment 1 GS placed both stemleaves to the object rightfor the upright and left-facing daisy but not on his sec-ond copy of either daisy In Experiment 2 he includedthe left stemleaf on the left daisy in both the unconnectedand the connected trials but omitted it on the right daisyin the unconnected display and contrapositioned it in theconnected display We also see some variability associ-ated with this stemleaf in Patient 2 of Marshall and Hal-ligan (1993) in that he included the left stemleaf on theright but not on the left daisy in the unconnected dis-play and contrapositioned it on the right daisy in the con-nected display Under these conditions of variability itmight be unreasonable to expect the model to reproducethe variability but the issue of contrapositioning is an im-portant one This pattern in which stimuli delivered to thecontralesional side are referred to the symmetrical loca-tion on the ipsilesional side also termed allochiria wasrecognized over a century ago (Obersteiner 1882) and maybe observed across multiple sensory modalities (Bisiach ampGeminiani 1991) Clearly not all patients exhibit this phe-nomenon as is evident in our data and as is confirmed byKawamura Kirayama Shinohara Watanabe and Sugishita(1987) who documented this pattern in 20 out of 123 pa-tients who had sustained a cerebral haemorrhage Al-though it has been suggested that there is correct activa-tion of the contralesional information with impreciselocalization the mechanisms underlying contraposition-ing are not well understood nor is the variability from pa-tient to patient (Bisiach amp Vallar 2000) The failure of themodel to reveal this pattern is perhaps not surprisingthen and this issue awaits further clarification

A second noticeable failure of the model is in account-ing for the occasional fragmentation of component parts(as specified in the object hierarchy) For example in Ex-periment 2 on the left-facing daisy GS omitted the leftstroke of the pot defined in object-centered coordinatesThis fragmentation of components into strokes is not verycommon and there are only a few examples in the patientdata reported here Note that patients almost never drawonly the right half of the circle for the head of the flower


or the right part of a petal (eg when the petal is verti-cal) nor do they omit the lip of the pot (if the base isdrawn) even if it occupies a position on the left of the spa-tial reference frame Similarly in clock drawing or copy-ing even if patients neglect to fill in the numbers on theleft of the clock they invariably draw the entire perime-ter of the clock (see Figure 1) A possible explanation forthe rarity of this fragmentation however may be attrib-utable to the apparent preservation of grouping mecha-nisms in these patients For example Vuilleumier et al(2001) reported that some patients are able to judge themidpoint of illusory Kanisza stimuli despite their failureto detect the left-sided inducers in explicit matchingjudgments Several recent studies have also shown thatpatients with neglect remain sensitive to other Gestaltproperties of the stimulus Thus if a feature on the left ofthe objectrsquos midline can be grouped together with a fea-ture on the right to form a ldquogoodrdquo figure on the basis ofprinciples such as good continuation symmetry or clo-sure the left-sided feature is less likely to be neglected(Ward Goodrich amp Driver 1994) Similar effects are ob-tained when the left item can be grouped with the itemson the right by color brightness proximity or collinear-ity for example (Gilchrist Humphreys amp Riddoch1996 Mattingley David amp Driver 1997) The strengthof grouping according to Gestalt heuristics could po-tentially be incorporated into the hierarchical represen-tation adopted here Under conditions of very severe ne-glect or when the elements do not strongly make up a moreglobal conf iguration fragmentation into lower levelstrokes (and neglect thereof ) would still be observed butwhen the neglect is less severe or when the grouping isstrong fragmentation would be resisted An obvious fu-ture direction of research would be to address this issueby manipulating the image to be copied One might pre-dict an increase in neglect when the image is not sym-metrical (as it is here) and indeed neglect might providea useful assay for when elements of a whole are stronglyintegrated and when they are subject to fragmentation

The final difficulty encountered by the model concernsthe petals on the daisy head In Experiment 1 when themisoriented daisies were copied and the daisy head wasdrawn first petals to the left of the head were neglectedThis sometimes gave rise to unusual patterns since whenthe entire daisy was complete the omitted petals occu-pied a position on the right defined within a referenceframe defined by the viewer or by the entire daisy Thispattern was evident in GSrsquos copies in JMrsquos right-facingdaisy and in the performance of a 3rd patient VD forwhom we had only limited data We suggest that this pat-tern emerged because when the symmetrical daisy headalone represents the display the exact reference frame isambiguous and petals to the left are deleted As we havesuggested previously one possible solution to this wouldbe to impose temporal order on the model since in thesecases the patients are following a daisy-head-first strat-egy In this case the reference frame would be ambigu-ous and the petals on the left would be associated with

low probability of inclusion Once other subparts are in-cluded their constrained reference frames would thenhave an impact in subsequently determining what is ne-glected and what is preserved

Object-Based Neglect RevisitedOne of the critical issues dealt with in this paper is the

existence of a frame of reference that is aligned with themidline of an individual object Such a reference framein which the spatial position of object parts depends onthe extent or shape of the object and is independent of theviewer is particularly useful for object recognition andwould serve an important role in viewpoint independenceIn some of his seminal work on structural-descriptiontheory of object recognition Marr (1982 Marr amp Nishi-hara 1978) postulated the presence of a representation inwhich object parts are related directly to each other Atthe outset we provided numerous examples from empir-ical studies from both human and nonhuman primateswhich appear to support such a representation

The existence of an object-centered representation hasnot however gone without challenge Driver and colleagues(Driver 1999 Driver amp Pouget 2000) for example havesuggested that there is no need to invoke a referenceframe that is tied to an individual object Rather theyhave argued that the left and right of an object may becoded solely from onersquos initial egocentric (and viewpoint-dependent) encounter with the object The claim is thatwhen an object is viewed left and right are assigned in apurely egocentric manner in accordance with the strengthof an underlying attentional gradient akin to the one weuse here but defined with respect to the retina (Driver1999 for additional evidence of an attentional gradientsee Kinsbourne 1993) A similar claim is made byPouget and Sejnowski in their modeling work (PougetDeneve amp Sejnowski 1999 Pouget amp Sejnowski 1997)because the left of the object always appears at the poorerend of the gradient relative to the right of the object inboth absolute and relative egocentric space the ipsile-sional information will always dominate over the con-tralesional information which will then be neglected

This view suggests that object-centered coding is notnecessary and that the same pattern of data may be obtainedfrom simply assuming an egocentric gradient IndeedMozer (in press) has conducted simulations of so-calledobject-centered neglect in the context of a computationalmodel MORSEL which assigns spatial position purelyegocentrically (by virtue of a retinotopic attentional gra-dient) and does not have any object-centered representa-tion He shows that this implementation can account fora host of object-centered neglect effects (eg Arguin ampBub 1993a Driver et al 1994 Driver amp Halligan 1991Pavlovskaya et al 1997) In all of these cases the left of theobject always appears further left than the object rightboth absolutely and relatively and so is less activated

Perhaps a more challenging situation is that of the bar-bell data from Behrmann and Tipper (1994 1999 Tip-per amp Behrmann 1996) in which the left of the object does


not always appear further left than the right of the objectIn this paradigm a barbell appears on a screen with theleft and right circles colored in blue or red (and the colorremains constant for a single subject but is counterbal-anced across subjects) In the first static condition a po-sition on the right or left is probed and this position isboth right and left in both viewer and object coordinatesand serves as a baseline against which to compare per-formance in the second condition In the critical rotat-ing condition the barbell is previewed and then under-goes a rotation of 180ordm so that the left defined by thebarbell appears on the right of the viewer and the rightof the barbell appears on the left of the viewer When aspatial position on the viewer-defined right and left isprobed both accuracy and speed of detection are influ-enced by whether this position occupies a right or a leftposition defined by the object Thus when the probe ap-pears on the viewerrsquos right but is on the left of the barbell(which rotated to that side) detection is poorer thanwhen the position is both viewer- and object-right Sim-ilarly when the probe appears on the viewerrsquos left de-tection is better when the position occupies the right ofthe barbell (which rotated in) than when it is both viewer-and object-left In this barbell experiment because the leftof the barbell does not fall further left than the right asimple egocentric gradient cannot obviously account forthe data Instead Mozer (in press) simulated the find-ings in the following way When the barbell appears ini-tially the activation of the left and right is set by thestrength of the egocentric gradient As the barbell turnsbecause of hysteresis of the system the initial activationis pulled along with it and through covert attention iscarried to the new location Probing the new location(end state) then yields poor performance even when theprobe appears on the right since the activation associ-ated with that location has been carried there by thecovert tracking of the moving barbell According toMozer then these simulations demonstrate that the re-sults of the barbell studies do not necessarily implicateobject-based representations

An outstanding question however is what mechanismallows for the representation of the object and its partsunder conditions of misorientation When objects are trans-lated in the picture plane the left of the object always re-mains to the relative left of the right of the object butthis is not true when objects are rotated Two potentialprocesses have been suggested to compensate for thisMozer (in press) suggests that covert attentional trackingrepresents the left and right initially defined egocentri-cally as the objects rotate The second suggested processis mental rotation For example Buxbaum Coslett Mont-gomery and Farah (1996) have suggested that in the caseof misoriented stimuli the stimulus is first normalized toits upright orientation through mental rotation and thenthe relative left is neglected According to their view thenan egocentric gradient can still explain the empirical re-sults in the case of the barbell the patients transform the

rotated barbell to its canonical upright position and thenneglect the left of the ldquouprightrdquo barbell (ie defined grav-itationally or egocentrically now) They base their claim onthe fact that only when they specifically instructed a ne-glect patient to do the mental transformation on the bar-bell paradigm did they obtain the object-centered results

There are problems however with both of the suggestedmechanisms With regard to covert tracking explanationsit is now well established that these patients have prob-lems directing covert (and overt) attention contralesion-ally (Arguin amp Bub 1993b Posner Walker Friedrichamp Rafal 1984) Functional imaging studies have alsoshown that the right parietal region plays a critical rolein directing attention to the left (Corbetta Miezin Shul-man amp Petersen 1993 Nobre et al 1997) and henceafter damage to this region as in the case of neglect at-tentional monitoring either covert or overt would becompromised There is also the problem of how such atracking system might operate when stimuli are staticand do not need to be trackedmdashfor example when astimulus is displayed inverted as with the daisies herethe faces in the study by Young et al (1990) or the wordsin the study by Caramazza and Hillis (1990a) In thesecases there is no opportunity for covert attention tocarry the activation of the egocentric gradient along withit It is precisely under such conditions that one mightthen invoke a process of normalization via mental rota-tion But the involvement of mental rotation to accountfor the results is in itself problematic Unlike Buxbaumet al (1996) Behrmann and Tipper (1994) did not ex-plicitly instruct the patients to perform mental rotationand yet they still obtained the critical pattern of resultsMoreover nothing in the demands of the task (simple lightdetection) would have prompted patients to engage in whatis generally considered to be an effortful time-consumingprocess Furthermore it has been repeatedly demon-strated that the right parietal lobe plays a critical role inmental rotation (Alivasatos amp Petrides 1997 Tagaris et al1997) and that when damaged mental rotation is signif-icantly impaired (Farah amp Hammond 1988) Because theneglect patients typically have extensive damage to theparietal cortex it is unlikely that they are capable of ex-ploiting mental rotation processes Consequently it is un-likely that object-centered effects emerge from covert at-tentional tracking or from normalizing via mental rotation

We have suggested that the results emerge from thefact that subjects represent the structure of viewed ob-jects in terms of a spatial coordinate system that has amidline def ined by the object itself Following braindamage to regions that represent spatial information thecontralateral side of such a representation is adverselyaffected We have also suggested that the use and salienceof such a representation depends importantly on the na-ture of the task The notion that the frame of reference useddepends on the goals of the user or the effector requiredby the task is not novel and is applied in the case of otherreference frames as well (eg see Vecera amp Farah 1994


for normal subjects) For example it has been suggestedthat the ability to attend to various locations in space de-pends on brain areas that are involved in organizing goal-directed actions to them (Colby 1998 Rizzolatti amp Ca-marda 1987 Snyder Batista amp Andersen 1997) Thuswhen a task requires eye movements we expect to ob-serve a robust influence of a frame of reference that isretinocentric and gaze dependent Likewise when thetask requires a reach limb-based coordinates are invokedand are relevant We would suggest that copying an ob-ject especially one that is hierarchically complex is aparticularly good example of a situation in which object-centered representations might need to be invoked Onemight also imagine that object recognition itself requiressuch a representation Indeed as was recognized by Mozer(in press) ldquosurely if demanded by the task people can men-tally construct visual object-based representationsrdquo Wewould suggest that many tasks employ this form of spatialrepresentation and it is on these tasks that object-centeredneglect would be obtained

Individual VariabilityA final issue to be addressed concerns the individual

variability across and within patients Specifically wehave obtained different coefficients and different weight-ings for the two reference frames for each subject andthe question is what determines these weightings acrossthe different patients There are a number of possibilitiesFor example the site of the lesion may be an importantdeterminant of the extent to which different referenceframes are affected given that neuronal populations in dif-ferent regions of the parietal cortex are responsible forcoding spatial position in different coordinate frames Forexample neurons in the lateral intraparietal (LIP) sulcusin monkeys have receptive fields at locations def inedrelative to the retina (and modulated by orbital position)whereas neurons in the ventral intraparietal sulcus rep-resent locations in a head-centered frame (Colby 1998)In addition in LIP a small proportion of neurons are sen-sitive to locations defined in a reference frame tied to anobject (Sabes et al 2002) Thus depending on the siteof the lesion various forms of spatial coding may be dis-rupted We have no clear way of verifying the correla-tion between lesion site and pattern of reference-framecoding in different individuals given the lack of spatialresolution on neuroimaging that is available for humansand so this remains speculative It is interesting to notehowever that there is an asymmetry between the formsof coding suggested by the neurophysiology data in thatobject-fixed effects represent a small proportion of thespatial code in the LIP whereas the stronger effects areviewer centered We should note that consistent with thisasymmetry none of the patients reported in this paperhas pure object-centered neglect the neglect is eitherviewer centered or a mixture of viewer- and object-centeredneglect (this same asymmetry was evident in Behrmannamp Tipper 1999) A prediction from the neurophysiology

data is that it would be rare perhaps impossible to finda patient whose deficit reflected only object-centered ne-glect without any viewer-centered neglect The conversehowever might not be uncommon and Patient VDwhose data we include in Experiment 1 and Patient 1 ofMarshall and Halligan (1993) from Experiment 2 appearto fit this pattern

The final issue concerns the variability within an in-dividual subject We obtained two copies of each figurefrom each patient whose data are reported in full in thispaper In most cases performance was not substantiallydifferent between the various versions of the figures al-though there were occasions on which neglect was some-what milder or more severe (eg we show in Experi-ment 1 that GS showed differences in the extent ofneglect in his two renditions of the right-facing daisy butoffer an explanation for why this is the case see also twoinstances of VDrsquos right-facing daisy) We should notethat there does not seem to be a reversal of the pattern ofdata in any individual patient and the differences appearto be quantitative rather than qualitative A clear exten-sion of the work we have described here is to determinethe degree to which the human performance is stochas-tic and to examine whether the model has predictive gen-erality over numerous trials A more stochastic versionof the model may be able to generalize over differencesin individual performance and to capture the variabilityin a straightforward fashion but this remains to be veri-fied empirically

CONCLUSIONS

This paper presents an approach by which to charac-terize systematically the behavior of a mechanism inwhich hierarchical object representations and multiple ref-erence frames interact to codetermine performance of asystem and its output under damage The simulations arenot intended to be an explicit instantiation of the neuralmechanism underlying neglect or to parallel directly thefunction of the parietal lobe The principles embodied inthis work however are consistent with many views thatargue that the parietal lobe integrates and transformsdata from one set of coordinates to another (Colby 1998Karnath 1994 Stein 1992) How the brain might actu-ally implement a hierarchical representation and how itmight achieve the dynamic reassignment of the compo-nents to parts and wholes are difficult research issues (al-though see Hinton 1990 for a connectionist approach tothese problems) and we have attempted to address thesein the context of hemispatial neglect

The task of copying the figure of a daisy was used inthis research because it is standardly used in the clinicalassessment of neglect and because much is known aboutthe performance of neglect patients on this task The prin-ciples governing the joint effects of neglect in more thanone reference frame as proposed here however are be-lieved to apply more generally Indeed the more general


goal of this research endeavor is to be able to use thismodel to fit data from other copying tasks and from otherstandard tasks that elicit neglect (eg line bisection let-ter cancellation) The work reported here represents aninitial step in this direction and we believe leads to anumber of testable predictions and potential constraintson a system that is thought to underlie spatial represen-tations and its relationship to object recognition

REFERENCES

Alivasatos B amp Petrides M (1997) Functional activation of thehuman brain during mental rotation Neuropsychologia 35 111-118

Andersen R A Essick G K amp Siegel R M (1985) Encoding ofspatial location by posterior parietal neurons Science 230 456-458

Arguin M amp Bub D N (1993a) Evidence for an independent stimulus-centered spatial reference frame from a case of visual hemineglect Cortex 29 349-357

Arguin M amp Bub D N (1993b) Modulation of the directional at-tention deficit in visual neglect by hemispatial factors Brain amp Cog-nition 22 148-160

Attneave F (1971 December) Multistability in perception Scien-tific American 225 63-71

Bartolomeo P amp Chokron S (1999) Egocentric frame of refer-ence Its role in spatial bias after right hemisphere lesions Neu-ropsychologia 37 881-894

Behrmann M Ghiselli-Crippa T amp Di Matteo I (2002) Im-paired initiation but not execution of eye movements in patients withhemispatial neglect Behavioral Neurology 13 1-16

Behrmann M Ghiselli-Crippa T Sweeney X Di Matteo I ampKass R (2002) Mechanisms underlying spatial representation re-vealed through studies of hemispatial neglect Journal of CognitiveNeuroscience 14 272-290

Behrmann M amp Moscovitch M (1994) Object-centered neglect inpatients with unilateral neglect Effects of leftndashright coordinates ofobjects Journal of Cognitive Neuroscience 6 1-16

Behrmann M amp Tipper S P (1994) Object-based attentional mech-anisms Evidence from patients with unilateral neglect In C Umiltagraveamp M Moscovitch (Eds) Attention and performance XV Consciousand nonconscious information processing (pp 351-375) CambridgeMA MIT Press Bradford Books

Behrmann M amp Tipper S P (1999) Attention accesses multiple ref-erence frames Evidence from neglect Journal of Experimental Psy-chology Human Perception amp Performance 25 83-101

Beschin N Cubelli R Della Sala S amp Spinazzola L (1997)Left of what The role of egocentric coordinates in neglect Journalof Neurology Neurosurgery amp Psychiatry 63 483-489

Bisiach E Capitani E amp Porta E (1985) Two basic properties ofspace representation in the brain Evidence from unilateral neglectJournal of Neurology Neurosurgery and Psychiatry 48 141-144

Bisiach E amp Geminiani G (1991) Anosagnosia related to hemiple-gia and hemianopia In G P Prigatano amp D L Schacter (Eds)Awareness of deficit after brain injury (pp 17-39) New York OxfordUniversity Press

Bisiach E amp Vallar G (2000) Unilateral neglect in humans InF Boller amp J Grafman (Eds) Handbook of neuropsycholog y (2nded Vol 1 pp 459-502) Amsterdam North-Holland

Black S E Ebert P Leibovitch F Szalai J Blair N amp Bon-dar J (1994) Recovery in hemispatial neglect [Abstract] Neurol-ogy 45 A178

Buxbaum L J Coslett H B Montgomery M W amp Farah M J(1996) Mental rotation may underlie apparent object-based neglectNeuropsychologia 34 113-126

Caramazza A amp Hillis A E (1990a) Levels of representation co-ordinate frames and unilateral neglect Cognitive Neuropsychology13 391-446

Caramazza A amp Hillis A E (1990b) Spatial representation ofwords in the brain implied by studies of a unilateral neglect patientNature 346 267-269

Cate A amp Behrmann M (2001) Hemispatial neglect Spatial andtemporal influences Manuscript submitted for publication

Chokron S amp Imbert M (1995) Variations of the egocentric refer-ence among normal subjects and a patient with unilateral neglectNeuropsychologia 33 703-711

Colby C L (1998) Action-oriented spatial reference frames in cortexNeuron 20 15-24

Colby C L amp Goldberg M E (1999) Space and attention in pari-etal cortex Annual Review of Neuroscience 22 319-349

Corbetta M Miezin F M Shulman G L amp Petersen S E(1993) A PET study of visusospatial attention Journal of Neuro-science 13 1202-1226

Deneve S amp Pouget A (1998) Neural basis of object-centered repre-sentations In M I Jordan M J Kearns amp S Solla (Eds) Advancesin neural information processing systems (Vol 10) Cambridge MAMIT Press

di Pellegrino G (1995) Clock-drawing in a case of left visuo-spatialneglect A deficit of disengagement Neuropsychologia 33 353-358

Driver J (1999) Egocentric and object-based visual neglect InN Burgess K J Jeffery amp J OrsquoKeefe (Eds) The hippocampal andparietal foundations of spatial behavior (pp 67-89) Oxford OxfordUniversity Press

Driver J Baylis G C Goodrich S amp Rafal R D (1994) Axis-based neglect of visual shape Neuropsychologia 32 1353-1365

Driver J amp Halligan P W (1991) Can visual neglect operate in object-centered coordinates An affirmative study Cognitive Neu-ropsychology 8 475-496

Driver J amp Pouget A (2000) Object-centered visual neglect or rela-tive egocentric neglect Journal of Cognitive Neuroscience 12 542-545

Farah M J Brunn J L Wong A B Wallace M amp Carpenter P(1990) Frames of reference for the allocation of spatial attention Ev-idence from the neglect syndrome Neuropsychologia 28 335-347

Farah M J amp Hammond K M (1988) Mental rotation and orientation-invariant object recognition Dissociable processes Cognition 2929-46

Gainotti G Messerli P amp Tissot R (1972) Qualitative analysisof unilateral spatial neglect in relation to laterality of cerebral lesionsJournal of Neurology Neurosurgery amp Psychiatry 35 545-550

Gilchrist I D Humphreys G W amp Riddoch M J (1996) Group-ing and extinction Evidence for low-level modulation of visual se-lection Cognitive Neuropsychology 13 1223-1249

Grabowecky M Robertson L C amp Treisman A (1993) Preat-tentive processes guide visual search Evidence from patients withunilateral visual neglect Journal of Cognitive Neuroscience 5 288-302

Halligan P W Marshall J C amp Wade D T (1992a) Contrapo-sitioning in a case of visual neglect Neuropsychological Rehabilita-tion 2 125-135

Halligan P W Marshall J C amp Wade D T (1992b) Left on theright Allochiria in a case of left visuo-spatial neglect Journal of Neu-rology Neurosurgery amp Psychiatry 55 717-719

Haywood M amp Coltheart M (2000) Neglect dyslexia and theearly stages of visual word recognition Neurocase 6 33-43

Hillis A E amp Rapp B (1998) Unilateral spatial neglect in dissocia-ble frames of reference A comment on Farah Brunn Wong Wallaceand Carpenter Neuropsychologia 36 1257-1262

Hillis A E Rapp B Benzing L amp Caramazza A (1998) Dis-sociable coordinate frames of unilateral neglect ldquoViewer-centeredrdquoneglect Brain amp Cognition 37 491-526

Hinton G E (1990) Mapping partndashwhole hierarchies in connection-ist networks Artificial Intelligence 46 47-75

Humpreys G W amp Heinke D (1998) Spatial representation in thebrain Neuropsychological and computational constraints VisualCognition 5 9-47

Humphreys G W amp Riddoch M J (1993) Interactions between ob-ject and space systems revealed through neuropsychology In D E Meyer amp S Kornblum (Eds) Attention and performance XIVSynergies in experimental psychology artificial intelligence andcognitive neuroscience (pp143-162) Cambridge MA MIT Press

Humphreys G W amp Riddoch M J (1994) Attention to within-objectand between-object spatial representations Multiple sites for visualselection Cognitive Neuropsychology 11 207-241


Humphreys G W amp Riddoch M J (1995) Separate coding of spacewithin and between perceptual objects Evidence from unilateral vi-sual neglect Cognitive Neuropsycholog y 15 238-311

Karnath H O (1994) Disturbed coordinate transformation in theneural representation of space as the crucial mechanism leading toneglect In P W Halligan amp J C Marshall (Eds) Spatial neglect Po-sition papers on theory and practice (pp 147-150) Hove UK Erl-baum

Karnath H O amp Niemeier M (in press) Task-dependent differencesin the exploratory behavior of patients with spatial neglect Neuropsy-chologia

Karnath H O Schenkel P amp Fischer B (1991) Trunk orientationas the determining factor of the ldquocontralateralrdquo deficit in the neglectsyndrome and as the physical anchor of the internal representation ofbody orientation in space Brain 114 (Pt 4) 1997-2014

Kawamura M Kirayama K Shinohara Y Watanabe Y amp Su-gishita M (1987) Alloaesthesia Brain 110 225-236

Kinsbourne M (1977) Hemi-neglect and hemisphere rivalry In E Wein-stein amp R Friedland (Eds) Hemi-inattention and hemispheric spe-cialization Advances in neurology 18 (pp 41-49) New York Raven

Kinsbourne M (1987) Mechanisms of unilateral neglect In M Jean-nerod (Ed) Neurophysiological and neuropsychological aspects ofspatial neglect (pp 69-86) Amsterdam North-Holland

Kinsbourne M (1993) Orientational bias model of unilateral neglectEvidence from attentional gradients within hemispace In I HRobertson amp J C Marshall (Eds) Unilateral neglect Clinical andexperimental studies (pp 63-86) Hove UK Erlbaum

Kinsbourne M (1994) Mechanisms of neglect Implications for re-habilitation Neuropsychological Rehabilitation 4 151-153

Kooistra C A amp Heilman K M (1989) Hemispatial visual inat-tention masquerading as hemianopia Neurology 39 1125-1127

Kosslyn S M (1987) Seeing and imagining in the cerebral hemispheresA computational approach Psychological Review 94 148-175

Lee M (1989) When is an object not an object The effect of lsquomeaningrsquoupon copying of line drawings British Journal of Psychology 80 15-37

Marr D (1982) Vision San Francisco FreemanMarr D amp Nishihara H K (1978) Representation and recognition

of the spatial organization of three-dimensional shapes Proceedingsof the Royal Society of London Series B 200 269-294

Marshall J C amp Halligan P W (1993) Visuo-spatial neglect Anew copying test to assess perceptual parsing Journal of Neurology240 37-40

Mattingley J B David G amp Driver J (1997) Pre-attentive fill-ing in of visual surfaces in parietal extinction Science 275 671-674

McGlinchey-Berroth R (1997) Visual information processing inhemispatial neglect Trends in Cognitive Sciences 1 91-97

Monaghan P amp Shillcock R (1998) The cross-over effect in uni-lateral neglect Brain 121 907-921

Mozer M C (in press) Frames of reference in unilateral neglect andvisual perception A computational perspective Psychological Re-view

Mozer M C amp Behrmann M (1990) On the interaction of selec-tive attention and lexical knowledge A connectionist account of ne-glect dyslexia Journal of Cognitive Neuroscience 2 96-123

Niemeier M amp Karnath H-O (2002) The exploration of space andobjects in neglect In H-O Karnath A D Milner amp G Vallar (Eds)The cognitive and neural bases of spatial neglect Oxford OxfordUniversity Press

Nobre A C Sebestyen G N Gittleman D R Mesulam M MFrackowiak R S J amp Frith C D (1997) Functional localizationof the system for the visuospatial attention using positron emissiontomography Brain 120 515-533

Obersteiner H (1882) On allochiria A peculiar sensory disorderBrain 4 153-163

Olson C R (2001) Object-based vision and attention in primatesCurrent Opinion in Neurobiolog y 11 171-179

Olson C R amp Gettner S N (1995) Object-centered directionalselectivity in the macaque supplementary eye field Nature 269 985-988

Olson C R amp Gettner S N (1996) Brain representation of object-centered space Current Opinion in Neurobiology 6 165-170

Olson C R Gettner S N amp Tremblay L (1999) Representationof allocentric space in the monkey frontal lobe In N Burgess KGeffrey amp J OrsquoKeefe (Eds) Spatial functions of the hippocampalformation and parietal cortex (pp 359-380) New York Oxford Uni-versity Press

Palmer S E (1977) Hierarchical structure in perceptual representa-tion Cognitive Psychology 9 441-474

Pavlovskaya M Glass I Soroker N Blum B amp Groswasser Z(1997) Coordinate frame for pattern recognition in unilateral spatialneglect Journal of Cognitive Neuroscience 9 824-834

Philbeck J W Behrmann M Black S E amp Ebert P (2000) In-tact spatial updating during locomotion after right posterior parietallesions Neuropsychologia 38 950-963

Posner M I Walker J A Friedrich F J amp Rafal R D (1984)Effects of parietal injury on covert orienting of visual attention Jour-nal of Neuroscience 4 1863-1874

Pouget A Deneve S amp Sejnowski T J (1999) Frames of refer-ence in hemineglect A computational approach Progress in BrainResearch 121 81-97

Pouget A amp Driver J (2000) Relating unilateral neglect to the neuralcoding of space Current Opinion in Neurobiology 10 242-249

Pouget A amp Sejnowski T J (1997) A new view of hemineglectbased on the response properties of parietal neurones PhilosophicalTransactions of the Royal Society of London Series B 352 1449-1459

Reuter-Lorenz P Drain M amp Hardy-Morais C (1996) Object-centered attentional biases in the normal brain Journal of CognitiveNeuroscience 8 540-550

Riddoch M J Humphreys G W Luckhurst L Burroughs Eamp Bateman A (1995) ldquoParadoxical neglectrdquo Spatial representa-tions hemisphere-specific activation and spatial cueing CognitiveNeuropsychology 12 569-604

Rizzolatti G Berti A amp Gallese V (2000) Spatial neglectNeurophysiological bases cortical circuits and theories In F Bolleramp J Grafman (Eds) Handbook of neuropsycholog y (pp 503-538)Amsterdam North-Holland

Rizzolatti G amp Camarda R (1987) Neural circuits for spatial at-tention and unilateral neglect In M Jeannerod (Ed) Neurophysio-logical and neuropsychological aspects of spatial neglect (pp 289-313) Amsterdam North-Holland

Sabes P N Breznen B amp Andersen R A (2002) The parietalrepresentation of object-based saccades Manuscript submitted forpublication

Smania N Martini M Gambina G Tomelleri G Palmara ANatale E amp Marzi C (1998) The spatial distribution of visual at-tention in hemineglect and extinction patients Brain 121 1759-1770

Snyder L H Batista A P amp Andersen R A (1997) Coding ofintention in the posterior parietal cortex Nature 386 167-170

Stein J F (1992) The representation of egocentric space in the pos-terior parietal cortex Behavioral amp Brain Sciences 15 691-700

Tagaris G A Kim S G Strupp J P Andersen P Ugurbil K ampGeorgopolous A P (1997) Mental rotation studied by functionalmagnetic resonance imaging at high field (4 Tesla) Performance andcortical activation Journal of Cognitive Neuroscience 9 419-432

Taylor H A amp Tversky B (1992) Descriptions and depictions ofenvironments Memory amp Cognition 20 483-496

Tipper S P amp Behrmann M (1996) Object-centered not scene-based visual neglect Journal of Experimental Psychology HumanPerception amp Performance 22 1261-1278

Vallar G (1998) Spatial hemineglect in humans Trends in CognitiveSciences 2 87-96

Vallar G Rusconi M L amp Bisiach E (1994) Awareness of con-tralesional information in unilateral neglect Effects of verbal cue-ing tracing and vestibular information In C Umiltagrave amp M Mosco-vitch (Eds) Attention and performance XV Conscious and nonconsciou sinformation processing (pp 377-391) Cambridge MA MIT PressBradford Books

Van Sommers P (1989) A system for drawing and drawing-relatedneuropsycholog y Cognitive Neurospsychology 6 117-164

Vecera S P amp Farah M J (1994) Does visual attention select ob-jects or locations Journal of Experimental Psychology General 123146-160


Vuilleumier P amp Sagiv N (2001) Two eyes make a pair Facial or-ganization and perceptual learning reduce visual extinction Neu-ropsychologia 39 1144-1149

Vuilleumier P Valenza N amp Landis T (2001) Explicit and im-plicit perception of illusory contours in unilateral spatial neglect Be-havioral and anatomical correlates of preattentive grouping mecha-nisms Neuropsychologia 39 597-610

Vuilleumier P Valenza N Mayer E Perrig S amp Landis T(1999) To see better when looking more to the right Effects of gazedirection and frames of spatial coordinates in unilateral neglect Jour-nal of the International Neuropsychological Society 5 75-82

Walker R Findlay J M Young A W amp Lincoln N B (1996)

Saccadic eye movements in object-based neglect Cognitive Neu-ropsychology 13 569-615

Ward R Goodrich S amp Driver J (1994) Grouping reduces visualextinction Neuropsychological evidence for weight-linkage in visualselection Visual Cognition 1 101-129

Young A W Hellawell D J amp Welch J (1992) Neglect and vi-sual recognition Brain 115 51-71

Young A W Newcombe F de Haan E H Newcombe F amp HayD C (1990) Facial neglect Neuropsychologia 28 391-415

(Manuscript received June 4 2001revision accepted for publication October 15 2001)


are defined with respect to the midline of the viewer (egaligned with the gaze head orientation or trunk of theviewer) the environment (eg based on landmarks such asthe walls of a room or defined gravitationally) or the object(eg determined by the intrinsic characteristics of objectssuch as principal axes of elongation or symmetry) Undermost viewing conditions these frames are all aligned sothere is no way to evaluate which reference frame deter-mines the spatial coding of stimuli and the subsequentneglect behavior Recent evidence however obtainedfrom a host of neuropsychological studies suggests thatneglect behavior is sensitive to spatial information de-fined with respect to multiple distinct reference framesIn light of this evidence it becomes important to understandhow information coded in these different referenceframes is integrated to yield coherent behavior

In this paper we examine specifically how informationappearing on the left defined by multiple reference framesmanifests in patientsrsquo figure-copying performance Wefocus on two major reference frames one defined ego-centrically by the midline of the viewer and the other de-fined allocentrically by the midline of an object In ad-dition we explore how spatial coding is affected by thecomplexity of the object being copied We start off byreviewing current findings that support the influence ofthese two forms of reference frame in neglect Then inthe first experiment we present empirical data from a taskin which the neglect patients copy a single daisy presentedin differing orientations To account for these data we for-mulate a computational account of the way in which theactivation of spatial information defined in multiple ref-erence frames may be synthesized to subserve behaviorand we present data obtained from such a computationaldemonstration We extend the account in the second ex-

periment by including empirical data on a more complexfigure-copying task and by showing that the critical as-sumptions underlying the computational model are suf-ficiently general to account for neglect performance underthese more challenging conditions

Object-Centered NeglectEvidence for neglect that is defined with respect to the

midline of the viewer is well established and not partic-ularly controversial For example there is a general con-sensus that early visual information is encoded with re-spect to retinal location and gaze direction (AndersenEssick amp Siegel 1985 Colby amp Goldberg 1999) andthere now exist numerous studies reporting neglect forspatial information appearing to the left of the retinalhead andor trunk midline (eg Bartolomeo amp Chokron1999 Behrmann Ghiselli-Crippa Sweeney Di Matteoamp Kass 2002 Beschin Cubelli Della Sala amp Spinaz-zola 1997 Bisiach Capitani amp Porta 1985 Chokron ampImbert 1995 Hillis amp Rapp 1998 Karnath Schenkel ampFischer 1991 Kooistra amp Heilman 1989 VuilleumierValenza Mayer Perrig amp Landis 1999)

The more controversial question concerns the role of areference frame centered on the midline of an individualobject so that spatial position is located with respect to arepresentation that depends on the objectrsquos extent shapeor motion Within such a representation the relationshipof object parts is defined with respect to each other allo-centrically and independently of the viewerrsquos position Manyrecent studies have examined whether neglect occurs forinformation appearing to the left of a midline defined byan individual object The result of many although not allsuch studies (Farah Brunn Wong Wallace amp Carpenter1990 however see Hillis amp Rapp 1998 for a reanalysis ofthese data) is that patients fail to report information ap-pearing to the left of the object midline even when thisinformation is located to the right of the midline of theviewer andor the environment (Behrmann amp Mosco-vitch 1994 Behrmann amp Tipper 1994 Driver amp Halli-gan 1991 Humphreys amp Riddoch 1995 PavlovskayaGlass Soroker Blum amp Groswasser 1997 YoungHellawell amp Welch 1992)

One of the earliest documented examples of object-based neglect is from Patient NG who had right-sidedneglect and who failed to read the rightmost letters of aword This was true when the word was presented verti-cally in mirror-reversed format and even when she wasrequired to spell words backwards (Caramazza amp Hillis1990a 1990b Hillis Rapp Benzing amp Caramazza 1998)Arguin and Bub (1993a) also showed that their patientrsquosinability to report a target letter in a horizontal array of fourelements depended on the object-relative position of theletter and not on the viewer-relative position In a morerecent series of studies Humphreys Riddoch and theircolleagues have explored several aspects of object-basedneglect and have shown that patients neglect letters posi-tioned to the left of individual words (Humphreys amp Rid-doch 1994 1995 Riddoch Humphreys Luckhurst

Figure 1 Representative examples of left-sided neglect duringa figure-copying task by a neglect patient












































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES




















































































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES











































































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES





































































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES






















































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES















































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES







































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES


































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES
































































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES
























































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES
















































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES



































































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES






























































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES

























































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES

















































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES











































































































GENERAL DISCUSSION





























CONCLUSIONS





REFERENCES






























































































































CONCLUSIONS





REFERENCES





















































































































CONCLUSIONS





REFERENCES















































































































CONCLUSIONS





REFERENCES











































































































REFERENCES















































































































































































Date post:	18-Jul-2020
Category:	Documents
Upload:	others
View:	1 times
Download:	0 times

The interaction of spatial reference frames and ...€¦ · 1987, 1993; Pouget & Driver, 2000;...

Documents