PergamonVisionRes., Vol. 36, No. 23, pp.3805–3814, 1996

Copyright01996 ElsevierScienceLtd.All rightsreservedPII: SO042-6989(96)OOO03-X Printedin GreatBritain

U042-6989/96$15.00+ 0,00

The Special Role of Distant Structures inPerceived Object VelocityELI BRENNER,*~A.V. van den BERG*

Received 31 March 1995; in revisedform 21 September 1995

How do we judge an object’s velocity when we ourselves are moving? Subjects compared thevelocity of a moving object before and during simulated ego-motion. The simulation consisted ofmoving the visible environment relative to the subject’s eye in precisely the way that a staticenvironment would move relative to the eye if the subject had moved, The ensuing motion of thebackground on the screen influenced the perceived target velocity. We found that the motion of the“most distant structure” largely determined the influence of the moving background. Relying onretinal motion relative to that of distant structures is usually a reliable method for accounting forrotations of the eye. It provides an estimate of the object’s movement, relative to the observer. Thisstrategy forjudging object motion has the advantage that it does not require metric information ondepth or detailed knowledge of one’s own motion. Copyright O 1996 Elsevier Science Ltd

Opticflow Perspective Stereopsis Motion Eyemovements

INTRODUCTION

How do we judge a visible object’s velocity when weourselves are moving? The most obvious possibilitywould be to make some kind of “prediction” (thoughnotnecessarily a conscious one) of how our movementswould have shiftedthe object’sretinal image if the objectwere stationary. The difference between the predictedand the actual retinal motion can then be attributed tomotion of the object.

Knowing our own movementswould help make suchpredictions. However, knowing our own motion is notenough. We also need to know the object’s distance. Toavoid confusion,we will use the terms eye-rotation andeye-translation to refer to the rotation and translation ofour eyes relative to the surroundings.We do so to avoidthe term “eye movements”, which is used to describetherotation of the eyes relative to the head. It is easy topredict how eye-rotation influences the object’s retinalmotion. Rotations shift the whole image on the retina tothe same extent. In contrast,withoutknowingthe object’sdistance, it is impossible to predict how eye-translationshifts the object’s retinal image. Translation shifts theimages of structures in the environment in inverseproportion to their distances from the eye. To predictthe object’s retinal motion, therefore, requires indepen-dent information on the object’s distance.

*Departmentof Physiology,ErasmusUniversity,P.O.Box 1738,3000DR, Rotterdam,The Netherlands.

TTo whom all correspondence should be addressed [Fax 31 104367594;ErnaiZbrenner@fysl.fgg.eur.nl].

If we are moving through a rigid, stationary environ-ment, the changingperspectiveas a result of our motiongives rise to systematic changes in the image of theenvironment on our retina (Gibson, 1979; Koenderink,1986).These systematicchanges (the optic flow)provideus with informationon the structure of the surroundings(Rogers & Graham, 1979; Cornilleau-P6r?x& Droulez,1994)as well as on our own motion (Warren & Hannon,1988; Van den Berg, 1992). Additional information onour own motion is normallyavailablefrom variousextra-retinal sources, such as vestibular stimulation, proprio-ception, and so on (e.g. Mergner et al., 1992).Similarly,extra-retinal information on the orientation of our eyescan help us localise the object when we fixate it.

In the present paper we will concentrate on targetmotion in the frontal plane and lateral eye-translations(parallel to the target’s trajectory). When subjects onlyhave extra-retinalinformationon their own motion (i.e.,when they make real lateral movementsin the dark), thetarget distance specified by ocular convergence influ-ences the perceived target motion (though clearly not tothe extent that wouldbe requiredfor accountingfor one’sown movements;Gogel, 1982;Schwarz et al., 1989). Incontrast,when subjectsonly have retinal informationontheir own motion (i.e., when ego-motion is simulatedbymovingthe environment),the target distancespecifiedbyocular convergence (and relative disparity) does notinfluence the perceived target motion (Brenner, 1991).The latter findingcannot be due to the simulationhavingbeen interpreted as motion of the environment(which itactually was) rather than as ego-motion, because themoving environment did influence the perceived target

3805

3806 E. BRENNERand A. V. van den BERG

velocity considerably.A possible explanation is that thevisual information that is used to judge an object’svelocitywhile one is moving does not consistof separatejudgments of ego-motion and target distance, but ofaspects of the image that provide direct estimates of theobject’s motion. In this study we consider two suchpossibilities.

If the object is moving across a surface that is part ofthe stationary environment, one could rely on localrelative motion. The retinal image of the part of thesurface that the object is moving acrosswill undergo thesame shift due to both eye-rotationsand eye-translationsas does the object itself, because they are at the samedistance. In this way, local relative motion could providejudgments of an object’s motion relative to the.szmwundings.However, if the structuresthat have retinalimages adjacent to that of the object are not at the samedistance from the observer as the object, the object’svelocity will be misjudged.

Another way of judging object motion without usingmetric information on distance is by relying on retinalmotionrelative to that of the most distantstructure.To doso, the observer has to determinewhich visible structureis furthestaway from himself.This informationcould,forinstance, be obtained from perspective. The retinalimages of distant structures are hardly shifted by eye-translations.Eye-rotationsshift them to the same extentas they do any other structure. The retinal motion ofdistantstructures(when expressedas an angularvelocity)therefore provides a direct estimate of the influence ofeye-rotations on all retinal motion. This estimate isreliable as long as the distant structures are indeed faraway (in terms of the velocity of eye-translation).Wecould, therefore, account for eye-rotationsby judging allretinal velocities relative to that of the most distantstructure available. In doing so, we would obtain anestimate of object motion relative to our (translating)eye; irrespectiveof changes in the orientationof the eye(eye-rotations).Although this is contrary to our intuitiveimpression of perceived motion, because it implies—forinstance—that a stationary target will appear to movewhen we ourselves move, it could still be the basis forperceived motion, with the distinction as to what hadactually moved (oneself or the object) deferred to a laterstage. An implication of this option is that when thedistant structuresare not far away, object motion will besystematicallymisjudged during eye-translation.

In our previous study, in which we found no influenceof target distance (Brenner, 1991), the target was at thecentre of a distant, frontal plane, well above a simulatedhorizontal surface. Thus, the target’s local surroundingwas the most distant surface. The finding that thesimulated distance had no influence is, therefore,consistentwith judging target velocity both on the basisof local relative motion and on the basis of motionrelative to the most distant structure. We previouslypresentedsome evidence that the resultswere unlikelyto(only) be due to the use of local relative motion. In thepresent study we examine this in more detail, with an

emphasis on whether subjects use the retinal motion ofthe most distant structure in the proposed manner whenestimatinga target’s velocity.

EXPERIMENT1

In the first experimentwe examinewhether modifyingthe stimulus so that the target’s local surrounding is nolongerthe most distantsurface influencesthe results.Thestimuluswas similar to that of the previousstudy,but thetarget moved across a horizontal, ground surface. Whentarget distance was varied, the target’s angular velocityremained the same. This was achieved by scaling thetarget’s simulated velocity together with its simulateddistance. The angular velocity of the most distantstructure was independent of target distance, becausethe simulated ego-motion was always the same. Thus,judging motion relative to the most distant structurepredicts the same results (when expressed as angularvelocities) for all target distances,whereas local relativemotion predicts different angular velocities for differentdistances, because the local angular velocity of thebackgroundvaries across the scene.

Methods

The experiments were conducted using a SiliconGraphicsGTX-21OComputerwith an HL69SG monitor.The image on the screen was 34 cm wide (1280 pixels)and 27 cm from top to bottom (492 pixels). Subjects satwith their head in a chin-rest at 42 cm from the screen;resulting in an image of 44 x 36 deg of visual angle.Images were presented at a frame rate of 120 Hz. LCDshutter spectacles ensured that alternate frames werepresentedto the left and right eyes. Red stimuliwere usedbecause the LCD shutter spectacles work best at longwavelengths(about 33Y0transmissionwhen “open” and0.3% when “shut”). Screen luminancewas 13 cd/m2forlightpixelsand 0.02 cd/m2for dark ones.Each imagewasdrawn in appropriateperspective for the eye that saw it,and for the simulated positions of the target and theobserverat that instant.Apart fromthe stimulus,the roomwas completely dark.

The display is shown schematically in Fig. 1. Thetarget was a small cube that moved from left to rightacrossa simulatedhorizontalplane. This simulatedplaneand a simulated, distant, frontal surface were coveredwith small squares. During the first part of eachpresentation, these two surfaces were static. Only thetarget moved. During the second part of each presenta-tion, the two surfaces could move to the left (with theappropriate changes in perspective). We refer to thisstimulusas a simulated eye-translation(to the right).

The targetwas simulatedto either be halfivaybetweenthe observerand the frontal surface, three-quartersof thedistanceto the surface, or immediatelyin front of it. Thefrontal surface, which was the most distant visiblestructure, was close enough for its image to moveconsiderably on the screen during the simulated eye-translation (see legend of Fig. 2). Relying on motionrelative to the most distant structure, in the manner

PERCEIVEDOBJECTVELOCITY 3807

Experiment1: translation

%

simulation

FIGURE1. Schematicrepresentationof the stimulusbefore andduringsimulated ego-translation. The simulation consisted of moving thebackgroundrelative to the subject’sstatic eye, in preciselythe way thatit would have moved relative to the eye if the eye had moved. Thebackground consisted of 105 squares distributed at random on ahorizontaland a frontal surface.Before the simulatedego-motion[seenfrom above in (a) and (b)], only the target movedon the screen [arrowin (c)]. Ego-motion[arrow at “eye“ in (d)] was simulatedby movingall the squares relative to the static eye [arrows in (e)]. The velocitywith which each square moved across the screen [length of arrows in(f)] wasinversely proportionalto the surface’s simulated distance, so

that the squaresnearbyon the horizontalsurface movedtwice as fast asthose at the back. We examinedhowfast the target had to moveduringthe simulatedego-motion(dashedarrow)for it to appear to continuetomove at the same velocity. Note that the simulation is a pure lateraltranslation of the eye [see thin outline in (d) for the eye’s simulated

position and orientation some time later].

proposed in the Introduction,would attributethis motionon the screen to eye-rotation,rather than translation,and,therefore, make subjectsmisjudge the target’s velocity.

The cube initially moved at slightly more than 6 deg/sec. It filled about 1.3 deg of visual angle (bothhorizontally and vertically). The extent to which thecube’s surfaces were visible depended on the cube’sposition and the distance between the observer’s eyes.Images were calculated separately for each subject (andposition), taking the distance between the individualsubject’seyes into account.Apart from the differencesinbinocularcues, the nearby targetwas lower on the screen,and the image of its upper surface accountedfor a largerpart of its vertical dimension.

Both optic flowand perspectiveonly providedistances

6+--------------- --------

-8<

I I I I I

40 50 60 70 80 90

simulated target distance (cm)

FIGURE2. Range of angularvelocities for which the object appearedto move at the same speed before and during simulated ego-motion(shaded area). Triangles pointing downwards and upwards arerespectively the upper and lower limits of the range (average of fivesubjectswith standarddeviationbetweensubjects).The target’s initialangularvelocity was slightlyover 6 deg/sec to the right (dashedline).Simulatedego-motionat 10cm/sec to the right shifts surfaces at 45 cm(distance of the nearest target) to the left at about 12 deg/sec and onesat 90 cm (distance of furthest target) to the left at about 6 deg/sec. Inorder to maintain the simulatedtarget velocity, subjectswouldhave tocompensate for such shifts (thin curve). For targets moving across asurface, they could do so by maintaining the local relative velocity.The judged object velocity would be relative to the surroundings.Inorder to judge objects’ velocities relative to themselves, subjects onlyhave to account for their eye-rotations. If they use extra-retinalinformationto estimate their eye-rotation,they shouldsimplymaintainthe target’s angularvelocity(dashedline). If they use the retinal slip ofthe image of the most distant structure to estimate eye-rotation, themovementof thebackgroundwill be mistakenfor the consequenceof arotation (thick line). The only proposal that falls within theexperimentally determined range of subjective equality is that ofjudging object velocity relative to the most distant structure. Thesimilarity between the data with (solid symbols) and without (opensymbols)distance informationfrom binocularstereopsis suggests thatperspective determineswhich structure is consideredthe most distant.

relative to a scaiing factor. The sizes, distances andvelocities given below are all based on the assumptionthat subjects use the distance between their eyes as thescaling factor. This places the simulated horizontalsurface 10cm below the subject’s eyes, and the distantfrontal surface (50 x 20 cm) at a distance of 91 cm. Thisis the only scaling factor for which the relationshipsbetween distancesspecifiedby perspectiveand binocularstereopsisare consistent.However, if subjectsdo not usethe distancebetween their eyes as the scaling factor, but,for instance, use their eye height instead (assuming thatthe horizontalplane is the ground they are standing on),all simulated sizes and velocitieswill be about 17 timeslarger. The angular velocity obviously does not dependon the scaling factor.

With the distance between the eyes as the scalingfactor, the cube moved at simulateddistancesof 45,67.5

..-

3808 E. BRENNERand A. V. van den BERG

or 90 cm from the observer.To be certain that we do notconfound the influence of retinal size and angularvelocity with that of simulated distance, the target’sangularsize and initialangularvelocitywere the sameforall distances. As a result, the simulated target size andvelocity changed in proportion to the simulated targetdistance:at 45 cm the cube had sides of 1 cm and movedat 5 cm/see; at 67.5 cm this was 1.5cm and 7.5 cm/see;and at 90 cm it was 2 cm and 10 cm/sec. The frontal andhorizontal planes contained no visible structures otherthan 35 (frontalplane) and 70 (horizontalplane) 2 x 2 cmsquares.

The experiment was conducted under two conditions:with and without binocular stereopsis. In the formercondition,the imagespresentedto each eye correspondedwith that eye’s position, as described above. In the lattercondition, images presented to both eyes were identicalshowing the view from a position midway between theeyes (alternate images were presented to the two eyeswith the LCD spectacles, but this did not provide anyadditional depth information). The images were super-imposed on the screen, so that the vergence anglerequired to fuse the images correspondedwith the screendistance of 42 cm.

The way in which the velocities before and duringsimulatedego-motionwere comparedwas essentiallythesame as in previous studies (Brenner, 1991, 1993;Brenner & van den Berg, 1994).Subjectswere presentedwith a target moving to the right across a staticbackground for between 500 and 750 msec (randomvariations in duration were used to discourage subjectsfrom relying on the target’s initial and final position,rather than on its velocity). When the target was at theverticalmidline,the backgroundsuddenlystartedmovingto the left, simulating rightward motion of the observer(at 10cm/see). At the same moment the target’svelocitycould change.The target and observermoved at theirnewvelocitiesfor another250-500 msec, after which subjectshad to indicate whether the target moved faster, at thesame speed, or more slowly during the simulated ego-motion.

For finding the velocity at which subjects switchedfrom seeingno change in velocityto seeingan increaseinvelocity, the staircase procedure was as follows: if thesubject reported that the target accelerated, the target’sfinal speed was set lower on the next presentation.If thesubject either reported that it did not change its speed, orthat its motion during the final interval was slower, itsfirialspeed was set higher on the next presentation.Themagnitude of the increase or decrease was reduced (to80% of the previous value) after each trial (in 11 stepsfrom 5 to 0.5 cm/see). The value onto which the staircaseconverged was taken as the upper limit of the range ofsubjective equality (the transition from no perceivedchange to a perceived increase in velocity). The lowerlimit of the range of subjectiveequality (transition fromno perceived change to a perceived decrease in velocity)was determined in the same manner, except that reportsof no change in speed resulted in a lower (rather than a

higher) velocity on the next presentation (for additionaldetails see Brenner, 1991).

The staircases for all distances, for presentationswithand without binocular information on distance, and forboth the upper and lower limitsof the range of subjectiveequality, all ran simultaneously,with the specific stair-case to be tested determined at random (from those notyet completed) for each presentation.

Subjects

Subjects were one of the authors (EB) and fourcolleagues who did not know the purpose of theexperiment. The only instruction subjects received wasthat they shouldindicatewhether the targetmoved faster,at the same speed, or more slowly during‘thesimulatedego-motion.They were not instructedon what to do whenin doubt,but had to chooseone of the three responses.Allsubjectshave normal binocular vision.

ResultsFigure 2 shows the range of angular velocities during

the simulated ego-motion for which the target appearedto continue to move at the same speed (the range ofsubjective equality; shaded area). This range wasinfluenced by the target’s simulated distance, but onlyslightly.

If subjectshad ignoredthe backgroundaltogether,theywould haverequiredthat the targetmore or lessmaintainsits angularvelocity relative to themselvesfor it to appearto continuemoving at the same speed (thin dashed line).They did not. In fact they required a decrease in angularvelocity that is close to the decrease that maintains thetarget’s retinal motion relative to that of the most distantstructure (thick line).

The conditions in the experiment were such that thedecrease in angular velocity that maintains the retinalmotion relative to that of the most distant structure wasindependentof the target’s distance: the velocity of ego-motion (10 cm/see) and the distance of the most distantstructure (91 cm)-and thereby the most distant struc-ture’s angular velocity—were always the same. Theactual requireddecreasein angularvelocity (shadedarea)does appear to depend slightlyon the target distance,butthis is much less than would be needed to maintain thelocal relative velocity (thin curve). Note that theconditionswere favorable for relying on local relativemotion: the target was small and seen slightly fromabove; the horizontal surface was quite densely struc-tured; and the top of the cube was separatedby almost 5deg of visual angle from the bottomof the frontal surfacewhen the cube was on its nearest path.

The results were extremely similar for binocularsimulations(solid symbols)and for simulationsin whichbinocular information specified that the image was flat(open symbols). This supports the notion, raised in theIntroduction, that subjects use a strategy that does notrequire metric information on depth. After the experi-ment, subjectswere asked whether they had experiencedvection (that they themselveswere moving) at any time

PERCEIVEDOBJECTVELOCITY 3809

during the experiment. None ever did. They saw thetarget move to the right and the surroundings to theleft; in complete agreement with all extra-retinalinformation. Nevertheless, their judgments of objectvelocity were influenced by the simulated ego-motion.This too supports the use of aspects of the image thatprovide direct estimates of the object’s motion, ratherthan separate judgments of ego-motion and targetdistance.

DiscussionIt is evident from Fig. 2 that subjects do not maintain

the simulated target velocity; neither relative to theenvironment(for instanceby relying on the local relativevelocity) nor relative to themselves (by ignoring thebackground). The perceived velocity was maintainedwhen the relative motion between the target’s retinalimage and that of the distantfrontal surfacewas identicalbefore and during the simulated ego-motion. Thissuggests that the retinal motion of the image of the mostdistantsurface is used to estimate the rotationof our eyesrelative to the surroundings.As the axes of rotation aredifferent for different parts of our body, and we seldommove only our eyes (e.g. Land, 1992), it may not alwaysbe feasible to obtain reliable extra-retinalpredictionsofthe retinal motion caused by our rotations.The proposedmechanismonly requiresthatwe identifythe most distantstructure; presumably from the depth order provided byperspective. The resultingjudgments of object velocityare relative to the observer’seye, disregardingchangesinthe eye’s orientation.

One shortcoming of this experiment is that theoutcome could also be interpreted as a compromisebetween local relative motion and absolutemotionbasedon extra-retinal information. Such a compromise (oftenreferred to as a low gain for the influenceof backgroundmotion) has been found for various tasks (e.g. Raymondet al., 1984;Post& Lott, 1990;Smeets& Brenner, 1995).Although we initially found that retinal informationdominates the perceived velocity in this task (Brenner,1991),we have since found that extra-retinalinformationcan be quite importantunder someconditions(Brenner&van den Berg, 1994). A compromise between localrelativemotion and the actual angularvelocitycould alsoaccount for the (modest) effect of target distance inFig. 2.

A secondshortcomingof the firstexperimentis that themost distant structure is very large in terms of visualangle, so that the retinal motion of the most distantstructure is also the most preponderous retinal motion.We therefore conducted a second experiment in whichthere was no frontal plane at the end of the horizontalplane, and the predicted direction of the effect wasdifferent for the two hypotheses proposed in theintroduction.

EXPERIMENT2

In order to have opposite directions of backgroundmotion for the most distant structures and for the

Experiment2: combinedtranslationand rotation

FIGURE3. Schematicrepresentationof the stimulusbefore andduringa combinationof simulatedegotranslationandrotation.Thehorizontalsurface (represented by squares) actually consisted of 100 triangles.The rightward rotation and leftward translation [arrows in (d)] aresimulatedby movingthe triangles in the appropriatemannerrelative tothe observer [(e): thin arrows correspond with the simulatedtranslation; thick arrows with the simulated rotation]. The influenceof the simulatedtranslationis larger than that of the simulatedrotationfor nearby structures,and smaller for distant structures, so that nearbystructuresmove to the right whereas distant structuresmove to the left[on the screen; arrows in (~]. The thin outline in (d) shows the eye’ssimulated position and orientation some time later. Note that thesimulatedego-motionis a translationto the left while fixatinga point

behind the target.

structuresclosest to the target, we simulated a combina-tion of ego rotation and translation.As the influenceoftranslation depends on the simulated distance, whereasthat of rotation does not, we can combine simulatedrotation and translation in such a way that the mostdistant structuresmove to the left at the same velocity asthe structuresclosest to the target move to the right. Thiscorresponds with moving to the left while maintainingfixation on a point behind the target (see Fig. 3). Theinfluenceof this complex pattern of background motionwas compared with that of uniform background motion(simulated ego-rotation; see Fig. 4). The most distantstructuresmoved at the same velocity in both conditions(Table 1). In this experimentwe also used a larger fieldofview, with the simulated floor coinciding with the realfloor, in an attempt to make the simulation more“realistic”.

3810 E. BRENNERand A. V. van den BERG

Experiment2: rotation

simulation

FIGURE4. Schematicrepresentationof the stimulusbefore andduringsimulated ego-rotation. The horizontal surface (represented bysquares) actually consisted of 100 triangles. The rightward rotation(d) is simulated by moving the triangles in the appropriate mannerrelative to the observer (e). This results in coherent leftwardmotionof

all the triangles on the screen (~.

MethodsThe experimentwas very similar to the first one. This

time, the stimuluswas projectedonto a large screenusinga Sony VPH-1270QM Multiscan Video Projector. Theimage on the screen was 174cm wide (1280 pixels; 82

deg at 100cm) and 188cm from top to bottom (492pixels; 86 deg at 100cm; bottom 42 cm above the floor).Subjectsstoodwith their backs againsta frame at 100cmfrom the screen. Images were back-projected onto thescreen at a rate of 120 Hz. LCD shutter spectaclesensured that each frame could only be seen with one eye.The eye that was stimulated alternated between frames.Each frame provideda new image;calculatedfor that eyeand simulated displacement. Different images werepresented to the two eyes, taking account of theindividual’sinter-oculardistance.

The groundplanewas simulatedto correspondwith thefloorlevel, taking the individualsubject’seye-heightintoaccount. One hundred randomly oriented triangles (withsides of 25 cm) were distributed in a semi-randomfashion across the ground plane. Only these triangleswere visible. Each triangle was first assigned a randomdistance lying between the closest position we couldpresent on the screen (about 125cm, depending on thesubject’seye height) and the most distantposition in oursimulated environment, which we set at 600 cm. Thetriangle was then assigned a random lateral positionwithin the range of positionsthat would be visible on thescreen.This procedurewas necessaryto ensure that therewere always structures on the ground surface in thevicinity of the target.

The target was a cube with sides of 20 cm. It alwaysmoved to the right, 100cm behind the screen (200 cmfrom the subject). Its initial simulated velocity (beforesimulated ego-motion) was always 1 m/see (thus itsimage moved at 50 cm/sec—about 27 deg/sec-acrossthe screen). The target’s velocity during the simulatedego-motionwas varied as in the firstexperiment,with thestep size decreasing from 0.5 to 0.01 m/see.

There were nine conditions (see Table 1). The onlydifferencebetween the conditionswas the kind and speedof simulated ego-motion. There was one conditionwithout any simulated ego-motion, four with simulatedrotation (turning to the right at four different velocities)and four simulating a combination of translation to theleft and rotation to the right. In the latter four conditions,

TABLE 1. The simulated ego-motion,and how the simulationinfluencesthe motion of selected parts of the background’simage on the screen

Simulated Backgroundvelocity atRotation(deg/see) Translation (m/see) target distance (deg/see) largest distance (deg/see)

Static observer o

Simulatedrotation 59

1418

Simulatedrotation and translation 9182733

0

0000

–0.5–1– 1.5–2

o

–5–9–14–18

591418

0

–5–9–14–18

–5–9–14–18

For the simulated ego-motion,rotating rightwards(as when one pursues a target movingto the right) and translating to the right are consideredpositive.For the background,rightwardmotionof the image is consideredpositive.The target initially movedat about27 deg/sec across thescreen. The simulated target distance was 2 m. The largest simulated distance was 6 m.

PERCEIVEDOBJECTVELOCITY 3811

angularvelocityof floorat distanceof targetduringsimulatedtranslationandrotation(O/s)

o 5 10 15 20L I 1 1 I

-.344-.346

-.536-.742

I \1 I 1

0 -5 -lo -15 -20

angularvelocityof horizon(“/s)

FIGURE 5. Means and standard errors of the outcomes of threereplications for each of the nine conditionsin experiment2 (see Table1) for one subject (EW). White triangles: simulated rotation. Blacktriangles: combined rotation and translation. Shaded triangles: nosimulated ego-motion. Thick line: constant velocity relative to thehorizon. Triangles pointing downwardsand upwards are respectivelythe upper and lower limits of the range of simulated target velocitiesfor which subjects reported perceiving no change in speed. The thinlines are the best fitting lines for the upper and lower limits for eachkindof simulation.The numbersat the rightare these lines’ slopes.Theaverages of the two slopes for simulatedrotation (in this case –0.536and —0.742) and of the two slopes for combined rotation andtranslation(–0.344 and –0.346) are shownfor each subject (with theslopes’ average standard errors) in Fig. 6. The angular velocity of thebackgroundat the distance of the target (upperaxis) is identical to that

of the horizon (lower axis) during simulated rotation.

the rates of simulated translation and rotation werecalculated to result in the same leftward motion of theimages of structures at 600 cm (the most distantstructures) across the screen as for the simulatedrotations, but with the structures at 200 cm (the targetdistance) moving at the same velocity in the oppositedirection. All other aspects, such as the randominterleavingof staircases,were as in the first experiment.

SubjectsTen subjects took part in the second experiment. All

except the two authors were unaware of the hypothesisbeing tested, but were aware of the fact that we werestudying the role of background motion on perceivedvelocity. Four of the subjects performed the completeexperiment three times, whereas the other six performedit once (the variability within subjectswas considerablysmaller than that between subjects). In contrast with ourusual procedure, subjects were explicitly asked to onlyindicate that the target appeared to continue moving atthe same speed when they were quite sure that this wasso. We hoped that this explicit instructionwould reducethe variability between subjects (which it did not).

IT-.5

I

0

-.5-

,,

-1,,

I I I !

,,,.

...,,

,.,,

‘absolute

.,.,,,

.,,,

-1.5 -1 -.5 0 .5

slope for simulated rotation

FIGURE 6. Influence of the background under the two kinds ofsimulations. Individual subjects’ slopes for the change in angularvelocity of the target (requiredto maintain the perceivedvelocity)as afunction of the angularvelocity of the horizon (see Fig. 5). The opensymbolsshowwhere the pointswouldbe expectedif onewere to judgeobject motion exclusively in terms of the object’s displacementrelative to oneself (absolute), local relative motion (local), or motionrelative to the most distant structure(horizon).Note that most subjectsappear to base theirjudgmentson a compromisebetweenthe absolute

velocity and the velocity relative to the horizon.

ResultsFigure 5 shows one subject’sdata for the two types of

simulations.The average outcome of the staircases foreach kind of simulated ego-motion are shown by thetriangles(thiswas one of the subjectswho performedtheexperiment three times). The numbers on the right givethe slopesof regressionlines for each of the four kindsofsymbols(the shadedsymbolswere includedwith both theopen and solid symbols, because the two kinds ofsimulations are obviously identical when the simulatedvelocity of ego-motionis zero).

If the subject had maintained the velocity relative tohimself, the slope would be zero for both simulations.Ifhe had maintained the retinal velocity relative to that ofthe most distantstructure,the slopewould alwaysbe –1.If he had maintained the local relative velocity it wouldhave been 1 for the simulationof combined rotation andtranslation, and – 1 for the simulated rotation. Theaverage of the slopes for the two transitions(from fasterto sameperceivedvelocity [downwardpointingtriangles]and from same to slower perceived velocity [upwardpointing triangles])was determinedfor each subject andeach kind of simulation (ego-rotation; combined ego-rotation and translation). These averages are shown inFig. 6.

The three open symbolsin Fig. 6 indicatewhat subjectswould set if they relied exclusively on the target’svelocity relative to themselves (absolute), relative to theadjacent surrounding (local) or relative to the most

3812 E. BRENNERand A. V. van den BERG

distant structure (horizon). It is evident that subjects donot rely exclusively on any one of these sources ofinformation. Moreover, there are considerable differ-ences in the extent to which subjects rely on retinalinformation. Our main interest was in the retinalcomponent. If subjects combined either of the proposedsources of retinal information in a fixed manner withextra-retinal information,their data should fall on one ofthe dotted lines.

There is a clear tendency to rely on the most distantstructure (negative values of the slope for simulatedrotation and translation) for the retinal contribution,butthe slopes for the combined rotation and translation aregenerally smaller (less negative)than those for simulatedrotation. Two subjects show similar slopes for bothconditions, one shows a slightly higher and three aslightly lower slope for the combined rotation andtranslation. The remaining four subjects show almostno influenceof the background motion in the combinedrotation and translation condition, although they wereinfluenced by the background for simulated rotation.Again, subjects never experiencedvection.

Discussion

The results of the second experiment confirm thatmotion of the images of distant structures provide themost important visual contributionwhen accounting forone’sown motion(negativeverticalvalues in Fig. 6). It isalso evident that subjects do not rely exclusivelyon thismeasure.

When the whole background shifted at a singlevelocity to the left (simulatingego-rotationto the right),the perceived velocity of the target was increased byabouthalf of the velocityof the movingbackground.Thisis the approximate magnitude of the influence of amoving background when subjects are asked to matchvelocities presented during separate intervals (e.g.Smeets & Brenner, 1995). In our previous studies inwhich subjects were asked to make judgments onchanges in target velocity at the onset of simulated ego-motion (Brenner, 1991;Brenner & van den Berg, 1994),the influenceof the movingbackgroundwas considerablylarger.

The apparentlylarger influencein the previousstudies,and in the first experimentof the present study, is partlydue to a shift in emphasis. Until now, we haveemphasised that the range of velocities for which thetarget appeared to continue to move at the same velocityincluded the values at which relative velocity wasmaintained. However, this range usually extendedasymmetricallyaround the value predicted from relativemotion, with most of the range lying in the direction ofthe actualvelocity (as in Fig. 2 of the present study).As aresult, the slope of target velocity as a function ofbackgroundvelocity is less steep. The fact that the rangeof velocities for which the target was reported to appearto continue to move at the same velocity in thisexperiment (e.g. Fig. 5) often did not include the valueone would expect on the basis of relative motion, is

probably partly due to our explicit instructions to keepthis range as small as possible.

The larger contributionof extra-retinal information inthe present experiment may also have to do with thehigher target velocity (27 deg/see), although targetvelocity appeared to make little difference at lowervelocities(6–12deg/see;Brenner& van den Berg, 1994).Alternatively, the differences may not really be due toextra-retinal information at all. The large projectionscreen has the disadvantagethat it is impossibleto keepthe roomdark enoughto preventsubjectsfrom seeinganystationarycontours (such as the edges of the screen andtexture on the floor in front of the screen). Such staticcontoursshouldbe irrelevant(assumingthat we base ourjudgments of objectvelocity on motion of the structuresthat are perceived to be most distant), because thesecontours are always very close to the subject. However,several subjectsexplicitlyreported that the visibleborderof the screen influenced their judgments. Presumablythey were influencedto some extent by the target’s finalposition on the screen.

A more important issue for our attempt to determinehow retinal information is used to account for our ownmotion is why we often found a larger influence ofbackgroundmotion for simulatedrotation alone, than forthe combined rotation and translation. We proposeseveral possibleexplanations.

First, the conflict between retinal and extra-retinalinformation is smaller for the simulated rotation. Thesimulatedrotationsin the combined rotation and transla-tion are twice as large as those for simulated rotationsalone, and they are accompanied by fast simulatedtranslations(Table 1).

Second, the triangles are distributedat random on thefloor. Whenever the most distant triangle is nearer than600 cm, the influenceof the simulatedcombinedrotationand translationis reduced [becausestructuresnearer than600 cm move more slowly to the left; see Fig. 3(f)], butthat of the simulated rotation is not [see Fig. 4(f)].Moreover, subjectsmay misjudgewhich triangle is mostdistant, or use the average velocity of several distanttriangles, which would decrease the influence of thesimulatedcombinedrotation and translationfor the samereason.

Third, we may not ignore local motion altogether. Inparticular,a perceptualconflictmay arisewhen the targetstops moving relative to the local surrounding texture,and thereby becomes part of the static environment.Forsimulated rotation at 27 deg/sec and translation at – 1.5m/see (see Table 1), the target would be expected tomove to the right at 15 deg/see, while the floor is alsomoving at 15 deg/sec to the right (see Fig. 5).

Taking these arguments into consideration, we con-clude that six of our subjects’results are consistentwiththe hypothesis that motion is primarily judged bycombining extra-retinal signals with the motion of thetarget’s retinal image relative to that of the most distantstructures. The other four subjects’ results are lessconclusivebecause they showed very little influence of

PERCEIVEDOBJECTVELOCITY 3813

backgroundmotion for simulatedrotationand translation(although they showed a similar influence to that of theothersfor simulatedrotation).We thereforeconcludethatwhenever the visual surroundingdoeshave an effect, thiseffect is dominated by the most distant structures.Undoubtedly,the extent to which subjects rely on visualinformation—when in conflict with extra-retinal infor-mation—dependson many factors (Brenner & van denBerg, 1994; Brenner et al., 1996), so that differencesbetween subjects are to be expected. The balancebetween retinal and extra-retinal information probablyalso depends on the target’s velocity and the kind andspeed of simulatedego-motion.We thereforedo not wishto concludeanythingabout this balance, but only that thebalance is primarily between extra-retinal informationand retinal motion relative to that of distant structures.

GENERALDISCUSSION

Retinal and extra-retinal information are combinedwhen judging a moving object’svelocity. We now showthat the retinalcontributionis dominatedby motionof thetarget’s retinal image relative to that of the most distantstructure.This providesan estimateof the target’smotionrelative to ourselves, without requiring detailed knowl-edge of the target’s distance or of our own motion.

Motion of the most distant structure’sretinal image isused to estimate eye-rotation. This estimate is probablycombined with an extra-retinal estimate of eye-rotation(Brenner & van den Berg, 1994). There is no evidentneed to estimate eye-translation.This may explain whyeye-translation is not adequately accounted for on thebasis of extra-retinalinformation:In the dark, duringself-induced lateral ego-motionsimilar to the motion thatwassimulated in the present study, subjects systematicallymisjudged the lateral motion of a single light source(Gogel, 1982). The misjudgements of the targets’distances that would account for the errors were verydifferent from the distances that subjects indicatedwhenasked to point at the targets, so it is unlikely that theerrors are (only) due to misjudgementsof distance.

Despite the fact that we did not instruct the subjectsonwhich frame of reference they should use, all subjectsappeared to be judging object motion relative to themoving eye. This is in accordance with a similarexperiment on perceived motion in depth duringsimulatedego-motionin depth (Brenner& van den Berg,1993).In that experiment,subjectsspontaneouslyjudgedthe target’svelocity relativeto the eye (on the basisof therate of expansionof the target and the vergence requiredto maintain fixation),completely ignoring the expansionof and changing disparity in the background (simulatingego-motion in depth). In fact, when we showed subjectstheir performance (in that study), and repeated theexperiment with the explicit instruction to report onmotion relative to the surrounding,they had difficultieswith the task and performed very poorly.

Our results are consistent with some recent findingsconcerningthe use of visual informationto determinethedirectionin which we are heading.Althoughthe extent to

which we can determine our simulated direction ofheading from visual displays in which combinationsofeye-rotation and eye-translation are simulated is still amatter of some controversy (Warren & Hannon, 1988;Van den Berg, 1992;Roydenet al., 1992,1994),there areclearly some conditions in which we can do so. Thisrequires an ability to separate the retinal flow field intoinfluencesof translationand of rotation.One way to do sowouldbe to considerall motionrelativeto (themotionof)the mostdistantstructure,as here suggestedfor perceivedobject motion. This would at least partly account for theinfluence of eye-rotation,because the retinal motion ofthe most distant structures is least influenced by eye-translation. It will, however, give rise to considerablesystematic errors if the most distant structure is nearby.Moreover, it requires independent information onstructures’distances.

Several aspects of our ability to determine ourdirection of heading from the retinal flow field supportthis hypothesised mechanism of isolating the transla-tional flow field. We can tolerate larger disturbancestothe flow field when perspective(Van den Berg, 1992)orstereopsis (Van den Berg & Brenner, 1994b’)provideinformationon structures’distances, than when they donot. Moreover, for simulated motion across a groundplane, limiting the visible range makes us misjudge ourdirection of heading in the way that is predicted by theuse of retinal motion relative to that of structures at thehorizon (Van den Berg & Brenner, 1994a).

The present results are also consistentwith reports thatthe most distant structure (or the one that appears to bemost distant) determines whether subjects experiencecircular vection when two structures of a display are inconflict;one movingat a constantangularvelocity acrossthe subject’sfield of view, and the other static (Brandt etal., 1975; Ohmi et al., 1987). Thus, estimates of eye-rotation based on the most distant structure appear toaccount for one’s circular vection, as well as providingthe basis for dealing with one’s rotations when judgingobject.motion and one’s directionof heading.It providesan estimate of the object’s angular velocity relative toourselves,without requiringmetric informationon depthor on our own motion. This estimate will normallyconform with estimates based on extra-retinal informa-tion. Obviously, in order to obtain an estimate of theactual velocity relative to ourselves, this measure ofangular velocity must be combined with information ondistance(Brenner, 1993;see Sedgwick,1986for a reviewon distance cues) and motion in depth (Brenner & vanden Berg, 1994; see Regan et al., 1986 for a review ofcues for perceiving motion in depth).

REFERENCES

Brandt, T., Wist, E. R. & Dichgans, J. (1975). Foreground andbackground in dynamic spatial orientation. Perception andPsychophysics, 17, 497–503.

Brenner, E. (1991). Judgingobject motion during smooth pursuit eyemovements:The role of optic flow.Vision Research, 31, 1893–1902.

Brenner, E. (1993). Judging an object’s velocity when its distancechanges due to ego-motion.Vision Research, 33, 487–504.

3814 E. BRENNERand A. V. van den BERG

Brenner,E. & van den Berg, A. V. (1993).Perceivedobject motion indepth during simulated ego-motion.Perception, 23 Suppl., 51-52.

Brenner, E. & van den Berg, A. V. (1994). Judging object velocityduring smooth pursuit eye movements. Experimental BrainResearch, 99, 316-324.

Brenner,E., van den Berg,A. V. & van Damme,W. (1996).Perceivedmotion in depth. Vision Research, 36, 699–706.

Cornilleau-P4r&,V. & Droulez, J. (1994). The visual perception ofthree-dimensionalshapefrom self-motionand object motion.VisionResearch, 34, 2331–2336.

Gibson, J. J. (1979). The ecological approach to visual perception.Boston, MA: HoughtonMifflin.

Gogel,W. C. (1982).Analysisof the perceptionof motionconcomitantwith a lateral motionof the head.Perception and Psychophysics, 32,241–250.

Koenderink,J. J. (1986). Optic flow. Vision Research, 26, 161–180.Land, M.F. (1992).Predictable eye–headcoordinationduringdriving.

Nature, 359, 31%320.Mergner, T., Rottler, G., Kimmig, H. & Becker, W. (1992). Role of

vestibular and neck inputs for the perception of object motion inspace. Experimental Brain Research, 89, 655–668.

Ohmi, M., Howard, I. P. & Landolt,J. P. (1987).Circularvection as afunction of foreground–backgroundrelationships. Perception, 16,17–22.

Post, R. B. & Lott, L. A. (1990). Relationshipof induced motion andapparent straight-ahead shifts to optokinetic stimulus velocity.Perception and Psychophysics, 48, 401406.

Raymond, J. E., Shapiro, K. L. & Rose, D. J. (1984). Optokineticbackgrounds affect perceived velocity during ocular tracking.Perception and Psychophysics, 36, 221–224.

Regan,D. M., Kaufman,L. & Lincoln,J. (1986).Motionin depth and

visual acceleration. In Boff, K. R., Kaufman, L. & Thomas, J. P.(Eds),Handbook ofperception and human performance: Volume 1.Sensory processes and perception (pp. 19.1-19.46). New York:Wiley Interscience.

Rogers,B. J. & Graham,M. (1979).Motionparallax as an independentcue for depth perception.Perception, 8, 125–134.

Royden,C. S., Banks,M. S. & Crowell,J. A. (1992).The perceptionofheading during eye movements.Naturej 360, 583–585.

Royden, C. S., Crowell, J. A. & Banks, M. S. (1994). Estimatingheadingduring eye movements. Vision Research, 34, 3197–3214.

Schwarz,U., Busettini, C. & Miles, F. A. (1989).Ocular responses tolinear motion are inversely proportional to viewing distance.Science, 245, 1394-1396.

Sedgwick,H. A. (1986)Spaceperception.In Boff, K. R., Kaufman,L.& Thomas, J. P. (Eds), Handbook of perception and humanperformance: Volume 1. Sensory processes and perception (pp.21.1–21.57).New York: John Wiley.

Smeets, J. B. J. & Brenner,E. (1995).Perceptionand action based onthe same visual information: Distinction between position andvelocity. Journal of Experimental Psychology: Human Perceptionand Performance, 21, 19–31.

Van den Berg,A. V. (1992).Robustnessof perceptionof headingfromoptic ffow. Vision Research, 32, 1285–1296.

Van den Berg,A. V. & Brenner,E. (1994).Humanscombinethe opticflowwith static depth cues for robust perception of heading. VisionResearch, 34, 2153–2167.

Van den Berg, A. V. & Brenner, E. (1994). Why two eyes are betterthan one for judgments of heading.Nature, 371, 700-702.

Warren, W. H. & Hannon, D. J. (1988). Direction of self-motion isperceived from optical flow.Naturej 336, 162–163.