2/10/2005 ZOL 867 1

High-Level Vision

What is “High-level” vision?General IssuesScenesObjectsFacesSelective Attention

2/10/2005 ZOL 867 2

Defined by Cognitive Scientists

From MIT Encyclopedia of Cognitive ScienceAspects of vision that reflect influences from memory, context, orintention are considered "high-level vision," a term originating in ahierarchical approach to vision. In currently popular interactive hierarchicalmodels, however, it is almost impossible to distinguish where one level ofprocessing ends and another begins. This is because partial outputs from lowerlevelprocesses initiate higher-level processes, and the outputs of higher-levelprocesses feed back to influence processing at the lower levels (McClelland andRumelhart 1986). Thus, the distinctions between processes residing at high,intermediate, and low levels are difficult to draw. Indeed, substantialempirical evidence indicates that some high-level processes influence behaviorsthat are traditionally considered low-level or MID-LEVEL VISION. With thiscaveat in mind, the following topics will be considered under the heading"high-level vision": object and face recognition, scene perception andcontext effects, effects of intention and object knowledge on perception,and the mental structures used to integrate across successive glances atan object or a scene. (Mary Peterson)

2/10/2005 ZOL 867 3

Defined by Visual Neuroscientists

• … “high-level” perception such asrecognition and categorization—i.e.,visual processes that rely on neuralactivities in inferior temporal cortex andbeyond.– Li, VanRullen, Koch, & Perona, PNAS,2000, 99 (14), 9596-9601.

http://www.mit.edu/~kardar/research/seminars/CorticalMaps/VisualSystem.html

2/10/2005 ZOL 867 4

Defined by Visual Neuroscientists (cont’d)

Low Level Vision (Eye to Cortex)Retina Optic nerve Subcortical structures Cortex

lateral geniculate nucleus (LGNd) V1 and V2 superior colliculus (SC)

Division of labor in the LGNd:Magno: Colorblind, fast, highcontrast, low spatial resolutionParvo: Color selective, slow, lowcontrast, high spatial resolution

Division of labor in V1 and V2:Layers 2 and 3, blob, thin stripe:ColorLayers 2 and 3, interblob,interstripe: color-formLayer 4b, thick stripe: motion anddynamic-form

http://webvision.med.utah.edu/VisualCortex.html

2/10/2005 ZOL 867 5

Defined by Visual Neuroscientists (cont’d)

Intermediate Level Vision (Color, Form, Movement)V1 and V2 parcel out the visual signal into separate regions in prestriatecortex, whose cells respond to different features:• V5: Motion• V4 (via blob regions of V1): Color• V4 (via interblob regions of V1): Color-form• V3 and V3a: Dynamic-form

2/10/2005 ZOL 867 6

Defined by Visual Neuroscientists (cont’d)

High Level Vision (Recognition and Action)

Ventral System. V4 inferior temporal cortexForms representations that are “abstract,” i.e.independent of a specific vantage point, lighting,location and, perhaps, orientation; implicated in objectrecognition

Dorsal System. V5 posterior parietal cortex (PP):• coordination of visually guided actions (e.g., eye

movements and orienting grasp)• localization (perhaps)• modulation of selective attention (perhaps)

These pathways are sometimes calledthe “what” and “where” pathways

2/10/2005 ZOL 867 7

Topics in High-Level Visual Processing

• Object recognition• Face recognition• Scene recognition (new)• Effects of semantic knowledge

(beliefs,interpretation) on perception• Transsaccadic Integration• Visual Attention and Active Vision (new)

2/10/2005 ZOL 867 8

Meta-Issue in Visual Perception

Perception vs Visual ProcessingIs consciousness critical to study of perception?• A central concern in classic study of perception

is why things look the way they do… i.e.,phenomenology of perception (closely relatedto consciousness

• Marr’s view: vision... is a process thatproduces from images of the external worlda description that is useful to the viewer andnot cluttered with irrelevant information. (Marr,1982)

2/10/2005 ZOL 867 9

Meta-Issue in Visual Perception

Perception vs Visual ProcessingThe world is experienced as a unified whole,but sensory systems do not deliver it to thebrain in this way…. Signals from differentsensory modalities are initially registered inseparate brain areas. How does thisinformation become bound together inexperience? [emphasis added]– Lynn Robertson (2003), Trends in CognitiveSciences, 4, 93-102.

2/10/2005 ZOL 867 10

Classic Example: Perceptual Learning

Visual system improves inability to recognize object inambiguous or noisy pattern

This is thought to require highercortical processing

But bees can do it too

Hard

Easy

Easy

2/10/2005 ZOL 867 11

Meta-Issue in Visual Perception

Computational Approach• What are the internal visual representations

that are available for additional visual andcognitive computations (and implemented inthe brain), without concern for whether theygive rise to perceptual experience or are opento awareness.

• Representations that directly control behavior• Representations that support other cognitive• processes• Computational approach is compatible with

computer vision, animal (at least invertebrate)vision

2/10/2005 ZOL 867 12

Meta-Issue in Visual Perception

Computational Approach• We don’t “see” or experience the operation of

gabor filters or edge detection algorithms orstructural descriptions (that’s why we have todevise clever experiments).

• Behavioral indicators (whether they give rise toawareness or not) are critical for evaluating ofunderlying visual processing. These includereaction time, errors, “implicit” measures likeeye movements.

2/10/2005 ZOL 867 13

Scenes: Transaccadic Stability

Why and how do we experience acomplete, detailed, full color visualworld despite the fact that (a) theretinas cannot deliver thishigh-fidelity input within a givenfixation, and (b) the visual systemcannot fuse together discreteretinotopic images to generate acomposite internal picture

Data are experiential: did you seethe world shift?

2/10/2005 ZOL 867 14

Scenes: Transaccadic Integration

• What is the nature of the internalrepresentation that is retainedacross a saccade and integratedwith the visual input acquired duringthe next fixation– regardless ofwhether this representation isfunctional in generating experience.

• Data are behavioral: was somebehavioral measure affected by themanipulation, e.g., fixation time

2/10/2005 ZOL 867 15

Objects(“If it looks and walks and quacks like a duck…”

Some Terminology:

“Type” A general class or category of object, such as coffee cup,car, postage stamp

“Token” A particular object within a given class

http://users.rcn.com/jkimball.ma.ultranet/BiologyPages/V/VisualProcessing.html

2/10/2005 ZOL 867 16

Objects (cont’d)

I. Eidetic, or pictoral images: system recognizes whole shape,but is confused by rotational changes

Bees confuse these patterns

2/10/2005 ZOL 867 17

Objects (cont’d)

II. Featural models: system recognizes diagnostic features(ethologists would call these “sign stimuli,” which could belearned or hardwired)

http://www.pigeon.psy.tufts.edu/avc/kirkpatrick/default.htm#RBCHerring gull chicks innately

recognize adult gull head (redspot is sign stimulus)

Pigeons confuse these patterns

2/10/2005 ZOL 867 18

Objects (cont’d)

III. Structural models: systemrecognizes assemblage of features,according to the form and arrangement ofindividual components (akin to visual“words” and “syntax”_

Example: Recognition by components(Biedermann) (Geon model)

• Recognition is achieved whenrelative relationships amongcomponents matches

• Rules are defined according torelative spatial relationships, so thissystem is robust to variance in size,rotation, or translation

Same features,different structure

2/10/2005 ZOL 867 19

Objects (cont’d)

III. Statistical models: systemrecognizes object based on degree ofresemblance (in whatever visualdimensions) to previously experiencedexemplars

This leads to recognition that is robustto variance in rotation, etc.,provided experience includesexposure to a wide variety ofviewpoints, contexts, etc.

2/10/2005 ZOL 867 20

Faces

2/10/2005 ZOL 867 21

Faces

Functional NeuroimagingProbably single-most important methodologyin the modern era of cognitive neuroscience