Visuospatial Representation
Spatial Knowledge, Imagery, Visual Memory
Representation
What is a representation? Four aspects of representation
The represented world The representing world Set of informational relations on how
the two correspond Set of processes that extract and use
information from the representation
Meaning Mental representations are carriers of
meaning In order to interact appropriately with the
environment we represent info from it and manipulate those representations
Correspondence Meaning derived from how representation
stands in consistent relation to the represented world
Conceptual Meaning determined by relations to other
representations
Spatial Knowledge
How we represent and use spatial information
Separate from strictly verbal knowledge Semantic propositions
Dependent on the linear dimension of space.
Spatial Cognition
How is the representing world like the represented world?
The represented world is a space The representing world is a space
What kinds of processes might be involved?
Space as a representation Spatial representation Representing world is a space. What is a
space? Geometric entity in which locations are specified
relative to a set of axes Dimensionality defined by the number of axes that
can point in independent directions Of interest is the distance between items, which can
be measured in different ways Euclidian
Straight line Non-independent dimensions
Saturation and brightness City-block
Distinct dimensions Color and size
Space as a representation Physical world experienced (at least
perceptually) has three dimensions (+ time)
However, the representing world is not confined to any number of dimensions
Represented world does not need to be spatial Conceptual info can be represented spatially More on that later
Spatial Representation
Analog representation Representation mimics the structure of
the represented world Multidimensional scaling
Propositional Abstract assertions regarding the state
of the represented world Not tied to a particular sensory modality
Multidimensional Scaling (MDS)
MDS Mathematical technique for taking a set of distances and
finding the best-fitting spatial configuration that corresponds to those distances
Input: a distance or proximity matrix that describes how close every object in a set is to every other object
N objects are represented by N(N-1)/2 numbers (distances) Output: a geometric representation where
every object is represented as a point in D-dimensional space
Each object is represented as a point in space N objects are represented by ND numbers (coordinates)
Purposes of MDS Give psychological interpretations to the dimensions Reveal the dimensionality of a data set
Difficult to get a sense of relative distance by means of this information
MDS
MDS recovers absolute original locations for the objects from the distances
Flipping on horizontal axis would give us a rough approximation of NSEW
Analog representation
MDS
Propositional Representation
(A,B) 10 miles east
(E,C) 20 miles south, 10 miles east
(F,D) 10 miles south, 10 miles west
Analog vs. Propositional Analog
Good for configural info Easy incorporation of new info
Propositional Time-consuming Lots of info must be represented
E.g. one point added may require many propositions
Allows for communication of spatial knowledge and incorporation of additional information not related to distance
Going south on I35, one must pass through Denton to get to either Fort Worth or Dallas
Cognitive Maps
Where is Seattle? Where is Terrill Hall?
Large vs. small-scale space Hierarchical representation
Small vs. Large-scale space
Maps of small-scale (navigable space) Cognitive geography
Maps of large-scale space What is our sense of the locations of
items in the world?
Small scale space Survey knowledge
Bird’s eye view (map knowledge) Good for global spatial relations Easy acquisition Not so great for orientation
Route knowledge Gained from navigating through the environment
Locate landmarks and routes within a general frame of reference
Landmark knowledge Salient points of reference in the environment
More difficult to acquire but better for navigation in irregular environments
May lead to survey knowledge Perhaps a different type Cognitive collage vs. orientation free
Large scale space
Which is farther north: Denton, TX or Chicago, IL? Portland, OR or Portland, ME?
Hierarchical representation of locations
Hierarchical representations Relative locations of smaller regions are
determined with respect to larger regions. States are superodinate to cities, countries
superordinate to states USA is south of Canada
Maine is just south of Canada Oregon is well south of Canada
Oregon must be south of Maine Cities in Oregon must be south of cities in Maine In this case such cognitive economy works against us
Portland OR is north of Portland ME
Hierarchical representations Judge relative
position of cities (Stevens and Coupe)
When superordinate info congruent with question, performance better
Is x north of y when one of right side maps presented
Hierarchical coding Huttenlocher & Hedges
Category-adjustment model Combine info across hierarchical levels
If info at subordinate is known with near certainty, there is no appeal to categorical info
If info at subordinate (fine-grained) levels is at all uncertain, people use categorical info in estimation
Bias toward center of category Bayesian approach utilizing prior knowledge
Gist: errors in estimation are due to categorization rather than nonmetric spatial relations
How are maps learned From descriptions
Taylor & Tversky: People learned maps from survey and route descriptions
From navigation People can assess distance and direction
traveled Integration of information
Visual information Vestibular information
Maps formed from video games are less accurate than maps in which people really move
Rotation is particularly important
Using spatial cognition Adaptive context
Locating and way finding Tool Use Mental rotation vs. mental movement
Symbolic representations of space Drawings, maps, models Language
Thinking
Adaptive context Locating and way finding Consider
Hatchling sea turtles finding the sea Salmon finding way back home
However these are more behavioral instinct and imprinting than pure navigation
Desert ant finding direct route home after meandering paths in featureless environment
Marsh tit stores seeds in holes in a hundred various places for later retrieval
Locating and way finding
Ego-centered systems Environment-centered systems Hierarchical coding
Locating and way finding Ego-centered system
Location of objects coded relative to self Updated as we move through the world Nonconscious
Rieser, Guth, Hill (1986) Participants asked to point out previously learned
locations in unfamiliar room after blindfolded and led along path
Did not matter whether previously told which location they’d be asked about, suggesting attentional focus did not assist in the process
Problem: may not always be accurate over larger distances without detailed environmental information
Locating and way finding Environment-centered system
Object location coded in relation to stable features of the environment
Requires feature-rich environment providing info to dominate sense used by organism
If conditions met, then superior to ego-centered
Allows for rechecking of position (no drift from accumulation of small errors)
Works better for retaining info over long periods of time
Cognitive maps Both humans and animals display errors
in judgment that cast doubt in positing a true ‘cognitive map’
Animal studies suggest approximation of distance from a single landmark
Humans make many errors in spatial judgments that suggest no real metric representation Distance from A to B judged different from B
to A Though again this sort of distortion may be
related to categorization (hierarchy)
Tool use
Making, using and designing tools for interaction with the environment involves cognitive processes such as mental rotation and imagery for success
Shephard & Metzler (1971)
Mental rotation vs. Mental Movement
Logically equivalent However evidence suggests that mental
rotation and perspective-taking/mental movement are psychologically distinct
Selection task Which these arrays/models would be the correct
view from over there? Item question
What object would be nearest to you if you were over there?
Specify frame of reference of relative to the observer
Mental rotation vs. Mental Movement
Selection task Piaget
Kids (< 10) not so hot at such a task Usually pick egocentric view
Huttenlocher & Presson They do much better when asked to do mentally rotate Can also physically move to new location that matches
a particular array Suggests conflict between current physically
present perspective and the new (imagined) one they are trying to obtain
MR allows them to stay put in the physically present room
Physical movement physically transforms that perspective
Mental rotation vs. Mental Movement
Item questions If kids do not move item questions help (even
as young as 3) Again, this helps them maintain that egocentric
perspective If asked to mentally rotate, item questions can
actually hurt performance compared to selection tasks
It may be that in item questions, whole array must be rotated to determine object relations vs a simple ‘rotation’ of the person or single object in selection task
Gist: mental rotation and mental movement can be differentially affected depending on the nature of the question asked, suggesting there may be different underlying processes involved
Drawings maps and models Spatial learning from maps differs from
learning by means of navigation Map learning may aid configural knowledge and allow
for better estimates of distance between points while navigational learning allows for better route distance estimation and location of unseen points
Recall survey vs. route knowledge Orientation-specific vs. orientation-free learning
Studies show evidence that navigational learning is more a collection of multiple views than orientation-free, though may lead to a sort of orientation-free type of knowledge
Sholl & Friedman
Spatial Language Contrasting experience with communication Experience spatial relations continuously,
but language is usually discrete (e.g. near vs. far) Spatial terms function much like other categories
(e.g. fuzzy boundaries, prototypes) Experience multiple spatial relations
simultaneously, but speak of one relation at a time
A frame of reference must be agreed upon in order to communicate spatial relations
Spatial Language Despite the difficulties in communicating spatial
knowledge, ambiguities are generally overcome and information encoded (survey, route knowledge)
However it does seem that spatial language may bias or constrain spatial representation, and may even affect the development of spatial concepts and categories
Even so, the actual link between spatial language and spatial representation is not entirely clear
Impaired sight individuals may have difficulties with a variety of spatial tasks but have intact spatial language
Thinking Spatial cognition also contributes to
logical reasoning, metaphor, and creativity
Transitive reasoning A > B, B > C A ? C
Metaphor The future stretched in front of them My heart is a flame turned upside down Structural alignment of spatial and temporal
concepts Diagrams as aids to understanding
Show conceptual similarity of items, connections amongst various concepts etc.
Creativity E.g. visualization for problem solving
Taking someone else’s point of view?
Imagery Some information in memory is purely
verbal Who wrote the Gettysburg address?
Other memories seem to involve mental images Trying to recall a procedure Making novel comparisons of visual items
What is a mental image? How are mental images represented and
processed? Are mental images like visual images?
Evidence for use of visual imagery
Selective interference Segal & Fusella Imagery interferes with detection of
stimuli (sensitivity decreased) Auditory imagery interfered with
auditory detection, visual imagery with visual stimuli
Manipulation of images Mental rotation studies
Evidence for use of visual imagery
Kosslyn Learn a map Mentally travel from
one point to another Measure time to
make this mental trip
Results Time to make trip
increases with distance
Times increase with imagined size of the map.
Evidence for use of visual imagery Moyer 1973
Subjects were given the names of two common animals and asked to judge which was larger
Which is larger, a moose or a roach?
Wolf or Lion? The time delays as a
function of size difference were similar to those usually found for perceptual judgments.
Kosslyn Kosslyn 1975 Scenario I: Imagine an elephant
standing next to a rabbit. Does a rabbit have a beak?
Scenario II: Imagine a fly standing next to a rabbit. Does a rabbit have an eyebrow?
People made faster judgments when relying on a larger mental image (such as the rabbit next to the fly) than when using a smaller mental image (such as the rabbit next to an elephant)
Kosslyn suggested that the size of an image is an important factor in determining how fast we can make judgments about it.
RT
0Elephant Fly
True
False
Inconsistent
Consistent
Paivio's Dual-Coding Theory Information is mentally represented
either in a verbal system (propositional) or a nonverbal (analogical) system (or both). Each system contains different kinds of
information. Each concept is connected to other
related concepts in the same system and the other system.
Activating any one concept also leads to activation of closely related concepts.
Paivio
The hypothesis of multiple codes (verbal and spatial) is based on the demonstration of independence of effects. Pictures of objects Words of objects
Paivio (1975) compared reaction times for consistent and inconsistent visual stimuli
If the stimuli are processed semantically, there should be no difference between consistent and inconsistent presentations.
If stimuli are processed spatially, inconsistent stimuli should require a mental conversion to appropriate size. Which takes time
Consistent
Inconsistent
Results “Which is larger?”
RT
0
Inconsistent Consistent
Paivio Consistent
RABBIT FLY
Paivio
Inconsistent
RABBIT ELEPHANT
Paivio
Congruity Effect only for Pictures (not words) Imagery relies on perceptual detail and semantic does not Such findings as this and picture superiority effect
(pictures are better recognized than words), and that verbal + imagery encoding leads to best recall, suggest a Dual Code Theory
RT
0
Picture Words
Inconsistent
Consistent
Santa 1977 More evidence of
dual coding Ss presented array of
objects or words On test presentation
asked whether the elements were same as studied
E.g. In geometric condition first two would be yes responses
Santa 1977 Results of positive
responses Spatial configuration
is preserved in geometric encoding
Compared to verbal presentation, which was encoding in typical English reading style and benefited from the linear configuration
Representation of images
What is the relationship between imagery and perception?
Can imagining interfere or facilitate detection of stimuli?
Similar processes involved?
Spatial Knowledge Symbolic Distance Effect (Moyer 1973)
Process of imagery = process of perception As perceptual distance increases so does psychological
distance (RT). Items “farther apart” are more quickly distinguished Which is larger?
Rabbit-Elephant Rabbit-Dog
Representation of images Contrary evidence Chambers and
Reisberg Images are (committed
to) a particular interpretation
E.g. The rabbit comes once drawn but was only a duck as imaged
Contrast with perception which requires interpretation, images are already interpretations
Are visual images visual? Plenty of evidence to suggest a spatial
component to visual imagery, but perhaps the visual part is represented propositionally
Kerr Congenitally blind also take longer to
imagine longer map routes like the one in Kosslyn
Are visual images visual? Images are also not as sharp as
real pictures Form a mental image of a tiger
Does it have stripes? How many? It is hard to examine details of
mental images that would require eye movements
Making new pictures Finke, Pinker, Farah Example
Imagine a capital letter H and a triangle
Rotate the H 90 degrees Place the triangle on top of it What is it?
Suggests images can take on new interpretations
Are visual images visual? Facilitation and interference (Farah)
Have people imagine a letter (H or T) Present one of the letters to the screen briefly
(20 ms), or present nothing, followed by mask Asked if they saw a letter
People are more likely to detect the stimulus if it was the same as what they were imaging, suggesting that visual and imaginal representations joined or fed into the same process
Are visual images visual? Evidence from neuroscience Patients with lesions of
visual cortex that lead to perceptual problems also have problems with mental imagery
ERP evidence PET evidence: Visual imagery leads to activation of visual cortex. Auditory imagery does not
In general, results of studies from mental rotation to brain imaging support the idea of both visual and spatial representation of images
Translating Words to images
Franklin and Tversky Create a mental image
based on the description Asked to identify location of
items in that imagined environment
Results are what one might expect given an imagined spatial environment
Up-down, front-back more relevant in navigating real world
Left-right confusion in real world and imagined world
Visual memory
Although our visual memory seems to be excellent, it turns out not to be that great in many respects
In general, our memory for details is lost, much like with other types of memory
Visual memory
Memory for pictures is quite good generally Again, don’t get too detailed Standing
Presented 10000 photos over several days Old-New memory over 80%
Picture superiority effect Better memory for pictures than words