Date post: | 06-Jul-2018 |
Category: |
Documents |
Upload: | pablo-perez-donoso |
View: | 226 times |
Download: | 0 times |
of 10
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
1/10
~ P e r g a m o n
Neuropsychologia,Vol. 33, No. 11, pp. 1335-1344, 1995
ElsevierSeienoeLtd
Printed n G reat Britain
0028-3932(95)00067--4
T H E C O G N I T I V E N E U R O S C I E N C E O F M E N T A L I M A G E R Y
S . M . K O S S L Y N ,* ~f M . B E H R M A N N : ~ a n d M . J E A N N E R O D §
t D e p a r t m e n t o f P s yc h o l o g y , H a r v a r d U n i v e r si t y , C a m b r i d g e , M a s s a c h u s e t t s , U . S .A . ;
: ~ D e p a rt m e n t o f P s y c h o l o g y , C a r n e g i e M e l l o n U n i v e r s i ty , P i t t s b u r g h , P e n n s y l v a n i a
an d §Vis ion e t M ot r i c i t6 , 16 , avenue du Doy en l_~p ine , 69500 B ro n , F ranc e
Received 12 November 1994; accepted 10 January I995
Thi s spec i a l i s sue i s devo t ed t o r ecen t work on men ta l image ry . To ge t a s ense fo r t he
s u b j e c t m a t t e r , a n s w e r t h e f o l l o w i n g q u e s t i o n s : H o w m a n y w i n d o w s a r e t h e r e i n y o u r
l iv ing room ? W ha t socks d id yo u wea r ye s t e rday? I s t he r e r oom in you r r e f r i ge r a to r f o r a
l it re o f mi lk? These ques t i ons t yp i ca l l y evoke v i sua l men t a l image ry . Fo r exam ple , when
answer ing t he f ir s t, peop l e u sua l l y v i sua l iz e the room , and t hen s can ove r e ach wa l l,
s ee ing each w in dow wi th t he i r m ind s eye . S imi la r ly , i f a sked wh e the r a c a t s m eow has
a h ighe r p i t ch t han t he sou nd o f a b lende r , mo s t peop l e r epo r t hea r i ng t he sounds i n
the i r m ind s e a r . Such t a sks evok e aud i t o ry m en ta l image ry . V i sua l men t a l ima ge ry i s
s ee ing i n the absence o f t he app rop r i a t e imm ed ia t e s enso ry i npu t , aud i t o ry men ta l
image ry i s hea r i ng i n t he absence o f t he app ro p r i a t e imm ed ia t e s enso ry i npu t , and so
on . Im age ry i s d i s t inc t f r om pe rcep t i on , w h ich i s t he r eg i s t ra t i on o f phys i ca l ly p r e sen t
st imuli .
The example s i l l u s t r a t e t ha t image ry p l ays an impor t an t r o l e i n memory and spa t i a l
reasoning , bu t as we sha l l see , these func t ions mere ly sc ra tch the sur face . Imagery
a lso p lays a ro le in abs t rac t reasoning , sk i l l l ea rn ing , and language comprehens ion .
On e o f the m ajo r ins ights a bo ut im agery i s tha t- -- -even w i th in a s ingle senso ry
m o d a l i t y l i t i s no t a sing le , und i f f e ren t i a t ed ab i li t y . Ra the r , a s t he open ing examp le
sugges t s, image ry i nvo lves a h os t o f p roces se s wo rk ing t oge the r. The image o f on e s
l i v ing room mus t be fo rmed a t t he ou t s e t , i t t hen mus t be s canned and i n t e rp r e t ed ,
and m us t be ma in t a ined wh i l e t h is is occu r r ing . M oreov e r , i n ma ny s i t ua t i ons t he
i m a g e d o b j e c t m u s t b e t r a n s fo r m e d . F o r e x a m p l e , w h e n a s k e d w h e t h e r f r o g s h a v e
s tu bb y g r een t a il s, peop l e o f t en r epo r t men t a l l y r o t a t i ng t he am ph ib i an and l ook ing
a t i t s pos t e r i o r qua r t e r s .
I n t e r e s t i n me n ta l imag e ry ha s waxed and w aned fo r cen tu r ie s , and i s now un de rgo ing a
r e su rgence . U n l ike p r ev ious pe r i ods o f en thus i a sm fo r t he t op ic , t he p r e sen t one i s no t
d r i ven by ph i l o soph i ca l conce rns o r c l a ims ba sed on i n t ro spec t ion . Ra the r , t he p r e sen t
* A d d r e s s f o r r e p r i n t r e q u e st s : D e p a r t m e n t o f P s y c h o l og y , H a r v a r d U n i v e r s it y , 33 K i r k l a n d S t r e e t , C a m b r i d g e ,
M A 02138 , U .S .A .
1335
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
2/10
1336 S.M . KOSSLYN M. BEHRMANN and M. JEANNEROD
interest has been fueled by the emergence of new methods and new ways to conceptualize
imagery. As illustrated in this special issue, the confluence of these factors has produced a
new wave of discoveries not only about the nature and function of imagery, but also about
its neural bases. Indeed, because imagery has been shown to share mechanisms with like-
modality perception, theories of imagery have been able to stand on the shoulders of
theories of perception. This state o f affairs has resulted in the somewhat ironic fact that an
area that was considered beyond the pale 20 years ago is now of the first cognitive domains
to be firmly rooted in the brain.
Interest in imagery can be traced back at least to the time of Plato, who thought that
memories were based on images; on his view, memories are carved into the mind much like
pictures can be carved on a wax tablet. He even considered individual differences, likening
them to differences in the purity of the wax and in the ease of carving figures into it.
Interest in imagery continued on for centuries. For example, the British Associationists
conceived of thought itself as sequences of mental images, and the original psychologies of
Wundt and James placed considerable emphasis on imagery.
Imagery fell from grace, however, when psychology became obsessed with method. The
behaviorists stressed that the scientific method requires that one study phenomena that are
publicly accessible. This, of course, was not what was going on in other sciences, such as
physics--which were quite happy to draw inferences about unobservable phenomena by
studying their footprints (e.g. tracks in cloud chambers). But more than this, the
behaviorists argued effectively that it was not clear how to conceive of mental images.
Pictures in the head? Who looks at them? In retrospect, it seems that the eclipse of research
on imagery was a result of three events: A lack of sophisticated methodologies for studying
mental events, impoverished theoretical constructs, and the apparent success of the
behaviorist approach. This last factor was critical because it justified a shift in the way the
field of psychology was characterized. Whereas James conceived of psychology as the
study of mental life, Watson declared that it should be the study of behavior. There is no
place for mental events, let alone those that do not have distinctive behavioral sequeUae, in
a field that studies behavior
qu
behavior.
Imagery came into its own again in the early 1970s, riding the tide of the cognitive
revolution [see 10]. As the limitations of behaviorism became apparent, researchers once
again became receptive to theorizing about internal events. Three kinds o f work emerged
from the application of the methods of cognitive psychology to the study of imagery. First,
Paivio and his collaborators demonstrated that people can learn and remember highly
imageable materials better than abstract materials [e.g. see 16]. Second, Segal and her
colleagues used signal detection paradigms to show that imagery interferes more with like-
modality perception than with different-modality perception (e.g. visual imagery interferes
with visual perception more than with auditory perception and vice versa for auditory
imagery [18]). And third, Shepard and his collaborators [e.g. 21] demonstrated tha t images
act as surrogate objects during reasoning. For example, when subjects compared two
objects (e.g. three-dimensional block figures) to determine whether they had the same
shape or were mirror images, they required more time when the objects had to be mentally
rotated greater amounts to be aligned. In spite of the fact that images are not actual
objects that must obey the laws of physics, objects in images often behave like actual
objects.
A second innovation in imagery research took place during the 1970s, but this one was
conceptual, not methodological. Imagery was conceptualized as a type of data structure,
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
3/10
T H E C O G N I T IV E N E U R O S C I E N C E O F M E N T A L I M A G E R Y 3 3 7
w h i c h w a s e m b e d d e d i n a n i n f o r m a t i o n p r o c e s s i n g s y s t e m [ e.g . 2, 1 2 ] . T h i s p e r s p e c t iv e l e d
t o a h o s t o f e x p e r im e n t s t h a t w e r e d e s i g n e d t o r e v e a l s t r u c t u r a l p r o p e r t ie s o f im a g e s a s
s p e ci fi c t y p e s o f d a t a s t r u c tu r e s . A d e b a t e e n s u e d a b o u t t h i s is s u e, w h i c h q u i c k l y b o g g e d
d o w n w h e n r e s e a r c h e r s r e a li z e d th e l i m i t a t i o n s o f p u r e l y b e h a v i o r a l a p p r o a c h e s [ e.g . se e
1].
T h e d i ff i cu l ti e s i n c h a r a c t e r i z i n g t h e w a y i m a g e s a r e r e p r e s e n t e d a n d p r o c e s s e d s e t th e
s t a g e f o r t h e m o s t r e c e n t d e v e l o p m e n t in t h i s fi el d: T h e c o g n i t iv e n e u r o s c i e n c e a p p r o a c h .
R e s e a r c h e rs b e g a n t o u s e n e u r o p s y c h o l o g i c a l d a t a t o i n f o r m t h e o r i es o f th e s t r u c t u r e o f
t h e p r o c e s s i n g s y s t e m [ e.g . 7]. A n d m o r e r e c e n t l y , s u c h d a t a h a v e b e e n u s e d t o c h a r a c t e r i z e
t h e n a t u re o f t h e r e p re s en t a t i o n i t s e l f [ e.g . 1 3 ].
A s e a c h n e w a p p r o a c h h a s m a t u r e d , i t h a s s u p p l e m e n t e d - - n o t r e p l a c e d - - t h e e a r l i e r
a p p r o a c h e s . T h u s , n e w d e v e l o p m e n t s h a v e f l e s h e d o u t t h e f i e l d , w h i c h c a n n o w b e
c h a r a c t e r i z e d a s e n c o m p a s s i n g a n u m b e r o f r e l a t i v e l y d i s t i n c t d o m a i n s .
R E S E A R C H D O M A I N S
T h e s t u d y o f im a g e r y h a s n o w d e v e l o p e d i n t o a f i e ld i n it s o w n r i g h t . I n t h i s s e c ti o n w e
b r i ef l y r e v ie w t h e m a j o r a r e a s o f r e s e a r c h , w h i c h w e h a v e g r o u p e d i n t o f o u r m a j o r
c a t e g o r i e s : L e a r n i n g a n d m e m o r y ; p e r c e p t i o n a n d a c t i o n ; i n f o r m a t i o n p r o c e s s i n g ; a n d
r e a s o n i n g . I n a l l c a s e s , r e s e a r c h e r s b e g a n b y c h a r a c t e r i z i n g t h e f u n c t i o n i t s e l f , a n d o n l y
t h e n s o u g h t t h e n e u r a l b a s e s o f th e s e f u n c t i o n s . W e d o n o t b e li e ve t h a t i t is w is e to t r e a t
t h e n e u r o p s y c h o l o g y o f i m a g e r y a s a d i s t in c t fi el d o r a r e a ; r a t h e r , i t i s a n i n t e g r a l p a r t o f
t h e s t u d y o f i m a g e r y m o r e g e n e r a l ly . N e u r o p s y c h o l o g i c a l f in d i n g s a n d t h e o r i e s i l lu m i n a t e
t h e n a t u r e o f i m a g e r y r e p r e s e n t a t i o n s a n d p r o c e ss e s , a s w e l l a s t h e f u n c t i o n s o f i m a g e r y .
W e s e e k to a v o i d t h e t r a p t h a t h a s b e f al le n s o m e p a r t s o f d e v e l o p m e n t a l p s y c h o lo g y ,
w h e r e t h e o r i e s o f t h e d e v e l o p m e n t o f c o g n it iv e f u n c t i o n m a k e l it tl e c o n t a c t w i t h m o r e
s o p h i s t i c a t e d t h e o r i e s o f t h o s e f u n c t i o n s t h a t h a v e b e e n f o r m u l a t e d b y c o g n i t i v e sc i en t is t s.
Learni ng and m em ory
W e c a n d i s t i n g u i s h t h r e e r o l e s f o r i m a g e r y i n l e a r n i n g a n d m e m o r y .
Imagery and learning
T h e a n c i e n t G r e e k s d i s c o v e re d t h a t o n e s m e m o r y f o r a s et o f
o b j e c ts c o u l d b e g r e a t l y e n h a n c e d i f o n e v i s u a l iz e d t h e m i n t e r a c t i n g i n s o m e w a y . O n c e o n e
h a s v i s u a l iz e d a s c e n e a n d e n c o d e d i t i n t o m e m o r y , i t th e n c a n b e r e c a ll e d in a n i m a g e . F o r
e x a m p l e , i f y o u w a n t t o r e m e m b e r a s e t o f e rr a n d s , y o u c a n v i s u al iz e y o u r s e l f d o i n g t h e m
i n o r d e r . L a t e r y o u c a n v i s u al iz e t h e fi rs t p a r t o f t h e s e q u en c e , a n d s e e y o u r s e l f p r o g r e s s
t h r o u g h t h e e n t ir e s e q u e n c e , w h i c h w i ll h e lp y o u t o r e c al l th e e n t i r e s e t. B o w e r [ 4] s h o w e d
t h a t m u c h o f t h e m n e m o n i c p o w e r o f i m a g e r y c o m e s f r o m i ts a b i li ty to r e p re s e n t
a s so c i a t i o n s b e t w een d i s t i n c t o b j ec t s .
I m agery and m em ory
E v e n i f t h e y d i d n o t s e t o u t t o s t o re i m a g e s i n t e n t i o n a l l y , p e o p le
o f t e n w i l l u s e i m a g e r y l a t e r to r e c a ll i n f o r m a t i o n . I m a g e r y i s u s e d s p o n t a n e o u s l y t o r e c a l l
t h e sh ap e , co l o r , s i z e , an d t ex t u re o f o b j ec t s o r sp a t i a l r e l a t i o n s i n s cen es t h a t : ( a ) h av e
b e e n c o n s i d e r e d r e l a t i v e l y i n f r e q u e n t l y , a n d ( b ) c a n n o t e a s i l y b e i n f e r r e d f r o m f a c t s
a s s o c i a t e d w i t h a s u p e r o r d i n a t e c a t e g o r y o r f r o m v e r b a l m a t e r i a l . F o r e x a m p l e , i m a g e r y is
u s e d t o d e t e r m i n e w h e t h e r a G e r m a n S h e p h e r d d o g s e a r s p r o t r u d e a b o v e t h e t o p o f its
sk u l l , b u t n o t w h e t h e r i t h a s fo u r l eg s [ s ee 1 4 ] .
Using imagery to learn skills
I f o n e v i su a li z e s o n e se l f p e r f o rm i n g a sk i ll o n e i s a cq u i r i n g ,
o n e w i ll a c t u a l ly i m p r o v e a t t h e s k il l. F o r e x a m p l e , i f y o u w e r e l e a r n i n g a n e w d a n c e ,
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
4/10
1338 S.M. KOSSLYN M. BEHRMANN and M. JEANNEROD
v i sua l i z i ng t he s t eps wou ld l e ad you t o be a be t t e r dance r - -p rov ided t ha t imag ined
prac t ice i s in te rmixed wi th ac tua l prac t ice [e .g . see 17] .
Percept ion and act ion
I m a g e r y d r a w s o n m e c h a n is m s u s e d n o t o n l y i n p e rc e p t io n , b u t a l s o i n m o t o r c o n t ro l .
Shared mechanism with perception
I t is no t an acc iden t t ha t w e speak o f 's e e ing ' ob j ec t s
i n v i sua l images , o r ' h ea r ing ' sounds i n aud i t o ry images . The phe nom eno logy o f men t a l
image ry i s in m any ways l i ke t ha t o f l i ke -moda l i ty pe r cep t i on . Th i s f a c t ha s sugges t ed t o
m any t ha t t he s ame m echan i sms a r e i nvo lved i n a t le a s t som e a spec t s o f ima ge ry and l ike -
m oda l i t y p e r cep t i on . Th e re i s now a l a rge li t e ra tu r e a t t e s t i ng t o t he v a l i d it y o f th i s
inference [e .g . see 9]. For exam ple , C raver -L em ley and Ree ves [5] sho w ed tha t v i sua l iz ing a
l ine in te r fe res wi th the ab i l i ty to see how two l ine segments a re a l igned , which i s as
expec t ed i f image ry engages mech an i sms t ha t o the rwi se wou ld be u sed i n pe r cep t i on . I t i s
l i ke ly t ha t image ry p roces se s ac tua l l y p l ay a r o l e i n pe r cep t i on , e spec i a l l y when i npu t i s
deg raded - - i n wh ich ca se s image ry may he lp t o comple t e ' no i sy ' i npu t [ s ee 13 ] .
Such f i nd ings a r e cons i s ten t w i th s eve ra l imp or t an t f e a tu r e s o f the neu roan a tom y o f t he
v i sua l sy st em, such a s t he f ac t t ha t v i r t ua l ly eve ry v i sua l a r ea t ha t s ends i n fo rma t ion t o
ano the r a r ea a l so r ece ive s f ibe r s f r om tha t o the r a r e a - - an d t he f ibe r s r unn ing i n each
d i r ec t ion a r e o f comp arab l e s iz e [e . g . s ee 8 ]. Moreo ve r , va r i ous b r a in s cann ing t e chn iques
have r evea l ed t ha t v i sua l a r ea s i n t he occ ip i t a l , t empora l , and pa r i e t a l l obes a r e a l so
ac t i va t ed du r ing v i sua l men t a l im age ry [ fo r a r ev iew , s ee 13 ] . I n add i t ion , dam age t o
po r t i on s o f t he b r a in u sed i n v i sual pe r cep t i on o f t en a l so r e su lt i n co r r e spon d ing de f ic i ts in
visual imagery [e .g. see 15; for an interest ing exception, see 3] .
Motor control
G eorg op ou los and h i s co l le agues [1 I ] have sho wn tha t som e fo rms o f
image t r ans fo rma t ion a r e accompan ied by ac t i v i t y i n t he mo to r co r t ex . These
t r ans fo rma t ions i nvo lve an t i c i pa t i ng whe re t o r each , bu t t he mo to r sy s t em may p l ay a
ro l e more gene ra l l y i n image t r ans fo rma t ions . I ndeed , Deu t sch
et al
[6] found tha t the
r i gh t hemi sphe re , pa r t i cu l a r l y t he pa r i e t a l and f ron t a l l obes , a r e pa r t i cu l a r l y ac t i ve when
sub j ec t s pe r fo rm v i sua l men t a l r o t a t i on ; t he se reg ions o f co r t ex i nc lude s eve ra l a r ea s t ha t
p l ay key ro l e s in mo to r con t ro l . Thus , i t is pos sib l e t ha t t he mo to r sy s t em p l ays an
imp or t an t r o l e in image ry , spec if ic a ll y i n 'image t r an s fo rm a t ion ' p roces se s , a s no t ed
be low.
Informa tion processing
Images can be r ega rded a s a pa r t i cu l a r way i n wh ich i n fo rma t ion i s s t o r ed , and t he se
r ep re sen t a t i ons a r e p roces sed i n specif ic w ays . The sys t em in wh ich images a r e p ro ces sed
can be concep tua l i z ed i n t e rms o f f ou r ma jo r ab i l i t i e s , a s no t ed be low. A l though t h i s
app roach ha s been worked ou t i n t he mos t de t a i l f o r v i sua l men t a l images , co r r e spond ing
dis t inc t ions may a l so ex is t in o ther modal i t i es .
Image inspection
Imag e i n spec t ion i s the ab i l i ty t o ex t r ac t i n fo rma t ion t ha t i s dep i c t ed
in an image . Fo r example , one can de t e rmine whe the r t he f i rs t t h r ee no t e s o f Th re e b l i nd
mice go up o r dow n , o r whe the r t he r e a r e any cu rved l ines i n t he uppe rca se l e tt e r ' b ' . I f
one cou ld no t ' i n spec t ' pa t t e rn s i n images , images cou ld no t s e rve a s r epos i t o r i e s o f
i n fo rma t ion . I n t he cou r se o f in spec t ing imaged ob j ec t s , sub j ec ts r epo r t hav ing t o ' zoom
in ' t o ' s e e ' sma l l de t ai ls , and i n f ac t, peop l e r equ i r e more t ime t o ev a lua t e ob j ec t s imaged a t
smal l s izes than to eva lua te ob jec ts imag ed a t l a rger s izes . S imi la r ly , images can only be so
l a rge be fo r e t hey appea r t o ' ove r f l ow ' one ' s f ie ld o f v i ew . I n add i t i on , t he f a r t he r one m us t
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
5/10
THE COGNITIVE NEUROSCIENCE OF MENTAL IMAGERY 339
s h i ft o n e s m e n t a l g a z e t o s e e a p o r t i o n o f a n i m a g e d o b j e c t, t h e m o r e t i m e is t a k e n . A n d
s o m e p a r t s o f o b j e c t s a r e m o r e d i f f ic u l t t o s e e i n i m a g e s ; f o r e x a m p l e , a p a r a l l e l o g r a m is
h a r d e r t o s e e i n a n i m a g e o f t h e S t a r o f D a v i d t h a n is a t ri a n g l e [ fo r r e v ie w s o f s u c h
f ind ings , s ee 13] .
Image generation
L o g i c a l l y , im a g e s c a n a r i s e in o n l y t w o w a y s : F i r st , s e n s o r y i n p u t c a n
b e r e t a i n e d ( o v e r t h e c o u r s e o f a f e w s e co n d s , n o t s i m p l y a f r a c t i o n o f a s e c o n d ) ; s e c o n d ,
i n f o r m a t i o n s t o r e d in l o n g - t e r m m e m o r y c a n b e a c ti v a t ed . M o s t i m a g e r y a p p e a r s t o a ri se
w h e n i n f o r m a t i o n i n l o n g - t e r m m e m o r y i s a c t i v a t e d . I n d e e d , m a n y i m a g e s a r e n o v e l
c o m b i n a t i o n s o f o b j e c t s o r c h a r a c te r i st i c s t h a t w e r e e n c o d e d a t d i f f e r e n t t i m e s a n d p l ac e s .
F o r e x a m p l e , m o s t p e o p l e c a n v i s u a li z e th e i r f a v o r i te p o li t ic i a n r id i n g a d o n k e y , a n d
r e p o r t w h e t h e r h e o r s h e c o u l d s e e o v e r t h e t o p o f t h e a n i m a l s h e a d - - - e v e n t h o u g h s u c h a
s c e ne w a s n e v e r w i tn e s s ed . M a n y s t u d ie s h a v e n o w s h o w n t h a t s u c h i m a g e g e n e r a t i o n
i n v o l v e s s e r ia l p ro c e s s i n g ; i n d e e d , t h e t i m e t o f o r m i m a g e s t y p i c a l l y i n c r e a s e s l i n e a r l y f o r
e a c h a d d i t i o n a l p a r t o r p r o p e r t y o f t h e t o - b e -v i s u a li z e d o b j ec t , a n d t h e t im e t o v i su a l iz e a
g i v e n p a r t o f t e n c a n b e p r e d i c t e d b y t h e o r d e r i n w h i c h s u b je c ts t y p i c a l ly d r a w t h e p a r t s .
I m a g e , g e n e r a t i o n h a s b e e n t h e t o p i c o f in t e n si v e s tu d y i n n e u r o p s y c h o l o g y , p a r ti c u l a r l y
s in c e F a r a h [7 ] r e v i e w e d t h e c a s e s t u d y l i t e r a t u r e a n d c l a i m e d t h a t t h i s a b i li t y is s e le c t iv e l y
d i s r u p t e d b y d a m a g e t o t h e p o s t e r i o r le f t c e r e b r a l h e m i s p h e r e .
Image transformation I m a g e r y w o u l d b e o f li m i te d v a l u e i f w e c o u l d o n l y r e c al l p a s t
e v e n t s o r r e c o m b i n e t h e s e m e m o r i e s in s p ec if ic w a y s . I n m a n y c a s e s w e w a n t t o t u r n
s o m e t h i n g o v e r i n o u r m i n d s [ se e 19 ]. O n e o f t h e m o s t i n te r e s ti n g f a ct s a b o u t i m a g e
t r a n s f o r m a t i o n s is th a t t h e y o f t e n p r e se r v e th e t i m e c o u r s e o f t h e c o r r e s p o n d i n g a c t u a l
t r a n s f o r m a t i o n . F o r e x a m p l e , i f y o u r o t a t e d t h e u p p e r c a s e l e tt e r n 9 0 ° c l o c k w is e , w o u l d i t
b e a n o t h e r l e tt e r? W h a t i f y o u r o t a t e d a n u p p e r c a s e le t t er p 1 80 °? T h e f a r t h e r y o u w o u l d
h a v e t o r o t a t e t h e o b j e c t , m o r e t i m e y o u w o u l d r e q u i r e . S im i l a rl y , t h e m o r e s q u a r e s o n e
m u s t f o l d t o v i su a l iz e a s e t o f c o n n e c t e d s q u a r e s f o r m i n g a b o x , t h e m o r e t i m e i s r e q u i r e d
[2 0]. S u c h r e s u l ts a r e a s e x p e c t e d i f o n e i s s im u l a t i n g w h a t w o u l d o c c u r i f a n o b j e c t w e r e
a c t u a ll y m a n i p u l a t e d i n a sp e cif ic w a y , a n d - - a s n o t e d a b o v e - - t h e m o t o r s y st e m m a y p l a y
a r o l e i n im a g e t r a n s f o r m a t i o n s .
Image retention
M a n y i m a g e r y t a s k s r e q u i r e c o n s i d e r a b l e t i m e t o c o m p l e t e , a n d t h u s
t h e im a g e m u s t b e m a i n t a i n e d . M a i n t a i n i n g a n i m a g e d p a t t e r n r e q u i r e s e f f o rt , a n d o n l y a
l im i t e d a m o u n t o f i n f o r m a t i o n c a n b e v is u a li z e d a t th e s a m e t i m e . T h i s a s p e c t o f i m a g e r y
h a s r e c e i v e d r e m a r k a b l y l i tt le st u d y , h o w e v e r , a n d t h e a v a i l a b l e re s e a r c h m e r e l y s e rv e s to
v e r i f y t h a t s u c h i n t r o s p e c t i o n s a r e c o r r e c t [ f o r a r e v i e w , s ee 1 3 ].
Reasoning using imagery
O n e r e a s o n w h y im a g e t r a n s f o r m a t i o n s a r e i m p o r t a n t i s t h a t i m a g e s a re m o r e t h a n a
m e a n s b y w h i c h i n f o r m a t i o n i s l e a r n e d , s to r e d a n d r e t r ie v e d . T h e y a l s o p l a y a k e y r o l e in
s e v e r a l t y p e s o f r e a s o n i n g .
Concrete and abstract reasoning
I m a g e r y c a n p l a y a r o l e i n t w o d i s t i n c t t y p e s o f
r e a s o n i n g . F i r s t , o n e c a n r e a s o n a b o u t p e r c e p t u a l p r o p e r t i e s t h e m s e l v e s . F o r e x a m p l e ,
c o n s i d e r h o w y o u d e c i d e w h a t is th e b e s t r o u t e t o t a k e t o g e t t o t h e a i r p o r t a t a s p e ci fi c
t im e o f d a y f r o m w h e r e y o u w o r k , o r h o w y o u w o u l d d e c i d e w h e t h e r a s o f a s e en in t h e
s t o r e w o u l d f it i n y o u r l iv i n g r o o m . I n b o t h c a se s , i m a g e r y is u s e d t o c a r r y o u t a k i n d
o f m e n t a l s i m u l a t i o n . S e c o n d , i m a g e r y c a n b e u s e d i n a b s t r a c t r e a s o n i n g . I n d e e d ,
i m a g e r y a p p a r e n t l y p l a y e d a r o l e i n s o m e o f t h e k e y d i s c o v e r ie s i n s c ie n c e [s ee 1 9]. F o r
e x a m p l e , a n i m a g e o f s n a k e s b i ti n g t h e i r o w n t a i ls a p p a r e n t l y l ed K e k u l e t o i n f e r t h e
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
6/10
1340 S.M. KOSSLYN M. BEHRMANN and M. JEANNEROD
ring-structure of benzene. In this sort of reasoning images are used as symbols. Such
reasoning occurs when one visualizes Venn Diagrams or lines with dots on them as aids
to solving logic problems.
Using imagery to comprehend language
Another kind of reasoning involves the
inferences one makes when comprehending language. Imagery is often contrasted with
verbal abilities, but it is clear that the two faculties work together in many ways. We earlier
noted that imagery can help one to learn new information, including verbal information.
In addition, imagery can help one to comprehend verbal descriptions. This is most obvious
when the description specifies visual or spatial information. For example, imagery is likely
to be used when one understands a description of the arrangement of an apartment or the
design of a complex machine.
OVERVIEW
The articles in this special issue span the range of research on imagery, and either speak
directly to the brain bases of imagery or have direct implications for future
neuropsychological research.
Learning and memory
We begin with an article that harvests the results of decades of research on the role
of imagery in memory. Richardson turns to the use of imagery mnemonics as a way to
assist patients with memory impairments (caused by brain damage or disease). He
found that patients with less severe memory impairments benefited most from imagery
training, but the effectiveness of training was not strongly related to the locus of brain
damage or to the etiology of the disorder. More strongly motivated patients also did
better, but intelligence, imagery ability, and education were not very important
predictors of the degree of improvement. It is of great interest that brain-damaged
patients who could use mnemonics effectively nevertheless may not use this strategy
spontaneously.
The early work on the role of imagery in memory focused on the importance of
concrete vs abstract materials, or on the role of organizational factors in memory. De
Beni and Pazzaglia consider memory for different kinds of mental images, focusing
specifically on the role of contextual and autobiographical variables. They asked normal
subjects to visualize a series of items, and measured how quickly the subjects could form
the images, how well they could later recall the items, and also assessed subjective
ratings of image quality. The items were visualized in isolation or in a specific context
(with three kinds of context), and De Beni and Pazzaglia particularly were interested in
the possibility that images of one s own life experiences have a special status in memory.
And in fact such images required the most time to generate, were recalled very well, and
were very vivid. F rom a neuropsychological perspective, this work is interesting because
it invites researchers to attempt to understand the importance of this particular kind of
imagery. It is possible, for example, that the right hemisphere plays a special role in
storing images of specific exemplars; does it also play a special role for autobiographical
images?
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
7/10
THE COGNITIVE NEUROSCIENCE OF MENTAL IMAGERY 34
ercept ion a nd action
In some ways imagery may have the same relation to other processes that glasses have to
the nose: Once certain properties of a nose were present (presumably to warm air, direct
scents to different nostrils, and so on), they could be used to hold up glasses. Similarly,
once specific properties o f basic sensory and response mechanisms were in place, they
could be recruited into imagery--which then in turn may have become a function in its
own right, thereafter being the subject of natura l selection.
Goldenberg, Miillbacher and Nowak report the case of a brain-damaged patient who
had cortical blindness, but still had visual mental imagery. This patient denied that she was
blind, and this belief could have been based on her confusing visual mental images for
actual percepts. This patient is interesting in part because her primary visual cortex was
almost totally lesioned; only a small portion of the cortex at the occipital tip of the left
upper calcarine lip was spared. Thus, her imagery apparently did not depend on an intact
area 17, but rather relied (at least in large part) on higher-level visual areas that were not
damaged. The authors note that this patient s imagery could have been triggered by tactile
or acoustic perception. Once her vision recovered in the portion of the field that was
registered by the preserved cortex, nonvisual input no longer appeared to induce the
illusion of seeing. These findings suggested that connections from nonvisual cortex play a
complex role in evoking imagery.
Servos and Goodale also examine a patient who had spared imagery in the face of
impaired perception. This patient had a severe visual form agnosia, but nevertheless was
able to scan mental images to search for specific features and could generate novel images
by combining images of previously encoded stimuli. This patient had been shown earlier to
use information about shape to control skilled grasping movements, even though she
could not use this information to categorize or characterize the form. These findings
suggest that sensory input, imagery, and motor control mechanisms have distinct ways of
accessing stored visual representations; it is interesting that imagery and motor access were
preserved together in the face of impaired sensory access. This patient appeared to have
damage primarily to areas 18 and 19; most of area 17 was intact, as well most extrastriate
cortex. The results may indicate that pathways within areas 18 and 19 are not essential for
visual mental imagery, but play a critical role in providing input to higher visual areas
during perception.
The fact that both motor control mechanisms and imagery can access representations of
shape even when perceptual input cannot is consistent with results summarized in the
following paper by Annett. Annett focuses on the kind of motor imagery that is used when
one mentally practices performing a specific action. Annett reviews theoretical and
practical issues that bear on the subjective, behavioral, and physiological methods used to
study moto r imagery. He argues that motor imagery relies on mechanisms that serve both
spatial and motoric functions, and that the highly distributed nature of motor control
blurs the question of whether motor imagery is primarily perceptual or motoric in nature.
Jeannerod also explores the role of the motor system in imagery. His central idea is that
motor images have the same properties as the corresponding actual motor representations.
He presents much evidence that supports this hypothesis. For example, the timing of
simulated movements respects the same constraints as that of actual movements. In
addition, simply imagining that a limb is moving produces an increase in specific reflexes in
that limb, as occurs during actual movement. Moreover, imaging movement produces
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
8/10
1342 S.M. KOSSLYN M. BEHRMANN and M. JEANNEROD
changes i n co r t ex l i ke t hose t ha t occu r when one ac tua l l y execu t e s an ac t i on . J eanne rod
p rop oses a h i e r a rch i ca l mod e l o f t he o rgan i za t i on o f a c t i on t ha t p rov ides an a ccou n t f o r
t he s imi l a r i t i e s be tween imag ined movemen t and ac tua l movemen t ; t h i s mode l p rov ides
the t heo re t i c a l f oun da t i ons fo r t he e f f ect s o f men t a l p r ac t i c e r ev i ewed i n t he p r ev ious
pape r .
The f ina l pape r i n t h is s ec t i on add re s se s a d i f fe r en t so r t o f mo to r image ry , wh ich i s no t
r e l a ted t o t he man ipu l a t i on o f shape . Wi l son , Smi th and Re i sbe r t examine t he ro le o f
subvo ca l i z a t i on i n aud i t o ry image ry . They examine t he pa r t ne r sh ip be tween t he i nne r
e a r a n d i n n e r v o i c e , a n d f in d th a t t h e tw o f u n c ti o n s u s u a l l y - - b u t n o t a l w a y s - - w o r k
toge ther . Al though th i s paper i s no t expl ic i t ly neuropsychologica l in i t s o r ien ta t ion , i t
p rov ide s a beacon fo r f u r t he r r e sea r ch ; i t s eems c l ea r t ha t aud i t o ry image ry can be more
d e e p l y u n d e r s t o o d b y a d o p t i n g a p a ra l le l a p p r o a c h t o t h a t n o w u s e d t o s t u d y v i s u al
image ry . I ndeed , it m ay t u rn o u t t ha t t he pa r a ll e ls r un deepe r t han t h is ; t he i nne r e a r and
v i sua l image i n spec t ion mechan i sm s may be func t i ona l l y s imi la r, and t he i nne r vo i ce
and som e fo rm s o f v i sua l image gene ra t i on mechan i sms (pa r t icu l a r l y t hose i n wh ich eye
movemen t s c an p rompt images i n a s equence ) may be func t i ona l l y s imi l a r . Such
mech an i sms a r e d i s cus sed i n t he a rt ic l e s t ha t add re s s t he ro l e o f image ry i n i n fo rm a t ion
process ing .
nformation processing
E n o r m o u s p r o g r e ss h a s b e e n m a d e r e c en t ly in u n d e r s t a n d i n g t h e b r a in - b a s e s o f t h e
mech an i sms t ha t g ene ra t e v i sua l men t a l images. Fa r ah r ev iews r ecen t ev idence t ha t speaks
to tw o gene ra l i s sue s abo u t imag e gene ra t ion . F i r s t, she cons ide rs t he ques tiOn o f whe th e r
a d i s t i nc t componen t p roces s i s u sed t o gene ra t e v i sua l men t a l images , o r whe the r
mechan i sms t ha t s e rve o the r pu rpose s work t oge the r t o a ccompl i sh t h i s end . Second , she
r ev iews f i nd ings t ha t bea r on t he neu roana tom ica l l oca l iz a t i on o f t he p roces se s tha t
unde r l i e imag e gene ra t i on , an d co ns ide r s wh ich reg ions o f t he b r a in a r e r ec ru i t ed by t he se
processes . The fo l lowing ar t ic les repor t addi t iona l f ind ings tha t speak to these i ssues .
S t anga l i no , Semenza a nd M ond in i r e po r t a de t a i l ed i nves t iga t i on o f t he e f fec t s o f b r a in
dam age on t he ab i l i ty t o gene ra t e v i sua l men t a l images . They t e s t ed 70 un i l a te r a l ly b r a in -
dam age d pa t i en t s w i th a s e t o f ta sks t ha t w as de signed t o a s ses s d i s ti nc t com po nen t
p roces se s . Cons i s t en t w i th t he o r ig ina l r e t r o spec t ive ana ly s is o f Fa r a h [7] , t he se r e sea r che rs
foun d t ha t d am age t o pos t e r i o r po r t i on s o f the l ef t hemi sphe re m os t f r equen t l y we re
re la ted to def ic i t s in the ab i l i ty to genera te images .
K o s s l y n , H a m i l t o n , M a l j k o v i c , H o r w i t z a n d T h o m p s o n p r o v i d e e v i d e n c e t h a t v i s u a l
men ta l images can be fo rmed i n two ways . One me thod r e l i e s on s t o r ed de sc r i p t i ons t o
a r r ange p a r t s o f image ob j ec t s , and t he o the r r el ie s on s t o r ed i n fo rma t ion ab ou t me t r i c
d i s t ance t o a r r ange pa r t s . The fo rm er me thod is more e f f ec ti ve when s t imu l i a r e p r e sen t ed
in the r igh t v i sua l f ie ld (and hence a re processed in i t i a l ly by the le f t ce rebra l hemisphere) ,
whe rea s t he o the r m e tho d is more e f fec t ive when s t imu l i a r e p r e sen t ed i n t he l e f t v i sua l
f ie ld (and hence a re processed in i t i a l ly by the r igh t cerebra l hemisphere) . Accord ing to
the i r ana ly s i s , p r ev ious r e sea r ch w i th b r a in -damaged pa t i en t s ha s u sed ma te r i a l s t ha t
can no t e a s i ly be v i sua l iz ed u s ing t he r i gh t-hemi sphe re p roces se s , and hence pa t i en t s w i th
l e f t- hemi sphe re dam age have been s e lec t ive ly impa i r ed .
D e n i s , G o n c a l v e s a n d M e m m i f o c u s o n a n o t h e r a s p e c t o f im a g e g e n e r a ti o n , n a m e l y it s
accu racy . I n p r ev ious work , t h i s g roup ha s shown tha t peop l e can u se ve rba l de sc r i p t i ons
to con s t ruc t images , wh ich can t hen be s canned . B ecause time i s r equ i r ed t o s can each
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
9/10
THE COGNITIVE NEUROSCIENCE OF MENTA L IMAGERY 1343
increment of distance, scan time can be used as an index of how accurately the parts of the
image are positioned. Using this technique, Denis and his colleagues have shown that the
structure of the initial description affects how accurately an image can be generated. But
even when subjects are given poorly structured descriptions, their images improve after
they see the description repeatedly. The present article reports a quantitative model and a
computer simulation that accounts for this increase in image accuracy. In addition, new
data from humans are reported that bear on this account, and underline the importance of
'regions of uncertainty' (i.e. indeterminancies about the locations of parts) in the image
generation process.
Georgopolous and Pellizzer consider another aspect of information processing in
imagery, mental rotation. They compare the processes that underlie mental rotation to
those used to scan through memorized lists of digits. This paper integrates previous work
by this group with monkeys and recent work with humans. The monkey research relied on
single-cell recordings, and showed that motor processes are involved in at least some forms
of mental rotation. The present work shows that mental rotation is in fact distinct from
memory scanning in humans, which is consistent with the role of the motor system in
imagery.
Reasoning
Vecchi, Monticelli and Cornoldi examine one of the key mechanisms that allows us to
use imagery in reasoning: working memory. A major bottleneck in using imagery in
reasoning is the capacity of working memory, and these researchers hypothesize that this
capacity hinges on properties of a passive store and properties of active imagery
operations. To test these ideas, subjects were asked to visualize moving through two-
dimensional and three-dimensional matrices; the matrices had some filled cells, and the
subjects had to retain the image in order to perform the task. Experiments with
congenitally blind people and with sighted people revealed that image capacity is a result
of at least two different factors.
Geminiani, Bisiach, Berti and Rusconi argue that imagery plays such a central role in
thinking that language cannot entirely compensate for severe impairment of 'analogue
representations'. They argue that language does not have an independent role in thought,
and provide suggestive evidence that syntactic structures themselves can be represented in
a non-linguistic medium. This work is intriguing, and moves into an area that has received
very little attention heretofor.
fterword
We finish with some reflections by Cooper on wha t counts as a mental image . After
the wide and varied writings that have come before, this question gains force. Issues like
this can only be addressed in the context of empirical findings, and can help not only to
make sense of those findings but to guide one to consider additional research questions.
REFERENCES
1 . A nde rson , J . R . Argum ent s conce rn ing r ep re sen ta t i ons fo r m en ta l im age ry .
P s y c h o l . R e v .
85, 249-277, 1978.
2. Baylor ,
G . W . A T r e a t i s e o n t h e M i n d s E y e .
Vol . Ed.) See . Ed.). Ca rnegie Mel lon Univers i ty , Pi t t sburgh,
Pennsylvania , 1971.
3 . Behrm ann , M. , Winocur , G . and Moscov i t ch , M. Di ssoc i a t i on be tween m en ta l im age ry and ob j ec t
r ecogn i t i on i n a b ra in -dam aged pa t i en t.
N a t u r e
359, 63~637, 1992.
8/17/2019 The Cognitive Neuroscience of Mental Imagery (Kosslyn 1996)
10/10
1344 S.M . KOSSLYN , M. BEHRMA NN and M. JEANNEROD
4. B ower , G . H. M en ta l im age ry and a ssoc i a ti ve l e a rn ing . In Cognition in Learning and Memory, L . G r e g g
(Edi tor) , pp. 51-88. Wiley, New Y ork, 1972.
5. Crav er-Lem ley, C. and Reeves, A. V isual imagery se lec tively reduces vernier acuity . Perception 16, 533-614,
1987.
6 . Deu t sch , G. , B ourbon , W . T . , Papan i co l aou , A. C . and E i senbe rg , H. Vi suospa ti a l expe r im en t s com pared v i a
ac t iva t ion of regional cerebra l b lood f low. Neuropsychologia26, 445-452, 1988.
7. F arah , M. J . T he neu rologica l basis o f men ta l imagery: A com pon ent ia l ana lysis .
Cognition
18, 245-272,
1984.
8. Fe l lema n, D. J . and V an Essen, D. C. Dist r ibu ted hierarchica l processing in pr imate cerebra l cor tex.
Cerebr.
Cortex 1, 1-47, 1991.
9. Fink e , R. A. and Sh epard, R . N. Visual funct ions of men ta l imagery. In
Handbook of Perception and Human
Performance,K. R . Boff, L . Ka ufm an and J . P. Thom as (Edi tors) , pp. 37-1-37-55. W iley-Intersc ience , N ew
York, 1986.
10. Gardner , H.
The Mind s New Science: A History of the Cognitive Revolution.
Basic Books, New York, 1985.
11. G corg opo lous, A. P. , Lur i to , J . T . , Pe t r ides, M. , Schwartz, A. B. and Massey, J . T . Menta l rota t ion o f the
neurona l popu la t i on vec to r .
Science
243, 234-236, 1989.
12. Kosslyn, S. M. Informat ion representa t ion in visua l images. Cognit. Psychol.7, 341-370, 1975.
13. Kosslyn, S. M.
Image andBrain: The Resolution of the Imagery Debate.
M IT Press, Camb ridge , Mass. , 1994.
14. K osslyn, S. M. an d Jol icoeu r , P. A theory -based app roach to the study o f individual dif ferences in menta l
im age ry . In Aptitude, Learning and Instruction: Cognitive Processes Analysis of Aptitude, R. E . Snow, P . A.
Fed er ico and W. E. M ontag ue (Edi tors) . Law rence Erlbau m, H i l l sda le , Ne w Jersey, 1981.
15. L ev ine , D . N . , Warach , J . and Fa rah , M. J . T w o v i sual sys tem s in m en ta l im age ry : Di ssoc ia t i on o f 'wh a t ' and
'where ' in imagery disorders due to bi la tera l poster ior cerebra l lesions.
Neurology
35, 1010-1018, 1985.
16. Paivio, A. Imagery and Verbal Processes. Hol t , R ineha r t Wins ton , New York , 1971.
17. Richa rdson, A. M enta l prac t ice : A review and discussion (Par t I and I I ) .
Res. Quart. 38,
95-107; 263-273,
1967.
18. Segal , S. J . and Fuse l la , V. Inf luence of imaged pic tures and sounds on de tec t ion of visua l and audi tory
signals. J. Exp. Psyehol. 83, 458-464, 1970.
19. Shepa rd , R . N. and C oope r , L . A. Mental Images and their Transformations.M IT Pre ss , Cam br idge , Mass . ,
1982.
20 . Shepa rd , R . N. and Feng , C . A ch ronom e t r i c s tudy o f m en ta l pape r fo ld ing . Cognit. Psychol. 3, 228-243,
1972.
21. S hepard, R . N. and Metz ler , J . Menta l rota t io n of three-dimensional objec ts. Science 171, 701-703, 1971.