Vision after complete blindness: Mike May

Fine et al., Nature Neuroscience 2003; Robert Kurzon, ‘Crashing Through’

But understanding normal perception is the toughestand most important challenge for psychology

We tend to take normal perception for granted, while findingbizarre experiences of exceptional individuals intriguing…

Occasionally, though, an exceptional individual’s experience can advance our understanding of normal perception…

For instance, the case of Mike May helps us not to take perception for granted

To what extent does visual processing rely on visual

experience?

Ione Fine (UCSD USC)

Ione Fine (UCSD USC)

Geoff Boynton (UCSD/Salk): fMRI+

Ione Fine (UCSD USC)

Geoff Boynton (UCSD/Salk): fMRI+

Brian Wandell (Stanford)and

Alex Wade, Alissa Brewer (Stanford)

Ione Fine (UCSD USC)

Geoff Boynton (UCSD/Salk): fMRI+

Brian Wandell (Stanford)

Alex Wade, Alissa Brewer (Stanford)

Stuart Anstis (UCSD)

Subject Mike May

• Blinded by a chemical accident at age 3. • Light sensitive (no form vision) between ages of 3-43

Subject Mike May

Sight restored by a new procedure - Corneal epithelial stem cell replacement

• Resolution• 2 & 3d Form• Motion• Object/Face Recognition

Campbell & Robson (1968)

Spatial frequency (cycles/degree)

Sensitivity

Contrast-Sensitivity Function (CSF)

Resolution limit: 50cpd

Resolution limit < 2 cpd, despite good optics

0 1 2 30

0.5

1

1.5

2

Spatial frequency (c/deg)

Log

sen

siti

vit

y

MM (Post-operatively)Normal

0 1 2 30

0.5

1

1.5

2

Spatial frequency (c/deg)

Log

sen

siti

vit

y MM + 5 monthsMM +11 monthsMM +17 monthsMM +21 monthsCONTROL

No improvement over time

MM could identify simple shapes2D FORM

…but not shapes defined by illusory contours. 2D FORM

2D FORM

Mike can identify simple 2d forms (100% correct)

But “constructive” 2d perception is harder

MM = 80%; controls=100%, 90%, 95%)

MM = 73%; controls = 80%, 85%, 100%

guessing

Letters recognizable

3D FORM

Sensitive (100% correct)to occlusion …

Shading gave no automatic impression of depth:

3D FORM

The circle was seen as a flat disc, with non-uniform surface lightness

“A square with lines attached”

3D FORM

Fails with:

Shape from Shading:

Perspective:

Could NOT recognize a stationary cube from any angle - “square with lines”

MOTION

Couldn’t identify STATIC cube…but with a ROTATING one, “It’s a cube! …going in …going out”

Could NOT recognize a stationary cube from any angle - “square with lines”

MOTION

Could NOT recognize a stationary cube from any angle - “square with lines”

YET … Can exploit motion to construct 3D structure- “it’s a cube! …moving in, moving out”

MOTION

MOTIONCould make sense of…

Point-light walker

Rotational Glass patterns

Structure from motion(100% correct)

QuickTime™ and aGIF decompressor

are needed to see this picture.

MM = 90%; controls=95%, 80%, 85%).

Sophisticated processing of MOTION:

• Can see form from motion (KDE cube)

• Saw depth in face masks by rocking his head

• Could see Johansson’s walking man

• Can play catch

• Skiing: vision now helps!

Motion SB (Ackroyd et al)“His only signs of appreciation were to moving objects, particularly the pigeons in Trafalgar square… He clearly enjoyed … watching … the movement of other cars on the road …He spotted a speeder coming up very fast behind us”

Virgil (Sacks)“when [the gorilla] finally came into the open he thought that, though it moved differently, it looked just like a large man”

QuickTime™ and aCinepak decompressor

are needed to see this picture.

GenderMM 70% correctcontrol 100%

Expression (happy/sad/neutral) MM 61% correctcontrol 100%

MM 25% correctcontrol 100%

Poor object & face identification

Clinton&GoreBy Sinha &Poggio

These dissociations between form and motion tasks were consistent with the size and

activation of visual areas measured using fMRI

V1 and (especially) extrastriate areas in the temporal stream, thought to be responsible for form processing, were small and showed low activity levels.

The Medial Temporal complex, thought to be responsible for motion processing, was normal in both size and activation

Cortical area

Su

rfa

ce a

rea

(cm

2)

0

5

10

15

20

25

30

35

V1(L)

V1(R)

MT+(L)

MT+(R)

control observersMM

Size of V1 and MT+

“The eye of the artist”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

“flat world”Matches projective shapes, not real shapes, e.g. NOT susceptible to Shepard’s illusion:

Tables have the SAME projective shape, and to MM they look the same

Images from Ted Adelson

Mike correctly sees thediamonds as similar in lightness, and responds photometrically to illumination and shadow in pictures, seeing shadows as dark things.

http://psylux.psych.tu-dresden.de/i1/kaw/diverses%20Material/www.illusionworks.com/html/shadow.html

• Normally sighted subjects cannot retrieve any aspect of experience that is a function of retinal illuminance or projected size. But MM has these (and nothing else) available to undirected introspection. In this sense he is free (unfortunately), of the good ‘illusions’ on which normal vision is founded.

• One example of resulting difficulties: Shadows at the edges of sidewalks appeared to him as black ridges that could present a potential hazard in walking

Phenomenal Regression to the Real Object

• Why can’t we see and judge what’s present at the sensory input, as MM can?

• William James wrote: “Pure sensations can only be realised in the earliest days of life. They are all but impossible to adults with memories and stores of association acquired."

• For MM (though NOT necessarily for a newborn: Granrud), James may be right, perhaps because the irrepressible interpretative processes of the normally sighted brain are not involved.

• For the normally sighted, interpretation is not an integument that can be peeled away to reveal sensory bedrock: it penetrates all our consciousness, presumably thanks to the continuously bidirectional flow of information through the visual system.

• So we have no ‘pure sensations’…but those ares all that MM has.

Phenomenal Regression to the Real Object

V1

LGN

Parietal (action)

temporal (perception)

The visual process as a causal chain

V1

LGN

Parietal (action)

temporal (perception)

In the normal visual system, each neural representation depends on the later ones.

The visual process as a feedback system

MM was not sensitive to perspective cues

DEPTH

Yet he WAS susceptible to the Muller-Lyer and related illusions

Richard Gregory

“Dumbbell” variant of the Muller-Lyer illusion

Aesthetics

• Color– Variety and vividness were new and impressive

• Bodies– Innate sign stimuli vs. interest based on association

• Dust, waves, fireworks– Meaning confers no aesthetic advantage

Key points about Mike’s perception

• Mike has a severe neural resolution loss,improving slowly if at all.

• He doesn’t see 2D subjective contours.• 2D Geometrical illusions are present.• Both perspective and shape from shading are

ineffective for depth perception.• But motion cues are effective.• Recognizing faces and common objects is a challenge.• We think of Mike as having “the eye of the artist”, inhabiting

a world of abstract 2-dimensional shapes and colors.• This may be why he now uses his vision, as he puts it, “mainly for entertainment”

The End