Date post: | 03-Jan-2016 |
Category: |
Documents |
Upload: | magee-valdez |
View: | 31 times |
Download: | 1 times |
EE1411
Perception Perception - Vision- Vision
Janusz A. Starzyk
http://grey.colorado.edu/CompCogNeuro/index.php/CECN_CU_Boulder_OReillyhttp://grey.colorado.edu/CompCogNeuro/index.php/Main_Page
Based on book Cognition, Brain and Consciousness ed. Bernard J. Baars
courses taught by Prof. Randall O'Reilly, University of Colorado, andProf. Włodzisław Duch, Uniwersytet Mikołaja Kopernikaand http://wikipedia.org/
Cognitive Neuroscience Cognitive Neuroscience and Embodied Intelligenceand Embodied Intelligence
EE1412
MotivationMotivationPerception is comparatively the easiest to understand although for many specific questions there are no clear answers.
General questions:
Why does the primary visual cortex react to oriented edges?
Why does the visual system separate information into the dorsal stream, connected to motion and representation of object locations, and the ventral stream, connected to object recognition?
Why does damage to the parietal cortex lead to spatial orientation and attention disorders?
In what way do we recognize objects in different places, orientations, distances, with different projections of the image onto the retina?
EE1413
IntroductionIntroduction
The purpose of vision: ‘to know what is where’ (David Marr)
The visual perception is far more complicated than simply taking a picture with digital camera
The camera doesn’t really do anything with this image and doesn’t have any knowledge about what is stored in the image
EE1414
Knowing what: perceiving features, groups and objects Studies of human visual perception and neuroscience suggest that
there are many levels of perception. The human brain appears to process basic visual features, such as
color, orientation, motion, texture and stereoscopic depth. Neurons are highly tuned to specific features like a line at
particular angle, a particular color, or particular motion detection. The activity of each neuron represents only a small part of the
visual field. How is the brain able to combine this information across many neurons? The brain is able to organize basic feature elements into organized
perceptual groups. Psychologists proposed the Gestalt laws of perceptual grouping, such as the
laws of similarity, proximity, good continuation, common fate and so forth
IntroductionIntroduction
EE1415
Perceptual groupingPerceptual groupingGrouping by similarity:
White dots grouped with white dots, squares with squares
Grouping by proximity:Here we perceive two separate groups of dots that are near each other
Grouping by good continuation:
On the left we perceive a single object. When the same lines are separated we do not
EE1416
Functional organization of the visual systemFunctional organization of the visual system
Visual pathway
EE1417
Visual pathwaysVisual pathwaysVisual pathways: retina => lateral geniculate nucleus (LGN) of the thalamus=> visual radiation => area of the primary cortex V1 => higher levels of the visual system => associative and multimodal areas.
V1 cells are organized in ocular dominance columns and orientation columns, retinoscopic. Simple layer 4 cells react to bands with a specific slant, contrasting edges, stimulus from one eye. A substantial part of the central V1 area reacts to signals from fovea, where the density of receptors is the greatest.
EE1418
The hierarchical organization begins in the retina, passes through the lateral geniculate nucleus (LGN - part of the thalamus), reaching the primary visual cortex V1, from where it's distributed further.
Functional organization of the visual systemFunctional organization of the visual system
EE1419
Functional organization of the visual systemFunctional organization of the visual system
Objects in environment are projected to the back of the eye – the retina.
Retina contains millions of photoreceptors
EE14110
Cones cone-shaped less sensitive operate in high light color vision
Rods rod-shaped highly sensitive operate at night gray-scale vision
Two types of light-sensitive photoreceptors
cone
rod
The retinaThe retina
EE14111
The signals from photoreceptors are processed by a collection of intermediary neurons, bipolar cells, horizontal cells and amacrine cells, before they reach the ganglion cells
The retinaThe retina
http://www.iit.edu/~npr/DrJennifer/visual/retina.html
EE14112
RetinaRetina The retina is not a passive camera registering images. Crucial rule: enhancing contrasts underlining changes in space and
time, strengthening edges, uniformly lit areas are less important. Photoreceptors in rods and cones, 3-layer network, ganglion cells =>LGN.
Receptive fields: areas, which stimulate a given cell. The combination of signals in the retina gives center-surround receptive fields (on-center) and vice versa, detects edges. Each individual field of cells can be modeled as a Gaussian model, so these fields are obtained as a difference of Gaussians (DOG).
EE14113
On-center off surround ganglion cellsOn-center off surround ganglion cellsNo stimuli:
both fire at base rateStimuli in center:
ON-center-OFF-surround fires rapidly
OFF-center-ON-surround doesn’t fire
Stimuli in both regions: both fire slowly
Stimuli in surround: OFF-center-ON-
surround fires rapidly ON-center-OFF-
surround doesn’t fire
EE14114
David Hubel & Torsten WieselDavid Hubel & Torsten WieselReceived Nobel price for their discovery of on-center off-surround cells in retina
http://www.physiology.wisc.edu/yin/public/
on-center cell
On-center off surround ganglion cellsOn-center off surround ganglion cells
EE14115
Retina ganglion cells receive both excitatory and inhibitory inputs from bipolar neurons
In the figure shown, the ganglion cell receives excitatory inputs from cells corresponding to the on-center region, and inhibitory inputs from the off-center region
On-center off surround ganglion cellsOn-center off surround ganglion cells
EE14116
Lateral inhibition is important in enhancing neural representation of edges, where the light intensity changes sharply and indicate a presence of contours, shapes, or objects.
Uniform parts of a picture are less interesting.
On-center off surround ganglion cellsOn-center off surround ganglion cells
Original image Image based on edges
EE14117
Sometimes this later inhibition leads to a surprising visual illusions as shown on this figure.
Notice black dots appear on intersection of white lines.
On-center off surround ganglion cellsOn-center off surround ganglion cells
EE14118
Retinal ganglion cellsRetinal ganglion cells There are two types of ganglion cells in the retina:
Large magnocellular ganglion cells, or M cells, carry information about:
– Movement– Location– depth perception.
Smaller parvocellular ganglion cells, or P cells, transmit signals that pertain to:
– Colour– Form– texture of objects in the visual field.
EE14119
Lateral geniculate nucleus (LGN)Lateral geniculate nucleus (LGN) From the eye, retinal ganglion cells send
their axons to a structure in the thalamus called lateral geniculate nucleus (LGN)
The inputs from the nasal portion of each retina must cross at the optic chiasm to project to the opposite LGN
EE14120
LGN pathway LGN pathway The M cells send their
information to layers 1 & 2 of LGN.
The P cells send their information to layers 3-6.
So, layers 3-6 are involved in processing information concerning fine detail and color.
Layers 1 & 2 process information concerning movement.
EE14121
Pathways in visual systemPathways in visual system
Propagation of the visual input from the left and right visual fields.
Signals propagate through eye, retina, optic nerve, chiasm, optic tract, LGN to visual cortex V1
EE14122
Lateral geniculate nucleusLateral geniculate nucleus Signal compression – partly already done in the retina. Different types of information find their way to different LGN layers. Intermediate station – all sensory signals (except olfactory) go
through different nuclei of the thalamus. Dynamic information processing: steering attention and fast large-
celled pathway reacting to motion. Retroactive projections V1=>LGN are an order of magnitude more
numerous than LGN=>V1 (role - prediction).
The competitive dynamic selects signals from the visual field, especially involving motion.
EE14123
LGN of the ThalamusLGN of the Thalamus
Parvocellular layers 3-6
Magnocellular layers 1& 2
EE14124
Neurons in V1 are sensitive to a whole host of visual features, not seen in the LGN, like orientation, direction of motion, color differences, or binocular disparities.
Orientation helps to detect edges and contours. Direction of motion is important to determine dangerous
moves of an attacker. Color helps to differentiate and identify objects
particularly in a camouflage environment. Binocular disparities between images in two eyes allow us
to perceive stereo-depth when we look at object with both eyes.
Primary visual cortex V1Primary visual cortex V1
EE14125
Primary visual cortex V1Primary visual cortex V1
Neurons in V1 respond with different strength to orientation edges, depending on location of their receptive fields.
Neuron’s response is strongest if the excitation aligns with its receptive field.
EE14126
Edge detectorsEdge detectorsContrasting signals points from the LGN are organized by the V1 cortex into edge detectors oriented at a specific angle.
Simple V1 cells combine into edge detectors, enabling the determination of shapes, other cells react to color and texture.
Properties of edge detectors: different orientation; high frequency = fast changes, narrow bands;
low frequency = gentle changes, wide bands;
polarization = dark-light or vice-versa, dark-light-dark or vice-versa.
EE14127
Representation in the V1 cortexRepresentation in the V1 cortex
Oriented edge detectors can be created by correlational Hebbian learning
based on natural scenes.
What happens with information about color, texture, motion?
EE14128
Retinoscopic maps in V1Retinoscopic maps in V1
The spatial position of the ganglion cells within the retina is preserved by
the spatial organization of the neurons within the LGN layers. The
posterior LGN contains neurons whose receptive field are near the fovea.
EE14129
Area V1: The Primary Visual CortexArea V1: The Primary Visual Cortex
V1 is made up of 6 layers (no relation to 6 layers in LGN).
LGN sends axons to layer IV of V1. M and P cells are
separate. Right and Left eye
are separate.
EE14130
Hierarchy of visual processingHierarchy of visual processingFrom retina, LGN, V1, through V4 and to ventral temporal cortex (VTC) neurons gradually respond to more complex stimuli:
Retina and LGN extract small dotsIn V1 small dots are combined into edges, In V4 edges are combined into simple shapes and color featuresIn VTC simple shapes are combined into objects
EE14131
Extrastriate visual areas – outside of V1Extrastriate visual areas – outside of V1
V1 sends signals to many higher visual areas, including areas such as V2, V3, V4 and motion-sensitive area MT.
Area V4 is important for the perception of color and some neurons in V4 respond to more complex features or their combination (like corners or curves).
The middle-temporal area (MT), is important for motion perception. Almost all of the neurons in MT are direction-selective, and respond
selectively to a certain range of motion directions or patterns of motion.
Flattened map of higher visual areas
EE14132
The ventral and dorsal pathways: The ventral and dorsal pathways: knowing knowing what what and and wherewhere
The projections from V2 to higher areas in the cortex can be roughly divided according to two major parallel pathways: a ventral pathway to temporal lobe (what) and a dorsal pathway to parietal lobe (where)
where
what
The ‘what’ pathway includes ventral areas like V4, LOC, and IT
EE14133
depth
direction
“where” pathway
“what” pathway
orientation
shape
color
RecognizedObject ready for perception
“Where" = large-celled pathway, heading for the parietal lobe.
"What"= small-celled pathway heading for the temporal lobe (IT).
Two streams whereTwo streams where?/what??/what?
EE14134
In the dorsal pathway, signals from V1 travel to dorsal areas like MT and V3A, which then send major projections to many regions of the parietal lobe.
In the ventral pathway, many signals from V1 travel to ventral areas V2, V3 and V4 and onward to many areas of the temporal lobe.
The ventral and dorsal pathways: The ventral and dorsal pathways: knowing knowing what what and and wherewhere
Dorsal and ventral pathways in a monkey brain
EE14135
Two streams where?/what?Two streams where?/what?
Milner and Goodale (1995): visual pathways don't so much determine where and what, as much as they enable action and perception.
There is also the old limbic pathway, enabling rapid action in dangerous situations (after which follows a wave of fear).
What? - temporal lobe
Where? - parietal lobe
EE14136
Two streamsTwo streamsUngerleider and Mishkin (1982): there exist two notably divided pathways for processing visual information, running from the eye. Large-grained PA retina cells, 3 types of photoreceptive cones, large receptive fields, rapidly-conducting axons, activation for light in a wide band. Small-grained PB cells, 1 or 2 types of photoreceptive cones, small receptive fields, slowly conducting axons, recognize color oppositions.
Large-celled pathway: runs to two large-celled LGN layers, it's characterized by a low spatial resolution, high sensitivity to contrast, rapid signal transfer, without information about color. The small-celled pathway has 4 small-grained layers in the LGN, high spatial resolution, color, slower information transfer, low sensitivity to contrast.
EE14137
Dorsal pathwayDorsal pathway
Large-celled pathway: from the occipital lobe through the dorsal pathway to the parietal cortex. Arrives at the 4B layer in V1, from here to the thick dark stripes of the V2 region, analyzes information about object motion.
In V1, layer 4B => V5, localization in the field of vision, motion.
V5 stimulates the parietal lobe, PPC (posterior parietal cortex), regions 7 and 5; this enables spacial orientation, depth and motion perception(eye orientation).
EE14138
Ventral pathwayVentral pathway
Small-celled pathway: the ventral pathway, to the inferior temporal cortex.
V1 => V2 interblob region, reacts to line orientation, gives a large visual acuity, without color. V1 => V3 blob region, reacts to shapes, reaction to color in the neurons in the dark stripes of V3. V2 => V4, main area of color analysis, information arrives at the inferior temporal cortex (IT). The IT area in the inferior temporal lobe has neurons which react to complex objects.
EE14139
Model v1rf.proj.gz, Model v1rf.proj.gz, ChapterChapter 8 8How receptive fields are formed in V1?
Project description in chapter 8.3.2. Natural shapes and textures lead to specific
receptive fields: from here reactions to edges.
Inputs: 12x12, signals from LGN cells
on (pos) and off (neg) center.
Input samples: randomly selected
parts of 24x24 from 4 600x800 natural pictures.
Hidden layer 14x14;
links: random excitatory connections.
EE14140
Areas involved in object recognitionAreas involved in object recognition
Human neuroimaging studies have revealed many brain areas involved in processing objects.
These object-sensitive areas, which lie anterior to visual areas V1-V4, respond more strongly to coherent shapes and objects, as compared to scrambled, meaningless stimulus.
Neurons in V1 respond to scrambled pictures of a cat equally well or even stronger
While neurons in lateral occipital (LOC) respond much less to scrambled pictures than to a picture of a cat
EE14141
Lateral occipital complex (LOC) The lateral occipital complex seems to have a general role in object
recognition and responds strongly to a variety of shapes and objects. Presumably neurons in this region respond best to different kinds of
objects. Fusiform face area (FFA)
Human neuroimaging studies have shown that there is a region in the fusiform gyrus, called the fusiform face area (FFA) that responds more strongly to faces than to just about any other category of objects.
This region responds more to human, animal and cartoon faces than to a variety of non-face stimuli.
Neurons in this area specialize in facial expression, particular identity or viewpoint (e.g. profile)
Parahippocampal place area (PPA) The parahippocampal place area is another strongly category-selective
region that responds best to houses, landmarks, indoor and outdoor scenes. This area responds weakly to faces, body parts, and animals.
Areas involved in object recognitionAreas involved in object recognition
EE14142
Hierarchical and interactive theories of visionHierarchical and interactive theories of vision
Hierarchical and interactive theories of vision According to hierarchical theory, visual consciousness is organized in a
hierarchical fashion with increasingly higher visual areas being more closely related to our internal conscious experience. But if this is the case how to explain awareness of all details in the observed image?
The interactive theory of visual consciousness emphasizes interactions between lower and higher visual areas where higher areas send feedback signals down to early visual area.
Is it possible to invoke visual experience without seeing? Yes we can bypass retina and
LGN and stimulate area V1. However, it seems impossible to
recover full visual experience from higher visual areas bypassing primary visual cortex.
EE14143
Some answersSome answers Why does the primary visual cortex react to oriented edges?
Because correlational learning in a natural environment leads to this type of detector.
Why does the visual system separate information into the dorsal pathway and the ventral pathway?
Because signal transformations extract qualitatively different information, strengthening some contrasts and weakening others.
Why does damage to the parietal cortex lead to disorders of spatial orientation and attention (neglect)?
Because attention is an emergent property of systems with competition.
How do we recognize objects in different locations, orientations, distances, with different images projected on the retina?
Thanks to transformations, which create distributed representations based on increasingly complex and spatially invariant features.
EE14144
Model v1rf.proj.gz, Chapt. 8Model v1rf.proj.gz, Chapt. 8How do receptive fields form? Where do these V1 properties come from?
Description of the project in Chapt. 8.3.2. Natural shapes and textures
lead to specific receptive fields: from this come reactions to edges.
Inputs: 12x12, signals from LGN
cells: on- and off-center.
Input images: randomly chosen
from a natural 512x512 scene.
Hidden layer 14x14; connections:
coincidental with the input,
excitatory with the surroundings.
EE14145
Model propertiesModel propertiesThe V1 cortex receives from the LGN an on/off signal with heightened
contrast, input to V1 through layer 4, processing in this model responds to
overlapping processes mainly in layers 2 and 3.
The model includes one hypercolumn, analyzing a small sector of the image
from images of landscapes and plants => all elements see the same thing.
Properties: spherical geometry, i.e. top = bottom, left = right; independent inputs for on/off cells, in accordance with biology;
strong and widespread excitatory horizontal connections – like in SOM; kWTA leaves ~10% active neurons.
Contrast for weights is small ~1, because these aren't decision-making neurons, thresholds are large (~2) to force sparse representations, strong correlations.
Noise helps in avoiding weak solutions.
EE14146
Exercises fromExercises from v1rf v1rf
Check the structure, connection weights (r.wt): strong activations within
the hidden layer, random connections with on/off inputs.
LoadEnv to load the 512x512 image - for the training 10 images were
used, here is one random one, processed into on/off points.
StepTrain – observe the oscillation of learning for phases – and + Complementarity of on/off: stronger "on" activation for images which are brighter in the middle than on the edges, dark = extra "off" activation.
Question: what can we expect if horizontal connections will dominate? Check your predictions, temporarily changing lat_wt_scale 0.04 => 0.2.
LoadNet to load the trained network, after 100,000 presentations of images and several days of calculations...
EE14147
Receptive fieldsReceptive fieldsHow do receptive fields form? Where do these V1 properties come from?
View, PROBE_ENV shows 4 different probe stimuli, StepProbe will show
activation of hidden units.
Check the activation r.wt, change the color scale so we can better see the field orientation, check several hidden elements, bi- and tri-polar fields of both types.
Load all: View, RFIELDSactivation on=red, off=blue. Orientation, position, size, polarity are 4 different traits of receptive fields.
Radial orientation changes (pinwheels), singular points.