Master’s Thesis 1 : Simulating the McCollough Effect in a Self–Organizing Model of the Primary Visual Cortex Julien B. Ciroux (s0456262) Master of Science Informatics School of Informatics Edinburgh University June 6, 2006 1 This document is a slightly revised and reformatted version of the thesis submitted for the Master of Science degree in Informatics at Edinburgh University. Apart from minor edits to the text, some figures and references have been reordered and renumbered. All results are the same as in the original thesis.
Transcript
Master’s Thesis1:Simulating the McCollough Effect
in a Self–Organizing Model ofthe Primary Visual Cortex
Julien B. Ciroux (s0456262)Master of Science
InformaticsSchool of InformaticsEdinburgh University
June 6, 2006
1This document is a slightly revised and reformatted version of the thesis submittedfor the Master of Science degree in Informatics at Edinburgh University. Apart fromminor edits to the text, some figures and references have been reordered and renumbered.All results are the same as in the original thesis.
Abstract
The McCollough effect, an orientation–contingent color aftereffect, is a useful toolfor studying the functionality of the visual system. There have been several at-tempts to model and understand the neural mechanisms underlying it, but none ofthem give a satisfactory functional explanation of the effect. Moreover, numerousexperiments seem to indicate that this aftereffect involves the primary visual cortex(V1), and there have been no reported efforts to simulate it with a computationalmodel of this cortical area. The present document constitutes the first compu-tational study of the McCollough effect with a self–organizing map model of V1.LISSOM (Laterally Interconnected Synergetically Self–Organizing Map) is a bio-logically plausible, self–organizing map model of V1. Fundamental assumptions ofthis model are that lateral connections between cortical neurons play an importantrole in cortical processing, and that it self–organizes through Hebbian learning. Ithas already successfully accounted for important anatomical and functional fea-tures of V1, as well as for the tilt aftereffect, another visual effect. It has beenrecently used to model the development of color and orientation maps at the V1level and is therefore appropriate for modeling the McCollough effect. This thesispresents a simulation of the effect with LISSOM, and the first rigorous compari-son of the simulated effect with available psychophysical data on the McCollougheffect. The model suggests that Hebbian learning of lateral inhibitory connectionsbetween orientation and color selective cells in V1 is the neural mechanism un-derlying the effect, thus lending strong support to the theory that the processingmechanisms involved in the ME are mostly located in the primary visual cortex.Moreover, the structure of the orientation and color map in V1, as well as theinteractions between them, are also examined further, and it will be shown thatthe model accounts for both structural self–organization in color and orientationmap and functional characteristics of adult vision. Thus, this thesis constitutesthe first psychophysically and biologically grounded simulation of the ME, andthe first to simulate complete topographic maps like those in the visual cortex.Importantly, the model was not originally developed with the ME in mind, whichstrongly suggests that the effect is a byproduct of the general cortical processingmechanisms in the model. The model also shows for the first time how the short-term adaptation in the ME is related to the long-term developmental processes,providing an important link between development and adult visual function.
Contents
1 Introduction 6
2 Previous Work 102.1 Overview of human color vision . . . . . . . . . . . . . . . . . . . . 10
4.3 Interpretation of the results in terms of neural mechanisms . . . . . 57
2
5 Discussion and Future Work 705.1 Further study of the simulated ME . . . . . . . . . . . . . . . . . . 715.2 Further study of the color and orientation map . . . . . . . . . . . . 74
6 Conclusion 77
3
Declaration
I declare that this thesis was composed by myself, that the work contained hereinis my own except where explicitly stated otherwise in the text, and that this workhas not been submitted for any other degree or professional qualification exceptas specified.
(Julien B. Ciroux)
4
Acknowledgment
I would like to express my sincere gratitude to Dr. James A. Bednar for giving methe opportunity to perform this study and guiding me throughout it. He greatlycontributed to gain insights and pertinence on the outcome of the simulations,and let me benefit from his knowledge and technical expertise when implementingthem.
JULIEN B. CIROUX
5
Chapter 1
Introduction
The McCollough Effect (hereafter ME), is a visual aftereffect discovered in 1965by Celeste McCollough [50]. She demonstrated in her celebrated paper that bypairing a simple grating with color, it was possible to produce a negative coloraftereffect contingent on the orientation of the grating. The effect was producedby exposing subjects to vertical black stripes on an orange background, whichalternated every few seconds with horizontal stripes on a blue background. Aftera few minutes of induction, subsequently viewed black and white (achromatic)test gratings will be perceived as desaturated blue when vertical and desaturatedorange when horizontal. Thus, the desaturated colors seen in the white sectionof the test patterns are approximately complementary to the color in which thepattern was presented during induction. Furthermore, the colors would exchangeplace on the gratings if the test stimulus, or the subject’s head, is rotated 90◦,disapearing altogether at approximately 45◦. A variety of complementary hueshave been subsequently used for inducing the ME [65, 66]. In particular, greenand magenta have been proven to generate strong aftereffects, and are thereforetraditionally used in ME experiments [65]. Figure 1.1 demonstrates the effect usingthese two colors.
McCollough’s experimental setup aimed to show that a color aftereffect canbe made dependent on the orientation of lines in the adapting stimulus, and theME has therefore been characterized as an orientation–contingent color aftereffect.Numerous visual effects have been reported on various visual stimuli dimensions,such as the tilt aftereffect involving orientation, or the motion direction afteref-fect involving direction of motion. Besides being a visual amusement, these visualaftereffects are useful psychophysical tools for probing the neural mechanisms un-derlying phenomenal visual experience, and for developing functional theories ofhuman visual processing. The ME is of particular interest because it involves twovisual stimuli dimensions, color and orientation, and therefore reveals interactingproperties between orientation coding and color perception.
6
Figure 1.1: The McCollough Effect. To experience the effect, first look at theblack and white pattern to check that you do not see any colors. Then gaze atthe two colored patterns for a few minutes, switching back and forth now andthen without staring at a particular point. Note that it really works better ifyou do it for at least 5 minutes. Return at the black and white pattern. Youshould see the horizontal pattern ‘greenish’ and the vertical one ‘pinkish’. Now,if you rotate the sheet by 45◦, it will look white again; by 90◦ the colors willexchange places on the gratings. It is also interesting to try to see it again inone hour, so as to experience the duration of the effect. (Figure reprinted fromhttp://research.lumeta.com/ches/me/, last visited 29/06/2005).
7
The ME has been the subject of a large body of literature, and several theo-ries and model have been developed in an attempt to account for it. One of themost well grounded theoretical explanations of the ME and other aftereffects isdue to Barlow and Foldiak [8, 9]. They propose that aftereffects are a reflectionof functional processes involved in a continuous calibration of the visual system.They also suggest that the neural mechanisms underlying this calibration are in-hibitory connections between neural units. In the case of the ME, the inductionpattern would increase inhibition between neurons coding for color and orienta-tion, therefore creating a shift in color perception. Another explanation relies onthe discovery of “double–duty” neurons in the monkey visual cortex, tuned to bothcolor and orientation [37, 52]; adaptation of these units could lead to effects like theME. Concerning the processing level involved in the ME, much evidence suggeststhat it arises in the primary visual cortex (V1) [38, 68, 12], but the anatomicallocus of the ME is far from being identified and other authors focus on V4 [55, 41].This thesis will help in clarifying the mechanisms giving rise to the ME, as well aslocating the neural substrate giving rise to the effect.
Although psychophysical experiments have considerably contributed to the un-derstanding of the ME, the problem of studying in detail the neural substrate thatgives rise to the effect in humans has come up against ethical boundaries. Dur-ing the last two decades, it has become possible to use computational simulationfor modeling neurons and help gaining insight in various cognitive mechanisms,such as visual perception and aftereffects. Particularly, LISSOM (Laterally In-terconnected Synergetically Self–Organizing Map) is a biologically plausible, self–organizing map model of V1. Fundamental assumptions of this model are thatlateral connections between cortical neurons play an important role in corticalprocessing, and that V1 self–organizes through Hebbian learning. It has alreadysuccessfully accounted for important anatomical and functional features of V1such as the input–driven development of topographic, ocular dominance, orien-tation and motion direction maps, as well as the development of patchy lateralconnectivity [16, 53, 63]. It has also been used to perform a successful computa-tional study of the tilt aftereffect [15, 13], and so has demonstrated its capacityto probe adult visual perception. The model has recently been extended furtherto include dichromatic (red and green) color processing in the retina and LGN,and has been shown to develop “blobs” of laterally connected color selective cellswithin the orientation map [17]. The observed blobs and receptive fields are similarto those found experimentally in the cortex [46]. Therefore, this extended modelis an appropriate model to perform a computational simulation of the ME sinceit provides the required interaction between color and orientation coding and is amap model of V1 that have already been shown to match experimental data well.
The present work will focus on the study of the ME within the LISSOM model.
8
The results suggests that a model of V1 is able to exhibit the effect and accountfor some of its fundamental properties, and gives a functional interpretation of theeffect based on inhibitory lateral interactions between color selective and orienta-tion selective cells. Performing the study has also helped examining in more detailthe structure and organization of color and orientation map in LISSOM, whileexploring further the model’s ability to simulate adult visual function. The resultsfrom this work support the anatomical and functional predictions described above.
The thesis is organized as follows:
• Chapter 2 starts with a quick review on the visual pathway and the mecha-nisms of color vision; then, the psychophysical properties and experimentaldata on the ME will be presented, followed by a discussion on the processinglevel involved in the effect and the proposed theories that have been devel-oped. Then, previous attempts to model the ME with neural networks willbe outlined.
• Chapter 3 presents previous results obtained with LISSOM and the archi-tecture of the model. The process leading to the self–organization in colorand orientation map will then be examined, as well as the structural andfunctional properties of the map.
• Chapter 4 reports the results obtained when simulating the ME in the modeland compare them with the psychophysical data introduced in Chapter 2.The neural mechanisms responsible for the effect are then studied in moredetail.
• Chapter 5 is a discussion about the outcome of this thesis and future stud-ies that will help to clarify the ME and the organization of the color andorientation maps in V1.
• Chapter 6 concludes the thesis.
9
Chapter 2
Previous Work
This chapter constitutes a review of the theoretical and experimental results thatare relevant for the results presented in this thesis. It starts with a quick reviewof the anatomy and physiology of the human visual pathway and of the neuralprocessing involved in color vision. Then, the fundamental psychophysical char-acteristics of the ME will be outlined, as well as the discussion on the anatomicallocus of the effect. Finally, proposed theories and previous attempts to simulatethe ME with neural networks will be critically reviewed.
2.1 Overview of human color vision
The human visual system and the mechanisms of vision are the result of a longevolution, and provide us with an efficient and reliable representation of the worldaround us. Particularly, color perception is a remarkable characteristic of consciousvisual experience that allows the brain to acquire knowledge instantaneously abouta constant physical property of objects and surfaces, namely their reflectance forlight of different wavelengths. This section will briefly outline the anatomy of theearly visual pathway, as well as the neural processes involved in color vision.
2.1.1 Visual pathway
Phenomenal visual experience is primarily elicited by the neuronal processing ofthe light entering the eye and hitting the retina. The retina is a layer of specializedcells, called photoreceptors because they absorb photons and encode light levels aselectric signals. There are two types of such photoreceptors: rods and cones. Rodsare very light sensitive and respond to a wide range of wavelength. Therefore, theyare particularly useful at low light conditions, and are believed to have evolved fornight vision; they are incapable of wavelength discrimination and thus are not in-
10
Figure 2.1: The normalized peak spectral sensitivity of the three cone typesin most primate retina. There is overlap in the absorption spectrum,and perception of intermediate colors results from a simultaneous activa-tion from more than one type of cone receptor. (Figure reprinted fromhttp://www.chm.bris.ac.uk/webprojects2003/white/excitation of cones.htm, lastvisited 23/08/05).
volved in color vision. Cones are less light sensitive and adapt over a wider rangeof light intensities than rods. Humans have three populations of cones with dif-fering wavelength sensitivity: long–wavelength (L or “red”), medium–wavelength(M or “green”), and short–wavelength sensitive (S or “blue”) cone photoreceptors.Figure 2.1 shows the spectral sensitivity of the three cone types. Cones evolved forvision in day light and form the basis of color vision. The color signature of anypoint in a scene is represented in the nervous system by three numbers that specifythe photon absorbtions in each of the three classes of cones [47]. This is knownas trichromacy, and is a peculiarity of most diurnal primates, including humans;most mammals are in fact dichromats [61].
Photoreceptors are then connected to the ganglion cells that form the opticnerve and constitute the second stage of visual processing. The neural signal isthen carried through the lateral geniculate nucleus (LGN) of the thalamus, locatedat each side of the brain, to the primary visual cortex (commonly referred to asV1), which is the first cortical area where visual information is processed. V1 isat the rear of the brain and contains prominent stripes of white matter consistingof the myelinated axons of the LGN neurons. For this reason, V1 is also calledstriate cortex, as opposed to higher cortical areas of the visual system referred to
11
Figure 2.2: The visual pathway from the retina to V1 (reprinted fromhttp://www.fz-juelich.de/ibi/ibi-1/Color vision, last visited 23/08/05).
as extrastriate or peristriate areas. Figure 2.2 shows the anatomical location ofthe visual pathway from the retina to V1. The visual cortex is generally brokendown into at least five main functional levels from V1 through V5, constitutingphysically separated cortical areas. From V1, the visual information is fed throughthe remaining levels for various stages of processing that contribute in visual per-ception in ways that are still unclear. For the purpose of the present work, we willonly concern ourselves with the first area V1, but also discuss briefly the role ofareas V4 and V2 that have both been argued to be central in the perception ofcolor [11, 44, 73].
At each stage of the visual pathway, neurons respond optimally to different setsof spatial features at particular location, called their receptive fields. Thus, neuronsin the optic nerve and in the LGN respond best to a circular light or dark spot in acertain location in the visual field, whereas neurons in V1 have more complicatedreceptive fields. For instance, most neurons in V1 respond strongly to an orientedstimulus such as an edge or a line. Other characteristics are sometimes included inthe definition of the receptive field: some neurons are more sensitive to chromaticstimuli, others activate more strongly when viewing moving stimuli. Some neuronsalso exhibit monocular properties (they respond to stimuli in only one eye), whileothers are binocular (they respond to stimuli in both eyes). However, the receptivefield primarily stands for a spatial area in the visual field that most activates theneuron.
12
2.1.2 Retinal circuitry and channels of the LGN
The mechanisms that underly the early stage of color vision take place in the retinalcircuitry and the LGN, and are relatively well understood. It has been recognizedfor over a century (due to Hering in the mid 1800s) that the mechanisms of colorvision are organized in three pairs of polar opposite: light–dark, red–green andyellow–blue [47]. Although the principles of this opponent organization are wellestablished within the neuroscience community, its neural basis remains elusive andcontroversial. The output of the three cone color mechanisms is transformed intoan achromatic channel (also referred to as a luminance channel) and two chromaticchannels, so that the transmission of the color information is nearly optimal andthe additive effect of noise is minimized [49]. Indeed, it has been shown thatopponent–color coding can be derived by a principal–components transformationwhich orthogonalizes the initial three retinal color mechanisms, such that theiroutputs are decorrelated and the color information is maximally compressed [23].
Building of the three parallel opponent–channels starts with the ganglion cells.The receptive fields of these ganglion cells have been shown to be organized witha center and an antagonistic surround, for both luminance contrast and for colorcontrast of types yellow–blue or red–green. Figure 2.3 shows the center–surroundorganization of luminance cells, but it also illustrates the mechanisms underlyingred–green (or blue–yellow) opponent processing. It is important to note here thatthere are no blue cones in the fovea, and therefore that foveal color vision relies onlyon the red–green channel [61]. It is not yet clear what specific connection patternsare responsible for the opponent processing; for instance some authors have arguedthat surrounds are not cone specific but would instead contain a mixture of redand green cones [61]. For simplicity, we model surrounds as containing a singletype of cone, but we do not expect the results to change significantly if the model’ssurrounds were to contain both cone types.
Neurons in the LGN are believed to exhibit the same center–surround receptivefield configuration. More precisely, the LGN seems to be organized in two parallelchannels referred to as the ON and OFF channels. In the ON channel, receptivefields of neurons have an excitatory center and an inhibitory surround, whereasin the OFF–channel they have an inhibitory center and an excitatory surround.Therefore a neuron of the ON–luminance channel respond optimally to a whitespot on a dark background, while a neuron in the OFF–luminance channel respondoptimally to a dark spot on a white surround. In the chromatic pathway, red–ONand green–OFF cells respond optimally to a red spot on a green background,whereas green–ON and red–OFF cells respond optimally to a green spot on a redbackground.
13
Figure 2.3: Center–Surround organization of luminance ganglion cells. Luminancecontrast signal is a result of the center–surround organization of ganglion and LGNcells. The same processing is used to produce red–green opponent signals with red(or green) replacing light and green (or red) replacing dark. (Figure reprinted from[61]).
2.1.3 Organization of V1
The anatomical structure of V1 is well studied, as are the receptive field proper-ties of individual neurons in V1. Particularly, it is well established that there areneurons in V1 that are selective to orientation of luminance edges, motion direc-tion, and color [57, 53, 29]. It is also known that neurons in vertical columns ofV1 have similar receptive fields and feature preferences. The feature preferencesgradually vary across the surface of the primary visual cortex in characteristicspatial patterns referred to as cortical feature maps. Several authors have arguedthat the primary visual cortex has such a structure because it captures the cor-relational structure of the visual environment during developmental input–drivenself–organization [58, 53]. Thus, V1’s representation of a visual scene is consti-tuted of basic features such as edges, colors, and motion directions. Subsequentlevels of processing use these basic features of the correlational structure to detectmore complicated features in the visual scene.
The structure and organization of the orientation map is relatively well stud-ied and understood. Neurons in these maps are connected intracortically throughspecific long–range lateral connections that have been found to link cells with sim-ilar orientation preference [18]. This is believed to allow suppressing redundancyin the input and improves the cell’s ability to detect changes in stimuli [63, 15].
14
Simulations with LISSOM also suggest that these long–range lateral connections,particularly inhibitory ones, play a central role in the emergence of the tilt after-effect [15].
Less effort has been put in the study of the organization of the color selec-tive cells. Discovery of cytochrome oxidase–rich blobs has greatly enhanced ourunderstanding of color representation in V1, since they have been shown to con-tain neurons having monocular, color–selective and unoriented response properties[36]. Recent experimental findings have shown for the first time how color–selectivecells organized at the map level, and how they relate to ocular dominance mapand orientation selective cells [46]. These studies have also refined the relationshipbetween CO blobs and color maps, by showing that color selective cells are notnecessarily found within CO blobs, but these results are still very controversial.The receptive field properties of color selective cells in V1 have also been studied.In particular, in addition to simple color–opponent selective cells, neurons that aretuned both to color and orientation have been found in the monkey striate cortex[37, 52, 45].
It is not yet known how this organization is constructed during development(i.e. whether it is environment driven or genetically specified), and computationalmodels have been built in an attempt to account for it. It has been recentlydemonstrated that such color blob organization can develop through activity–dependent self–organization [17, 56]. Particularly, simulation with LISSOM hasdemonstrated how “blobs” of color–selective cells can develop within unselectiveareas of the orientation map. LISSOM also accounts for the presence of “double–duty” units tuned for both color and orientation, located on the borders betweencolor “blobs” and highly orientation selective areas of the map. Furthermore,LISSOM suggests that color blobs found in V1 connect laterally to other colorselective areas of V1 [17]. By studying the ME within the LISSOM model, therole of these connections in the development of color selectivity and in adult colorperception will be explored further, as well as the interacting properties betweencolor and orientation maps in V1.
2.2 Previous work on the McCollough Effect
In this section, psychophysical findings on the ME will be reviewed, and the exper-imental data that will be used to assess the simulation will be examined in moredetail. In light of these experimental results, the neural substrate and mechanismsinvolved in the ME will be discussed, and previous attempts to model the MEwith neural networks will be outlined.
15
2.2.1 Psychophysical properties of the ME
The ME is one of the most studied contingent aftereffects, under the assumptionthat it may provide insight into the neural processes underlying everyday colorperception. Therefore, many psychophysical experiments have been performed onthe ME, revealing many properties of the effect. In a first part, the principal andmost studied features of the ME will be reviewed, followed by a quick review ofmore complicated and controversial properties.
The ME is of special interest because it presents a number of characteristicsthat distinguish it from other classical aftereffects such as the tilt aftereffect orthe motion aftereffect. Particularly, the ME associates two visual dimensions: ori-entation and color. Other contingent aftereffects can be established between anumber of visual dimensions, such as texture density and orientation or texturedensity and color of surround [27]. In particular, color aftereffects have also beenproduced by adaptation to colors paired with moving stimuli, and by adapta-tion to colors paired with gratings of different spatial frequency [67, 20, 69, 65].These color aftereffects are sometimes referred to as motion–contingent and spatialfrequency–contingent McCollough effect. Although the mechanisms responsible forthese effects are probably similar (i.e. chromatic adaptation of two different pop-ulations of feature–detector neurons), the present work is restrained to the studyof the classical orientation–contingent aftereffect.
One of the most striking aspect of the ME is its persistence over time. Ithas been found that the effect can last for hours, days and weeks depending onthe adaptation duration and stimulation after simulation [43]. In particular, anaftereffect that lasts up to 6 weeks after 20 minutes of adaptation to a red andgreen moving grating has been reported [67], which is longer than any other clas-sical aftereffect’s temporal resistance. Another particularity of the ME is that itdoes not require fixation at a particular point of the induction pattern. However,the color aftereffect has been reported to be greatest when the test grating is inapproximately the same retinal location as the adapting gratings had been [65].
The ME also presents unusual properties concerning its interocular transferand binocular interactions. In her original paper, McCollough reported that theME was monocular; that is, adaptation of only one eye resulted in the effect inthe adapted eye but not in the unadapted eye [50]. Typically, figural and motionaftereffects show substantial interocular transfer and so the ME seems to be anexception. However, subsequent studies have shown that the ME reveals binocularinteractions, although interocular transfer does not occur under monocular adap-tation. When pure orientational information and pure color information is givento each eyes separately (i.e. adapting one eye with an achromatic grating and theother with an homogeneous field), the ME is nevertheless induced [48]. Moreover,testing the achromatically adapted eye leads to an aftereffect referred to as an
16
“anomalous ME” in which the aftereffect exhibits the same hue as the adaptingcolored field, instead of the complementary. Therefore, in this experiment, boththe orientation and color information transferred interocularly. Finally, the ex-tinction of the effect has been shown not to transfer interocularly. That is afterbinocular induction of the ME, one eye is presented with an achromatic gratingof the same orientation and spatial frequency, while nothing is presented to theother. After 20 minutes, the strength of the effect is measured for each eye, anda significantly weaker effect is found only for the eye that was presented with theachromatic grating, meaning that there have been no interocular transfer duringthe monocular deadaptation procedure [62].
A few articles have reported “reverse” McCollough Effects; that is, spatial af-tereffects that are contingent on color [34, 40]. Particularly, Held and Shattuck(1971) reported that the direction of a tilt aftereffect could be made color depen-dent: after subjects have looked at red stripes tilted off clockwise and vertical andgreen stripes tilted equally but counterclockwise, the vertical test stripes appearedto be tilted counterclockwise when red but clockwise when green [34]. However,the magnitude of the effects was much smaller than that usually found in tilt af-tereffect experiments. In the same way, color–contingent motion aftereffects havebeen reported [30]. After viewing a red contracting spiral alternating with a greenexpanding spiral for a few minutes, a red stationary spiral appears to be expand-ing, while a green stationary spiral appears to be contracting. This reciprocity ofeffect between dimension also happens with classical ME’s induction procedure,giving rise to a color–contingent pattern aftereffect. That is, presentation of anhomogenous chromatic field produced a faint image of the patterns that have beenpaired with this color [40]. However, this effect is very weak, and is thereforedifficult to measure reliably.
In fact, it is not necessary to use two chromatic gratings for the induction oforientation–contingent color aftereffects [5, 66]. Following repeated presentationof a single chromatic grating (e.g red horizontal), an achromatic horizontal gratingappears “greenish”. This seems obvious when thinking of the ME as a chromaticadaptation of orientational units. Nevertheless, a more surprising effect also oc-curs following the induction with a single grating: an illusory color is viewed onthe orthogonal grating even when this orthogonal grating has not been presentedduring induction. This aftereffect elicited by the non-induced pattern has beenlabeled the Indirect McCollough Effect (IME, [26]). Despite being a very interest-ing phenomenon, there have been few reports of the Indirect ME, and there areinconsistencies among the results of the few published studies. However, it seemswell established that a complementary illusory color is seen on the non-inducedorthogonal grating following adaptation on a single chromatic grating [2, 5, 66].Furthermore, the study of the ME in the case of a single adapting grating gives
17
rise to the question whether the effects produced when two gratings are alternatedduring induction are independent from each other. Thus, it is of special interestto study this phenomenon in the model, so as to shed light on these unansweredquestions.
The ME also exhibits more complicated properties, that are sometimes alsomore controversial. For instance, the ME has been produced with a variety ofpatterns others than gratings: spokes and concentric circles, or clockwise andcounterclockwise spirals have been used to induce MEs. More generally, it has beenfound that two geometric figures could be used for inducing the ME if they are bothlocally orthogonal. These types of figures are derived from previous applications ofthe mathematics of continuous transformation (called Lie transformation groups)to the stimulus dynamic of perceptual invariance. For an introduction on thisaspect of the ME, see [21, 40].
It has also been suggested by some researchers that there are some values ofspatial parameters that give rise to optimal MEs. For instance, the ME would bestronger for a given spatial frequency, or for a given ratio between the black bars’width and the light bars’ width on the gratings [69, 65]. Nonetheless, it seemsunlikely that these parameters would be equal from an individual to another, andmore studies are needed to confirm these hypotheses. Some studies have focusedon how the effect varies with hues chosen from different locations in a color space[35, 66], but it has not yet been possible to measure enough points in this spaceto establish clearly how the effect varies for different adaptation colors. However,these studies do suggest that the effect involves separate detectors for color andorientation [35], which is compatible with the results that will be presented forLISSOM.
2.2.2 Experimental data
This section will present in detail the experimental data that will be used toassess the results from the ME simulations. It first starts with a review of thetechnique and method used for measuring the ME in humans, and then examinesexperimental data that have been reported in two psychophysical papers on theME. These data will be used for comparing the simulation in the model with theeffect experienced by humans.
Measuring the ME in human
A major problem facing researchers who investigate the ME is one of finding asatisfactory independent variable. Although it is relatively easy to demonstratethe effect, it is rather more difficult to quantify it adequately. In many publishedreports, the data are simply whether or not subjects give the appropriate verbal
18
response when viewing the achromatic test pattern. Therefore, the results of thesestudies cannot be efficiently and reliably compared to data from a computingsimulation of the effect.
Some studies, however, used a measurement method that gives a more directestimate of the strength of the effect, called the color–cancellation technique, firstintroduced by White in 1976 [72, 59]. It is achieved by varying the colorimetricpurity of the light bars of the test gratings: variable amounts of desaturated lightsthat are complementary to the color of the aftereffect are added to the test patternsuntil the gratings appear achromatic. A special color–mixing projector makes itpossible to produce the relative amount of white, green and red lights so that toobtain an achromatic appearance of the test field. The amount of complementarylight that is required to cancel the effects is then used as an index of the effect’smagnitude [59, 28].
Hue matching has also been used by some experimenters, and is also an effi-cient way of measuring the ME. Whereas the color cancellation technique gives usa measure of the effect’s strength on a saturation scale, use of a projection col-orimeter to match the ME enables to express the strength of the effect in terms ofCIE chromaticity coordinates [65, 35]. However, when the independent variable isnot the induction patterns’ hues (e.g. studying the ME when varying orientationof the gratings), the magnitude of the ME can be reliably expressed on a saturationscale and there is no need to use the CIE coordinates. The present work will focuson results from studies using the color cancellation technique, because no studiesof the ME with varying hues will be performed in the model.
In general, the induction procedure is preceded by a pre-inspection procedureconsisting of a sequence of several null matches on the test patterns. Each time, anachromatic test grating is presented, and the subject has to adjust the objectivecoloration of the test pattern (using the color–cancellation apparatus) to cancelany subjective coloration seen. The results for each test pattern are then averagedto provide a pre-inspection baseline, and the aftereffect strength is measured as adeviation from this baseline. The results of these pre-induction null-matches giverise to the question whether individuals have a perfect color balance among thecells tuned for a given orientation. In a study by Riggs, White, and Eimas (1974),some subjects consistently made pre-inspection settings that were significantlydifferent from baseline, indicating that (for example) vertical gratings were con-sistently seen as more magenta than horizontal gratings [59]. It is conceivable forinstance than someone who happens to spend the formative year of his childhoodwith green vertical striped wallpaper in his room, might have his color equilibriumshifted toward the magenta side for vertical perception [59]. This variation of thecolor equilibrium with the orientation of the test stimulus has also been observedin LISSOM, i.e., the color saturation value computed when presenting an achro-
19
matic grating varies with the orientation, such that the color saturation baselinewas different for each orientation. To reduce the possibility of bias in the model,the strength of the aftereffect will also be measured as a deviation from this base-line, for any given orientation of the test grating as done in the psychophysicalexperiments.
Finally, although the ME can be induced without fixation, the subject’s headis generally stabilized by a chinrest and a head restraint, allowing him to view thegratings without much variation of the retinal coordinates of the image. Some-times, an artificial pupil is also used to monocularly view all stimuli [28]. Moreover,the gratings are generally projected on a relatively low degree circular screen (i.e.around 9◦), minimizing the variation of the retinal coordinates of the gratings.The assumption that the fluctuation of the retinal coordinates of induction andtest gratings is minimized will be used, when appropriate, for simulating ME ex-periments in the model.
The ME as a function of the angle between test and adapting gratings
The fact that the ME is dependent on the orientation of the test grating is one ofits defining characteristics. With red–horizontal and green–vertical induction pat-terns, the achromatic horizontal test pattern is subsequently viewed faintly green;then the effect decreases when rotating the test pattern toward 45◦, disappearsat 45◦, moves toward red when the test pattern is rotating toward the vertical,and finally appearing faintly red at the vertical. Early determinations of the ME’stuning on tested orientation were hampered either by the variability or the sub-jectivity of the method that was used for measuring the aftereffect’s strength.Consequently, there is only one study that provides a reasonably objective andreliable measure of the ME as a function of the angle between the axes of the testand the adapting gratings, by Ellis in 1977 [28].
Ellis measured the strength of the ME both as a function of the luminancecontrast of the adapting gratings and as a function of the angle between the axesof the test and adapting gratings. The idea was to combine both functions in orderto express an equivalent contrast transformation which converts the measurementof the orientation tuning into a unit comparable to that used for other kinds oforientation–specific aftereffect. For this work we will focus on the data concerningthe dependence of the ME on the angular difference between test and adaptinggratings, because the current LISSOM model is designed to match only the high–contrast behavior of visual cortex neurons [53]. Figure 2.4 shows the curve obtainedby Ellis for one individual. More sample curves have been reported, but they arenot as complete and will be shown later when comparing to the results obtainedwith the model.
As described above, the results in the study are reported as sample curves for
20
Figure 2.4: The ME as a function of the angle between test and adapting gratings.This is a representative curve obtained for an individual in Ellis (1977). Samplecurves are preferred to curves averaged over different individuals, because of thefluctuation of the effect between individuals. This curve is an average over 25experiments with the same subject. The procedure is the following. First, the MEis induced for a magenta–horizontal and green–vertical grating, or the reversedsituation. Then, a test grating (as shown in the picture) is presented. Specialapparatus allows adding magenta at the top and the same amount of green atthe bottom, depending on the situation. The test grating can be presented with0◦, 5◦, 10◦, 15◦, 30◦, and 45◦ angular difference with the adapting grating, clock-wise or counterclockwise. Any time the angular difference is changed, the wholeinduction procedure is performed again. Strength of the effect is measured asa deviation from a pre-induction setting, and is expressed in colorimetric purity.(Figure reprinted from [28]).
21
one individual, which is generally the case in reports about the ME. Results varyconsiderably between different individuals, although it is possible to reveal similar-ities between them (i.e., common properties of the functions; such as linearity andcommon regression). Experimental findings also suggest that a given individualcan show weak, intermediate or strong aftereffects and that the results are usuallyconsistent [59]. Consequently, it does not seem to be appropriate to average theresults over all tested individuals, and experimenters generally choose to displaysample curves. Sample curves for a given ME experiment with LISSOM have beenused for comparison with human data.
Effect is independent of absolute orientation but decreases for smallangular divergence.
The ME can be induced with a single induction pattern, whose orientation canbe varied without changing the effect [64]. Similarly, the effect is independent oforientation if two adapting gratings are used (for instance, one green and one red),as long as they are kept at 90◦ angular divergence [31]. It is believed that in thiscase the effect is stronger because the effects for each of the complementary huesadd up [66, 2, 5].
However, the effect varies systematically as the angular divergence between theinduction patterns is varied. For instance, with a zero angular divergence betweenred and green induction patterns, no effect is seen, presumably because the twoaftereffects cancel each other [31]. Assuming that the ME is due to the adaptationof orientation tuned cells, Fidell (1970) hypothesized that adapting patterns ofcomplementary hues but similar orientations should excite the same population ofcortical neurons, and that the net effect on each neuron would be zero.
Fidell tested this hypothesis (1970), but unfortunately the reported data arefar from being complete. Nevertheless, he reached some interesting conclusionsthat can be assessed further within LISSOM. In his report, the orientation inde-pendence of the ME is tested by performing experiments with adapting gratingsat four different orientations, with a constant 90◦ angular divergence between bothgratings. The produced effects are reported not to show any significant differencein strength, as expected. Then, three more induction processes are studied: one ofthe gratings is kept at the vertical, and the second one is set with 45◦, 22◦, and 11◦
angular divergence with it. By doing so, Fidell aimed to determine the minimumangular divergence between adapting gratings necessary to produce a ME. He ob-served that the effect seems to show significant decrease at 22◦, and breakdown at11◦. Nevertheless, although this conclusion seems to be verified when people areasked to characterize the effect, the data that are available suffer from a lack ofdata points and a poor statistical assessment. Moreover, the strength of the effectis reported as the number of dial settings used to cancel the effect, rather than in
22
colorimetric purity. To avoid confusion, the original plot from this study is notreported in this thesis, but the data points will be re-plotted for comparison whenpresenting the results of the ME simulation (figure 4.6).
It is easy to reproduce Fidell’s experiment with LISSOM, using more datapoints so that to obtain a more continuous curve, and testing in many differentconditions so that to reach a better statistical significance. This allows producinga more detailed curve of the strength of the ME as a function of the angulardivergence between induction gratings, and to locate more precisely at which anglethis decrease starts to take place.
For comparison with the model, it is possible to make a rough prediction fromanimal data about where this decrease should take place. The rate of firing ofmost orientation–specific neurons in cat and monkey V1 has been estimated to fallto half of peak value for stimuli 14◦ to 26◦ from the neuron’s preferred orientation[24]. If the ME arises through adaptation of orientation–tuned neurons, adaptationpatterns differing by more than 26◦ would be expected to activate disjoint sets ofneurons. Thus we should not expect the effect to break down until the angulardivergence was less than about 26◦ in cat and monkey. Neurons in the modelhave an orientation selectivity half-width of about 30◦ , and the model does notshow any significant breakdown of the effect until about 25◦ of angular divergence,which is consistent with this estimate from animal data.
2.2.3 Processing level involved in the ME
The ME indicates that color and orientation interact at some point during visualprocessing; but the question remains as to whether this interaction occurs at anearly or later stage in the cortical visual pathway. Several hypotheses have beenproposed concerning the processing level involved in the ME, the earliest and mostcommon one being that it arises in the striate cortex (V1). However, more recentstudies suggest that neural processing in V1 alone may not be sufficient to explainentirely the ME. Evidences and experiments supporting both hypotheses will bereviewed in this section, and it will be seen that the hypothesis of an early locationin the visual system cannot be ruled out by the new discoveries.
The hypothesis of an early visual mechanism (V1)
Considerable evidence suggests that many aftereffects reflect changes in mecha-nisms at an early or low level in the primary visual pathway [15, 38]. It is com-monly accepted that structures up to and including V1 are low level, and mostresearchers agree that the ME is a low–level phenomenon in this anatomical orphysiological sense.
23
When the effect was first introduced by McCollough, it was proposed thatthe early visual area V1 played a central role [50]. The hallmark of the effect isits dependence on contour orientation, but this orientation contingency dependson the orientation with respect to the retina, because there is no correction forhead-tilt [12]. Neurons in V1 are the first to show orientation selectivity, thusthe effect likely involves cortical structures. But, because there is no perceptualcorrection for head tilts, it is likely that the effect is subserved by orientationsensitive mechanisms in V1. Indeed, most neurons in V1 appear to be tuned tooriented contours in retinal rather than real world coordinates [38].
Furthermore, it has been well established that the ME depends upon the wave-length of light coming from the gratings, rather than the perceived color [68, 71].Again, some researchers suggest that color sensitive cells in V1 respond to thewavelength characteristics of a stimulus, while cells further along in the visualsystem, such as many of those in V4 or V2, respond to more complex aspects ofa color display and appear to be associated with color constancy 1 [38, 11, 73].Hence, the fact that the effect seems to depend on wavelength characteristics ofthe input suggests that the adaptation involves early color coding mechanisms.
The study of individuals who have sustained damage to a particular visualpathway gives another indication that the ME involves area V1. For example,patients suffering from a profound visual agnosia experience the ME [38]. The el-ementary visual functions and general cognitive ability of these patients are quiteintact, but visual form discrimination and recognition are dramatically impaired,due to a diffuse damage in the peristriate area of the visual cortex with relativelyless damage in V1. In the same way, a patient with cortical blindness who had aprofound impairment in form perception and was unable to use orientation infor-mation for grasping or reaching an object, but could discriminate different colors,experienced the ME [39]. Given the massive injury to the extrastriate cortex inthis patient, it is likely that the anatomical loci of the mechanisms underlyingthe ME are within the primary visual cortex or even earlier in the visual path-way. These cases also demonstrate that orientation selectivity is still operating inthese individuals, but at a level that is inaccessible to conscious perception [38].Therefore, the aftereffect seems to be induced without conscious perception of theinduction patterns, which seems to agree with the hypothesis that the activity inthe primary visual cortex is not directly accessible to consciousness [25].
Controversy on the role of V1
Although the mechanisms in V1 seem necessary for the perception of the ME,several results appear to show that they are not sufficient.
1the perception of stable and uniform object colors despite variable lighting conditions
24
To help localize the cortical visual structures that may be directly involved inthe ME, recent functional magnetic resonance imaging (fMRI) studies have beenconducted. The results show that different portions of human V4 presented signif-icant activation during different stages of color stimuli processing and perception.Particularly, activity in the posterior LV4 (L=left) appeared to be related to theprocessing of chromatic simulation and the induction of the ME, while activity inthe anterior LV4 appeared to be related to conscious color perception [55]. Thisdissociation between the induction of the ME and the perception of the illusorycolor suggests that separate neural substrates underlie these processes. These re-sults seem confirmed by another fMRI study runs by Humphrey and his colleagues(1999), which aims to locate the region responsible for ME adaptation. All of thesubjects except one showed significant activity in area V4 while not in area V1 [41].Moreover, two neural models presenting interesting results have been developed,linking the ME to broader visual processing which are believed to occur in V4 (seesection 2.2.5).
However, one cannot conclude from these studies that the ME is not linkedwith neural activity in V1. The fMRI technique does not show the actual activitylevel of each part of the brain. It only detects areas whose activity is significantlydifferent between a tested task and a similar control task, at a scale of the orderof at least one cubic-millimeter. As will be seen in the model, the effect of the MEoccurs at a sub-millimeter scale, and those would not result in changes that couldbe measured using fMRI.
Therefore, although the ME is correlated with activity in V4, this correlationdoes not necessary imply that neurons in V1 are not involved. Subtle changes inthe profile of the activity of the neural elements coding for color and orientationin V1 could actually give rise to the ME, which becomes detectable only at theV4 level for current fMRI technology.
2.2.4 Theoretical account of the ME
In spite of all the experimental data that have been collected about the ME, thenature of the mechanisms underlying it are far from being clear. Among the theo-ries that have been proposed, Barlow and Foldiak’s seems to be the most plausible.This section will present the different theories that have been developed since thediscovery of the ME, and will explain why the theory involving adaptation throughinhibitory interactions is the most attractive. The problem of exactly knowingwhich neural units are responsible for this phenomenon will also be emphasized.
25
Edge–Detector theory
McCollough’s original intention when she discovered the effect was to link it tothe aftereffects stemming from adaptation to chromatic fringes produced by pris-matic spectacles. The dispersion of light passing through a base–left wedge prismproduces bluish fringes on white–black edges of objects (i.e., luminance edges)and yellowish fringes on black–white edges. When one wears such a prism, thesefringes are visible on first looking but tend to disappear when it is worn for any ex-tended period of time. When the prisms are removed, complementary contingentaftereffect (commonly called “phantom fringes”) appear at luminance edges (i.e.yellowish fringes on white–black edges and blueish fringes on black–white edges).To interpret it, McCollough assumed the existence of orientation–specific edge–detector mechanisms in the visual system that are subject to color adptation.She then explained the aftereffect by the fact that such detectors respond withdecreased sensitity to the wavelengths with which they have been most stronglystimulated [20]. However, she did not clarify the functional mechanisms underlyingthis adaptation.
Neural Fatigue
The first functional explanation of the ME relies on neural fatigue. This theoryrefers to the general fact that repeated stimulation with a pattern appears tofatigue neural mechanisms that encode the pattern. Thus, extending the pointof view of McCollough, it was proposed that the ME arises from the fatigue ofneurons that are tuned to both orientation and color, after such neurons werediscovered in V1 [37, 52]. However, numerous researchers have argued that modelsbased on simple fatigue alone cannot provide an explanation for the ME [43, 59,64]. In particular, this model cannot account for one of the major functionalcharacteristics of the ME: its duration. Indeed, the recovery time of simple neuralprocesses is far from being long enough to explain the long-term persistence of theeffect. Therefore, the neural fatigue model is unlikely to be correct.
Associative model
The ME is viewed by some researchers to be an instance of learning. The earli-est explanation in that class appealed to Pavlovian conditioning: the process ofbehavior modification by which a subject comes to respond in a desired mannerto a previously neutral stimulus that has been repeatedly presented along with anunconditioned stimulus that elicited the response [1]. According to this theoreti-cal framework, the color would be an unconditioned stimulus while the orientationgrating is a conditioned stimulus. After some pairings, the conditioned stimulusalone (achromatic grating) elicits the color aftereffect as a conditioned response
26
[3, 4]. Nevertheless, it appears that the terms and concepts of the conditioning ac-count are too flexible to propose crucial experiments that could potentially validateor discount it.
However, classical conditioning is not the only type of associative model thathave been proposed for the ME. Notably, Barlow (1990) has proposed that corti-cal neurons coding for oriented luminance contrast and color contrast may inhibitone another respectively so as to “decorrelate” persistently associated stimulusproperties. On this account, the decorrelation process eliminates redundant stim-ulus information in the cortex [9, 10, 70]. The ME could therefore be explainedas follows. During the induction phase, when a horizontal black and red gratingis shown, neurons coding redness and neurons coding horizontal orientation areactivated simultaneously, and the strength of the mutual inhibition between theseneurons increases. Thus, during the test phase, the neurons tuned to horizontalinhibit the redness neurons, and the lack of redness biases the output of the colorsystem towards green. According to Barlow, this would be a specific case of a moregeneral principle called the “law of repulsion”. It states that when two stimuli fre-quently occur together, their representation in the brain repel each other, meaningthat the representations inhibit each other so that each is weaker than it is wheneach stimulus is presented alone. It is supposed that the degree of repulsion isconstantly modifiable (following an “anti–Hebbian” rule) and therefore representsthe average strength of the association between the two stimuli over some periodin the past [10]. This theory has been shown to give good results in the modellingof the self–organizing neurons in V1, providing an interesting explanation of thetilt aftereffect [15]. The present thesis will also find results that agree with thistheory, adding more evidence for the importance of lateral inhibitory interactionsin the emergence of aftereffects.
Dodwell and Humphrey (1990) proposed a variant model of the ME close toBarlow’s. They assumed that the ME was a manifestation of error–correctionmechanisms that function to learn and maintain long-term statistical correlationsbetween object form and color in the face of short-term violations of these long-term correlations [26, 70]. In perceptual environments, correlations between colorand orientation are usually zero. In the ME, color and orientation are highlycorrelated. The model assumed that the ME is generated because the zero cor-relation between color and orientation that exists in the long run is violated. Tomaintain the internal representation of the long run correlation between the twodimensions to zero, the visual system recalibrates by decorrelating color and ori-entation. While this theory is conceptually appealing, it fails to specify the neuralmechanisms that implement such an error–correction mechanisms, and is thus notdirectly testable.
Overall, Barlow’s theory is functionally appealing and can be related to the
27
neural mechanism implemented by lateral interactions. Nevertheless, the questionremains whether it is the lateral connectivity between orientation and color units,or instead adaptation of single doubly–tuned orientation and color selective units,that is primarily responsible for producing the ME. Later sections will show thatin the LISSOM model, increases in strength between orientation and color unitsare responsible for most of the effect, although neurons tuned to both stimulusdimensions also take part.
2.2.5 Neural network model of the ME
Using a computational model can be very useful to investigate cognitive phenom-ena that are difficult to understand experimentally.Therefore, there have beenseveral attempts to account for the ME using neural networks. However, eitherthey are models developed in order to specifically reproduce the ME, or they aretoo simplistic to be of interest. Two network models will be reviewed in detailbecause they rely on interesting theoretical interpretations of the ME, while theothers will only be briefly presented.
Color at edges and color spreading
Broerse and O’Shea (1999) proposed a complicated theory of the ME which couldbe considered as an extension of the original proposition of McCollough [22]. Theysuggest that subjective colors in MEs consist of two components: edge colors ap-pearing along the edges of contours, and spread colors radiating from edge colorsinto adjacent uncountoured regions of test patterns. They propose that edge colorsarise from a mechanism that normally functions to correct chromatic aberration(differences in convergence of the different colored constituents of white light, whenrefracted through a lens), generated by the optics of the eye. In fact, such a mecha-nism could also account for the aftereffect reported in experimental investigationsrequiring observers to wear optical prisms, as introduced in section 2.2.4. Theappearance of “phantom fringes” after the removal of the prism would reveal theaction of this correction mechanism. According to this error–correcting point ofview, conventional ME induction stimuli would simulate differences in the wave-length composition of chromatic aberration at different edge–orientation, due toslight irregularities in lens curvature. Once generated in the visual system, edgecolors are treated as signifying the presence of real color in a pattern or surface,and may subsequently involve filling-in processes to create what is assumed to bethe characteristic uniformly–colored appearance of conventional ME. They con-jecture that filling-in is mediated by processes similar to those underlying colorspreading from small colored lines in the neon color spreading illusion (see figure2.5) [22]. They also assumed that color spreading reveals a neural filling-in process
28
Figure 2.5: Neon color spreading illusion. This illusion reveals the filling-inphenomenon: when viewing the picture, one should see three blue stripes cre-ated by the alignment of the little blue dashes. Nevertheless, the space be-tween the little blue lines is black on the picture. (Figure reprinted from:http://www.blelb.com/english/blelbspots/spot05/expspot05 en.htm)
operating to achieve color constancy.In other words, they propose that the ME represents a failure of the visual
system to correctly assign surface color on the basis of information at edges. Thatis, the ME represents a departure from veridical color perception that arises fromthe normal perception of filling-in and error–correction under conditions that differgreatly from those experienced during everyday life. Indeed, the persistent pairingof surface color with luminance edges throughout ME induction, and the suddenremoval of induction colors at the test session, represents a set of viewing circum-stances not normally encountered in the natural environment [70]. This raises aninteresting question about the ME: is the illusion simply the expression of such aspecific correcting mechanism, or the expression of a more general learning rulethat produces this outcome in this particular context? In any case, this type oftheory is of particular interest concerning the lack of interocular transfer in theME, because it makes intuitive sense that the compensation processes for chro-matic aberrations is not transferred interocularly because the two eyes differ intheir aberration [33].
Interesting experiments have been undertaken to validate the conjecture thatthe distribution of color in ME test patterns may be due to some form of spreading
29
Figure 2.6: Edge color spreading In the middle column of the top row, fine red lineshave been added to both sides of the dark bars to simulate color fringes. Whenviewed from about three times picture height, these lines are almost invisible, butthe overall appearance of this grating is that it is suffused with a desaturated redcolor, remarkably similar to that experienced with the ME. In the left and rightcolumns of the middle row, fine blue and green lines have been added , respectively.Likewise, the middle column of the bottom row contains yellow colored–lines,imparting a yellowish tint to the whole grating. The top left, centre and bottomright gratings have no added colored lines and appear achromatic. The top rightgratings has red lines on black/white edges and green lines on white/black edgesand the bottom left grating has yellow lines on black/white edges and blue lineson white/black edges. In both cases, these complementary–colored lines appear toneutralize each other. (Figure reprinted from [22])
30
process. Broerse and O’Shea demonstrated that fine colored lines located imme-diately adjacent to the edges of otherwise achromatic square-wave gratings aresufficient to induce MEs comparable in strength to MEs induced with traditionaluniformly colored gratings (see figure 2.6). Furthermore, they confirm the con-jecture that two spatial components are involved in the ME. One component isthe induction of illusory color at the local luminance discontinuities in the testgrating (edge color). The other is the spreading of these induced colors away fromthe edges into uncontoured regions of the surface (spread color). Indeed, theyobserved that ME colors were slightly more saturated near luminance edges thanaway from edges [22].
To test these proposals, a neural network model incorporating many of thereceptive-field profiles of neurons in primate color vision, from those found in theLGN, V1, V2, and V4 was built by Broerse and Vladusich [70]. This model iscomposed of two parallel processing streams. One stream encodes color contrastto facilitate filling-in and color constancy; the other selectively encodes (spuri-ous) color fringes at luminance boundaries, and learns to inihibit the filling-inof these colors within the first stream (error–correction). They show that theirmodel can account for the prism experience aftereffect, which suggests functionallinks between color constancy and the ME. It also accounts for the lack of inte-rocular transfer of the ME, but does not deal with the long-term persistence andthe dimension reciprocity of the effect. They also failed to illustrate the subtlebinocular transfer reported by MacKay and MacKay (1975). The other problemcomes from the fact that they focused on functionalities (i.e. color constancy andfilling-in) rather than on the neural organization of the visual system. This is asignificant liability, because no system specifically processing color fringes has beenexperimentally observed in the cortex.
FACADE theory
An interesting explanation of the ME comes from a larger theory which seeks toaddress broad theoretical issues such as colour constancy, filling in, and 3D vision.Grossberg and his colleagues (2002) have developed a neural network model ofpreattentive Form-And-Colour-And-DEpth vision (FACADE) [33]. A key hypoth-esis of this model is that the ME arises using visual mechanisms whose primaryfunctions are to adaptively align boundary and surface representations which arepositionally shifted with respect to one another. This position shift is due to theprocess of binocular fusion and allelotropia (see figure 2.7), i.e., a displacement inthe boundary system but not in the surface system of the visual cortex.
Thus, the proposed model replaces an orientation and color system by a bound-ary and surface system. Since boundaries are used to form the compartmentswhich surface brightness and color signals fill in, the signals between the boundary
31
Figure 2.7: Grossberg’s model of allelotropia. (a) The stimulus is composed ofthree bars (the three black spots). The two outermost bars fall onto correspondingretinal points, whereas left and right eye locations of the middle one are disparate.So, when fused, the middle bar is seen at a midpoint between the two disparatemonocular positions. (b) The brain needs to map between the binocular boundaryposition of the fused middle bar and disparate position of the two monocular middlebars. This mapping is thought to occur through learning based on their mutualcorrelation during visual development. (Figure reprinted from [33].)
and surface systems must be positionally aligned. The FACADE model predictsthat the signals between the boundary system and the surface system undergoperceptual learning in order to compensate for their mutual displacement. Themain principle of this model is that the brain’s representation of a visual scene isgenerated by interactions between two main subsystems: the Boundary ContourSystem (BCS), and the Feature Contour System (FCS). The BCS forms binocularboundary segmentations that do not carry any visible signals; the FCS fills in visi-ble surface properties at spatial locations whose boundaries are determined by theBCS. Anatomically, the BCS models properties of the interblob cortical stream(i.e., orientation columns), and the FCS models properties of the blob corticalstream (i.e., color columns). The ME is shown to arise as an emergent propertyof such mechanisms.
This model is particulary effective at clarifying why interocular transfer of theME does not occur under monocular adaptation, and succeeds in explaining thelong persistence of the effect. Indeed, this property arises because the model usesa type of habituative synapses. It also provide the only comprehensive account ofbinocular interactions in ME. Another strong advantage of this model is that itsmain concepts and mechanisms were not derived to explain ME data. Rather, they
32
are natural consequences of the FACADE theory prediction that boundaries andsurfaces are computed by parallel cortical processing streams. However, it fails toexplain the reciprocity between dimensions of the ME. That is, the model does notaccount for the color–contingent tilt aftereffect. It is also based on the assumptionthat the brain adopts a boundary and surface model, which, unlike a color andorientation model, has no known physiological basis. Finally, another drawback ofthis model is that it does not use realistic retina stimuli, and it therefore cannotbe tested with realistic natural scenes to see if the ME persists after a realisticdeadaptation procedure.
Other neural networks
The first attempt to model the ME with a neural network is due to Montalvo (1976)[54]. It is an extension of a more general feature–extracting network applied tothe ME, and is of minimal complexity since it uses a set of only 11 parameters.Even if it presents some interesting results, it is too simplified to be a biologicallyplausible account of the ME. In the same way, a recent attempt to model theME with a feature–extracting network is due to Ans (2001) [6]. It is based onsource separation, using Independent Component Analysis (rather than the PCAimplemented by Hebbian learning). It gives an interesting view of the neuralmechanisms that may underly the basic processing of color and orientation, andits approach and network structure are closer to what is used in LISSOM, incomparison to other models of the ME. However, its complexity is minimal, andit is far from being a model of the visual cortex. Consequently, both networksneither give a satisfactory account of the ME nor include it within a more generaltheory of vision. Finally, another neural network was proposed by McLoughlin(1995) [51]. Like the FACADE model, it first aims to account for 3-D perception.Whereas it presents a lot of similarities with the Grossberg’s model, it accountsfor only a few of the properties demonstrated by the FACADE model. Except forthe ones listed above, they have been no previous attempt to account for the MEin a computational way.
Although much is known about the ME, and computational models have beenable to account for some of this data, a complete and convincing model of the MEis not yet available. Firstly, no model has ever achieved a detailed comparisonbetween psychophysical experimental data and the modeled effect. Secondly, noneof them implements a realistic 2D processing based on the known anatomy ofthe cortex. Consequently, all previous models suffer from a lack of physiologicalgrounds. Moreover, as it has been said, much evidence suggest that V1 is theanatomical locus of the ME. Yet, no model of V1 has ever been used to simulatethe effect. Instead models have been especially built for reproducing the ME. TheLISSOM model is therefore the first map model of V1 to be used for studying
33
the ME. Also, the LISSOM model will show how the short-term adaptation in theME is related to long-term developmental processes, providing an important linkbetween development and adult visual function.
34
Chapter 3
The LISSOM model of color andorientation in V1
LISSOM has already proven to be a successful model of V1, both from a structuraland a functional point of view. Previous work and principal assumptions behindthe model will be quickly reviewed, and the architecture of the model for color andorientation maps will be presented. Then, the process that leads to the formationof such a map will be studied, as well as its characteristics and its use for modelingadult visual perception.
3.1 Previous work with LISSOM
The LISSOM model of V1 is a simulated neural network that receives afferentconnections from several sheets of neural units modeling the visual pathway fromthe retinal photoreceptors to V1, and has lateral interconnections within V1. Fun-damental assumptions of this model are that long-range interactions at high con-trasts are inhibitory, while short-range interactions are excitatory, and that theconnections in the model self–organize through Hebbian learning. In this process,afferent connections learn activity correlations that are contained in the input soas to form a map of the input space, and lateral connections learn activity cor-relations between cortical neurons so as to store long-term correlations in V1’sactivity.
The model has been primarily used for simulating the input–driven self–organiz-ation of V1, and has successfully accounted for the formation of topographic,orientation, ocular dominance, and direction of motion maps, as well as the de-velopment of patchy lateral connectivity in the primary visual cortex [15, 63, 53].Particularly, the extensive study of orientation map in the model has led to theobservation that lateral connections link cells with similar orientation preferences,
35
as it has also been found in a study of the shrew striate cortex [18]. This sug-gests a functional role for the lateral connections: learning the correlation betweencortical units during development enables them to later filter visual processing bysuppressing redundancy in visual information, thus providing an efficient sparsedistributed representation of the visual input.
In addition to being a good model of the primary visual cortex, LISSOM hasbeen successfully used to model visual function and cortical plasticity. For sim-ulating early development of the cortex, Hebbian learning is highly active duringtraining and formation of the map model. When this training is over and the mapis formed, the model can be used to investigate phenomena in the adult primaryvisual cortex. The plasticity that is observed in adult cortical structures is thensimulated in the model by the same Hebbian learning process, but at a much lowerrate than during the development process. The study of cortical plasticity in LIS-SOM has shed light on the mechanisms that drive the reorganization of the visualcortex after retinal or cortical lesions [53], while functional study with LISSOMhas simulated the tilt aftereffect and identified the neural processes that give riseto the effect [13, 15].
The study of the tilt aftereffect by Bednar was the first functional study of LIS-SOM, and the present work (as well as the thesis of my fellow student, Chris Ball,on the motion aftereffect [7]) explores further the suitability of the model for sim-ulating visual function. In the tilt aftereffect study, it has been found that lateralinteractions, and particularly the inhibitory connections, were playing a centralrole in the emergence of the effect. When adapting on an oriented grating, corti-cal plasticity modifies the lateral interaction (more precisely, increases inhibition)between neurons having preference for the orientation used during adaptation, re-sulting in a “shifted” representation of the orientation of the test input (see Bednarfor a better description of the phenomena and the detailed study of both the directand indirect tilt aftereffect [15, 13]).
The biological plausibility of LISSOM is well established and the model pro-vides a good trade-off between being computationally efficient and biologicallyrealistic. It models the interactions of small group of cortical units rather thanindividual neurons and synapses. It is known that neurons in vertical columnsof V1 have similar receptive fields and feature preferences that gradually varyacross the surface of the primary visual cortex to form the cortical feature maps.Therefore, one can model V1 as a two-dimensional sheet, with each model neuroncorresponding to a cortical column of cells through the six anatomical layers ofV1. This simplification is appropriate given the present low level of knowledge ofcortical organization, but also for studying and understanding the basic computa-tion that are being performed in the visual cortex. The biological plausibility ofmany other assumptions of the model (such as the use of a weighted sum for com-
36
puting the activity, Hebbian learning or a sigmoidal activation function) has beenexamined in detail by other researchers. One of the controversial points concernswhether the direct long-range connections are inhibitory, but this issue has alreadybeen reviewed in detail in previous studies with LISSOM, and current anatomicalknowledge is consistent with LISSOM approach for high–contrast stimuli [53, 13].
The model has been recently extended further to include dichromatic (red andgreen) color processing in the retina and the LGN, as well as the ON and OFFchannels of the LGN. In a first study of this model, it has been shown that “blobs”of laterally connected color selective cells develop within the map of orientation[17]. The observed blobs and receptive fields in the model are similar to thosefound experimentally in the cortex [52, 46, 45]. Furthermore, results with themodel suggest that color selective blobs connect laterally to other color-selectiveareas. These first results formed the basis for the current work, and have beenrefined when performing the computational simulation of the ME. Further analysishave revealed more properties of the color map’s structure, as well as interactingproperties between color and orientation units. Moreover, the study of the effecthas allowed the suitability of this model to be assessed for simulating adult colorperception and visual function.
3.2 Architecture of the model
LISSOM is a general framework for a family of models. Each different versionhas been implemented for different purposes, but all are based on the same fun-damental assumptions and computational principles (for review of the differentLISSOM models, see [53]). The model that includes dichromatic color processingand that is used for ME’s simulation is called HLISSOM (H standing for Hierar-chical model). The architecture and equations under the model will be reviewedin this section, following the description in the paper from Bednar, De Paula,and Mikkulainen that examined for the first time the formation of color selective“blobs” with LISSOM [17].
Figure 3.1 shows the architecture of the HLISSOM model. It consists of two-dimensional sheets of neural units modeling different regions of the visual system.The input is presented on two sheets of retinal photoreceptors, each modeling adifferent foveal cone type (red or green), several paired sheets of ON-center/OFF-surround and OFF-center/ON surround LGN units, modeling the luminance andred-green opponent channels of the LGN, and a sheet of cortical units representingV1. The blue cones have been omitted in the model for simplicity, and becausethe majority of cones in the retina are L (red) or M (green), with no blue conesat all in the fovea [61]. Nevertheless, although the red–green channel is primarilyresponsible for most of color vision, blue cones certainly play an important role,
37
GreenChannel
RedChannel
V1
ON OFF
Luminosity
Green/Red
Red/Green
Color Image
Retina
LGN
Figure 3.1: Architecture of the HLISSOM model. The model is a hierarchy ofsheets of neural units that model the visual pathway from the red and greenphotoreceptors to V1 (Figure reprinted from [17].)
and will be included in the model eventually.The model simulates the three common types of ganglion cell receptive fields
constituting the chromatic red–green opponent pathway and luminance (achro-matic) pathway: luminosity (light–center/dark–surround and dark–center/light–surround), red–center/green–surround, and green–center/red–surround for bothON-center and OFF-center cells. When a pattern of activity is presented in thetwo photoreceptor sheets, the activity levels of all LGN cells are calculated as fol-lows: each cell (i, j) computes its response ηij as a scalar product of a fixed weightvector and its RFs on the photoreceptor sheets:
ηij = σ
(∑ρ
γρ
∑ab
Xρabwij,ρab
), (3.1)
where ρ identifies the input receptive field (on either the red or green photore-ceptors in the retina), σ is a piecewise linear sigmoid activation function, γρ is aconstant scaling factor, Xρab is the activation of input unit (a, b) on sheet ρ, andwij,ρab is the corresponding weight value.
Each V1 neuron computes its initial response like an LGN cell, then the V1activity settles through short-range excitatory and long-range inhibitory lateral
38
interaction:
ηij(t) = σ
(∑ρ
γρ
∑ab
Xρab(t− 1)wij,ρab
), (3.2)
where ρ identifies a receptive field on one of the six LGN sheets or the lateralexcitatory or lateral inhibitory weights to V1, γρ is a constant scaling factor foreach ρ (negative for inhibitory lateral weights), and Xρab(t−1) is the activation ofinput unit (a, b) during the previous settling step. The V1 activity pattern startsout diffuse, but within a few iterations of equation 3.2, converges into a smallnumber of stable focused patches of activity, or activity bubbles.
After the activity has settled, the connection weights of each V1 neuron aremodified. All V1 weights adapt according to the Hebb rule, normalized so thatthe sum of the weights from each sheet ρ is constant for each neuron (i, j):
w′ij,ρab =
wij,ρab + αρηijXρab∑ab [wij,ρab + αρηijXρab]
, (3.3)
where ηij stands for the activity of neuron (i, j) in the final activity bubble, w′ij,ρab
is the new connection weight, and wij,ρab the current connection weight, α is thelearning rate for each type of connection, and Xρab is the presynaptic activity. Thelarger the product of the pre- and post-synaptic activity ηijXρab, the larger theweight change, as a result of the Hebbian learning. This property of learning isresponsible for the learning of long-term correlations contained in the input andmakes the primary visual cortex organizes to represent the correlational structureof the input space. At long distances, few neurons have correlated activity andmost long-range connections eventually become weak. The weakest connectionsare eliminated periodically during the training simulating early development ofthe cortex, resulting in patchy lateral connectivity similar to that observed in thevisual cortex.
For the experiments reported in this paper, three 36×36 ON-center/OFF-surround (red-on/green-off, green-on/red-off and light-ON/dark-OFF) and three36×36 OFF-center/ON-surround (red-ON/green-OFF, green-ON/red-OFF andlight-ON/dark-OFF) cell sheets received input from two 110×110 photoreceptorsheets. Each ON/OFF cell had fixed Difference of Gaussians receptive fields (RFs)within the photoreceptor arrays. Before training, the afferent weights of the 64×64V1 neurons are random, and the lateral weights have a smooth circular Gaussianprofile. The radius of the Gaussian for lateral interaction is higher for inhibitoryconnections than for the excitatory ones. In the original paper from Bednar et al.[17], the radius was quite high, covering most of the simulated part of the cortex.Nevertheless, for computational efficiency, it is convenient to work with shorterinhibitory connections. Moreover, results obtained with shorter connections cangenerally reproduced in the long-range network, but the effects will be stronger
39
with the more numerous connections of the long-range network. This difference inthe amplitude of the effect can be controlled by increasing the learning rate whensimulating cortical plasticity in a short-range network. The radius for inhibitoryconnections was set to 15 units, compared to an initial value of 6.3 for excitatoryones.
3.3 Training the color and orientation map
Starting with a random afferent and lateral weight distribution for neurons in V1,training is performed by: presenting an activity pattern in the retina, computingthe settled response of the network, and modifying the weights, all repeated for agiven number of iterations. The training inputs for this study have been a set ofnatural images, from which a small region is randomly selected at each iteration,before being presented to the retina. This process models early exposure to coloredvisual stimuli. Other maps such as orientation are known to be present at birth,possibly driven by prenatal spontaneous activity [53], but color vision is thoughtto mature later, and in any case it is not yet known if there are color maps thatdevelop before visual experience.
The natural images set that is used for training is a series of color bitmapsseparated into red, green, and blue color channels. The red and green part ofthe image are presented on the corresponding cone photoreceptor sheets. Whenperforming the study further and trying to determine hue selectivity in the model,I noted that among the orientation selective cells, the color preference was pre-dominantly green. I thought that it could reflect a bias in the image dataset, andtherefore studied further the general level of activity in both the red and greenphotoreceptor sheets during training. I observed that the overall activity level inthe green channel was slightly higher than in the red one, and that this differencewas responsible for the shift toward green in the distribution of red and greenselectivity pattern in regions outside of the color blobs.
Although there is no data about the very detailed organization of color blobsin V1, it seemed unlikely that such imbalance was realistic, while also being po-tentially an issue for studying the ME. Indeed, one of the consequences of thisimbalance was that almost no orientation and red selective neurons were foundafter training, whereas numerous orientation and green selective cells had devel-oped. This difference was obvious when observing red and green blobs on thecolor selectivity map: green blobs were constituted by highly selective areas (verybright on the selectivity map), surrounded by less selective areas forming continu-ous transitions from the green blob centers to the orientation selective regions; redblobs, on the contrary, were only constituted of highly selective areas, immediatelysurrounded by units very unselective to color. The surrounding area of slightly
40
color selective neurons that is found in green blobs, is an intermediate borderingregion between orientation and color selective regions, and is the only area where“double–duty” units have been found in the model. Therefore, if the neural sub-strate of the ME were to be these cells, it seemed unlikely that the model wouldaccount for it if missing red “double–duty” units. In addition, visual phenomenonand studies of color cells in the red and green channel suggest that the process-ing mechanisms resulting in green and red perception are very similar [61], whichwould probably not be the case if there were such a crucial difference of substrateand structural organization as early as in V1.
For the reasons presented above, I compensated for this bias toward green byadding a slight amount of activity in the red channel: during training, I increasedby 2.4% the scale parameter in the red photoreceptor sheet. In other words, Imodified the original RGB values of the images by slightly increasing the red com-ponent until reaching a similar activity level in both color channels. This hasenabled the production of map showing an even distribution of red and green pref-erences: blobs of both color had similar structure and there was approximatelythe same amount of “double–duty” units preferring red as “double–duty” unitspreferring green. It might well be possible that such a re-calibration of the colorchannels is taking place in the visual pathway, considering that green is probablythe predominant color in our natural environment (at least when not living inSiberia, Sahara, or also London). It also would not be surprising, considering itsdegree of complexity and the number of similar tasks that we know it performs,that the brain has evolved such a mechanism. The luminance contrast processingof the ganglion cells of the retina is such an example of the extraordinary capac-ity of the brain in developing efficient solution for processing visual information.Nonetheless, nothing is yet known about such a re-calibration of the red and greenchannels, and this is just an hypothesis suggested both by the model and the studyof the ME.
With these changes to the training inputs, the training was performed for20,000 iterations, the learning parameters used being similar to those found inearlier V1 orientation model [15], scaled for this cortex size using the model scalingmethodology for LISSOM [14]. During training, the cortical sheet self–organizesand the map structure emerges progressively. The afferent weights of the majorityof neurons develop oriented synaptic weight patterns, thus forming a receptive fieldtuned to an oriented stimulus in the unit’s retinal field. Nevertheless, some neuronsdo not develop orientation selectivity, and form blob–shaped patches of unselectivecells within the orientation map. These cells are more receptive to activationin one of the photoreceptor sheets. That is, they respond best to stimuli thatdrive activity in the “red-ON channel” (red-ON/green-OFF and green-OFF/red-ON channel, which is most activated by activation of red cones only), or in the
41
“green-ON channel” (green-ON/red-OFF and red-OFF/green-ON channel, whichis most activated by activation of green cones only). Therefore, it is more accurateto describe these cells as cone selective rather than color selective.
In order to reveal the structure of the orientation map on the cortical sheet,each neuron is labeled with its orientation preference. For finding the orientationpreference of a neuron, a set of oriented stimuli (i.e., oriented gratings), that variesin orientation, phase and spatial frequency are presented to the network. Then, theorientation of the stimulus that activates it the most is taken to be its preference,while the magnitude of the response for this preferred stimulus is taken to be theselectivity of the neuron (see [53] for details of this procedure). In the same way,color selective neurons were labeled with their hue preference. It is not knownwhether neurons in V1 already develop preferences for a given hue, but it has beensuggested that such neurons exist in area V2 of the visual cortex [73].
A function for finding hue preference was also implemented to produce a huemap of V1. It presents an intermediate mix of red and green cone activation onthe retina, therefore representing the different proportion of each wavelength atdifferent point of the hue scale. I found that the hue preference always classesneurons into two categories, red or green selective, and that no intermediate wave-length composition was represented in the network, whatever the number of pointsused on the hue scale. It has been therefore chosen to compute the hue preferenceonly in terms of red or green cone selectivity. The receptive fields of these unitstherefore organizes to be color selective, or, more precisely, wavelength or conesensitive.
Figure 3.2 shows the color and orientation maps obtained after training themodel. The orientation map is comparable to those produced in previous stud-ies with LISSOM [53], and presents important similarities with those found inmacaque monkey. It contains pinwheels (points around which the orientationpreferences changes continuously), linear zones (bands were the orientation prefer-ences change continuously) and fractures (points where the orientation preferencechanges abruptly between two distant orientations) [63]. The color map is consti-tuted by highly selective areas organized in blobs. These blobs have been found tobe red selective or green selective, as previously explained. It is a prediction of thismodel that a single blob only contains one type of color selective cell. (Previousexperimental studies in V1 only located cells responding to color–opponent stimuli(red–green gratings), which does not enable discrimination between the two colorpreference types.)
Figure 3.3 shows in detail how the color map relates to the orientation map.The unselective regions in the orientation map correspond with the color selectiveblobs. However, the periphery of the blobs overlap with selective zones for orienta-tion. The model therefore contains neurons that are selective both to color and to
42
(a) Orientation map (b)
(c) Color map
R
G
(d)
Figure 3.2: The orientation and color map in HLISSOM. These figures show thecolor and orientation maps that have been produced after 20,000 training itera-tions on patches of natural images. From left to right in (a) and (c) there are: thepreference map, the combined preference and selectivity map, and the selectivitymap. The preference map colors the units with the orientation (or color) prefer-ences ((b) is the orientation preference key, and (d) is the color (or cone) preferencekey, where R indicates red preference and G green preference), while the selectivitymap shows the selectivity of the units for their preferred stimulus. The map inthe middle is the superposition of both, color being the preference, and saturationof the color being the selectivity. The histogram below each map shows the dis-tribution of the preferences or of the selectivity. Therefore, the histograms belowthe orientation preference map shows the distribution of all orientation preferenceon a cyclic hue scale: red corresponds to horizontal and cyan to the vertical. Notethat the orientation’s preference distribution is not perfectly flat, indicating somebiases in the training images (toward orientations close to vertical). Because thereare only two possible color vectors (red or green), the color map’s histogram onlyshows two bars: red indicates a red preference and cyan indicates a green prefer-ence. All neurons prefer either red or green, but neurons outside the color blobsare very unselective for color. In the same way, each neuron prefers an orientation,but neurons in color blobs are typically unselective to orientation. By comparingboth maps, it is possible to view that the color blobs correspond to relatively uns-elective areas in the orientation map, and also that units that are slightly selectiveto both color and orientation can be found at the blobs extremities.
43
(a) (b) (c) (d)
Figure 3.3: Structural detail of the organization of color and orientation maps.These three images show how the color map relates to the orientation map inHLISSOM. (a) superposes the location of the blobs (white circles) on the orienta-tion preference map. (b) is the same picture but using the orientation preferenceand selectivity map; it clearly shows that blobs are centered on unselective regionsof the orientation map. Finally, (c) has been obtained as (a), but the orientationmap has been smoothed using a circular Gaussian filter, in order to reveal the lo-cation of the pinwheels (points around which the orientation preferences changescontinuously) and linear zones (bands where the orientation preferences changescontinuously). It shows that blobs are generally centered on pinwheels, thoughsometimes the blobs are centered on linear zones. (d) is the orientation preferencekey.
orientation, which matches the observation of such “double–duty” units in monkeycortex [37, 52]. I observed that the color selective blobs were generally situatedsuch as to be centered on pinwheels, or on linear zones. Then, when studyingthe lateral connectivity of individual color selective neurons, i observed that cellsin the center of the blobs connect to cells with orientation preferences coveringall possible orientations. That is, the pattern of lateral weights for one of thoseneurons looks like a pinwheel. These observations constitutes a prediction, as noprevious experimental paper has ever reported such a structural property yet.
Finally, sample afferent weights of the four main types of neurons (orientationtuned, red and green selective, and neurons tuned to both color and orientation)are shown in Figure 3.4. It is also important to note that, because of the useof shorter–range connections in this study, it is not easy to show that lateralinhibitory connections link cells with similar preferences, but the result has beendemonstrated earlier and still appears to be true on these maps [53, 63, 17].
44
3.4 Measuring perceived color
In order to assess and compare the ME in LISSOM with human psychophysicalexperiments, a method for measuring the perceived color in the model needed tobe developed. In other words, given a pattern of activation in V1, an estimate ofthe color perceived in the model needs to be computed. The method that has beenimplemented to compute the perceived orientation consists of a weighted averageover the orientation preferences of all the orientation–selective units activated bya given input. Experiments on neurons in monkey V1 suggest that they use sucha statistical encoding of orientation [32]. However, as it was said in the previoussection, the hue map on the cortical sheet only showed two types of color selectivecells: red or green. Therefore, the only representation of the color dimension ofthe stimulus is on a red–green linear scale, or a weighted average of two oppositecolor vectors, in other words, the difference between the sum of activity in red cellsand activity in green cells.
This kind of saturation scale has been used in previous neural network simu-lations of the ME, where the perception of red was associated with an excess ofactivity in the red units of the network over the green ones, and perception ofgreen associated with the reversed situation [70]. The magnitude of the resultingdifference vector then represents the amount of color saturation. It has been veri-fied and will be shown in the next chapter that a red stimulus or a green stimulusresults in a perception that is shifted toward the red side or the green side of thissaturation scale; i.e. a red (or green) grating results in a red (or green) vector thatshows a significantly high magnitude, while a white (achromatic) grating resultsin a color vector with a non-significant magnitude.
45
(a) Orientation–selective cell (tuned to 90◦)
(b) Red–selective cell
(c) Green–selective cell
(d) “Double–duty” unit (tuned to green and vertical)
Figure 3.4: Four different type of receptive-field in V1. This figure shows the af-ferent weights for typical, orientation–selective, red–selective, green–selective andcolor–orientation selective cells in LISSOM’s cortical sheet. White indicates anON subregion of the RF (i.e., an area of the receptive field where ON-cell inputwill excite the neuron), black indicates an OFF subregion, and gray indicates asubregion without significant net preference for ON or OFF. The afferent weightsare presented as follows (from left to right): luminance ON, luminance OFF,Red-ON, Red-OFF, Green-ON, Green-OFF. The orientation–selective cell (a) re-sponds to a vertical bar on any photoreceptor sheet, the red selective cell (b)responds to a bright spot on the red photoreceptor sheet or a dark spot on thegreen photoreceptor sheet, the green selective cell (c) responds to a bright spot onthe green photoreceptor sheet or a dark spot on the red photoreceptor sheet, andthe “double–duty” cell (d) responds most strongly to a vertical bar on the greenphotoreceptor sheet.
46
Chapter 4
McCollough effect simulations
In this chapter, the results of the ME simulations with LISSOM will be describedand compared to the human data previously introduced. At first, the parametersand methods used for simulating ME experiments will be described, and it willbe shown that the model exhibits color aftereffects with similar characteristicsas those in humans. Then, the mechanisms that give rise to the effect will beexamined, and the lateral inhibitory connections between color and orientationcoding units will be shown to play a central role in the emergence of the effect.This constitutes a prediction of the model, with a level of detail that is not yetavailable from human or animal experiments.
4.1 Parameters and method
The simulations used the trained orientation and color map described in the pre-vious section. The map is a model of one individual’s primary visual cortex, andthe ME experiments will be performed on this particular individual. It would bepossible to test the ME on different individuals by creating maps with other setsof images, or by changing the order of presentation of the images during the de-velopment process, so as to simulate different early–life visual experience. In anycase, the simulations in this thesis represent the testing of the ME on a particularindividual under different conditions, and will be compared to single–individualdata from psychophysical experiments.
The induction procedure of the simulation has been set up to reproduce thesituation in which humans are tested for the ME. Furthermore, the parametershave been set in order for the present study to be consistent with the tilt afteref-fect experiments that were previously performed with LISSOM [15]. In that way,the different results obtained with LISSOM can be considered together, thereforeproviding a model of the primary visual cortex that exhibits several important
47
characteristics of adult visual perception, in addition to being a good developmen-tal model of V1’s feature maps.
To induce the ME, chromatic square-wave gratings (either green or magenta)are typically used in human experiments. To avoid discretization effects, the modelsimulations used sine-wave gratings, thus effectively modeling a single spatial fre-quency channel in the retina. To determine the appropriate strength of the gratingin the red or green channels, we must consider in detail the activation pattern ofboth green and red photoreceptors when a red or a green grating is presented tothe human retina. Input to the model actually represents the population of redcones and the population of green cones for the retinal area covered by the model,rather than the wavelength composition of the light that reaches it; thus, thesepopulation activities must be computed in order to simulate a given stimulus inthe model. As shown on Figure 2.1, the absorption spectrum sensitivity of redand green cones very much overlap. Therefore, even when a pure red stimulus ispresented to the eyes, green cones are also activated in some extent. Similarly,when a pure green stimulus is presented, some of the red cones activate. Thus, weneed to estimate the population responses to red and green gratings.
The psychophysical experiments that are compared to the simulation resultsuse a green grating at 523nm and a red (magenta) grating at 650nm. They makeuse of different projectors or of special chromatic filters to ensure that the appro-priate wavelength is reflected by the induction grating in both cases. The plotof the normalized wavelength sensitivity of the cone photoreceptors (as shown inFigure 2.1) has been used to determine the relative amount of activity in thetwo cone populations for these two wavelengths (assuming that the retinal cir-cuitry normalizes in some way the signal created by the photon absorption of bothcones). A ratio of 0.5 has been found to be a good approximation of the relativeamount of activity at 523nm and 650nm. The fact that this ratio is approximatelythe same at both wavelengths, and that it also is relatively high, might explainwhy this pairing of wavelength is so efficient for inducing MEs. Each chromaticchannel activates most when the difference between its activity level and the otherless–activated cone populations’ activity is high. Therefore, the effect might beenhanced when using wavelengths that trigger such peak responses in the two conepopulations. The fact that this ratio is almost equal for both wavelengths mightalso explain why effects of similar amplitude are observed on both test gratingswhen using this color pairing for inducing the ME.
To match these conditions, for a red (magenta) stimulus, a sine grating ispresented to the red sheet of retinal photoreceptors and a sine grating of half theamplitude and half the offset is presented to the green sheet. The opposite is donein case of a green stimulus. Furthermore, the sum of the amplitude of both signalsis constant and taken to be equal to the sum of the amplitude of the achromatic
48
sine gratings presented to both cones when testing. Thus, the total amount ofactivity in the retina is always equal in the three cases (i.e., when presenting a red,green or achromatic grating), just as the total amount of light (i.e., luminance) isconstant during the human experiments.
Finally, the scale parameter controlling the total amplitude has been chosen sothat response of the model V1 neurons will be biologically plausible. That is, therepresentation of a visual scene has been shown to be sparsely distributed in V1,and the activity in the model should show a similar sparse representation in orderto be realistic. The scale parameter of the stimuli was set so that V1’s response wassufficiently large to measure an averaged orientation and color preference reliably,without being so high that it activates all neurons indiscriminately [53]. A morecomplicated model would be less sensitive to this parameter, as explained in [53],but for simplicity this version of LISSOM does not include these extensions. Figure4.1 shows the three stimuli as previously described, and Figure 4.2 shows a typicalsettled response of the cortical sheet to each of the three stimuli. Much weakerscales would result in only a little activation, and much larger scales would result inwidespread, unselective activation, while the chosen scale has robust yet selectiveresponses.
To simulate adapting with the two oriented chromatic stimuli as in psychophys-ical experiments, red and green gratings were alternately presented to the retinafor a number of iterations. The number of iterations were set to be consistent withthe tilt aftereffect simulation and to approximately match what is generally usedin ME experiments, as described in the following. The speed of the alternationhas been shown not to influence the results [65], but it was also chosen to be asin most experiments. When it is not otherwise specified, each chromatic gratingwas presented for 300 iterations, switching from one to the other every 5 itera-tions. The total of 600 iterations is low compared to the 20,000 iterations used fortraining the map, and considering that the tilt aftereffect necessitates 90 iterationsto match a three minute psychophysical experiment, 600 iterations approximatelymatches the average 15 minutes of most ME experiments.
During the simulation, the learning rates of the input and excitatory weightswere set to 0, leaving only the inhibitory connections with a learning rateαi=0.00005. There is not yet experimental data showing the relative levels ofplasticity for afferent and lateral connections. However, previous work with theLISSOM model strongly suggests that the lateral inhibitory connections are themost important for aftereffects in the model, and similar effects are seen even whenafferent and lateral excitatory connection plasticity is disabled [15].
During an ME experiment, the retinal position of the gratings is always slightlyvarying, because of head or eye movements. In the model, the position of theadapting inputs is not important as long as it activates the area that will later be
49
0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
0.7red photoreceptors
0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
green photoreceptors
(a) Red grating stimulus
0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
red photoreceptors
0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
green photoreceptors
(b) Green grating stimulus
0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
red photoreceptors
0 5 10 150
0.1
0.2
0.3
0.4
0.5
0.6
green photoreceptors
(c) Achromatic grating stimulus
Figure 4.1: Stimuli used for the simulations. Sine gratings that are used for in-duction ((a), red grating stimulus and (b), green grating stimulus) and for testingof the ME (c). Two-dimensional sine gratings were used, of which only a one-dimensional cross-section is shown here. They are also oriented in the model andit is possible to set their phase, which will be done to simulate movements duringinduction and experiments under different conditions.
50
tested for the ME. When it is not otherwise specified, the phase of the sine gratingon the retinal sheet is randomly varying during the induction period, with a givenmean and variance. Then, testing of the ME is done for 5 phases within thisdistribution, and an average is done to produce the result of the ME experiment.To simulate different measurements of the ME, the same procedure is repeatedwith a different mean phase each time. The variance is chosen to match thecircumstances of each experiment. In general, the head is kept fixed and thegratings are projected with a relatively large size (e.g 9◦ of visual angle at a viewingdistance of 1 m [28]), therefore the variance is taken to be relatively small (1
8of
the total period of the sine grating). The only time when a different variance hasbeen used has been when checking that fixation was not necessary for inducingthe ME (in this case, the variance is taken to cover the entire phase distribution).
Finally, a pre-induction test procedure is performed before each testing, i.e.the test pattern is presented to the model and the perceived color is measuredon the red–green scale, as described in section 3.4. After induction, the samemeasurement is done and the strength of the ME is reported as the deviation fromthe pre-induction test baseline.
4.2 Results
To assess how well the simulated ME relates to the effect experienced by humans,an evaluation process has been set up making use of most of the psychophysicalproperties and experimental data that have been presented in section 2.2.1 and2.2.2. It consists of a series of assessments, going from the simple (and principal)hallmark of the effect to more precise and detailed properties.
First, the model will be shown to exhibit the specific ME responses followinga classical induction with red–horizontal and green–vertical gratings, or green–horizontal and red–vertical gratings. It will also been checked that the ME can beestablished without fixation of the test grating. Then, the ME will be studied as afunction of the orientation test angle for comparison with the results of Ellis’ study[28], and it will be shown that the simulated effect varies similarly to the effectexperienced by humans. This will be followed by a study of the independence ofthe adapting orientation of the effect (i.e., the ME can be induced with every pairof oriented gratings differing by 90◦) and a study of the breakdown of the effectwith decrease of the angular difference between the two induction patterns. Bydoing so, the ideas advanced by Fidell will be analysed and refined further [31], andthe computational model will provide more detailed data than the ones previouslyobtained in the human experiments. Finally, the indirect McCollough Effect (IME)observed following a single grating induction will be studied in detail, and it willbe shown that the effect observed in the two–grating induction case can be seen
51
as the superposition of two independent effects. Even though the properties of theME have not yet been exhaustively examined in the model, these results indicatethat the ME obtained in LISSOM exhibits most of the principal psychophysicalproperties of the human effect, and that these properties can be explained as aresult of adaptation in inhibitory lateral interactions.
4.2.1 The classical ME
Most of the psychophysical ME experiments only test if complementary illusorycolors are seen on the achromatic test gratings, when they have the same orienta-tion as the adapting ones. The bar graphs presented in Figure 4.3 are the resultsof similar simple ME experiments with the LISSOM model. Three cases have beenstudied:
• Induction with red–horizontal and green–vertical gratings with head andgaze fixed (using the phase variance as described in section 4.1).
• Induction with green–horizontal and red–vertical gratings with head andgaze fixed (using the phase variance as described in section 4.1).
• Induction with red–horizontal and green–vertical gratings, without fixation(using a phase variance spanning the full sine grating period).
• Induction with green–horizontal and red–vertical gratings, without fixation.
Each time, tests have been performed for 12 different experimental conditions(i.e., changing the mean phase for any experiment), and the results are the averagesover these twelve experiments. More experiments could have been performed toobtain higher statistical significance, but most of the phases that are tested overlap,and almost the same population of neurons is activated any time the mean phase isonly slightly changed. Therefore, more data points would not considerably changethe results.
These results show that the model exhibits the ME for a classical experimentalinduction and test procedure and for two different orientation–color pairings. Italso demonstrates that no fixation is required to produce the effect, as long as theneurons that are tested have been adapted at some point during induction. Theonly difference between the two cases is the amplitude of the effect: it is obviouslyless strong when the phase has been varying over a larger interval, because theneurons tested are less often activated during induction. This result agrees withfindings that the effect is stronger when the test grating is in approximately thesame retinal location as the adapting gratings had been [65].
52
4.2.2 The ME as a function of the angle between test andadapting gratings
As in Ellis’ study, the ME was also studied in LISSOM as a function of the angulardifference between induction and test gratings [28]. Ellis used the top half of thetesting field for displaying the horizontal achromatic grating, and the bottom halffor displaying the vertical one. Moreover, he sometimes paired red with horizontaland green with vertical, and sometimes did the contrary during induction. Thecolor–cancellation apparatus that he used allowed him to add magenta at the topand green at the bottom at the same time (or the contrary, depending which colorwas paired with horizontal). Therefore, he could not really differentiate betweenthe effects seen on the two different test gratings. Testing was performed byrotating both half fields of a certain angle between adaptation orientation minus45◦ and adaptation orientation plus 45◦, and then recording the strength of theeffect for the given angular difference. No more than twelve different orientationsof the test pattern were reported for each individual, and the number reported wasthe average over 25 experiments.
To reproduce this situation in the model, the following procedure was followed:a pre-induction test was done for 24 oriented gratings between 0◦ and 180◦ (usinga step of π
24), then induction was performed as previously described, and the post-
induction test procedure was performed for the same 24 oriented gratings. Forcomputational efficiency, learning is turned off in the model for the pre-inductiontests and the oriented patterns are all presented in sequence. This is an advantageof the model over human experiments, where one must ensure that the cortex isreturned to equilibrium between adaptation periods, and can only test one orien-tation at each adaptation. For each oriented pattern, it has been chosen to countas positive a deviation of the perceived color toward green, and negative towardred.
The experiment was done for two different orientation–color pairings, as in El-lis’s study: red–horizontal and green–vertical or green–horizontal and red–vertical.For each pairing the experiment was done for 12 different phases of the sine grat-ing, and the method described in the previous section was used to obtain theresults. Figure 4.4 is the plot that resulted. In all cases, the plot starts 45◦ be-fore the orientation that was adapted with red. Therefore, 0 represents 0◦ for the12 experiments with red–horizontal and green–vertical gratings, and 90◦ for the12 experiments with the other pairing. The plot shows that, within an intervalof 90◦ centered on the adapting orientation, the model ME increases smoothlyas the orientation of the test grating moves toward the orientation of the adapt-ing grating, reaching its maximum strength when the test grating is oriented asfor adaptation. Moreover, the curve obtained with the two inducing colors willapproximately overlap, if plotted in the same quadrant.
53
The curve in Figure 4.4 constitutes a prediction of the model, because theapparatus and method used in Ellis’ study did not allow him to produce such adetailed curve. In order to compare the results with their data, the average ofthe red and green aftereffects have been computed, and the results were presentedas Ellis reported it: from an angular difference equal to -45◦ for both inductiongratings (i.e., horizontal and vertical) to an angular difference equal to 45◦ (seeoriginal plot in section 2.2.2). Figure 4.5(a) presents the plot that was obtainedwith the model, plotted against the representative individual curve that was shownin section 2.2.2. Because the value of the aftereffect strength in the model isdifficult to relate to the chromatic purity value reported by Ellis, it has beenchosen to re-scale both human data and simulated data so that the maximumvalue in both cases was equal to 1. (The red–green scale used in the model reflectsthe difference in activity between the population of red and green selective cells;a value on this scale is difficult to relate directly to the wavelength compositionof light). Re-scaling is also appropriate because the strength of the effect differsbetween individuals, as described below.
Data from two additional individuals is reported in Ellis’s paper, but onlyranging from 0◦ to 45◦ angular divergence between test and adapting gratings.Figure 4.5(b) plots all the available data against the results from the model, takingthe data points from 0◦ to 45◦ from the graph of Figure 4.5(a). Again, all the datahave been re-scaled to have their maximum equal to 1, so that the shapes can becompared.
These plots show that the results from the ME simulation are comparable withthe results from human experiments. The effect is stronger around 0◦ angulardivergence, and decrease progressively to zero at -45◦ and 45◦. The main differencebetween human data and the results from the simulation are that curves fromhuman data decreases approximately linearly, whereas plots from the simulationpresents are slightly convex 1. This is particularly clear in Figure 4.5(b). However,the similarities are quite striking, and the comparison between simulated and realME is much closer than any previous attempt to simulate the ME, despite themodel not having been specifically designed to reproduce the effect.
1Simulations performed since this thesis was submitted show that by making the phase vari-ance be a better match to the human experiments, the difference in curvature becomes negligible.The resulting curves are nearly indistinguishable from the human results, and will appear in aforthcoming publication.
54
4.2.3 Effect is independent of absolute orientation but de-creases for small angular divergence.
Figure 4.6 shows the results obtained when reproducing Fidell’s experiments de-scribed in section 2.2.2. The results here have been averaged over 6 differentexperiments for each curve. The axis represents the orientation of the red gratingsused during induction. The strength of the aftereffect is here an average betweenthe green and red aftereffect on both test gratings, as was done in the original ex-periment. For the red curves, the angle between red and green gratings is alwaysequal to 90◦: we clearly see that the strength of the aftereffect is approximatelyconstant whatever the orientations of both induction gratings, provided that theangle between them is constant and equal to 90◦. This is a prediction of the model,because although it is known that the ME could be induced with every orienta-tion, no study has precisely measured the strength of the effect at each orientation.It would have been reasonable to think that the effect could have been strongerfor the vertical and horizontal induction gratings (because both humans and themodel seem to have more neurons tuned for these orientations), but such an ef-fect did not occur in the model. This result may indicate that the mechanismsunderlying the ME are well balanced across different orientations.
The blue curve is the result from the experiment studying the strength of theME as a function of the angular divergence between the two induction patterns.Here, the green grating is kept at 90◦, and the red grating varies from 0◦ to 90◦.It is clear that the effect starts to decrease significantly around 25◦ of angulardivergence (when the red orientation is at 65◦ on the graph), and that the effect isconsiderably reduced for small angular divergence. These results agree with Fidell’sconclusion that for small angular divergence the effect considerably decreases, butthey provide more detail that suggests that the ME is an approximately continuousfunction of the angular divergence. Furthermore, the minimum angular divergencefor producing the ME is shown not to exist in the model since the two effectsdo not quite cancel when the same orientation is used during induction. Here,we observe only a small residual effect, probably due to the fact that the twopopulations of neurons are not exactly balanced within the model (which is likelyto also happen in human V1). The orientation tuning of the neurons in LISSOMis approximately 60◦ (which gives a half tuning of 30◦), which is comparable tothe orientation tuning of 52◦ that have been found experimentally [24], and thatis probably why the decrease of the effect starts at 30◦ of angular divergence.
In order to link the results in LISSOM with Fidell’s data, data points from hisexperiments have been added on the plot. His data shows the same breakdownof the effect when decreasing the angular divergence between adapting gratings,and similar constancy when the angular divergence is kept equal to 90◦. However,the addition of this plot only enables a qualitative comparison with the one from
55
the LISSOM simulation, because it is composed of only a few data points anddoes not show statistical significance. Nevertheless, results from this simulationclearly confirm the hypothesis that both effects nearly cancel when adapting onthe same population of oriented cells (blue curve on Figure 4.6, at x = 90◦), andthe observation that the ME can be induced with all pairing of orientations thatdiffer by 90◦ (red curve on Figure 4.6).
4.2.4 Indirect McCollough Effect
In order to verify that the model exhibits the Indirect McCollough Effect (IME),simulations were performed with single grating induction procedures. Two differ-ent induction patterns were tested: red–horizontal only, and green–vertical only.This way, it was possible to check if the effect produced by the two grating proce-dure alternating between red–horizontal and green–vertical can be interpreted asthe sum of the two effects produced by each grating alone.
The same experiment as in the study of the dependence of the effect on the testorientation was performed, plotting the ME as a function of the angular differencebetween adapting and test gratings. For each single grating induction procedure,the results have been averaged over 12 experiments. Figure 4.7 shows the curveobtained in both cases. It is clear that there is perception of an illusory color onthe orthogonal non-induced grating, which correlates with the reported findings onthe IME. The simulation gives us details not yet tested experimentally in humans,and thus these results are a prediction of the model.
Note that when performing the single grating induction, the effect is equalto zero at 30◦ angular difference between test and adapting grating. Given thatorientation tuning in LISSOM is approximately 60◦, it makes sense that the effectis null 30◦ away from the adapting orientation. However, when adding the twocurves of Figure 4.7, the indirect effect that is observed for the orientation morethan 30◦ away from the adapting orientation compensates each other and producethe slightly different shape that is observed for a two grating induction procedure(red–horizontal and green–vertical). Moreover, the direct and the indirect effectsat both adapting orientations add up to produce a stronger aftereffect, as wassuggested by many researchers [5, 66, 2, 28].
The fact that the model exhibits the IME is an additional evidence that the MEsimulated with LISSOM is similar to the effect experienced by humans. Moreover,study with a single grating enables us to explore further the properties of theeffect in LISSOM. The model suggests that the two effects add up in the normalcase of two induction gratings. The next section will examine in more detail theneural mechanisms giving rise to the direct effect, and speculate on the mechanismsunderlying the indirect effect.
56
4.3 Interpretation of the results in terms of neu-
ral mechanisms
The ME that arises in the model has been shown to be very similar to the effectexperienced by humans. In particular, the dependence of the ME on the orientationof the test grating reveals the change in the overall activity distribution for a givenoriented test grating. In order to study in more detail which changes in the corticallateral connectivity are responsible for the effect, the lateral inhibitory weightsdistribution of individual neurons will be examined.
In order to characterize what happens when adapting with each grating, asingle grating induction has been performed for both cases: red–horizontal andgreen–vertical. Each induction has been split in three periods, changing the meanphase each time in order to be sure to activate most of the orientation units duringthe experiment. Then, the changes in weight patterns of orientation units and colorand orientation selective units were examined. To make the changes more obviousin the plots, the learning rate has been increased to αi = 0.005. The mechanismsunderlying the direct ME were studied first, and it is clear how these effects arise.Because the indirect effect observed on the non-induced orthogonal orientation islikely to be due to very small changes in inhibitory patterns, the mechanisms givingrise to it are only suggested since no conclusive observations have been made whenplotting the lateral weights.
For these examples, a single induction with a green–vertical grating was per-formed. Figure 4.8 shows the lateral inhibitory weights of a vertical unit beforeand after induction. The inhibitory weights are colored with the color preference ofthe neurons that they link to, the saturation of the color specifying the selectivity,and the value (brightness) the strength of the connection. We clearly see that afterinduction with the green vertical grating, the inhibitory weights to green-selectiveunits have increased, whereas the inhibitory weights to red selective units havedecreased, as a result of the normalization process (for a review about the biolog-ical plausibility of normalization, see [53]). Before adaptation, the weights to redand green selective units are approximately evenly distributed, because long-termcorrelations in the environment (i.e., the image data set that was used for trainingthe map) do not favor any orientation–color association. After adaptation, thehigh correlation between green and vertical has modified the weight distributionby increasing the inhibition to red selective cells, as a result of Hebbian learning.Since total weight strength for all inhibitory weights is constant (see Equation 3.3),the strength of connections to red selective cells has decreased.
The phenomenon observed for the vertical cell drawn in Figure 4.8 is observablefor all vertical units in the network, provided that they have been activated duringadaptation. Thus, when an achromatic vertical grating is presented for testing,
57
the vertical units activate and inhibit the green units more than before, while alsoinhibiting the red units less. This creates a shift of the perceived color towardred, resulting in the illusory color of the ME (i.e., an achromatic vertical gratingappears red). The orientation tuning of an orientation–selective cell in LISSOMis about 60◦ (i.e., the half bandwidth is 30◦), therefore this increase of inhibitionbetween green and orientation units affects units preferring orientation between60◦ and 120◦, which seems clear when looking at the angular function obtained fora single grating induction procedure in the IME experiment (section 4.2.4, figure4.7).
The indirect ME could therefore be explained as follows: the green selectiveunits have increased inhibition to vertical units, so the inhibition to all otherorientation have decreased because of the normalization process. Therefore, forall orientations farther than 30◦ off the vertical, inhibition to green units hasdecreased, causing a shift of the perceived color toward green, as observed withthe indirect ME. Unfortunately, it is difficult to see this effect in the plots, so thisinterpretation of the IME remains only suggestive, and it will need more analysisin future study in order to be validated.
Because other authors have proposed that ”double–duty” units could underliethe effect, it is also important to consider how such units behave in the model.Figure 4.9 examines the change in the inhibitory weights of a green–selective unitthat is also slightly tuned for vertical. It has been found that this type of cell(independently of its orientation preference) shows more significant changes thanother color selective cells, because it is situated at the limit of the color blobs, andtherefore is closer to pure orientation–selective cells than are other color selectivecells. The lateral inhibitory connections are first plotted colored with the orienta-tion preference of the neurons they connect to. One can see that the proportion ofinhibitory weights that connect to vertical units (blue–green) has increased, whilethe horizontal orientation (red–pink) are less represented. When coloring withcolor preference, we see that the inhibition to green selective cells has also slightlyincreased in strength. Thus, when considering these units as color selective, wesee that they have increased inhibition with vertical units. When consideringthem as orientation units, they only have increased slightly their inhibition toother color selective cells. However, because of the small proportion of such unitscompared to purely vertical units in the network, the effect of such change is neg-ligible compared to the change undergone by orientation units. Moreover, whenan achromatic test grating is presented, “double–duty” units respond only weaklyif at all, while pure orientational units respond strongly. The model thus predictsthat pure orientation–selective units are primarily responsible for the emergenceof the effect.
Similar effects are seen for the opposite condition, examining the effects of
58
red–horizontal induction on horizontal–selective cells. In this case, the lateral in-hibitory connection weights to red–selective units have increased, whereas weightsto green selective units have decreased. Again, this phenomenon is observable forall horizontal units that have been adapted during induction. For achromatic hor-izontal grating presented subsequently, the perceived color will be shifted towardgreen, as a result of the stronger inhibition between horizontal and red selectivecells.
The effects for “double–duty” units are also similar with red–horizontal induc-tion. In this case, for units tuned to red and also slightly to horizontal, the propor-tion of inhibitory weights connecting to horizontal units have slightly decreased,and the inhibition to red selective cells have also increased, but not dramatically.Therefore, these units contribute to the effect both as orientation cells and as colorcells. Even so, such cells cannot be the principal cause of the effect, for the samereasons as explained earlier for green–vertical “double–duty” units.
To conclude, we have seen that for the two induction procedures (green–verticaland red–horizontal), it is the change in the weights of inhibitory connections be-tween color selective and orientation selective units that is primarily responsiblefor the effect. Although we found “double–duty” units in the model, they con-tribute little to the effect, and their contribution is through the same mechanismsas for the units primarily selective for either color or orientation alone. Thus themodel replicates the ME in detail, illustrating how it can occur within a model forthe development of color and orientation selective maps and providing concretepredictions that can be tested in future experiments.
59
(a) Cortical activitytriggered by a red grating
stimulus
(b) Cortical activitytriggered by a green grating
stimulus
(c) Cortical activitytriggered by an achromatic
grating stimulus
Figure 4.2: Settled LISSOM response to the three types of stimulus used in thesimulation. The settled responses of V1 are shown for each type of stimulus.The gratings presented here were vertically oriented. The neurons are coloredwith their color preference, with intensity (dark or bright colors) indicating activeneurons and the brightness of the color indicating the level of hue selectivity. It isclear that the cortical representations of the gratings are sparse and distributed,which matches experimental observation of normal cortical activity. (a) A redinput grating activates primarily red–selective neurons (vector direction 0.0 andmagnitude 79.9). (b) Conversely, a green input grating activates green–selectiveneurons (vector direction 0.5 and magnitude 82.5). (c) An achromatic stimulusprimarily activates neurons without strong hue preferences (dark colors) and anequal mix of neurons with red and green preferences (vector direction 0.0 andmagnitude only 1.4).
60
0
1
2
3
4
5
6
7
8
9
ME M
ag
nit
ud
e
Horizontal Vertical
(a) Red–horizontal and green–verticalinduction
0
1
2
3
4
5
6
7
8
9
ME M
ag
nit
ud
e
Horizontal Vertical
(b) Green–horizontal and red–verticalinduction
0
1
2
3
4
5
6
7
8
9
ME M
ag
nit
ud
e
Horizontal Vertical
(c) Red–horizontal and green–verticalinduction (without fixation)
0
1
2
3
4
5
6
7
8
9
ME M
ag
nit
ud
e
Horizontal Vertical
(d) Green–horizontal and red–verticalinduction (without fixation)
Figure 4.3: LISSOM results for a classical ME experiment. The first two bargraphs (a) and (b) have been obtained when using a phase variance covering 1
8of
the total period of the sine gratings. A green bar indicates that a green illusorycolor is seen for the corresponding orientation of the test grating (vertical or hor-izontal). A red bar indicates that a red illusory color is seen. The two other bargraphs (c) and (d) have been obtained when using a phase variance equal to the to-tal period of the sine grating, therefore simulating the induction without fixation.For the four graphs, results have been averaged over 12 experiments. The strengthof the ME is expressed as the difference between color vector magnitude obtainin LISSOM for the pre-induction and the post-induction test, using the methodpresented in section 3.4. In each case, the direction of the effect matches that fromthe psychological experiments; the strength of the effect will be examined in laterfigures.
61
−45 −30 −15 0 15 30 45 60 75 90 105 120 135−8
−6
−4
−2
0
2
4
6
8
Orientation of the test pattern
Mag
nitu
de o
f the
ME
(in th
e m
odel
)
Figure 4.4: The ME in LISSOM as a function of the angle between test and adapt-ing gratings. Full results for the dependence of the ME on the angle betweenadapting and test gratings experiments simulated with LISSOM. This plot is anaverage over 12 experiments where red was paired with horizontal and green withvertical, and 12 experiments where green was paired with horizontal and red withvertical. In each case, the plot starts at 45◦ less than the orientation that wasadapted with red. Therefore, 0 on the x-axis represents 0◦ for the first 12 experi-ments and 90◦ for the last 12. The strength of the effect is reported as the valueof the deviation from the pre-test, expressed as the difference between color vectormagnitudes obtained in LISSOM (as explained in section 3.4). A deviation towardgreen is plotted as positive, and a deviation toward red as negative. Error barsare the standard deviations.
62
−45 −30 −15 0 15 30 45−0.2
0
0.2
0.4
0.6
0.8
1
1.2
Orientation of the test pattern
Mag
nitu
de o
f the
ME
simulated MEhuman data
(a) Comparison with curve presented in section 2.2.2
0 15 30 45−0.2
0
0.2
0.4
0.6
0.8
1
1.2
Orientation of the test pattern
Mag
nitu
de o
f the
ME
simulated MEhuman data 1human data 2human data 3
(b) Comparison with the remaining data
Figure 4.5: Comparison between human data and LISSOM simulation. In (a), thesimulated ME as a function of the angle between test and adapting gratings isplotted against the data from Ellis presented in section 2.2.2. In (b) more datapoints from Ellis’s experiment have been plotted; these range from 0◦ to 45◦. Inboth plots, all the curves have been re-scaled so that their maximum was equal to1. (Data-points re-plotted from [28])
63
0 10 20 30 40 50 60 70 80 900
0.2
0.4
0.6
0.8
1
Orientation of the red grating
Mag
nitu
de o
f the
ME
constant angular divergence (90deg)decrease in angular divergenceFidell data 1Fidell data 2
Figure 4.6: Effect is independent of absolute orientation but decreases for smallangular divergence.. The red curve was obtained by changing the adapting orien-tations, always keeping a 90◦ angle between both gratings. The blue curve wasobtained by only changing the orientation of the red grating, the green grating be-ing kept constant at 90◦. The x-axis represents the orientation of the red gratingin both cases. The plot is an average over 6 different experimental conditions. Thestrength of the aftereffect is reported as an average between effects on both testgratings (oriented as the adapting ones). The corresponding experimental datapoints by Fidell have been re-plotted in dashed–lines, re-scaled so that the maxi-mum for simulated data and human data were both equal to 1. In blue are the fourdata points obtained with a constant right angle between red and green gratings(corresponding to the red curve in the model). Despite the large variability, theresults in this human study did not differ significantly from a constant value, whichagrees with the results in LISSOM. In red, there are the four data points obtainedwhen decreasing angular divergence between adapting gratings (corresponding tothe blue curve in the model). Again, despite the lack of data points in the humandata, the same breakdown as in the LISSOM simulation is observed on this plot.Error bars are the standard deviations. (Human data re-plotted from [31].)
64
−45 −30 −15 0 15 30 45 60 75 90 105 120 135−3
−2
−1
0
1
2
3
4
5
6
7
Orientation of the test pattern
Illuso
ry c
olor
per
ceive
d in
the
mod
el
(a) Single red–horizontal induction
−45 −30 −15 0 15 30 45 60 75 90 105 120 135−7
−6
−5
−4
−3
−2
−1
0
1
2
3
Orientation of the test pattern
Illuso
ry c
olor
per
ceive
d in
the
mod
el
(b) Single green–vertical induction
Figure 4.7: Indirect ME in LISSOM. These plots were produced as for Figure4.4, with the difference that induction with a single adapting grating was used inboth cases. (a) shows the results obtained with a red–horizontal adapting grating,and (b) with a green–vertical adapting grating. Therefore, the x-axis can herebe considered as the real orientation of the test gratings. An average over 12experiments is used for both plots, and the time of induction was set to half theinduction time when using a two stimulus induction (300 iterations). Error barsare the standard deviations.
65
(a) Afferent weights of the vertical unit
(b) Lateral inhibitory weightsbefore induction (colored with
color preference)
(c) Lateral inhibitory weightsafter induction (colored with
color preference)
R
G
(d)
(e) Lateral inhibitory weightsbefore induction (colored with
orientation preference)
(f) Lateral inhibitory weightsafter induction (colored with
orientation preference)
(g)
66
Figure 4.8: (Facing page) Changes in the lateral inhibitory weights distributionof a vertical–selective cell after induction with a green–vertical grating. (a) Theafferent weights of a vertical–selective achromatic model V1 cell (see figure 3.4for the plotting order of the afferent weights). (b,c,e,f) The lateral inhibitoryweights of this neuron, before (b,e) and after (c,f) induction. Each weight is colorcoded with the target unit’s hue (b,c) or orientation (e,f) preference. Weak ornon-existent connections are shown as white. Strong connections to units highlyselective for hue or orientation are brightly colored, while strong connections toweakly selective units are dark. The framed area shows the entire sheet of LISSOMV1 units, and the black circle shows the maximum possible range of connectionsto this neuron (which is located at the center of the circle). In (e), it is clear thatthis neuron starts out with strong connections to other vertical–preferring neurons(colored cyan, blue and green). Induction with a vertical stimulus further amplifiesthis tendency (f), strengthening the connections to vertical neurons and decreasingconnections to others. Before induction the strongest connections were to neuronswith only weak hue preferences (b), and the neuron connected to both types ofhue selective cells (colored red and blue). After induction with the green grating,the neuron has greatly increased its connections with green–selective cells (coloredcyan), while decreasing the connections to red–selective cells. These changes inconnection strength lead to the McCollough effect in the model, by systematicallybiasing the perception of orientation and color.
67
(a) Afferent weights of the green–vertical “double–duty” unit
(b) Lateral inhibitory weightsbefore induction (colored with
color preference)
(c) Lateral inhibitory weightsafter induction (colored with
color preference)
R
G
(d)
(e) Lateral inhibitory weightsbefore induction (colored with
orientation preference)
(f) Lateral inhibitory weightsafter induction (colored with
orientation preference)
(g)
68
Figure 4.9: (Facing page) Changes in the lateral inhibitory weights distribution ofa green-selective cell also slightly tuned to vertical, after induction with a green–vertical grating. This figure shows the same type of data with the same color codingas in figure 4.8, for a neuron primarily selective for green, but also weakly selectivefor vertical. In (b) it is clear that the neuron connects almost exclusively to othergreen–selective cells before induction. After induction, there are connections toother nearby cells (c), which are clearly vertical preferring cells (f). Thus neuronsin color blobs become more strongly connected to the orientation present duringinduction for that color, which contributes to the McCollough effect.
69
Chapter 5
Discussion and Future Work
The results presented in the previous chapter demonstrate that lateral inhibitoryinteractions between orientation and color selective units in V1 could be responsiblefor the ME, and also suggest that units tuned to both stimulus dimensions areonly playing a minor role in the emergence of the effect. This brings additionalevidence for the anatomical locus of the effect to be the early stage of color andorientation processing in the striate cortex. This interpretation of the ME thereforeimplies that the effect is not a flaw of the visual system but the consequence ofthe same self–organizing process that drives the development of an efficient sparsedistributed representation, as well as the plasticity of the adult primary visualcortex. In addition to identifying the neural substrate of the effect, this account ofthe ME provides additional support for the theoretical account due to Barlow andFoldiak [8]. Moreover, the ME simulation has allowed LISSOM to be validatedmore thoroughly as a model of adult visual function. The fact that it was successfulraises the hope of seeing more successful computational studies of phenomenalvisual experience with LISSOM.
Nevertheless, although the simulated ME have been rigorously compared tohuman experiments and shown to exhibit satisfying similarities with it, not all thepsychophysical findings concerning the ME have been studied in the simulation.This was for two reasons: the lack of time for performing long experiments, andlimitations of the current LISSOM model. Moreover, some results presented inthis thesis should be analyzed further. In particular, the mechanisms that giverise to the Indirect ME must be clearly identified. Other experiments and furtheranalysis that are needed to provide a full account of the effect will therefore bereviewed in the first part of this chapter.
Much of the work that was performed for simulating the ME with LISSOMhas also involved analysis of the color and orientation map, as well as the need toobtain a detailed hue map on the simulated V1. In a second section, the findingsrelevant to the hue map itself will be discussed, in light of the current experimental
70
knowledge, and proposed research will be directed toward the verification of whathas been predicted by the model.
5.1 Further study of the simulated ME
Even though LISSOM is primarily a structural and developmental model of theprimary visual cortex, results of this study and of the previous tilt aftereffectstudy have proven that it can be successfully used to model adult visual function.However, the ME needs to be studied further with the model in order to provide afull account of all the experimental properties of the effect. It is expected that withthe appropriate extensions, the model will be able to account for all the knowndata on the ME.
Tilt aftereffect contingent on color
The color and orientation map model used for this thesis would also be appropriatefor simulating the tilt aftereffect contingent on color. A similar comparison of theresults would be possible with the experimental data provided by Held and Shat-tuck (1971) [34]. The tilt aftereffect contingent on color can be considered as beinga “reversed” ME. Instead of testing the adaptation of the color selective neuronsto orientation, as it is done for the ME, the adaptation of the orientation selectiveneurons on color is observed here. In other words, instead of being an orientation–contingent color aftereffect, it is a color–contingent orientation aftereffect. Sinceboth the tilt aftereffect and the ME have been simulated with LISSOM, there isgood reason to expect that a study of the tilt aftereffect contingent on color willbe successful. It is believed that “double–duty” units could here play a centralrole. When presenting a chromatic grating, they respond optimally, and couldthen inhibit other orientation selective units, preferentially those that were pre-sented during adaptation, resulting in a tilt aftereffect contingent on the color ofthe grating.
Dependence of ME’s strength on the induction time
The study from Riggs, White, and Eimas (1974) reports data concerning the de-pendence of the ME on the induction time [59]. It has been found that the strengthof the effect increases with induction time. First results from the ME simulationimply that this property is also present in the model, but a more detailed studymust be performed to verify this. No data have been found on a saturation of theeffect in humans, but this phenomena occurs in the model, and it is expected thatit also occurs in humans.
71
Extinction of the effect
There are several studies concerning the extinction of the effect. They studythe duration and extinction of the ME under different stimulation conditions [43,59, 64, 62]. For instance, presenting achromatic gratings of the same orientationand spatial frequency as during the ME experiment, makes the effect disappearmore quickly [62]. It is possible to reproduce some of these experiments withLISSOM, but it would be very time–consuming, because of the long deadaptationperiods that must be simulated. However, studying the duration of the effect whenstimulating to both natural images (simulating everyday visual experience), andartificial stimuli (such as oriented gratings) is currently possible with the samemodel used in this thesis.
Binocular properties of the effect
A version of LISSOM modeling input from both eyes has already been developedand has accounted for the formation of ocular dominance maps that are very similarto those observed experimentally [53, 63]. This model can be developed further toinclude red and green dichromatic processing in the retina, as well as the ON andOFF channel of the LGN. With such a model, study of the interocular transfer ofthe effect and the binocular interactions reported by McKay and McKay (1975)[48] would be possible.
Color aftereffect contingent on spatial frequency
The ME has been reported to be dependent on the spatial frequency of the induc-tion gratings [19]; that is, using two complementary chromatic gratings of the sameorientation but different spatial frequency, a different negative color aftereffect isexperienced depending on the spatial frequency of the grating. This suggests thatthere has been a frequency–specific color adaptation during presentation of thegratings. This property can be tested with a version of the color and orientationmap model, extended to organize into a map of spatial frequency preferences [53].
Optimum parameters
It has been suggested that the ME was stronger for some values of the spatialfrequency of the gratings [65]. Similarly, the ratio between black bar’s widthand light bar’s width have been suggested to influence the strength of the effect[69]. This could be verified in the model, after having gathered more reliablepsychophysical data for this condition. Such a model could also be used to predictthe shapes of novel aftereffects not yet measured.
72
Motion–contingent color aftereffect
In parallel with this work, a simulation of the motion aftereffect in LISSOM hasbeen led by Chris Ball, a fellow Master’s student [7]. The results show that themotion aftereffect also arises in the model. This gives rise to some hope that amodel combining color, orientation and direction of motion maps can be developedin the future, and used to simulate the motion–contingent color aftereffect. In thisframework, the opposite effect could also be studied, namely the color–contingentmotion aftereffect.
Further analysis of the effect
Finally, although interesting speculation has been made on the mechanisms giv-ing rise to the Indirect McCollough Effect, sufficient evidence has not yet beenmeasured to support this interpretation. It has been suggested that inhibitoryconnections between orientation selective units further away the adaptation grat-ing (i.e., approximately 30◦ away) and the color selective units (of the inducedcolor) have decreased during the induction. This decrease would be caused bythe normalization process that keeps the total strength of all inhibitory weightsconstant. Because inhibition between the color selective units and the orientationunits that have been activated by the adaptation grating has increased, there isa decrease of inhibition between the same color selective units and all the otherorientation units that were not activated by the adapting grating (that is, unitswhose orientation tuning does not overlap with the orientation used during induc-tion). This interpretation explains the shape of the curves obtained when studyingthe dependence of the effect on the angle between test and adapting grating in thecase of a single–grating induction procedure (see section 4.2.4).
In fact, the direct effect is only observed for orientation within 30◦ angulardifference from the adapting orientation. Given that the orientation tuning inthe model is approximately 60◦, it seems logical that these orientation selectiveunits (which contain the adapted orientation in their tuning for orientation) see anincrease in inhibition from the color selective cells adapted during induction. Forall other orientations, the move toward the non-induced color is nearly constant,which agrees with the above interpretation that for all units not activated by theadapting grating, inhibition with the color selective cells (of the induced color) hasdecreased through the process of normalization. However, this interpretation needsadditional analysis to be completely justified, which would provide a completeaccount of the IME.
Finally, the small difference of curvature observed between the human dataand the simulated ME when studying the effect as a function of the angle betweentest and adapting gratings, have been found to be eliminated when working with
73
a higher phase variance. Results from this condition will be presented in a laterpublication, after further analysis.
Study with longer-range inhibitory connections
As mentioned previously, the present study has been performed with relativelyshort inhibitory connections compared to previous studies with LISSOM (eventhough they were still long-range compared to the excitatory ones). This wasdone for gaining computational efficiency, but also because the results are easilytransposable to the longer-range case. It would be of special interrest to performthe very same study with more long-range connections in the map. The inter-pretation of the results is likely to be easier: inhibitory connections will link cellsfurther away, so the increase in inhibition should be more obvious when examiningindividual neurons in detail.
5.2 Further study of the color and orientation
map
The study of the ME in LISSOM has led to further analysis of the structure andorganization of color and orientation maps. As a result, several new discoverieshave been made, sometimes constituting a prediction and sometimes corroborat-ing previous experimental findings. These observations are reported below, andshould be useful for redirecting the study of the primary visual cortex towards newexperiments.
• It has been confirmed than there were neurons tuned to both color andorientation in the model, and that they were found at the borders betweencolor “blobs” and orientation selective regions. These correlates with thediscovery of such “double–duty” units in the monkey striate cortex [37, 52].
• Further studies about how the color map relates to the orientation map re-vealed that blobs of color selective cells were generally centered on a pinwheelor, less often, a linear zone of the orientation map. This constitutes a pre-diction of the model since such relationship has not yet been conclusivelydemonstrated in animal maps.
• When trying to obtain a more detailed hue map in the model, it was foundthat color–selective cells were always classified in only two categories: red–cone selective cells or green–cone selective cells. This corroborates the studiessuggesting both that the ME is sensitive to wavelength rather than subjectivecolor [68, 71], and that neurons in V1 appears to be wavelength sensitive
74
[38]. A study of the area V2 of the visual system suggests that neuronsin V2 develop hue sensitivity and organize so that hue preference graduallyvaries to form “rainbow stripes” [73]. The present study suggests that V2would be the first cortical area to develop such a hue representation of colorstimuli. However, it is not yet clear what would be the functional significanceof having an additional stage of wavelength sensitive cells between the LGNand V2.
• Finally, the successful simulation of the ME indicates that interactions be-tween orientation and color coding mechanisms occur as soon as in V1. AsV4 has been argued to be central in the perception of color, some researchershave suggested that interaction between form and color perception starts inV4. However, results of this study suggest that the perception of color andform involves early neural mechanisms in the striate cortex.
• As specified in the thesis, training of the map has involved adding activity inthe red channel so that to obtain an evenly distributed color preference map.It was observed on this occasion that the natural images used for trainingcontained more green than red. This gives rise to the question whetherthe visual system compensates for a constant difference in the amount ofactivity in both chromatic channel, or if the bias toward green is representedin the cortex. There is also more red cones than green cones in the retina,and the ratio between them is highly variable [60]. Processes compensatingfor this difference might also compensate for the predominance of green inour environment. Numerous different methods were tried in LISSOM forobtaining balanced color maps. For instance, different set of images havebeen tried, resulting in maps that develop blobs of a single color. Suchenvironmental differences may account for some color vision deficiencies thatare still not understood [42]. In the same way, a map was produced by puttingexactly the same images in the red and green channel during training, i.e.swapping green and red channel alternately. The resulting map was perfectlybalanced, and MEs with this map were particularly close to human data.This suggests that mechanisms compensate for any imbalance early in thevisual pathway, and it would be interesting to try to find the location of suchphysical mechanisms.
The above observations are far from clarifying completely the structure andproperties of the color map and its relationship with other cortical feature maps.For instance, in order to study further the organization of color selective blobs inV1, and how they relate to other features maps, it would be of special interest todevelop a binocular and color model of the primary visual cortex, as was describedin the previous section. Experimental data are available on how color “blobs” relate
75
to the ocular dominance map [46], and it would be very interesting to observe therelationships between both maps in the model. This work would help to clarifylinks between development and the explanation of numerous visual aftereffects,providing a simple and consistent explanation for a wide variety of phenomena.
76
Chapter 6
Conclusion
This thesis constitutes the first comprehensive computational account of the ME.The simulated effect has been rigorously compared with psychophysical data andthe results have lent strong support to the theory that the ME results from Hebbianlearning of lateral inhibitory connections between orientation and color selectivecells in V1. Moreover, it has been shown in this thesis that a model of V1 thatwas primarily developed to account for the self–organization in features map, wassufficient to give a detailed account of the ME. This suggests that the processingmechanisms involved in the ME are mostly located in the primary visual cortex,even if the change in this early area of the visual system drives subsequent modi-fication in the activity pattern of higher cortical areas.
In addition to providing real progress in the understanding of the ME, thepresent work also reveals interacting properties between orientation and color cod-ing mechanisms. It was also demonstrated further that LISSOM and Hebbianlearning account for both structural self–organization in color and orientation mapand functional characteristics of adult visual perception. Furthermore, the simu-lations that have been performed in this thesis are consistent with the previousfunctional studies with LISSOM, therefore providing a single simple model of V1accounting for numerous observed cortical phenomena. The use of such a completeand well–tested model will be central for future understanding of visual processing,and eventually for future understanding of the cortex in general.
77
Bibliography
[1] L. G. Allan and S. Siegel. McCollough effects as a conditioned response:Reply to Skowbo. Psychological Bulletin, 100:388–393, 1986.
[2] L. G. Allan and S. Siegel. Characteristics of the indirect McCollough Effect.Perception and Psychophysics, 50(3):249–257, 1991.
[3] L. G. Allan and S. Siegel. Assessing a new analysis of contingent color after-effects. Cognition, 64:207–222, 1997.
[4] L. G. Allan and S. Siegel. Contingent color aftereffects: reassessing old con-clusions. Perception and Psychophysics, 59(1):207–222, 1997.
[5] B. A. Ambler and K. G. Foreit. Induction of the McCollough Effect II: Twodifferent mechanisms. Perception and Psychophysics, 24:466–470, 1978.
[6] B. Ans, J. Herault, and B. Sere. McCollough effect: A neural network modelbased on source separation. Visual Cognition, 8(6):823–841, 2001.
[7] C. Ball. Motion aftereffects in a self-organizing model of the primary visualcortex. Master’s thesis, University of Edinburgh, Edinburgh, Scotland, UK,2005.
[8] H. Barlow and P. Foldiak. Adaptation and decorrelation in the cortex. TheComputing Neuron, 1989. R. Durbin, C. Miall and G. Mitchison (Eds),Addison-Wesley, New York.
[9] H. B. Barlow. A theory about the functional role and synaptic mechanism ofvisual aftereffects. Vision: Coding and Efficiency, 1990. C. Blakemore (Ed.),Canbridge University Press, Cambridge, UK.
[10] H. B. Barlow. The knowledge used in vision and where does it come from?Philosophical Transactions of the Royal Society of London, B352, 1997.
78
[11] A. Bartels and S. Zeki. The architecture of the colour centre in the humanvisual brain: new results and a review. European Journal of Neuroscience,12:172–193, 2000.
[12] F. L. Bedford and K. S. Reinke. The McCollough Effect: Dissociating retinalfrom spatial coordinates. Perception and Psychophysics, 54(4):494–519, 1993.
[13] J. A. Bednar. Tilt aftereffects in a self-organizing model of the primary visualcortex. Master’s thesis, Department of Computer Sciences, The University ofTexas at Austin, Austin, TX, 1997. Technical Report AI-TR-97-259.
[14] J. A. Bednar. Learning to see: Genetic and Environmental influences on visualdevelopment. PhD thesis, Department of Computer Sciences, The Universityof Texas at Austin, Austin, TX, 2002. Technical Report AI-TR-02-294.
[15] J. A. Bednar and R. Miikkulainen. Tilt aftereffects in a self-organizing modelof the primary visual cortex. Neural Computation, 12:1721–1740, 2000.
[16] J. A. Bednar and R. Miikkulainen. Self-organization of spatiotemporal recep-tive fields and laterally connected direction and orientation maps. Neurocom-puting, 52-54:473–480, 2003.
[17] J. A. Bednar, J. B. De Paula, and R. Miikkulainen. Self-organization of coloropponent receptive fields and laterally connected orientation maps. Neuro-computing, 65-66:69–76, 2005.
[18] W. H. Bosking, Y. Zhang, B. Schofield, and D. Fitzpatrick. Orientationselectivity and the arrangement of horizontal connections in tree shrew striatecortex. Journal Of Neuroscience, 17(6):2112–2127, 1997.
[19] B. G. Breitmeyer and A. C. Lynn. Frequency-specific color adaptation in thehuman visual system. Perception and Psychophysics, 11(1B):95–96, 1972.
[20] J. Broerse and R. P. O’Shea. Local and global factors in spatially contingentcoloured aftereffects. Vision Research, 35(2):207–226, 1995.
[21] J. Broerse, C. Shaw, P. Dodwell, and D. Muir. Colored aftereffects contingentupon global transformations? Spatial Vision, 8:95–117, 1994.
[22] J. Broerse, T. Vladusich, and R. P. O’Shea. Colour at edges and colourspreading in McCollough effects. Vision Research, 39:1305–1320, 1999.
[23] G. Bunchsbaum and A. Gottschalk. Trichromacy, opponent colours codingand optimum colour information transmission in the retina. Proc. Royal So-ciety London, B220:89–113, 1983.
79
[24] F. W. Campbell, B. G. Cleland, G. F. Cooper, and C. Enroth-Cugell. Theangular selectivity of visual cortical cells to moving gratings. Journal of Phys-iology, 198:237–235, 1968.
[25] F. Crick and C. Koch. Are we aware of neural activity in primary visualcortex? Nature, 375:121–123, 1995.
[26] P. C. Dodwell and G. K. Humphrey. A functional theory of the McColloughEffect. Psychological Review, 97:78–89, 1990.
[27] F. H. Durgin. Visual aftereffect of texture density contingent on color offrame. Perception and Psychophysics, 58(2):207–223, 1996.
[28] S. R. Ellis. Orientation selectivity of the McCollough effect: Analysis byequivalent contrast transformation. Perception and Psychophysics, 22(6):539–544, 1977.
[29] E. Erwin, K. Obermayer, and K. Schulten. Models of orientation and occu-lar dominance columns in the visual cortex: A critical comparison. NeuralComputation, 7:997–919, 1995.
[30] O. E. Favreau, V. F. Emerson, and M. C. Corballis. Motion perception: Acolor-contingent aftereffect. Science, 176:78–79, 1972.
[31] L. S. Fidell. Orientation specificity in chromatic adaptation of human ”edge-detectors”. Perception and Psychophysics, 8(4):235–237, 1970.
[32] W. S. Geisler and D. G. Albrecht. Visual Cortex Neurons in Monkeys andCats: Detection, Discrimination, and Identification. Visual Neuroscience, 14:997–919, 1997.
[33] S. Grossberg, S. Hwang, and E. Mingolla. Thalamocortical dynamics of theMcCollough effect: boundary-surface alignment through perceptual learning.Vision Research, 42:1259–1286, 2002.
[34] R. Held and S. R. Shattuck. Color and edge-sensitive channels in the humanvisual system: Tuning for orientation. Science, 174:314–316, 1971.
[35] J. Hirsch and G. M. Murch. Variation in hue of a contour-contingent aftereffectdue to color adaptation during inspection of the stimulus patterns. Perceptionand Psychophysics, 11:406–408, 1972.
[36] J. C. Horton and D. H. Hubel. Regular patchy distribution of cytochromeoxydase staining in primary visual cortex of macaque monkey. Nature, 292:762–764, 1981.
80
[37] D. T. Hubel and T. N. Wiesel. Receptive fields and functional architecture ofmonkey striate cortex. Journal of Physiology, 195:215–243, 1968.
[38] G. K. Humphrey and M. A. Goodale. Probing unconscious visual processingwith the McCollough effect. Consciousness and cognition, 7:494–519, 1998.
[39] G. K. Humphrey, M. A. Goodale, M. Corbetta, and S. Aglioti. The McCol-lough effect reveals orientation discrimination in a case of cortical blindness.Current Biology, 5:545–551, 1995.
[40] G. K. Humphrey, R. Gurnsey, and P. J. Bryden. An examination of colour-contingent pattern aftereffects. Spatial Vision, 8(1):77–94, 1994.
[41] G. K. Humphrey, T. W. James, J. S. Gati, R. S. Menon, and M. A. Goodale.Perception of the McCollough effect correlates with activity in extrastriatecortex: A functional magnetic resonance imaging study. American Psycho-logical Society, 10(5):444–448, 1999.
[42] L. M. Hurvich. Color vision and its deficiencies. Impact of Science on Society,31(2), 1981.
[43] P. D. Jones and D. H. Holding. Frequency-specific color adaptation in the hu-man visual system. Journal of Experimental Psychology, 1(4):323–327, 1975.
[44] B. Kennedy. Chromatic chutes and ladders: how the McCollough effect mayinvolve feedback loops in the visual system. Psychology, 126:1–6, 2002.
[45] C. E. Landisman and D. Y. Ts’o. Color processing in macaque striate cortex:Electrophysiological properties. J. Neurophysics, 87(6):3138–3151, 2002.
[46] C. E. Landisman and D. Y. Ts’o. Color processing in macaque striate cortex:Relationships to ocular dominance, cytochrome oxydase, and orientation. J.Neurophysics, 87(6):3126–3137, 2002.
[47] P. Lennie. Color vision: Putting it together. Current Biology, 10:R589–R591,2000.
[48] D. M. MacKay and V. MacKay. Orientation-sensitive aftereffects of dichop-tically presented colour and form. Nature, 242:477–479, 1975.
[49] P. R. Martin. Colour through the thalamus. Clinical and Experimental Op-tometry, 87(4-5):249–257, 2004.
[50] C. McCollough. Color adaptation of edge-detectors in the human visual sys-tem. Science, 149:1115–1116, 1965.
81
[51] N. P. McLoughlin. Neural network model of 3-D surface perception: Da Vincistereopsis and the McCollough effect. PhD thesis, Boston University, 1995.
[52] C. R. Michael. Color vision mechanisms in monkey striate: simple cells withdual opponent-color receptive fields. Journal of Neurophysiology, 41:1233–1249, 1978.
[53] R. Miikkulainen, J. A. Bednar, Y. Choe, and J. Sirosh. Computational mapsin the Visual Cortex. Springer, 2005.
[54] F. S. Montalvo. A neural network model of the ME. Biological cybernetics,25:49–56, 1976.
[55] T. Morita, T. Kochiyama, T. Okada, Y. Yonekura, M. Matsumura, andN. Sadato. The neural substrates of conscious color perception demonstratedusing fMRI. NeuroImage, 21:1665–1673, 2004.
[56] H. Nakagama and S. Tanaka. Self-organization model of cytochrome oxydaseblobs and ocular dominance in the primary visual cortex. Cerebral Cortex, 14(4):376–386, 2004.
[57] B. A. Olshausen and D. J. Field. How close are we to understanding V1?Neural Computation, 2005. in press.
[58] R. C. O’Reilly and Y. Munakata. Computational Exploration in CognitiveNeuroscience. MIT Press., 2000.
[59] L. A. Riggs, K. D. White, and P. D. Eimas. Establishment and decay oforientation-contingent aftereffects of color. Perception and Psychophysics, 16:535–542, 1974.
[60] A. Roorda and D. R. Williams. The arrangement of the three cone classes inthe living human eye. Nature, 397, 1999.
[61] M. H. Rowe. Trichromatic color vision in primates. News in Physiol. Sci., 17(3):R589–R591, 2002.
[62] R. L. Savoy. Extinction of the McCollough Effect does not transfer interocu-larly. Perception and Psychophysics, 36(6):571–576, 1984.
[63] Joseph Sirosh, Risto Miikkulainen, and James A. Bednar. Self-organizationof orientation maps, lateral connections, and dynamic receptive fields in theprimary visual cortex. In Joseph Sirosh, Risto Miikkulainen, and YoonsuckChoe, editors, Lateral Interactions in the Cortex: Structure and Function.The UTCS Neural Networks Research Group, Austin, TX, 1996. Electronicbook, ISBN 0-9647060-0-8, http://nn.cs.utexas.edu/web-pubs/htmlbook96.
82
[64] D. Skowbo, B. N. Timney, T. A. Gentry, and R. B. Morant. The McCol-lough Effects: Influence of several kinds of visual stimulation on decay rate.Perception and Psychophysics, 16:47–49, 1974.
[65] D. Skowbo, B. N. Timney, T. A. Gentry, and R. B. Morant. McCollough Ef-fects: Experimental findings and theoretical accounts. Psychological Bulletin,82(4):497–510, 1975.
[66] C. F. Stromeyer. Further studies of the McCollough effect. Perception andPsychophysics, 6(2):105–110, 1969.
[67] C. F. Stromeyer and R. J. W. Mansfield. Colored aftereffects produced withmoving edges. Perception and Psychophysics, 7(2):108–114, 1970.
[68] P. Thompson and G. Latchford. Colour-contingent after-effects are reallywavelength-contingent. Nature, 320:525–526, 1986.
[69] J. J. Uhlarik and A. G. Osgood. The role of some spatial parameters ofgratings on the McCollough effect. Perception and Psychophysics, 15(3):524–528, 1974.
[70] T. Vladusich and J. Broerse. Color constancy and the functional significanceof McCollough effects. Neural Networks, 15:775–809, 2002.
[71] W. R. Webster and K. Willenberg. Orientation-contingent color aftereffectsare determined by real color, not induced color. Perception and Psychophysics,44(1):43–49, 1988.
[72] K. D. White. Luminance as a parameter in the development and testing ofthe McCollough Effects. Vision Research, 19:297–302, 1976.
[73] Y. Xiao, Y. Wang, and D. J. Felleman. A spatially organized representationof colour in macaque cortical area V2. Nature, 421, 2003.
![RBaker Thesis1] Copy](https://static.documents.pub/doc/80x56/577d21c91a28ab4e1e95e3e9/rbaker-thesis1-copy.jpg)
![Photoelectric effect [45 marks] - Peda.net](https://static.documents.pub/doc/80x56/61869499ebec7b11d64c02eb/photoelectric-eect-45-marks-pedanet.jpg)
