Modeling visual performance differences with polar angle: A ... · pass through simulated human...

Modelingvisualperformancedifferenceswithpolarangle:AcomputationalobserverapproachShorttitle:AcomputationalobservermodelofvisualperformancefieldsElineR.Kupers*,MarisaCarrasco,JonathanWinawerAuthoraffiliations:DepartmentofPsychologyandCenterforNeuralScience,NewYorkUniversity,NewYork,NewYork,UnitedStatesofAmericaKeywords:*Correspondingauthor:E-mail:[email protected]

.CC-BY-NC-ND 4.0 International licenseacertified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made available under

The copyright holder for this preprint (which was notthis version posted October 3, 2018. ; https://doi.org/10.1101/434514doi: bioRxiv preprint

https://doi.org/10.1101/434514http://creativecommons.org/licenses/by-nc-nd/4.0/

AbstractVisual performance depends on polar angle, even when eccentricity is held constant; on manypsychophysicaltasksobserversperformbestwhenstimuliarepresentedonthehorizontalmeridian,worst on the upper vertical, and intermediate on the lower vertical meridian. This variation inperformance ‘around’ the visual field can be as pronounced as that of doubling the stimuluseccentricity.Thecausesoftheseasymmetriesinperformancearelargelyunknown.Somefactorsintheeye,e.g.conedensity,arepositivelycorrelatedwiththereportedvariationsinvisualperformancewith polar angle. However, the question remains whether such correlations can quantitativelyexplain theperceptual differencesobserved ‘around’ the visual field. To investigate the extent towhich the earliest stages of vision –optical quality and cone density– contribute to performancedifferenceswithpolarangle,wecreatedacomputationalobservermodel.Themodelusestheopen-source softwarepackage ISETBIO to simulate anorientationdiscrimination task forwhichvisualperformancedifferswithpolarangle.Themodelstartsfromthephotonsemittedbyadisplay,whichpass through simulated human optics with fixational eye movements, followed by coneisomerizationsintheretina.Finally,weclassifystimulusorientationusingasupportvectormachinetolearnalinearclassifieronthephotonabsorptions.Toaccountforthe30%increaseincontrastthresholdsforupperverticalcomparedtohorizontalmeridian,asobservedpsychophysicallyonthesame task,ourcomputationalobservermodelwouldrequireeitheran increaseof~7dioptersofdefocusorareductionof500%inconedensity.Thesevaluesfarexceedtheactualvariationsasafunctionofpolarangleobservedinhumaneyes.Therefore,weconcludethatthesefactorsintheeyeonlyaccountforasmallfractionofdifferencesinvisualperformancewithpolarangle.Substantialadditionalasymmetriesmustariseinlaterretinaland/orcorticalprocessing.




AuthorSummaryAfundamentalgoalincomputationalneuroscienceistolinkknownfactsfrombiologywithbehavior.Here, we considered visual behavior, specifically the fact that people are better at visual tasksperformedtotheleftorrightofthecenterofgaze,comparedtoaboveorbelowatthesamedistancefromgaze.We sought to understandwhat aspects of biology govern this fundamental pattern invisualbehavior.Todoso,weimplementedacomputationalobservermodelthatincorporatesknownfacts about the front end of the human visual system, including optics, eyemovements, and thephotoreceptorarrayintheretina.Wefoundthateventhoughsomeofthesepropertiesarecorrelatedwithperformance,theyfallfarshortofquantitativelyexplainingit.Weconcludethatlaterstagesofprocessinginthenervoussystemgreatlyamplifysmalldifferencesinthewaytheeyesamplesthevisualworld,resultinginstrikinglydifferentperformancearoundthevisualfield.




1. Introduction

1.1 PsychophysicalperformancedifferswithvisualfieldpositionPsychophysicalperformance isnotuniformacrossthevisual field.The largestsourceof thisnon-uniformity is eccentricity: acuity ismuchhigher in the central visual field (fovea), limitingmanyrecognitiontaskssuchasreadingandfacerecognitiontoonlyarelativelysmallportionoftheretina.Asaresult,centralvisualfieldloss,suchasmaculardegeneration,canbedebilitating.Evenamodestdifference in eccentricity can have substantial effects on performance. For example, contrastthresholds on an orientation discrimination task approximately triple at 8º compared to 4ºeccentricity (1,2).Similareffectsare found forawiderangeof tasks (fora reviewonperipheralvision,see(3)).

Interestingly, visualperformancediffersnotonlyasa functionofdistance from the fovea(eccentricity), but also around the visual field (polar angle). The polar angle effects can be quitesystematic.At a fixed eccentricity, contrast sensitivity and spatial resolution arebetter along thehorizontalthantheverticalmeridian,andbetteralongthelowerthantheupperverticalmeridian.ThetwoeffectshavebeendescribedbyCarrascoandcolleaguesandcalledthe“horizontal-verticalanisotropy”andthe“verticalmeridianasymmetry”(2,4-8).Sucheffects,oftencalledperformancefields,arefoundinnumeroustasks,includingcontrastsensitivityandspatialresolution(5,6,9-23)(Fig1),visualsearch(24-31),crowding(32-34),motionperception(35),visualshort-termmemory(36),contrastappearance(7)andspatialfrequencyappearance(37).Theseeffectscanbelarge.Forexample, contrast thresholds at 8º eccentricity can be 5 times lower on the horizontalmeridiancomparedtothevertical(5),alargereffectthandoublingtheeccentricity,from4ºto8º(2).

Fig 1. Example of psychophysical task and performance differences ‘around’ the visual field. (A)Experimental design of a two-alternative forced choice (2-AFC) orientation discrimination task. Whileobserversmaintained fixation, a brief cue appeared, and after a 500ms ISI and after a Gabor stimulus (4cycles/º) was presented at one of four possible iso-eccentric locations (4.5º eccentricity). A response cueindicated to the observer the target location they were asked to make an orientation judgment on(clockwise/CWor counter-clockwise/CCWrelative tovertical).Contrastvariedacross trials.(B) Contrast-dependent psychometric functions of an example observer show best performance for the two horizontallocations (LeftandRight), andpoorestperformance foruppervertical.The increase in contrast thresholds(contrastatwhichperformancereached82%correct,dashedline)rangedfromabout2.5%(LeftandRight)to4%(Upper).Datafromfigure6inCameron,Tai&Carrasco(5).




Thecausesofperformancedifferenceswithpolaranglearenotknown.Someeyefactorsmay

contributetothemastheydotodifferencesacrosseccentricity.Forinstance,thedrop-offindensityofconesandretinalganglioncellswitheccentricitycontributestodecreasedacuity(38,39).Inthispaper, we take a modeling approach to quantify the extent to which optics and photoreceptorsamplingcontributetoperformancedifferenceswithpolarangle.

1.2 ConedensitydifferswithvisualfieldpositionInthehumaneye,conedensityvarieswitheccentricityandpolarangle.Fovealconeshaverelativelysmalldiametersandaretightlypacked,becomingsparserintheperipheryduetobothincreasedsizeandlargergapsbetweenthem(40,41).

Conedensityalsodiffersasafunctionofpolarangleatafixedeccentricity.From~2ºto7ºeccentricity,densityisabout30%greateronthehorizontalthantheverticalmeridian(40,41)(Fig2).This30%difference isaboutthesameastheconedensitydecreasefrom3ºto4ºeccentricityalongasinglemeridian.Asaresult,iso-densitycontoursareelongatedbyabout30%intheverticalaxiscomparedtohorizontal.Becauseconedensityishigheronthemeridianwhereperformanceisbetter (horizontal compared to vertical), one might be tempted to conclude that cone densityexplainstheperformancedifference.Wereturntothisquestioninsubsequentsections.

Fig2.Variationsinhumanconedensityasafunctionofeccentricityandpolarangle.Conedensityfallsoff sharplywith eccentricity, and also varies betweenhorizontal and vertical retinal axes.Data are pooledacrossbotheyes.Errorbarsrepresentstandarderroracross10observers.Arrowindicatesdifferenceinconedensitybetweenhorizontalanduppervisualmeridianat4.5ºeccentricity(matchingFig1).Figurerecomputedwithdatafromobserversbetween22-35yearsold(‘Group1’)reportedbySongetal.(41). 1.3 OpticalqualitydifferswithvisualfieldpositionBefore light hits the retina, it has already been transformed by refraction from passing throughdifferentmedia(cornea,vitreousandaqueoushumors),bydiffractionfromthepupil,aswellasbyopticalaberrations(chromaticandachromatic)ofthe lensandintraocular lightscattering(foran




overviewsee(42)).Thesetransformationsreducetheopticalqualityoftheimageprojectedontotheretina.

Opticalqualityisnotuniformacrosstheretina(43,44).Aclear,systematiceffectisthatbothdefocusandhigher-orderaberrationsbecomeworsewitheccentricity.Weassumethatmyopesandhyperopes wear corrective lenses to achieve good focus at the fovea. Defocus is the largestcontributionto imagequality(42).Theeffectsofdefocusare largest inthefarperiphery,butstillevidentinwhencomparingfoveatoparafovea(e.g.,0vs5º,Fig3).

Fig3.Variationsinopticalqualityasafunctionofvisualfieldlocationforexampleobserver.TheletterE was convolved with 5 location-specific point spread functions (PSFs) from an example observer, usingwavefront measurements of image quality and correcting for the central refractive error. The wavefrontmeasuresarebasedonapriorstudy(43),andprovidedcourtesyofPabloArtal.

Mostmeasurementsofopticalqualityinhumanareeitheratthefoveaoralongthehorizontal

meridian. These measurements show that in addition to the decline in optical quality witheccentricity,therearealsohemifieldeffects:Forexample,intheperiphery,thetemporalretinatendstohavepooreropticsthanthenasalretina(44).Therearesome(43),butmanyfewermeasurementsalong theverticalmeridian compared to thehorizontalmeridian.Toourknowledge, it isnot yetfirmlyestablishedwhethertherearesystematicdifferencesinopticalqualitybetweentheverticalandhorizontalmeridians.However,thefactthatopticalqualityvarieswitheccentricityaswellasbetweennasalandtemporalretinasuggeststhatoneshouldat leastconsideropticsasapossibleexplanatoryfactorforperformancedifferencesaroundthevisualfield.




1.4 Quantifying the contribution of components in the eye to behavioral performanceusingacomputationalobservermodel

Themeridiandifferencesinconedensityarecorrelatedwithmeridiandifferencesinpsychophysicalperformance,withhigherconedensityandbetterperformanceonthehorizontalaxis(adjustedR2=0.88,Fig4)(40,41).However,acorrelationdoesnotnecessarilyimplyanexplanation.Withoutanexplicitlinkinghypothesisormodelthatcanpredicthowadifferenceinconedensityshouldaffectvisual performance on a given task, we cannot knowwhether this correlation ismeaningful forexplainingthisbehavior.Shouldadecreaseinconedensityincreasecontrastthresholds?Andifso,shouldadecreaseofabout25%inconedensity(differencebetweenhorizontalandverticalat4.5ºeccentricity)leadtoanincreaseincontrastthresholdofabout25%,asobservedpsychophysically?

Fig4.Correlationbetweenperformanceandconedensityacrosspolarangles. Contrastthresholds(y-axis) were averaged across three observers reported in Cameron, Tai, and Carrasco (5). Thresholdswereobtained forstimuliat4.5ºeccentricity,above(gray),below(blue), left (red),andright (green)of fixation.ConedensityalongthefourmeridiansaretheaveragevaluesreportedbySongetal.(41)at4.5ºeccentricity.(Forrightandleft,conedensitywasaveragedforthenasalandtemporalretina,sinceobserversperformedthepsychophysicstaskbinocularly).Errorbars indicateonestandarderroracross10observers(conedensity)and3observers(contrastthresholds).

Answeringthesequestionsrequiresacomputationalmodel.Acomputationalmodeloftheeyecanquantifytheextentofeachcomponent’scontributiononvisualperformanceandpotentiallyrevealwhichcomponentslimitperformanceonagiventask.

In this study, we quantify the contribution of cone density and optical quality on visualperformance according to a computational observer model. We then compare the modeledcontributionstotheobservedquantities,andaskwhethertheobserveddifferencesinconedensityandopticalqualityasafunctionofpolaranglecanexplaintheobserveddifferencesinperformance.




To implement the computational observer model of the human eye, we used the ImageSystemsEngineeringToolboxforBiology(ISETBIO(45-47)),apubliclyavailabletoolbox,tosimulateencodingstagesinthefront-endofthehumanvisual(availableathttp://isetbio.org/).Weusedthismodel to simulate a 2-AFC orientation discrimination task using Gabor stimuli matched inparameters as reported by Cameron, Tai and Carrasco (5). Our computational observer modelconsistsofmultiplestagesrepresentingthefront-endofthevisualsystem:thespectralradianceoftheexperimentalvisualstimuli,opticalqualityofthelensandcornea,fixationaleyemovements,theconemosaicwithphotoreceptorsandtheirisomerizationrateforagivenstimuluspresentation.

The goal of a computational observer model is to calculate the effect of a stimulusmanipulationatparticularencodingstagesofthevisualpathway,whereeachstageisrepresentedasbiologicallyplausibleaspossibleand includingsourcesofnoise. Inourmodel,we includephotonnoise,noiseinthevisualsystemanduncertaintyaboutthedecisionwhendiscriminatingbetweentwostimulusclasses.Giventhatourcomputationalobservermodeldoesnothaveaccesstoalltheinformationofthestimuluswhenexecutingthetaskandmustlearnthestimulusclassfromnoisydata,itwillnotrepresentoptimalperformanceforthegiventask.Ourmodelisthereforedifferentfromidealobservermodels:i.e.modelsthathaveaccesstoallinformationwhenexecutingthetaskandquantifyoptimalperformanceatagivenencodingstagebecauseitisonlylimitedbybiophysicalconstraints.

With a computational observer model, one can show where along the visual pathwayinformationlosshappensandhowthislossofinformationisinheritedorpotentiallycompensatedfor in later encoding stages of the visual pathway. Here, we investigate to what extent visualperformance (contrast threshold) depends on variations in cone density and optical quality. Bysystematicallyvaryingconedensityandopticalquality(defocus)independently,wecancomparethecomputationalobservermodelperformancetoreporteddifferencesintheliteratureonthehumaneye to quantify the individual contribution of each of these two factors to differences in visualperformanceacrossthevisualfield.




2. Results

2.1 OverviewofcomputationalobservermodelToinvestigatetowhatextentperformancedifferencesinthevisualfielddependonvariationsinconedensityandopticalquality,wedevelopedacomputationalobservermodelofthefirststagesofthehuman visual pathway. The computational observerwas presentedwith oriented Gabor stimuli,tilted either clockwise or counter-clockwise from vertical, to simulate a 2-AFC orientationdiscriminationtask.Tocomparetheperformanceofthecomputationalobservertohumanobservers,wematchedthestimulusparameterstoapsychophysicsstudy(5).2.1.1SceneradianceThemodelstartsfromthephotonsemittedbyavisualdisplay,definedasthesceneradiance.Threeframesoftwoexampletime-varyingachromaticGaborstimuliareshowninthefirstpanelofFig5.TheleftcolumnshowstheresultofeachstageforaGaborwithhighcontrast(100%blackoutline)andmiddlecolumnforalowcontrastGabor(10%,orangeoutline).BothGaborstimuliareorientedclock-wise from vertical, contain a spatial frequency of 4 cycles/º and are presented at 4.5ºeccentricities.PhotonswereemittedfrompixelswhosespectralpowerdistributionsmatchedthoseofanAppleLCDdisplay.ThehighandlowcontrastGaborstimulicontainedthesamemeanradiance,but thehighcontrastGaborhasanamplitude that is10x larger than the lowcontrastGabor (1Drepresentationatdottedline,rightcolumnofpanel1).




Fig5.Overviewofcomputationsinobservermodel.Computationsareshownfora100%(blackoutline)or10% (orange outline) contrast Gabor stimulus. (1) Spectral radiance. The computation starts with thespectral radianceof a time-varying scene.The2D images shows the radiance summedacrosswavelengths(400-700 nm) during 1-ms windows. 1D slices through the images are plotted on the right. (2) Retinalirradiance.Theretinalirradianceistheimagethathaspassedthroughtheeye’soptics,beforeprocessedbytheretina.Theopticsaremodeledasatypicalhumanwavefrontwitha3-mmpupil,indicatedbytheschematicpointspreadfunction(PSF)onthelowerleft.Forillustrationpurposes,wavelengthsofthe2Drepresentationwere converted into RGB values. The yellowing is a result of the spectral filtering by the eye’s optics. (3)Isomerization.Thetime-varyingretinalirradianceistransformedintophotonabsorptionsusingarectangulargridofL-,M-andS-cones.ThedarkpixelscorrespondtoS-cones(seetextformoredetails).(4)Behavioralinference.The last stageof the computational observer is a simplifieddecision stage, performing a2-AFCorientation discrimination task. First, the absorption image at each time point was transformed into theamplitudedomainbytheFouriertransform.Theamplitudeimageswereusedtotrainalinearsupportvectormachine(SVM)classifierusing10-foldcross-validation.Theimagesshowanexampleofthetrainedclassifierweights.Theweightsarehighatlocationscorrespondingtothestimulusfrequency(4cycles/º)andorientation(±15º).Oncetrained,theclassifierpredictedtheleft-outdata.Thecross-validatedaccuracywasfitbyaWeibullfunction(rightpanel)todeterminethecontrastthreshold.

2.1.2RetinalirradianceThesecondstageofthemodelsimulatestheretinalirradiance:theresultofthetime-varyingradiancepassing through the simulatedhumanoptics (including refraction and aberrations causedby thepupil,cornea,andlens).Theretinalirradianceisthelightimagejustbeforethephotonsarecapturedbythephotopigmentintheretina.ThesecondpanelofFig5showstheretinalirradiancesummedacrossallwavelengths.TheeffectoftheopticsistoblurtheGaborstimuliandtoreducethefractionofshortwavelengthlight.Themeanirradianceisthesameforthetwostimuli.2.1.3IsomerizationThethirdstageimplementsaconemosaicandcomputesphotonabsorptionsforeachconeateachtimesample(panel3ofFig5).Theconemosaicisarectangularpatchwithafieldofviewof2x2ºat4.5º eccentricity. Each cone type absorbs apercentageof the emittedphotons, dependingon thewavelengthofthelightandtheefficiencyoftheconetype.

Themodelimplementstwosourcesofnoise.Thefirstsourcecomesfromsmallfixationaleyemovements.Theseeyemovementscauseshiftsofthestimulusontheconemosaicduringthetrial.The second noise source is from photons, which are inherently noisy and follow a Poissondistribution.

DuringtheonsetofahighcontrastGabor,theL-,M-andS-conesincreasetheirabsorptionson average by~50, ~30 and~4 photons/ms respectively. After stimulus offset, the absorptionsreturntobaselineat~110,~75and~12photons/ms.Theabsorptionratesforameanluminancescreen (~110 photons/ms for the L-cones) was validated by an independent computation ofisomerization given the luminance given byWyszecki and Stiles (48), implemented in the idealobservermodelbyGeisler(equation2,p.776,(49)),wheretheaverageL-coneabsorptionunder100cd/m2witha3-mmpupilispredictedas~108photons/ms.TheS-conesabsorbfewerphotonsthantheL-andM-cone.Thisisbecauseinertpigmentsinthelensandmaculaabsorbmorelightatshortwavelengths,andbecausethephotopigmentdensityislowerintheS-cones.Asaresult,thelocationsofS-conesinabsorptionarray(Fig5,panel3)aredark.




2.1.4BehavioralinferenceThelaststagecomputesa2-AFCinferenceofthestimulusorientationfromtheconeabsorptions.Anideal observer would have full knowledge of the response statistics (mean and distributions ofresponses of each cone at each time point from each of the possible stimuli), and uses this fullknowledgetomaketheoptimaldecisiongivenaparticularmeasurement.Itisunlikelythathumandecision-makinghas access to this full knowledge.Herewe implement a computationalobserverwhich learnspatterns from thedata, using a linear support vectormachine (SVM) classifier. Theclassifierusestheweightslearnedfromtrainingdatatoclassifythestimulusorientationoftheleft-outdata.

Becausethewithin-classstimulidifferinphaseandbecausetheeyesmoveduringthetrial,theoutputsofindividualconesarenotinformativeaboutthedecision.Hencealinearclassifiertraineddirectlyontheconeoutputswouldfail.Wethereforetransformtheconeoutputspriortotrainingtheclassifierbycomputingthe2DFouriertransformontheconearrayateachtimepoint.Weretainthe amplitudes anddiscard thephase information. Because the Fourier transform separates thephaseandamplitudeforeachspatialfrequencyandorientation,withsufficientsignaltonoiseitispossible to infer the stimulus orientation (irrespective of phase) from the amplitude spectrum.Transformingtheoutputsoftheconearrayinthiswaycanbethoughtofasgivingtheobservermodelpartial informationaboutthetask:namely,thatorientationandspatial frequency(butnotphase)mightberelevant.

Asaproofofconcept,theclassifiershowstwoexpectedpatterns.First,thelargestweightsoftheclassifierarecenteredonthepeakspatialfrequency(4cycles/º)andorientations(±15º)ofthestimuli(Fig5,panel4).Second,theclassifieraccuracyincreaseswithstimuluscontrast(Fig5,panel4, right). To summarize the data for a given simulated experiment, we computed the contrastthreshold by fitting aWeibull function to the cross-validated accuracy as a function of stimuluscontrast.Thecontrastthresholdforthecomputationalobserverwithtypicalhumanopticsandaconemosaicmatchedto~4.5ºeccentricitywas2.7%.Thisisslightlylower(thusbetterperformance)thanthresholdsreportedinthepsychophysicsexperimentwiththesamestimulusparameters(5),whichrangedfrom3.6-9%contrastforhumanobservers.

2.2 TheeffectofsmallfixationaleyemovementsinthecomputationalobservermodelOurcomputationalobservermodelincludessmallfixationaleyemovements(Fig6A).Weimplementdriftbasedona statisticalmodelbyMergenthalerandEngbert (50)andmicrosaccadesbasedonstatisticsreportedbyMartinez-Condeetal. (51,52).Thedisplacementof thestimulusduetoeyemovementsinoursimulationsisrelativelysmall:withinonetrial(asinglecoloredline),theretinaldisplacementtendstobeabout2-4conesorless(Fig6A).Thisissmallcomparedtothespatialscaleofourstimulus, forwhichafullcyclecorrespondsto~6conesat4.5ºeccentricity.Giventhatthetrialslastonly54ms,theprobabilityofamicrosaccadeislow.Hencewhenbothmicrosaccadesanddriftarepresent,eyemovementsaredominatedbydrift.




Fig6.Theeffectoffixationaleyemovementsonmodelperformance.(A)Horizontalandverticaland2Ddisplacementsinunitsofconesoftheretinalimageovertime.Colorsindicateeyemovementpathsofexample5 trials. A 1D slice of a single cycle of the Gabor stimulus is shown as the background (red box). (B)Computational observer performancewith no eyemovements (black), only drift (green) or both drift andmicrosaccades(red).Accuracyisaveragedacrossperformanceof2,000trialsperstimuluscontrast(30,000trialsintotal).Errorbarsrepresentstandarderrorofthemeanacross5simulatedexperimentswith400trialsforeachstimuluscontrast.

The fixational eye movements have a small but systematic effect on the computationalobserver model, making performance slightly worse (Fig 6B). It might be surprising that eyemovementshaveanyeffectonthemodelperformance:AnimagetranslationisequivalenttoaphaseshiftintheFourierdomainandthemodeldiscardsphaseinformation.However,becausetheretinalmosaic contains multiple cone types with different sensitivities, a shift in the stimulus causes achangeinboththeamplitudeandphasespectraoftheabsorptionimages,affectingtheinformationavailabletotheclassifier.2.3TheeffectofopticalqualityonorientationdiscriminationLarge levelsofdefocusworsenvisual acuity (53),wheredefocus levels larger than0.75diopters(corresponding to 20/40 vision on the Snellen acuity chart for near sightedness) are usuallycompensated forwith visual aids.Here,we tested the effect of defocus on the 2-AFCorientationdiscriminationtaskreportedbyCameron,TaiandCarrasco(5)toinvestigatewhethervariationsindefocuscouldexplainthedecreaseinperformancewithpolarangle.Ifthetaskisverysensitivetothelevelofdefocus,thensmalldifferencesinopticalqualityasafunctionofpolaranglemightexplaintheobserveddifferencesinperformance.

Defocusaffectsthemodulationtransferfunctionofatypicalhumanwavefrontbyattenuatingthehighfrequencies(Fig7A).TheGaborpatchesinourexperimenthadapeakspatialfrequencyof4cycles/º(dashedline).Forthisspatialfrequency,thesimulatedlevelsofdefocusintheobservermodelcauseamodestreductionincontrast.




Fig7.Theeffectofopticalqualityonmodelperformance.(A)Themodulationtransferfunction(MTF)ofourcomputationalobservermodelisshownusingdifferentlevelsofdefocus(anadditiontodiffractionandhigher order aberrations). TheMTFs are based on a typical humanwavefronts using the statisticalmodelprovidedbyThibos(54)basedondatafromThibosetal.(55).ThethreeexampleMTFsinblack,redandyellowlinesindicateadefocusZernikecoefficientof0,1or2μmrespectively(equivalentto0,3.08or6.16dioptersfora3-mmpupil).Dottedlinerepresents4cycles/º.A1Dslicethroughthestimulusshowstheeffectofthethree levelsofdefocus.(B)Classifieraccuracy forabsorption ratesasa functionof stimulus contrast for9different defocus levels for a cone mosaic simulated at 4.5º eccentricity. Accuracy is averaged acrossperformanceof2,000trialsperstimuluscontrast(30,000trialsintotal).Errorbarsrepresentstandarderrorofthemeanacross5simulatedexperimentswith400trialsforeachstimuluscontrast.(C)ContrastthresholdsfrompanelBasafunctionofdefocusindiopters,fittedwithalinearfunction.

Asexpected,largeincreasesindefocuscausethecomputationalobservermodeltoperformworse,evidencedbyarightwardshiftofthepsychometriccurve(Fig7B).Whencomparingcontrastthresholdsasafunctionofdefocuslevel,thecomputationalobservermodelshowsanapproximatelylinearrelationbetweencontrastthresholdanddefocus(R2=0.88,Fig7C).However, theeffectofdefocusonmodelperformanceissmall.Toexplainanincreaseof1.5%incontrastthreshold,similartowhatisobservedpsychophysicallyasafunctionofpolarangle,thecomputationalobservermodelwouldrequireanadditional7dioptersofdefocus.Thisisfarhigherthananyplausibledifferenceindefocusasafunctionofpolarangleat4.5º.Typically,defocusat4.5ºalongthehorizontalorverticalmeridianiswithin~0.2dioptersofdefocusatthefovea(43,44).Thedifferencebetweentheverticalandhorizontallocationsat4.5ºwouldbeevenless.Assumingadifferenceindefocusof0.2diopters,theopticalqualitywouldexplainonlyabout3%oftheeffectofvisualperformanceasafunctionofpolarangleforthistask.




2.4TheeffectofconedensityonorientationdiscriminationTheconemosaicvariessubstantiallywithretinallocation.Aseccentricityincreases,conediameterincreases,asdoesspacingbetweenthecones,resultinginlowerdensity.Weusedourcomputationalobservermodel to quantify the extent to which variations in the conemosaic could explain thechangesinperformancewithpolarangle.Wesimulatedalargerangeofconedensities,spanningarangefromabout3timeslowerto15timesgreaterthanthetypicaldensityat4.5ºeccentricity(i.e.,theretinallocationofthesimulatedpsychophysicalexperiment).Aswevariedtheconedensity,wealso varied the cone size and spacingbetween cones according to the reported relationbetweendensity and coverage (40). The denser mosaics sample the stimulus more finely, with fewerabsorptionspercone,becausetheconeareadecreasesitcaptureslessphotons.(Fig8A).

Fig 8. The effect of cone density onmodel performance. (A)Example conemosaics simulated by ourcomputationalobservermodel.Thehighesttestedconedensity(22,500cones/deg2)isequivalenttotheconedensity in the fovea (0º) whereas the lowest (466 cones/deg2) is equivalent to a cone density at 40ºeccentricity,accordingtodatafromCurcioetal.(40).Inourcomputationalobservermodel,lowerconedensityimplicitly results in larger absorption rates, because the cone area increases and therefore capturesmorephotons.(B)Classifieraccuracyforabsorptionratesasafunctionofstimuluscontrastfor13differentconedensity levels, each psychometric function is the average across performance of 2,000 trials per stimuluscontrast (30,000 trials in total). Error bars represent standard error of the mean across 5 simulatedexperimentswith400trialsforeachstimuluscontrast.(C)ContrastthresholdsfrompanelBasafunctionoflogconedensity(cones/deg2),fittedwithalog-linearfunction.

Ourcomputationalobservermodelshowsadecreaseincontrastthresholdasafunctionofconedensity(Fig8BandC).However,theeffectisrelativelysmall.Forevery5-foldincreaseinconedensity,thecomputationalmodelcontrastthresholdreducesby1percentagepoint(e.g.,from4%to




3%). Themeridional effect on human performance is~4.4% (upper verticalmeridian) vs 3.4%(horizontal).Forconedensitytoaccountforthisobserveddifferenceinhumancontrastthresholds,therewouldneedtobeabouta500%meridionaldifferenceinconedensity.Thisisfargreaterthanthe20-30%reporteddifferenceinconedensityat4.5ºeccentricity(40,41,56).Thisindicatesthat,according to our computational observermodel, cone density accounts for less than 10% of thedifferencesinvisualperformancewithpolarangleontheorientationdiscriminationtaskreportedbyCameronetal.(5).

3. Discussion

3.1 An explicit model is needed to link biological measurements with psychophysicalperformanceOurgoalwastoassessthedegreetowhichfront-endpropertiesofthevisualsystemexplainwell-established psychophysical performance differences around the visual field. In particular, wequantifiedthecontributionoftwofactorsintheeye–conedensityandopticalquality(defocus)–tocontrast thresholds measured at different polar angle in an orientation discrimination task asreportedby(5).Thesefront-endfactorshavebeenreportedtovarywithpolarangle,andinprinciple,theobservedperformancedifferencescouldbeaconsequenceofthewaythefirststagesofvisionprocess images. For instance, conedensity is higheron thehorizontalmeridian compared to theverticalmeridian(upto20ºeccentricity(40,41,56)).Nonetheless,withoutamodelto linkthesefactorstoperformance,howmuchexplanatorypowertheyhavecannotbeassessed.Wethereforedevelopedacomputationalobservermodeltotestthesepotentiallinks.Theunderlingsoftwareweused,ISETBIO,hasrecentlybeenusedtomodelanumberofbasicpsychophysicaltasks,includingcontrastsensitivity(46),Vernieracuity(57),illuminationdiscrimination(58),colorperception(59),chromaticaberration(60),visualperceptionwithretinalprosthesis(61),andspatialsummationinRicco’sarea(62).

3.2OpticsandconedensitycanexplainonlyasmallpartofperformancefieldsAlthoughconedensityalongthecardinalmeridianscorrelateswithbehavior,ourmodelshowedthatthiscorrelationhaslittleexplanatorypower:Differencesinconedensitycanonlyaccountforasmallfractionofthevariationinvisualperformanceasafunctionofmeridian.Similarly,variationinopticalqualitywithinaplausiblebiologicalrangehasonlyaverysmalleffectoncontrastthresholdsinthemodelofourtask.Ourobservermodelputsaceilingonthesetwofactorsat lessthan10%oftheobservedpsychophysicaleffects.To fullyexplain thesevisualperformancedifferenceswithpolarangle,ourcomputationalmodelwouldrequireadifferenceofmorethan7dioptersindefocusandadifference ofmore than500% in conedensity for the horizontal compared to the upper verticalmeridian.Suchlargedifferencesarefaroutsidetherangeofplausiblebiologicalvariation;defocusat4.5º eccentricity is typically within 0.1-0.2 diopters of the fovea (44) and cone density at thehorizontalmeridianis~20-30%morethantheverticalatthiseccentricity(40).




3.4DownstreamprocessingcontributestoperformancefieldsThefactthatneitheropticsnortheconesamplingarraycanexplainmorethanasmallfractionoftheeffectofpolarangleoncontrastthresholdsindicatesthatdownstreammechanismsmustexplainthemajorityofthiseffect.

A potential contributing factor is the spatial pooling of retinal ganglion cells. Likephotoreceptors,midgetretinalganglioncellssamplethevisualfieldasymmetrically.Forexample,at4.5ºeccentricity,thedensityofmidgetretinalganglioncellsonthehorizontalmeridianisreportedtobe1.4timesgreaterthanontheverticalmeridian(~1,330vs~950cells/deg2onthehorizontalvs.inferiorretina)(56,63).This40%meridionaleffectislargerthanthe20-30%effectatthelevelof the cones, indicating that polar angle asymmetries in cone density are accentuated in furtherretinalprocessing.Wehavenotincludedretinalganglioncellsinourmodel,butgivenourobservermodel with the cone array, we speculate that this furthermeridional difference in ganglion celldensitywillnotbesufficienttoexplainthereportedmeridionalpsychophysicaleffects.

A secondpotential factor is visual cortex. Someaspectsofperformance fieldsmanifest asamplitudedifferencesintheBOLDfMRIsignalinV1.Liu,HeegerandCarrasco(64)reporteda40%larger BOLD amplitude in V1 for stimuli on the lower than the upper vertical meridian. Thisasymmetry was found for high but now low spatial frequency stimuli, matching psychophysicalresults.Theydidnotreportdifferencesbetweenstimuliontheverticalversushorizontalmeridians.Performancefieldsmayalsobereflectedinthegeometryofvisualcortex.Forexample,atemplateoftheV1mapfittoapopulationof25observersshowedmorecorticalareadevotedtothehorizontalthantheverticalmeridian,althoughtheauthorsacknowledgedthatthiscouldbeafundamentalfactaboutV1oranartifactoftheflatteningprocessusedintheiranalyses(65).Thisarealdifferencehasbeenconfirmedinanindependentdataset(66).Thesedataalsoshowedthatpopulationreceptivefields(pRFs)inV1andV2are~10%smallerwhencomparinghorizontaltoverticalquadrants.Thegeometry and the pRF size effects are complementary: greater area and smaller pRFs along thehorizontalmeridianarebothconsistentwiththispartofvisualcortexanalyzingthevisualfieldingreaterdetail.However,therearealsopsychophysicaldifferencesbetweentheupperandthelowerverticalmeridian(2,6,7), forwhichnopRFdifferenceswerereported.Moreover, there isnotanexplicit model to link differences in pRF properties directly to performance on a particularpsychophysicaltask.Henceitisunknownwhetherandhowthesecorticaldifferenceswouldaccountfortheobservedbehavioralpatterns.

In addition to factors in early visual cortex, cognitive factors will also be important toconsider indevelopinga fullunderstandingofvisualperformanceacrosspolarangles.Exogenouscovertvisualattentiondoesnotcompensatefordiscriminabilitydifferencesacrosspolarangles(4-6,19),butendogenouscovertattentionmaydoso.Wearecurrentlyinvestigatingthispossibility.

3.5LimitationsofthemodelOurgoal inbuildingacomputationalobservermodelwastoexplicitly linkknownfactsaboutthebiologyofthevisualsystemwithpsychophysicalperformance.Thevalueofthemodelisevidencedby the difference in the inference one might have drawn from a purely correlational approach(performanceisbestwhereconedensityishighest)andtheinferencedrawnfromthemodel(littlerelationbetweenconedensityandperformance).Nonetheless,allmodelsaresimplifications,andoursisnoexception.




First, ourmodel containedonlyoneeye,whereasmostof thepsychophysical evidence insupportofperformancefieldscomesfrombinocularexperiments.Therehave,however,alsobeensome monocular experiments that vary stimulus polar angle, and these experiments confirmdifferencesinperformanceacrosspolarangleandshowasimilarmagnitudeoftheeffect(4,8).Hencethislimitationisunlikelytoaffectourconclusions.

Second,wemodeledtheconemosaicasarectangularpatchwithuniformdensityforeachsimulation,whereasthephotoreceptors inhumanretinaareorganizedinahexagonalgridwithagradualchangeindensityasafunctionofeccentricity.Theuniformlyspacedrectangulargridwasimplementedtosavecomputationalresources. Thedifferencebetweenaneccentricity-dependentmosaicandauniformmosaiccanbe important formodelingperformancenear the fovea(59),asdensitydeclinesrapidlyoverashortdistance(40).However,furtherintheperiphery,thedensitychanges are modest across a small patch. And given that our model showed that very largedifferencesintheconearraywereneededtoexplainvariationinpsychophysicalperformance,itisunlikelythatusingahexagonal,eccentricity-dependentarraywouldhavealteredourconclusions.

Third,wedidnotmodeldifferencesinphotopigmentdensityormacularpigmentdensityasa functionof retinalposition.Pigmentdensityhasaneffectonwavelengthsensitivityandoverallefficiency(48).Althoughourmodeldidnotvarypigmentdensity,itdidincludeposition-dependentefficiency, implemented by varying the cone coverage, which ranged from close to 1 (no gapsbetweencones)nearthefoveato~0.25inthefarperiphery.Hence,additionalvariationinefficiencyarisingfrompigmentdensitywouldbeunlikelytohaveasubstantialimpactonmodelperformance.Moreovermacularpigmentdensitydoesnotvarysystematicallywithpolarangleat iso-eccentriclocations(67).

Finally,ourcomputationalmodelonlydealswithvisualprocessesuptophotonabsorptionsby the cones. Processes up to this point, optics, photon noise, and cone sampling, are wellcharacterizedandcanbeaccuratelymodeled. In futurework,wewillbuildonourcomputationalobservermodeltoinvestigatethecontributionofdownstreamfactors,suchaspost-receptorretinalcircuitryandpoolingofsignalsbyretinalganglioncellsandvisualcortex.

3.6TheinferenceengineTheperformanceofaclassifierdepends,inpart,onhowmuchknowledgeofthetasktheclassifierhas access to. Our observer model had far less information than an ideal observer model. Bydefinition, the idealobservermodelhascompleteknowledgeabout thestimuliandsetsanupperlimitonperformance(68,69).Whenidealobservermodelsareappliedtoveryearlysignalsinthevisualsystemsuchasconeresponses,theytypicallyoutperformhumanobserversbyalargemargin,e.g.,byafactorof10ormore(49,70).Theincompleteknowledgeinourobservermodelledtopoorerperformancethanwouldbeobtainedbyanidealobservermodel,andsimilarperformancetohumanobservers(~2-4%contrastthresholdsinourtask).Moreover,althoughsomeobserversmayhavesome (explicit or implicit) knowledgeofperformancedifferencesacrosseccentricity, theydonothaveknowledgeofdifferencesacrossiso-eccentriclocations.Inthetaskreportedby(5),twooftheobserverswere theauthors (and trainedpsychophysicalobservers)andoneobserverwasnaïve.Thissuggeststhatknowledgeofthephenomenondoesnotalterthepatternofperformance. Our inferenceenginehas two typesofknowledgeabout the task,onemoregeneralaboutvisualprocessingandonemorespecifictotheparticularexperimentwesimulated.Thegeneral(and




implicit) knowledge arises from transforming the 2D time-varying cone absorption images toamplitudespectra.Transformingthedatainthiswayeffectivelygivestheobservermodelknowledgethatspatialfrequencyandorientation(butnotphase)mightberelevantforthetask.Thistransformdoesnotindicatewhichspatialfrequenciesororientationsarerelevant.AlthoughthevisualsystemdoesnotliterallycomputeaFouriertransformoftheconeresponses,cellsinvisualcortexaretunedtoorientationandspatialfrequencyinlocalpatchesoftheimage(71,72),andcomplexcellsinV1arerelativelyinsensitivetophase(73,74).Hencetheimplicitknowledgeweprovidetotheclassifierviatransformtotheamplitudespectraisanapproximationtogeneralprocessingstrategiesinthevisualsystem,ratherthanspecificknowledgeaboutourparticulartask.Pilotsimulationsinwhichtheclassifieroperateddirectlyontheabsorptionimagesresultedinnear-chanceperformance.Thisisexpected,becausethephaserandomizationofthestimulicausesthenumberofabsorptionsforanyparticularconetobeuninformativeastothestimulusorientation.

Morespecificknowledgeinthecomputationalobservermodelcomesfromthetrainingtrials,whichareusedtolearnthebestlinearseparation(hyperplane)betweenthetwoclasses.Theplaneisdefinedbyaweightedsumoftheclassifierinputs(amplitudespectrainourcase),whichcanbethoughtofasanapproximationtoreceptivefieldanalysisbydownstreamneurons.Thehighweightslearned by the classifier for this task correspond to oriented, band-pass filters, which matchpropertiesofthestimuli(Fig5,panel4).Becausethemodelhasincompleteknowledge,valuesfarfromthestimulus(veryhighorverylowspatialfrequency,andorientationsfarfromthestimulusorientations)havenon-zeroweights,whicharelearnedduringtrainingonafinitenumberofnoisytrials.

ConclusionOverall,ourmodelincludesarelativelydetailed,biologicallyplausiblefront-end,whichincorporatesrealisticdetailsabouttheoptics,photonnoise,smallfixationaleyemovements,andwavelength-andposition-sampling by photoreceptors. This front-end processing was combined with a linearclassifier that performs at levels comparable to thehumanwithoutproviding explicit knowledgeabout the tasks. Future work will incorporate more biologically explicit models of downstreamprocessing, including retinal and cortical circuitry. Such models are likely to reveal that laterprocessinginthenervoussystemamplifiesasymmetriesinprocessingaroundthevisualfieldthatbegin in the earliest stages of vision, and thus, to explain a larger portion of the psychophysicalasymmetriesfoundinmanyvisualtasks.




4. Methods4.1 ComputationalobservermodelsoftwareoverviewThecomputationalobservermodelreliesonthepubliclyavailable,MATLAB-basedImageSystemsEngineeringToolbox forBiology (ISETBIO (45-47)), available at http://isetbio.org/.The ISETBIOtoolboxincorporatestheimageformationprocess,wavelength-dependentfiltering,opticalquality,andthespatialarrangementandbiophysicalpropertiesofcones.WeusedtheISETBIOtoolboxforthecoremodelarchitectureandsupplemented itwithexperiment-specificcustomMATLABcode.Theexperiment-specificcodeimplementsstimulusparametersmatchedtoapriorpsychophysicalstudy(5),manipulationofbiologicalparameterstoassesstheirimpactonperformance,anda2-AFClinearsupportvectormachineclassifier.Intheinterestofreproduciblecomputationalmethods,theexperiment-specific code, for both simulation and analysis, is publicly available via GitHub(http://github.com/isetbio/JWLOrientedGabor). In addition, the data structures created by thesimulation and analyses are permanently archived on the Open Science Framework URL:https://osf.io/mygvu/.

4.2 PsychophysicalexperimentOursimulationswerecreatedtomatchapreviouspsychophysicalstudy(5).Inthatstudy,stimuliwereachromaticorientedGaborpatches.TheGaborswerecomprisedofharmonicsof4cycles/º,windowedbyaGaussianwithastandarddeviationof0.5º,presentedat4.5ºeccentricity,atoneof8locationsequallyspacedaroundthevisualfield(seealsoFig1A).Gaborpatchesweretiltedeither15ºclockwiseorcounter-clockwisefromvertical,andpresentedfor54msoneachtrial.ThecontrastoftheGaborpatchesvariedfromtrialtotrial.Thecontrastlevelswereselectedforeachobserverbasedonpre-experimenttesting,andusuallyrangedfromabout1%to10%Michelsoncontrastusinga method of constant stimuli. The observer’s task was to indicate the orientation of the Gaborstimulus relative to vertical (clockwise or counter-clockwise) with a button press. Data wereanalyzedbyfittingaWeibullfunctiontothemeanperformance(%correct)ateachcontrastlevel,independentlyatdifferentlocationsaroundthevisualfield.

4.3 Stimuli(scenespectralradiance)Theobservermodelstartswithadescriptionofthestimulus,calleda‘scene’inISETBIO.Thesceneisdefinedbythespectralradianceateachlocationinspaceandtime(the‘lightfield’).Thespectralradiancecontainedwavelengthsrangingfrom400-700nm,discretizedto10nmsteps,withequalphotonsateachwavelength(3.8x1015quanta/s/sr/nm/m2).Thestimuluswasdiscretizedinto2-mstime steps and1.8-arcminute spatial steps (32 samples per degree). The scene comprisedGaborstimuli with parameters described above (section 4.1.1), oriented either clockwise or counter-clockwise,representedwithinafieldofviewof2ºdiameter,andpresentedfor54mspertrial.Thedimensionsofthesceneweretherefore64x64x31x28(heightxwidthxwavelengthxtime).GaborpatchesvariedinMichelsoncontrastbetween0.05%and10%.Wealsoincorporatedastimuluswith0%contraststimulusasasanitycheckwhetherourmodelwouldperformatchancelevel.Forallstimuli, themean luminancewas100 cd/m2.Becausephotonnoise andeyemovementnoise areaddedlater(seesections4.14and4.15),andbecausewedonotmodelthescenebeforeorafterthestimulusonset/offset,thesceneisinfactidenticalatall28timepoints.




Machinelearningalgorithmscanexploitsourcesofinformationthatahumanobserverwouldbeunlikelytouse.Forexample,ifthevalueofasingleimagepixelhappenedtocorrelatewiththestimulusclass,aclassifiercouldsucceedbasedononlythevalueofthispixel.Wewantedtopreventourclassifierfromsucceedinginthisway.Inoursimulations(unliketheCameronetal.paper(5)),thephaseoftheGaborpatcheswasselectedfromtwovalues180ºapart(φ=90ºandφ=270º),randomizedacrosstrials.A180ºphasedifferencemeansthatthetwopossiblestimuliwithinaclasswereidenticalexceptforasignreversal.Asaresult,theexpectedvalueofeachpixelineachstimulusclasswas0(relativetothebackground).Similarly,theexpectedvalueoftheconeabsorptionratesateachlocationontheretinawithinastimulusclasswas0(relativetothebackground).Therefore,thelinearclassifiercouldnotsucceedusingtheabsorptionlevelfromanysinglecone.Webelievehumanobserversdonotperformthetaskthiswayeither,hencerandomizingthephaseislikelytomaketheobserverperformancemoresimilartothehumanperformance.

4.4 Optics(retinalirradiance)The optics transform the scene into a retinal image. We first describe the optics used for thesimulationsinsections2.2and2.4(Fig6andFig8).Forthesesimulations,theopticswerematchedtoatypicalhumaneyewitha3-mmpupil(diameter)infocusat550nmusingastatisticalmodelofwavefrontaberrations(54).Thisstatisticalmodelisbasedonmeasurementsfromhealthyeyesof100observers(55),anddescribedbyabasissetofZernikepolynomials(75).ThestatisticalmodelbyThibos contained the first 15Zernike coefficients (Z0-Z14, usingOSA standard indexing).Thesimulated humanwavefront was used to construct a point spread function (PSF). This PSFwasconvolvedwiththesceneateverytimepointtogeneratetheretinalimage.Afterthisspatialblurring,theopticalimagewasfurthertransformedbyspectralfiltering(lightabsorptionbyinertpigmentsinthe lens andmacula),whichprimarily reduce the intensityof short-wavelength light. Finally, theopticalimageswerepaddedby0.25ºoneachsidewiththemeanintensityateachwavelength.Thepadding isneededtohandleeyemovements,sothatconesneartheedgeof thesimulatedretinalpatchhaveadefined input evenwhen these conesaremovedoutside the sceneboundaries. Thedimensionsoftheopticalimagearethesameasthedimensionsofthescene,exceptforthespatialpadding:80x80x31x28(heightxwidthxwavelengthxtime),whichwasdiscretizedthesamewayasthescene.

To investigate the effect of optical quality on visual performance of our task, wesystematicallyaddedfurtherdefocustothemodelofhumanoptics(section2.3,Fig7).WedidthisbyincreasingtheZ4Zernikecoefficient(defocus)from0-2μminstepsof0.25μm(correspondingto0-6.16dioptersfora3-mmpupil),whilekeepingallotherZernikecoefficientsfromThibos’statisticalmodelunchanged.Note thatusingadefocus coefficientof0doesnot result inperfectdiffractionlimitedoptics,given that theotheraberrationsarestillnon-zero.Wemanipulateddefocusratherthan all the higher-order aberrations because at the stimulus eccentricity we simulated (4.5º),defocusisthelargestcontributortoopticalquality(44).

4.5 Conemosaic:spatialsamplingandisomerizationWeconstructedtheconemosaicasauniformlyspacedrectangularpatchwithafieldofviewmatchedtothestimulus(2x2º).EachconemosaiccontainedarandomdistributionofL-,M-andS-coneswitharatioof0.6:0.3:0.1.Weused theStockman-Sharp(76) functions toestimateconephotopigment




spectralsensitivity,assuming50%opticaldensity forL-andM-cones,and40%forS-cones.Peakefficiencywasassumedtobeequalforeachconeclass,66.67%multipliedbytheretinalcoverage(thefractionoflocalretinaoccupiedbycones).

For the simulations in sections2.2and2.3 (Fig6 andFig8), the conedensitywas1,560cells/deg2, approximatelymatched to the density at 4.5º on the horizontal retina as reported byCurcioetal.(40).Thisresultsinanarrayof79x79conesforour2ºpatch.ThepositionsoftheL-,M-,andS-coneswererandomizedwithinthearray(butheldtofixedratio).Forthesesimulations,weassumedacoverageproportionof0.49,meaningthattheconeinnersegmentssampledfromabouthalfoftheopticalimage,andmissedabouthalfduetothespacesbetweencones.Acoverageoflessthan1actslikeareductioninefficiency,sincephotonsarelosttothegapsbetweencones.Ingeneral,conecoveragedecreaseswitheccentricityasthedensityofrodsincreases,fillingthespacesbetweencones.

Inonesetofexperiments,wesystematicallyvariedconedensity(Resultssection2.4,Fig8),spanning22,500to466cones/deg2(correspondingtoconearraysrangingfrom297x297to43x43).Foreachconedensity,wedeterminedanequivalenteccentricitybasedontherelationbetweeneccentricityanddensityonthenasalmeridian fromCurcioetal. (40).Wethenadjustedtheconecoverageaccordingtothiseccentricity,assumingthatcoveragedeclinesexponentiallyasafunctionofeccentricity,from1(fovea,nogapsbetweencones)to0.25at40º.ThisapproximationissimilartothatusedbyBanksetal.(49),whichwasbasedondatafromCurcioetal.(40). Thenumberof absorptionswas computed foreach cone in twosteps.First, thenoiselessnumberofabsorptionswascomputedbymultiplyingtheappropriateconesensitivityfunction(L,M,orS)bythecorrespondinglocationintheopticalimage(hyperspectral),andscalingthisvaluebythepeakefficiency(66.67%).Theconecoveragewasaccountedforbyonlysamplingtheopticalimageatthelocationswithintheconeinnersegments.Second,thenoiselessvalueswereconvertedtonoisysamplesbyassumingaPoissondistribution.

Thedimensionsoftheconearrayabsorptionswere79x79x28(rowsxcolumnsxtime)forthesimulationsinsections2.2and2.3(Fig6andFig8).Whentheconedensityvaried(section2.4,Fig7),thefirsttwodimensionsoftheconearraysizealsochanged.

4.6 EyemovementsWe added small fixational eye movements (drift and microsaccades), before computing theisomerization rate for each cone at each time sample. The ISETBIO toolbox provides code thatgenerates eye movement samples based on a Mergenthaler and Engbert’s drift model (50) andmicrosaccadestatisticsreportedbyMartinez-Condeetal.(51,52). Thedriftmodel computes eyemovementpaths for a single trialwithmodifiedBrownianmotionprocess.Theeyemovementpathsweregeneratedinunitsofarcminutesandthenconvertedtodiscreteconeshiftsinthehorizontalandverticaldirection.Iftheamplitudeofaneyemovementwassmallerthanthedistancebetweentwocones,thedisplacementwasaccumulatedovermultipletimesamples,untilthethresholdwasreached,beforeanewshiftwasaddedtotheeyemovementpath.

Thedriftmodelwasimplementedbyaddingadisplacementvectortothecurrentpositionateachtimepoint.Thedisplacementvectorwasdeterminedbycombining3inputs:2DGaussiannoise,anautoregressivetermforpersistentdynamicsatshorttimescales,andadelayednegativefeedback




forantipersistentdynamicsatlongertimescales.TheparametersweusedforthismodelweretheISETIOdefaults,whichcontainedahorizontalandverticaldelaydefinedasX=0.07sandY=0.04s,feedback steepness of 1.1, and feedback gain of 0.15. The control function had amean of 0 andstandarddeviationof0.075andthegammaparameterwassetto0.25.Themeannoisepositionandstandard deviationwere set to 0 and 0.35, respectively. Before computing the velocity of a driftperiod,thedriftmodelappliedatemporalsmoothingfiltertotheeyemovementpathsusinga3rdorderSavitzky-Golayfilteroveravelocityintervalof41ms. Forperiodswherethedriftwasstabilized,theeyemovementcodecheckedformicrosaccadejumpstotheeyemovementpath.Whetherornotamicrosaccadewasaddeddependedonwhenthelastmicrosaccadewas.Inourexperiment,weusedtheISETBIOdefaultwheretheintervalbetweenmicrosaccadesfollowedagammafunctionwithameanof450ms,withaminimumdurationof2ms.Amicrosaccadewasdefinedasavectorwithameanamplitudeofamicrosaccadewas8arcminutes.Eachvectorcontainedanadditionalendpointjitterof0.3arcmininlengthand15ºindirection.Themicrosaccade jumpswereeither ‘corrective’ (towards thecenterof themosaic)or ‘random’ (anydirection).Themicrosaccademeanspeedwasdefinedas39º/s,withastandarddeviationof2º/s.Giventhatthedefinedintervalbetweenmicrosaccadeswaslongcomparedtothestimulusduration(54ms),mosttrialsdidnotcontainmicrosaccades. A216mswarmupperiodwasimplementedbeforethetrialsbegan.Eyemovementsduringthisperiodaffectedtheeyepositionatthestartofthetrialbutwerenototherwiseincludedintheanalysis.

4.7 SimulatedExperimentsandBehavioralInferenceAsimulatedexperimentcomprised6,000trials,with400trialsateachof15contrasts.The400trialsper contrast level included 200 clockwise and 200 counterclockwise stimuli, each of whichwasfurthersubdividedinto100trialsateachof2phases.Thedatafromasinglecontrastlevelwithinasingleexperimentwererepresentedasa4Darray(mrowsxmcolumnsx28time-pointsx400trials),inwhichmisthenumberofconesalongonesideoftheretinalpatch(79intheexperimentsforFig6andFig7,variablefortheexperimentsinFig8).

Withinthe6,000trialsofanexperiment,allparametersotherthanthestimulusorientation(clockwiseorcounterclockwise)andphase(90ºor270º)wereheldconstant,includingthespatialdistributionofL-,M-,andS-cones,theoptics,theconedensity,theconecoverage,andthepresenceorabsenceoffixationaleyemovements.Eachsimulatedexperimentwasrepeated5times,sothatasinglepsychometricfunctionsummarized30,000trials.ThearrangementofL-,M-,andS-coneswasregeneratedrandomlyforeachofthe5repeatedexperiments.ErrorbarsinFig6-8indicatestandarderrorsofthemeanacrossthe5experiments. Classification (clockwise vs. counter-clockwise) via cross-validation was performedseparately foreachstimuluscontrast level(setof400trials) ineachexperimentas follows.First,eachmxmimageofconeabsorptionswastransformedintoanmxmamplitudespectrumusingthe2D fast Fourier transform and discarding the phase information. This left the dimensionalityunchanged(mrowsxmcolumnsx28time-pointsx400trialswithinasinglecontrastlevel).Second,theamplitudeswereconcatenatedacrossspaceandtimeintoa2Dmatrix(400trialsx28m2valuesper trial). A 400-element vector labeled the trials by stimulus class (1 for clockwise and -1 forcounter-clockwise).Third,the2Dmatrixand400-elementvectorwithlabelswasusedfortraining




and testing a linear support vector machine (SVM) classifier on the amplitude images usingMATLAB’s fitcsvm with 10-fold cross-validation, kernel function set to ‘linear’, and the built-instandardization option (to z-score each row of the data matrix). The learned classifier weightsrepresentedthebestlinearseparation(hyperplane)betweenthetwostimulusclasses.Withthesetrainedweights,theclassifierpredictedthestimulusclasslabelfortheleftouttrialsinagivendatafold.WeusedMATLAB’skfoldLossfunctiontoaveragetheaccuracyacrossthe10-folds,whichyieldedoneaccuracymeasure(%correct)percontrastlevelperexperiment.

4.8 Quantifying the contribution of cone density and optics on the computationalobserverperformanceToquantifythecontributionofagivenfactorintheeye,weaveragedtheclassifieraccuracyforeachstimuluscontrast levelacross the5experiments to fitwithaWeibull function (Equation1).Thisresultedinfullpsychometricfunctionsforeachconedensityandopticalqualitylevel.Wecalculatederrorbarsforeachcontrastlevelasthestandarderrorofthemeanacrossthe5iterations.Foreachpsychometric function,wedefined thecontrast thresholdas thepowerof1over theslopeof theWeibullfunctionb,inourcaseb=3,oftheperformancelevelexpectedatchance(0.5fora2-AFCtask,seeEquation2).Thisresultsinathresholdof~80%(0.51/3=0.7937).

Equation1: ! = 1 − 1 − % ∙ e-)*+

,

Wheregistheperformanceexpectedatchance(0.5),tisthethreshold,bistheslopeoftheWeibullfunction,andkisdefinedas:

Equation2: ! = - log '()'(*+,

Thecontrast thresholds, t,were summarizedasa linear functionofdefocus (Fig7)oran

exponentialfunctionofconedensity(Fig8,representedasastraight-lineonasemi-logaxis).Theselinearorlog-linearfitsenabledustocomputethechangeinconedensityorthechangeindefocusneededtoachievea1%increaseincontrastthreshold–similartothemeridionaleffectobservedinhumanperformance(~4.4%attheupperverticalmeridianvs~3.4%atthehorizontalmeridianasseenin(5)).




AcknowledgmentsThis research was supported by the US National Eye Institute R01-EY027401 (M.C. and J.W.).Wewould like to thankBrianWandell,DavidBrainardandNicholasCottaris for theiradviceandencouragement.WethankPabloArtalforkindlyprovidingdataonhumanopticalquality.




References1. RovamoJ,VirsuV,NasanenR.Corticalmagnificationfactorpredicts thephotopiccontrastsensitivityofperipheralvision.Nature.1978;271(5640):54-6.

2. AbramsJ,NizamA,CarrascoM.Isoeccentriclocationsarenotequivalent:theextentoftheverticalmeridianasymmetry.VisionRes.2012;52(1):70-8.

3. StrasburgerH,RentschlerI,JuttnerM.Peripheralvisionandpatternrecognition:areview.JVis.2011;11(5):13.

4. Carrasco M, Talgar CP, Cameron EL. Characterising visual performance fields: effects oftransientcovertattention,spatialfrequency,eccentricity,taskandsetsize.SpatVis.2001;15(1):61-75.

5. CameronEL,TaiJC,CarrascoM.Covertattentionaffectsthepsychometricfunctionofcontrastsensitivity.VisionRes.2002;42:949-67.

6. Talgar CP, Carrasco M. Vertical meridian asymmetry in spatial resolution: visual andattentionalfactors.PsychonBullRev.2002;9(4):714-22.

7. FullerS,RodriguezRZ,CarrascoM.Apparentcontrastdiffersacrosstheverticalmeridian:visualandattentionalfactors.JVis.2008;8(1):161-.

8. Corbett JE, Carrasco M. Visual performance fields: frames of reference. PLoS One.2011;6(9):e24470.

9. RovamoJ,VirsuV.Anestimationandapplicationofthehumancorticalmagnificationfactor.Experimentalbrainresearch.1979;37:495-510.

10. VirsuV,RovamoJ.Visualresolution,contrastsensitivity,andthecorticalmagnificationfactor.Experimentalbrainresearch.1979;37(3):475-94.

11. RijsdijkJP,KroonJN,vanderWildtGJ.Contrastsensitivityasafunctionofpositiononretina.VisionRes.1980;20:235-41.

12. RobsonJG,GrahamN.Probabilitysummationandregionalvariationincontrastsensitivityacrossthevisualfield.VisionRes.1981;21(3):409-18.

13. LundhBL,LennerstrandG,DerefeldtG.Centralandperipheralnormalcontrastsensitivityforstaticanddynamicsinusoidalgratings.ActaOphthalmol(Copenh).1983;61(2):171-82.

14. ReganD,BeverleyKI.Visualfieldsforfrontalplanemotionandforchangingsize.VisionRes.1983;23(7):673-6.

15. Skrandies W. Human contrast sensitivity: regional retinal differences. Hum Neurobiol.1985;4(2):97-9.

16. Anderson RS,WilkinsonMO, Thibos LN. Psychophysical localization of the human visualstreak.OptomVisSci.1992;69(3):171-4.




17. Mackeben M. Sustained focal attention and peripheral letter recognition. Spat Vis.1999;12(1):51-72.

18. AltpeterE,MackebenM,Trauzettel-KlosinskiS.The importanceof sustainedattention forpatientswithmaculopathies.VisionRes.2000;40(10-12):1539-47.

19. CarrascoM,WilliamsPE,YeshurunY.Covertattentionincreasesspatialresolutionwithorwithoutmasks:supportforsignalenhancement.JVis.2002;2(6):467-79.

20. SeipleW, Holopigian K, Szlyk JP,Wu C.Multidimensional visual fieldmaps: relationshipsamong local psychophysical and local electrophysiological measures. J Rehabil Res Dev.2004;41(3A):359-72.

21. SilvaMF,Maia-LopesS,MateusC,GuerreiroM,SampaioJ,FariaP,etal.Retinalandcorticalpatternsofspatialanisotropyincontrastsensitivitytasks.VisionRes.2008;48(1):127-35.

22. Silva MF, Mateus C, Reis A, Nunes S, Fonseca P, Castelo-Branco M. Asymmetry of visualsensory mechanisms: electrophysiological, structural, and psychophysical evidences. J Vis.2010;10(6):26.

23. SilvaMF,d'AlmeidaOC,OliveirosB,MateusC,Castelo-BrancoM.Developmentandagingofvisualhemifieldasymmetriesincontrastsensitivity.JVis.2014;14(12).

24. CarrascoM,GiordanoAM,McElreeB.Temporalperformance fields:visualandattentionalfactors.VisionRes.2004;44(12):1351-65.

25. Chaikin JD, Corbin HH, Volkmann J. Mapping a field of short-time visual search. Science.1962;138(3547):1327-8.

26. KristjanssonA,SigurdardottirHM.On thebenefitsof transientattentionacross thevisualfield.Perception.2008;37(5):747-64.

27. KroseBJ,JuleszB.Thecontrolandspeedofshiftsofattention.VisionRes.1989;29(11):1607-19.

28. NajemnikJ,GeislerWS.Eyemovementstatisticsinhumansareconsistentwithanoptimalsearchstrategy.JVis.2008;8(3):41-14.

29. NajemnikJ,GeislerWS.Simplesummationruleforoptimalfixationselectioninvisualsearch.VisionRes.2009;49(10):1286-94.

30. Pretorius LL, Hanekom JJ. An accurate method for determining the conspicuity areaassociatedwithvisualtargets.HumFactors.2006;48(4):774-84.

31. RezecAA,DobkinsKR.Attentionalweighting:apossibleaccountofvisualfieldasymmetriesinvisualsearch?SpatVis.2004;17(4-5):269-93.

32. Fortenbaugh FC, Silver MA, Robertson LC. Individual differences in visual field shapemodulatetheeffectsofattentiononthelowervisualfieldadvantageincrowding.JVis.2015;15(2).




33. He S, Cavanagh P, Intriligator J. Attentional resolution and the locus of visual awareness.Nature.1996;383(6598):334-7.

34. ToetA,LeviDM.Thetwo-dimensionalshapeofspatial interactionzones in theparafovea.VisionRes.1992;32(7):1349-57.

35. FullerS,CarrascoM.Perceptualconsequencesofvisualperformancefields:thecaseofthelinemotionillusion.JVis.2009;9(4):131-7.

36. Montaser-Kouhsari L, Carrasco M. Perceptual asymmetries are preserved in short-termmemorytasks.Attention,perception&psychophysics.2009;71(8):1782-92.

37. Edgar GK, Smith AT. Hemifield differences in perceived spatial frequency. Perception.1990;19(6):759-66.

38. RodieckRW.TheFirstStepsinSeeing.Oxford:OxfordUniversityPress;1998.

39. WandellBA.FoundationsofVision:SinauerAssociates;1995.

40. Curcio CA, Sloan KR, Kalina RE, Hendrickson AE. Human photoreceptor topography. TheJournalofcomparativeneurology.1990;292:497-523.

41. Song H, Chui TY, Zhong Z, Elsner AE, Burns SA. Variation of cone photoreceptor packingdensitywithretinaleccentricityandage.InvestOphthalmolVisSci.2011;52(10):7376-84.

42. ArtalP.ImageFormationintheLivingHumanEye.AnnuRevVisSci.2015;1:1-17.

43. PolansJ,JaekenB,McNabbRP,ArtalP,IzattJA.Wide-fieldopticalmodelofthehumaneyewithasymmetricallytiltedanddecenteredlensthatreproducesmeasuredocularaberrations.Optica.2015;2(2):124.

44. JaekenB,ArtalP.Opticalqualityofemmetropicandmyopiceyesintheperipherymeasuredwithhigh-angularresolution.InvestOphthalmolVisSci.2012;53(7):3405-13.

45. FarrellJE,WinawerJ,BrainardDH,WandellB.27.2:DistinguishedPaper:Modelingvisibledifferences:Thecomputationalobservermodel.SIDSymposiumDigestofTechnicalPapers2014.p.352-6.

46. CottarisN,JiangH,DingX,WandellB,BrainardDH.Acomputationalobservermodelofspatialcontrastsensitivity:Effectsofwavefront-basedoptics,conemosaicstructure,andinferenceengine.bioRxiv.2018.

47. BrainardDH,JiangH,CottarisNP,RiekeF,ChichilniskyEJ,FarrellJE,etal.,editors.ISETBIO:Computational tools for modeling early human vision. Imaging and Applied Optics 2015; 20152015/06/07;Arlington,Virginia:OpticalSocietyofAmerica.

48. WyszeckiG,StilesWS.Colorscience.NewYork:Wiley;1982.

49. Banks MS, Sekuler AB, Anderson SJ. Peripheral spatial vision: limits imposed by optics,photoreceptors,andreceptorpooling.JOptSocAmA.1991;8(11):1775-87.




50. Mergenthaler K, Engbert R. Modeling the control of fixational eye movements withneurophysiologicaldelays.PhysRevLett.2007;98(13):138104.

51. Martinez-Conde S,Macknik SL, Hubel DH. The role of fixational eyemovements in visualperception.NaturereviewsNeuroscience.2004;5(3):229-40.

52. Martinez-CondeS,MacknikSL,TroncosoXG,HubelDH.Microsaccades:aneurophysiologicalanalysis.TrendsNeurosci.2009;32(9):463-75.

53. VillegasEA,AlconE,ArtalP.Opticalqualityoftheeyeinsubjectswithnormalandexcellentvisualacuity.InvestOphthalmolVisSci.2008;49(10):4688-96.

54. ThibosLN.Retinalimagequalityforvirtualeyesgeneratedbyastatisticalmodelofocularwavefrontaberrations.OphthalmicPhysiolOpt.2009;29(3):288-91.

55. ThibosLN,HongX,BradleyA,ChengX.Statisticalvariationofaberrationstructureandimagequalityinanormalpopulationofhealthyeyes.JOptSocAmA.2002;19(12):2329-48.

56. CurcioCA,SloanKR,PackerO,HendricksonAE,KalinaRE.Distributionofconesinhumanandmonkeyretina:Individualvariabilityandradialasymmetry.Science.1987;236(4801):579-82.

57. Jiang H, Cottaris N, Golden J, Brainard D, Farrell J,Wandell B, editors. Simulating retinalencoding: facors influencingvernieracuity.ProceedingsofElectronic Imaging;2017;Burlingame,CA.

58. DingX,RadonjićA,CottarisNP, JiangH,WandellBA,BrainardD.Computational-Observeranalysisofilluminationdiscrimination.2018.

59. BrainardDH,CottarisNP,RadonjicA.Theperceptionof colorandmaterial innaturalistictasks.BioRXiv.2018.

60. Wang X, PedersenM, Tomas JB. The influence of chromatic aberration on demosaicking.BioRXiv.2014.

61. Golden JR, Erickson-Davis C, Cottaris NP, Parthasarathy N, Rieke F, Brainard DH, et al.Simulationofvisualperceptionandlearningwitharetinalprosthesis.BioRXiv.2018.

62. Tuten WS, Cooper RF, Tiruveedhula P, Dubra A, Roorda A, Cottaris NP, et al. Spatialsummation in the human fovea: the effect of optical aberrations and fixational eye movements.BioRXiv.2018.

63. WatsonAB.Aformulaforhumanretinalganglioncellreceptivefielddensityasafunctionofvisualfieldlocation.JVis.2014;14(7).

64. LiuT,HeegerDJ,CarrascoM.Neuralcorrelatesofthevisualverticalmeridianasymmetry.JVis.2006;6(11):1294-306.

65. Benson NC, Butt OH, Datta R, Radoeva PD, Brainard DH, Aguirre GK. The retinotopicorganization of striate cortex is well predicted by surface topology. Current biology : CB.2012;22(21):2081-5.




66. Silva MF, Brascamp JW, Ferreira S, Castelo-Branco M, Dumoulin SO, Harvey BM. Radialasymmetriesinpopulationreceptivefieldsizeandcorticalmagnificationfactorinearlyvisualcortex.NeuroImage.2018;167:41-52.

67. RobsonAG,Moreland JD,PauleikhoffD,MorrisseyT,HolderGE,FitzkeFW,etal.Macularpigment density and distribution: comparison of fundus autofluorescencewithminimummotionphotometry.Visionresearch.2003;43(16):1765-75.

68. GreenDM,SwetsJA.Signaldetectiontheoryandpsychophysics.NewYork,:Wiley;1966.xi,455p.p.

69. Geisler WS. Contributions of ideal observer theory to vision research. Vision research.2011;51(7):771-81.

70. BanksMS,GeislerWS,BennettPJ.Thephysical limitsofgratingvisibility.Visionresearch.1987;27(11):1915-24.

71. Blakemore C, Campbell FW. On the existence of neurones in the human visual systemselectivelysensitivetotheorientationandsizeofretinalimages.JPhysiol.1969;203(1):237-60.

72. CarandiniM,DembJB,ManteV,TolhurstDJ,DanY,OlshausenBA,etal.Doweknowwhattheearlyvisualsystemdoes?JNeurosci.2005;25(46):10577-97.

73. HubelDH,WieselTN.Receptivefields,binocularinteractionandfunctionalarchitectureinthecat'svisualcortex.JPhysiol.1962;160:106-54.

74. RustNC, SchwartzO,Movshon JA, SimoncelliEP. Spatiotemporal elementsofmacaquev1receptivefields.Neuron.2005;46(6):945-56.

75. ZernikeF.Diffractiontheoryoftheknife-edgetestanditsimprovedform,thephase-contrastmethod.MonthlyNoticesoftheRoyalAstronomicalSociety.1934;94(377-384).

76. Stockman A, Sharpe LT. The spectral sensitivities of the middle- and long-wavelength-sensitive cones derived from measurements in observers of known genotype. Vision Res.2000;40(13):1711-37.



Date post:	22-Oct-2020
Category:	Documents
Upload:	others
View:	3 times
Download:	0 times

Modeling visual performance differences with polar angle: A ... · pass through simulated human...

Documents