+ All Categories
Home > Documents > Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability...

Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability...

Date post: 28-Jun-2020
Category:
Upload: others
View: 1 times
Download: 0 times
Share this document with a friend
18
Comparing models of contrast gain using psychophysical experiments Christopher DiMattina # $ Computational Perception Laboratory, Department of Psychology, Florida Gulf Coast University, Fort Myers, FL, USA In a wide variety of neural systems, neurons tuned to a primary dimension of interest often have responses that are modulated in a multiplicative manner by other features such as stimulus intensity or contrast. In this methodological study, we present a demonstration that it is possible to use psychophysical experiments to compare competing hypotheses of multiplicative gain modulation in a neural population, using the specific example of contrast gain modulation in orientation- tuned visual neurons. We demonstrate that fitting biologically interpretable models to psychophysical data yields physiologically accurate estimates of contrast tuning parameters and allows us to compare competing hypotheses of contrast tuning. We demonstrate a powerful methodology for comparing competing neural models using adaptively generated psychophysical stimuli and demonstrate that such stimuli can be highly effective for distinguishing qualitatively similar hypotheses. We relate our work to the growing body of literature that uses fits of neural models to behavioral data to gain insight into neural coding and suggest directions for future research. Introduction A large body of experimental and theoretical work has quantitatively analyzed the relationship between neural codes, perception, and perceptual decisions (Nienborg, Cohen, & Cumming, 2012; Parker & Newsome, 1998; Romo & de Lafuente, 2013). Typi- cally, these studies use physiological data to explain behavior by correlating neural performance with behavioral performance (e.g., Britten, Shadlen, New- some, & Movshon, 1992; Cohen & Newsome, 2009; Egger & Britten, 2013; Vogels & Orban, 1990; L. Wang, Narayan, Gra ˜ na, Shamir, & Sen, 2007) or by using the responses of a neural population to predict behavior (e.g., Bollimunta, Totten, & Ditterich, 2012; Kiani, Cueva, Reppas, & Newsome, 2014). However, in recent years, an ever-growing body of literature (reviewed in the Discussion) has taken a complementary approach by making use of behavioral data or theoretically optimal performance on well-defined behavioral tasks to inform and connect with models of neural encoding. This work has demonstrated that quantitatively char- acterizing behavioral data using neurally plausible models can yield insight into sensory receptive field properties (e.g., Burge & Geisler, 2014, 2015; W. S. Geisler, Najemnik, & Ing, 2009; Neri & Levi, 2006; Yamins et al., 2014), pooling of neural population responses (e.g., Goris, Putzeys, Wagemans, & Wich- mann, 2013; Morgenstern & Elder, 2012), attentional modulation (e.g., Murray, Sekuler, & Bennett, 2003; Neri, 2004; Pestilli, Carrasco, Heeger, & Gardner, 2011; Pestilli, Ling, & Carrasco, 2009), perceptual learning (e.g., Petrov, Dosher, & Lu, 2005), and near- optimal performance in perceptual tasks (e.g., Ma, Navalpakkam, Beck, Van Den Berg, & Pouget, 2011; Qamar et al., 2013). In this paper, we extend this growing body of literature by presenting a general methodology for using data obtained in psychophysical experiments to characterize contrast gain modulation in sensory neural populations. Although we focus on contrast gain in early vision, many sensory neural populations tuned to parameters of primary interest (tactile orientation, auditory frequency, etc.) also exhibit response modu- lation by stimulus amplitude or contrast (Barbour & Wang, 2003; Bensmaia, Denchev, Dammann, Craig, & Hsiao, 2008; Kiang, 1965; Muniak, Ray, Hsiao, Dammann, & Bensmaia, 2007; Sachs & Abbas, 1974; Sadagopan & Wang, 2008). We apply this methodology in real psychophysical experiments to analyze a simple model of orientation decoding from a population of contrast- and orientation-tuned neurons in order to demonstrate how psychophysical data may be used to (a) accurately recover neural encoding model parame- ters and (b) compare competing hypotheses of neural encoding. In particular, we demonstrate that we can Citation: DiMattina, C. (2016). Comparing models of contrast gain using psychophysical experiments. Journal of Vision, 16(9):1, 1–18, doi:10.1167/16.9.1. Journal of Vision (2016) 16(9):1, 1–18 1 doi: 10.1167/16.9.1 ISSN 1534-7362 Received November 14, 2015; published July 1, 2016 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Downloaded From: http://jov.arvojournals.org/pdfaccess.ashx?url=/data/Journals/JOV/935414/ on 07/05/2016
Transcript
Page 1: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Comparing models of contrast gain using psychophysicalexperiments

Christopher DiMattina $

Computational Perception LaboratoryDepartment of Psychology Florida Gulf Coast University

Fort Myers FL USA

In a wide variety of neural systems neurons tuned to aprimary dimension of interest often have responses thatare modulated in a multiplicative manner by otherfeatures such as stimulus intensity or contrast In thismethodological study we present a demonstration thatit is possible to use psychophysical experiments tocompare competing hypotheses of multiplicative gainmodulation in a neural population using the specificexample of contrast gain modulation in orientation-tuned visual neurons We demonstrate that fittingbiologically interpretable models to psychophysical datayields physiologically accurate estimates of contrasttuning parameters and allows us to compare competinghypotheses of contrast tuning We demonstrate apowerful methodology for comparing competing neuralmodels using adaptively generated psychophysicalstimuli and demonstrate that such stimuli can be highlyeffective for distinguishing qualitatively similarhypotheses We relate our work to the growing body ofliterature that uses fits of neural models to behavioraldata to gain insight into neural coding and suggestdirections for future research

Introduction

A large body of experimental and theoretical workhas quantitatively analyzed the relationship betweenneural codes perception and perceptual decisions(Nienborg Cohen amp Cumming 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Typi-cally these studies use physiological data to explainbehavior by correlating neural performance withbehavioral performance (eg Britten Shadlen New-some amp Movshon 1992 Cohen amp Newsome 2009Egger amp Britten 2013 Vogels amp Orban 1990 L WangNarayan Grana Shamir amp Sen 2007) or by using theresponses of a neural population to predict behavior(eg Bollimunta Totten amp Ditterich 2012 KianiCueva Reppas amp Newsome 2014) However in recent

years an ever-growing body of literature (reviewed inthe Discussion) has taken a complementary approachby making use of behavioral data or theoreticallyoptimal performance on well-defined behavioral tasksto inform and connect with models of neural encodingThis work has demonstrated that quantitatively char-acterizing behavioral data using neurally plausiblemodels can yield insight into sensory receptive fieldproperties (eg Burge amp Geisler 2014 2015 W SGeisler Najemnik amp Ing 2009 Neri amp Levi 2006Yamins et al 2014) pooling of neural populationresponses (eg Goris Putzeys Wagemans amp Wich-mann 2013 Morgenstern amp Elder 2012) attentionalmodulation (eg Murray Sekuler amp Bennett 2003Neri 2004 Pestilli Carrasco Heeger amp Gardner2011 Pestilli Ling amp Carrasco 2009) perceptuallearning (eg Petrov Dosher amp Lu 2005) and near-optimal performance in perceptual tasks (eg MaNavalpakkam Beck Van Den Berg amp Pouget 2011Qamar et al 2013)

In this paper we extend this growing body ofliterature by presenting a general methodology forusing data obtained in psychophysical experiments tocharacterize contrast gain modulation in sensory neuralpopulations Although we focus on contrast gain inearly vision many sensory neural populations tuned toparameters of primary interest (tactile orientationauditory frequency etc) also exhibit response modu-lation by stimulus amplitude or contrast (Barbour ampWang 2003 Bensmaia Denchev Dammann Craig ampHsiao 2008 Kiang 1965 Muniak Ray HsiaoDammann amp Bensmaia 2007 Sachs amp Abbas 1974Sadagopan ampWang 2008) We apply this methodologyin real psychophysical experiments to analyze a simplemodel of orientation decoding from a population ofcontrast- and orientation-tuned neurons in order todemonstrate how psychophysical data may be used to(a) accurately recover neural encoding model parame-ters and (b) compare competing hypotheses of neuralencoding In particular we demonstrate that we can

Citation DiMattina C (2016) Comparing models of contrast gain using psychophysical experiments Journal of Vision 16(9)11ndash18 doi1011671691

Journal of Vision (2016) 16(9)1 1ndash18 1

doi 10 1167 16 9 1 ISSN 1534-7362Received November 14 2015 published July 1 2016

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 40 International LicenseDownloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

use psychophysical data to correctly infer the physio-logically measured values of contrast gain functionparameters in visual neurons (Albrecht amp Hamilton1982) We further demonstrate experimentally thatadaptive stimulus optimization methods that haverecently gained traction in brain and cognitive science(eg Cavagnaro Myung Pitt amp Kujala 2010DiMattina 2015 DiMattina amp Zhang 2013 LewiButera amp Paninski 2009 Myung Cavagnaro amp Pitt2013 Paninski Pillow amp Lewi 2007 Z Wang ampSimoncelli 2008) can be used to find psychophysicalstimuli during the course of the experimental sessionwhich are optimized for distinguishing competinghypotheses of neural coding We find that presentingstimuli adaptively optimized for model comparisonmay in some cases be very helpful for discriminatingbetween qualitatively similar hypotheses of neuralencoding We discuss the limitations of the presentmethodology and suggest interesting directions forfuture research We believe that with further develop-ments of biologically motivated approaches to model-ing psychophysical data psychophysical experimentscan more directly inform investigations of neuralencoding

Methods and results

Defining biologically interpretable psychometricmodels

Here we present for didactic purposes a derivation ofthe psychometric function that makes explicit the factthat perceptual behavior is ultimately dependent on theparameters of the sensory neural population used toguide that behavior

For a population of N neurons a neural encodingmodel P(rjsh) specifies the probability of observingneural responses rfrac14 (r1 rN)

T as a function ofstimulus parameters s and neuronal population pa-rameters h (Borst amp Theunissen 1999 Paninski et al2007) Perhaps the simplest possible neural encodingmodel is a set of tuning curves specifying the expectedfiring rate of each neuron in the population as afunction of the sensory variable s for instance theorientation-tuning curves shown in Figure 1a In thiscase the population parameters h would represent theproperties of this set of tuning curves for instance thecenters l1 lN tuning curve width r and amplitudeA Similarly a neural decoding model P(sjrx) specifiesthe probability of a stimulus s being present as afunction of the observed neural responses r andpossibly additional parameters x (Paninski et al2007)

We define a behavioral decoding model P(bjrx) asspecifying the probability of a behavioral response b asa function of neural responses r as well as additionaldecoding parameters x In this formulation thestochastic neural responses r which is the output of theneural encoding model serves as the input to thebehavioral decoding model as illustrated in Figure 1band c The behavioral decoding model may determin-istically specify b as a function of r x as in the exampleshown in Figure 1b which compares the decisionvariable u frac14 RN

ifrac141xiri to a fixed decision thresholdAlternatively the behavioral decoding model may alsospecify b probabilistically in order to model stimulus-independent lsquolsquodecision noisersquorsquo (Shadlen Britten New-some amp Movshon 1996) The joint probability ofobserving a behavior b and neural response r as afunction of a stimulus s may be written as the productof a neural encoding model and behavioral decodingmodel using the basic probability law P(AB) frac14P(AjB)P(B) (Bishop 2006) yielding the expression

Pethb rjsx hTHORN frac14 PethbjrxTHORNPethrjs hTHORN eth1THORNBy marginalizing the joint probability P(brjsxh) overr we can express the probability of a behavior entirelyas a function of the stimulus parameters s and modelparameters h x without any dependence on unob-served neural responses This follows from the basicprobability law

RP(AB) dB frac14 P(A) Marginalizing

Equation 1 over r yields the equation

Pethbjs hxTHORN frac14RPethbjrxTHORNPethrjs hTHORN dr eth2THORN

Note that the integrand in Equation 2 is the productof the behavioral decoding model and the neuralencoding model integrated over all possible neuralresponses r conditioned on the stimulus s In the case offixed decoding model parameters x so that P(x)frac14 d(x x) (where d denotes the Dirac delta function) we canuse Equation 2 to derive an expression for the posteriorprobability of the neural encoding model parameters hgiven only psychophysical trial data Dw frac14 fsi bignifrac141(Appendix A)

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth3THORNIn the case in which one does not make informativeprior assumptions P(h) about the neural encodingmodel parameters Equation 3 becomes the likelihoodIn the application presented in this study we do notincorporate informative priors on h and simply attainmaximum likelihood point estimates

Although Equations 2 and 3 make explicit thedependence of the psychometric function on the neuralencoding model and show that one can in principleestimate neural parameters from behavior these

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 2

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

equations are of little practical use without specific

assumptions about the neural encoding ie P(rjsh) orbehavioral decoding ie P(bjrx) models Even with

such assumptions one must be aware that there are

practical limitations on the number of neuronal

parameters h that can be accurately estimated during

the course of a psychophysical experiment As studies

with classification images show (Ahumada 1996

Eckstein amp Ahumada 2002 Mineault Barthelme amp

Pack 2009 Murray 2011) binomial (eg yesno)

responses provide relatively little information per trial

necessitating a large number of trials to attain accurate

estimates of the perceptual filter However we dem-

onstrate here that it is very realistic to use psycho-

physical data to estimate and compare low-dimensional

analytical models (eg May and Solomon 2015a

2015b Pestilli et al 2011 Pestilli et al 2009) in a

process of focused hypothesis testing

Orientation discrimination model

We now consider the application of our modeling

framework to a simple orientation discrimination task

in which a subject has to determine in which of two

directions (clockwise counterclockwisethorn) a sinu-

soidal grating stimulus having contrast c (0 c 100) has been tilted (by d8) with respect to vertical

In order to do this we must specify concretely the

hypothesized neural code r the observable behaviors b

the hypothesized neural encoding model P(rjsh) andthe hypothesized behavioral decoding model P(bjrx)

Figure 1 Schematic illustration of neural encoding and behavioral decoding models (a) A neural encoding model P(rjsh) specifies theprobability of observing stimulus-dependent neural population responses r Bottom An oriented bar stimulus elicits noisy responses

from orientation-tuned neurons whose tuning curves are specified by parameters hfrac14 (Ar l1 lN)T Top Observed noisy single-

trial responses r frac14 (r1 r2 rN)T of each neuron (b) A behavioral decoding model takes as input the stimulus-evoked neural

responses r frac14 (r1 r2 rN)T and uses them to determine the probability of a behavior b In the deterministic model shown here

neural responses r are multiplied by weights xfrac14 (x1 xN)T and summed to form a decision variable (ufrac14Rixiri) which is compared

to a threshold (s) to predict a binary perceptual decision (c) One can define a biologically interpretable psychometric function by

using the output r of a neural encoding model as the input to a behavioral decoding model

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 3

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

A fairly straightforward derivation of a psychomet-ric function defined using the neural encoding modelshown in Figure 1 and with linear decoding (Fisherlinear discriminant) is given in Appendix B Thisanalysis is similar to those presented in several previousstudies (eg Ma 2010 Pestilli et al 2009) Ourderivation yields the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth4THORN

where w(c) denotes the contrast tuning (also calledcontrast gain) of neurons in the population and K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing population sensi-

tivity to changes in orientation around the verticalreference (0 frac14 p2) at 100 contrast

In this paper we consider three different functionalforms for the contrast gain function w(c) One formsuggested from neurophysiological findings (Albrechtamp Hamilton 1982) is the Naka-Rushton function

w1

cgeth1THORN

frac14 cn

cn thorn cn50

eth5THORN

having parameters g(1)frac14 (n c50)T This functional form

(Equation 5) is also sometimes referred to as thehyperbolic ratio function (Albrecht amp Hamilton 1982)Another form is the hyperbolic tangent (tanh) function

w2

cgeth2THORN

frac14 tanhethbcTHORN frac14 ebc ebc

ebc thorn ebc eth6THORN

commonly used in machine learning (Bishop 2006)having parameter g(2) frac14 (b)T Both of these functionalforms (Naka-Rushon Tanh) are shown in Figure 2Finally we consider a Gaussian form that allows forthe possibility of a nonmonotonic relationship betweencontrast and firing rate given by

w3

cgeth3THORN

frac14 exp 1

2r2ethc lTHORN2

eth7THORN

with parameters g(3) frac14 (l r)TOur interest in fitting multiple models to the same

data set is to test the efficacy of psychophysical data fordistinguishing between competing hypotheses of neuralencoding This approach follows previous work usingfits of multiple models to behavioral data to gaininsight into sensory or cognitive mechanisms (Qamar etal 2013 van den Berg Awh amp Ma 2014) Thecomparison between the Naka-Rushton model and theGaussian model is a coarse-grained qualitative com-parison because the two models are qualitatively verydifferent (monotonic vs nonmonotonic) whereas thecomparison between Naka-Rushton and Tanh is a fine-grained quantitative comparison because the twomodels are both monotonic and qualitatively verysimilar (Figure 2)

Recovering neural encoding model parameters

Fitting thresholds

Because we can write our psychometric function(Equation 4) in terms of d0 (25) we can use thresholdstaken at multiple contrasts to estimate the psychomet-ric function parameters using least-squares curvefitting Figure 3 shows the best fit of the model(Equation 4) with Naka-Rushton contrast gain(Equation 5) to the data from Skottun Bradley SclarOhzawa and Freeman (1987 their figure 1) We see inFigure 3 that this model provides an excellent fit totheir data (Supplementary Figure S1) We find that thevalues recovered for the Naka-Rushton contrastfunction parameters n c50 from their threshold data liewithin the range measured in previous neurophysio-logical work (Albrecht amp Hamilton 1982) as shown inFigure 4 (red circles)

Figure 2 Two competing hypotheses for the functional form of

contrast gain tuning Despite the qualitative similarity of the

Naka-Rushton (Equation 5) and Tanh (Equation 6) models we

observe a better quantitative fit to neurophysiological data by

the Naka-Rushton function particularly at lower contrasts (a)

Fits of both models (Equations 5 and 6) to contrast gain

responses of a representative V1 neuron Data points

graphically adapted from figure 3 of Albrecht and Hamilton

(1982) (b) Fits of both models (Equations 5 and 6) to contrast

gain responses of several V1 neurons Data points graphically

adapted from figure 1 of Albrecht and Hamilton (1982) (c)

Residual sum-of-squares error for the fits of both models in (b)

We see a better fit for the Naka-Rushton model (sign-rank test

n frac14 9 p frac14 00039 001)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 4

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Experiment 1 Direct estimation from psychophysicaltrials

The data in Skottun et al (1987) only providesthresholds and therefore our estimates of g(1) frac14 (nc50)

T were not obtained as in most psychophysicalexperiments in which one finds the maximum likeli-hood estimate of model parameters using stimulus-response data Dw frac14 fsi bignifrac141 (Kingdom amp Prins2010) In order to directly test the use of psychophysicaldata to recover the parameters of neural tuning curveswe ran an orientation discrimination experiment(Experiment 1) on nine subjects (seven naive) in whichwe covaried orientation and contrast Additionaldetails of Experiment 1 are described in theSupplementary Methods Contour plots of subjectperformance P(b frac14 1js frac14 (cd)TKg(1)) are shown inFigure 5 (and Supplementary Figure S3) with fits of themodel (Equation 4) with Naka-Rushton gain (Equa-

tion 5) to subject data in the middle column We foundin a subsequent experiment (Supplementary Material)that this model could also generalize reasonably wellfor most (but not all) subjects to predict responses to asmall validation set of novel stimuli (SupplementaryFigure S4)

We see in Figure 4 that the values of the Naka-Rushton parameters n c50 estimated from ourExperiment 1 data (black diamonds) lie within theneurophysiologically observed range Numerical val-ues of these parameters are given in SupplementaryTables S1 and S2 Interestingly we find that all of ourestimates of the half-saturation parameter c50 ob-tained in these experiments (along with five of sixestimates of c50 from Skottun et al 1987) lie towardthe lower end of the physiologically observed range(ie around 5 contrast see Albrecht amp Hamilton1982) This suggests the subjects may be using theneurons that are most sensitive to contrast when theyperform the task consistent with the lsquolsquolower enve-lopersquorsquo principle of sensory coding (Egger amp Britten2013 Mountcastle LaMotte amp Carli 1972 L Wanget al 2007)

Comparing competing models

Exploring model space

In Experiment 1 whose goal was to show that onecan estimate neural model parameters from psycho-physical data we assumed a known form (Equation 5)of the contrast gain function based on previous

Figure 3 Fits of the behavioral decoding model (Equation 4)

with Naka-Rushton contrast gain (Equation 5) to threshold data

(79 performance) graphically adapted from figure 1 of Skottun

et al (1987) Plot of residual sum-of-squares error for models

with Naka-Rushton (red) and Tanh (green) contrast gain

(Equation 6) are given in Supplementary Figure S1

Figure 4 Estimates of neural contrast gain function parameters

n and c50 (Naka-Rushton) from psychophysical data Red dots

denote estimates from threshold data (Skottun et al 1987)

black diamonds are estimates from fitting the model directly to

psychophysical trial data (Experiment 1) We see that all of the

estimates lie within the physiological range (blue lines frac14 l 6

196 r) (Albrecht amp Hamilton 1982)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

neurophysiological investigations (Albrecht amp Hamil-ton 1982) Supposing that the correct functional formof the contrast gain function w(c) was not knownbeforehand from physiological recordings we may beinterested in evaluating various possibilities by fittingthe model (Equation 4) to psychophysical data withdifferent choices for w(c) and seeing which bestaccounts for the observed results Such informationderived from relatively fast and inexpensive psycho-physical experiments could provide important clues toguide subsequent neurophysiology research

In order to test the ability of psychophysicalexperiments to compare competing models of neuralcontrast gain we will also consider two otherpossibilities for the contrast gain given by thehyperbolic tangent (Tanh) function (Equation 6) andthe familiar Gaussian tuning curve (Equation 7) Thesethree possible choices (Equations 5 6 and 7) ofcontrast gain function w(c) define a discrete space ofthree competing neural encoding models which weindex by i frac14 1 2 3 By fitting each model topsychophysical data we may evaluate their relativelikelihoods using the Akaike Information Criterion(AIC) which measures goodness-of-fit while penalizingmodel complexity (Akaike 1974 Burnham amp Ander-son 2003) Previous work has shown that it isimportant that any model comparison method takes

complexity into account because an overly complexmodel often fits training data well but fails to generalizeto novel observations (Bishop 2006 Pitt amp Myung2002)

We denote the value of the AIC for the i-th model byAICi with model i being preferred to model j if AICi AICj We define a model preference index

Pij frac14 AICi AICj eth8THORNwhere a positive value of Pindashj indicates model i ispreferred to model j and a negative value indicating j ispreferred to i The model preference index is definedimplicitly with respect to a fixed number of observa-tions ie Pindashjfrac14 Pindashj (n) where n is the number of trialsused to compute the AIC We define a change in modelpreference after k additional trials as

DPij frac14 Pijethnthorn kTHORN PijethnTHORN eth9THORNIn our analysis model 1 assumes Naka-Rushtoncontrast tuning (Equation 5) model 2 assumes Tanhtuning (Equation 6) and model 3 assumes Gaussiantuning (Equation 7)

Computing the AIC for fits of all three models tothe data collected in Experiment 1 allows us todetermine the model preferences P1ndash2 (Naka-Rush-tonndashTanh) and P1ndash3 (Naka-RushtonndashGaussian) Wesee in Figure 6a that the Naka-Rushton model ispreferred over the Gaussian model for all nine subjectsand over the Tanh model for seven of nine subjectswith the preference being quite strong for manysubjects Statistical tests show that over these ninesubjects both model preferences are significantlydifferent from zero (sign-rank test nfrac14 9 P1ndash2 0 pfrac14002 P1ndash3 0 p frac14 0004) Figure 6b shows how thismodel preference P1ndash2 evolves with the number ofexperimental trials We see that as more trials arecollected the model preference (for most subjects)seems to change in favor of the Naka-Rushton modelwhose better ability to fit the data overcomes thecomplexity penalty imposed by the AIC We also seefrom Figure 6b that the final model preferences areestablished after about 1000ndash1200 trials Similarresults were obtained using the Bayes InformationCriterion which more severely penalizes modelcomplexity (Bishop 2006) changing the final modelpreference for only one subject (SupplementaryFigures S5 and S6)

Experiment 2 Optimizing stimuli for model comparison

In Experiment 1 data was collected using themethod of constant stimuli which previous work hassuggested may be suboptimal for purposes of modelestimation and comparison (Watson amp Fitzhugh1990) Therefore we conducted a second experiment(Experiment 2) in order to determine if stimuli

Figure 5 Contour plots of the psychometric function P(bfrac14 1jsfrac14(c d)T) as a function of orientation (d) and contrast (c) for

three subjects in Experiment 1 Other subjects shown in

Supplementary Figure S3 Left Raw data Middle Fits of model

(Equation 4) with Naka-Rushton (Equation 5) contrast gain

(model 1) to data Right Fits of (Equation 4) with Tanh

(Equation 6) contrast gain (model 2) to data

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 2: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

use psychophysical data to correctly infer the physio-logically measured values of contrast gain functionparameters in visual neurons (Albrecht amp Hamilton1982) We further demonstrate experimentally thatadaptive stimulus optimization methods that haverecently gained traction in brain and cognitive science(eg Cavagnaro Myung Pitt amp Kujala 2010DiMattina 2015 DiMattina amp Zhang 2013 LewiButera amp Paninski 2009 Myung Cavagnaro amp Pitt2013 Paninski Pillow amp Lewi 2007 Z Wang ampSimoncelli 2008) can be used to find psychophysicalstimuli during the course of the experimental sessionwhich are optimized for distinguishing competinghypotheses of neural coding We find that presentingstimuli adaptively optimized for model comparisonmay in some cases be very helpful for discriminatingbetween qualitatively similar hypotheses of neuralencoding We discuss the limitations of the presentmethodology and suggest interesting directions forfuture research We believe that with further develop-ments of biologically motivated approaches to model-ing psychophysical data psychophysical experimentscan more directly inform investigations of neuralencoding

Methods and results

Defining biologically interpretable psychometricmodels

Here we present for didactic purposes a derivation ofthe psychometric function that makes explicit the factthat perceptual behavior is ultimately dependent on theparameters of the sensory neural population used toguide that behavior

For a population of N neurons a neural encodingmodel P(rjsh) specifies the probability of observingneural responses rfrac14 (r1 rN)

T as a function ofstimulus parameters s and neuronal population pa-rameters h (Borst amp Theunissen 1999 Paninski et al2007) Perhaps the simplest possible neural encodingmodel is a set of tuning curves specifying the expectedfiring rate of each neuron in the population as afunction of the sensory variable s for instance theorientation-tuning curves shown in Figure 1a In thiscase the population parameters h would represent theproperties of this set of tuning curves for instance thecenters l1 lN tuning curve width r and amplitudeA Similarly a neural decoding model P(sjrx) specifiesthe probability of a stimulus s being present as afunction of the observed neural responses r andpossibly additional parameters x (Paninski et al2007)

We define a behavioral decoding model P(bjrx) asspecifying the probability of a behavioral response b asa function of neural responses r as well as additionaldecoding parameters x In this formulation thestochastic neural responses r which is the output of theneural encoding model serves as the input to thebehavioral decoding model as illustrated in Figure 1band c The behavioral decoding model may determin-istically specify b as a function of r x as in the exampleshown in Figure 1b which compares the decisionvariable u frac14 RN

ifrac141xiri to a fixed decision thresholdAlternatively the behavioral decoding model may alsospecify b probabilistically in order to model stimulus-independent lsquolsquodecision noisersquorsquo (Shadlen Britten New-some amp Movshon 1996) The joint probability ofobserving a behavior b and neural response r as afunction of a stimulus s may be written as the productof a neural encoding model and behavioral decodingmodel using the basic probability law P(AB) frac14P(AjB)P(B) (Bishop 2006) yielding the expression

Pethb rjsx hTHORN frac14 PethbjrxTHORNPethrjs hTHORN eth1THORNBy marginalizing the joint probability P(brjsxh) overr we can express the probability of a behavior entirelyas a function of the stimulus parameters s and modelparameters h x without any dependence on unob-served neural responses This follows from the basicprobability law

RP(AB) dB frac14 P(A) Marginalizing

Equation 1 over r yields the equation

Pethbjs hxTHORN frac14RPethbjrxTHORNPethrjs hTHORN dr eth2THORN

Note that the integrand in Equation 2 is the productof the behavioral decoding model and the neuralencoding model integrated over all possible neuralresponses r conditioned on the stimulus s In the case offixed decoding model parameters x so that P(x)frac14 d(x x) (where d denotes the Dirac delta function) we canuse Equation 2 to derive an expression for the posteriorprobability of the neural encoding model parameters hgiven only psychophysical trial data Dw frac14 fsi bignifrac141(Appendix A)

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth3THORNIn the case in which one does not make informativeprior assumptions P(h) about the neural encodingmodel parameters Equation 3 becomes the likelihoodIn the application presented in this study we do notincorporate informative priors on h and simply attainmaximum likelihood point estimates

Although Equations 2 and 3 make explicit thedependence of the psychometric function on the neuralencoding model and show that one can in principleestimate neural parameters from behavior these

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 2

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

equations are of little practical use without specific

assumptions about the neural encoding ie P(rjsh) orbehavioral decoding ie P(bjrx) models Even with

such assumptions one must be aware that there are

practical limitations on the number of neuronal

parameters h that can be accurately estimated during

the course of a psychophysical experiment As studies

with classification images show (Ahumada 1996

Eckstein amp Ahumada 2002 Mineault Barthelme amp

Pack 2009 Murray 2011) binomial (eg yesno)

responses provide relatively little information per trial

necessitating a large number of trials to attain accurate

estimates of the perceptual filter However we dem-

onstrate here that it is very realistic to use psycho-

physical data to estimate and compare low-dimensional

analytical models (eg May and Solomon 2015a

2015b Pestilli et al 2011 Pestilli et al 2009) in a

process of focused hypothesis testing

Orientation discrimination model

We now consider the application of our modeling

framework to a simple orientation discrimination task

in which a subject has to determine in which of two

directions (clockwise counterclockwisethorn) a sinu-

soidal grating stimulus having contrast c (0 c 100) has been tilted (by d8) with respect to vertical

In order to do this we must specify concretely the

hypothesized neural code r the observable behaviors b

the hypothesized neural encoding model P(rjsh) andthe hypothesized behavioral decoding model P(bjrx)

Figure 1 Schematic illustration of neural encoding and behavioral decoding models (a) A neural encoding model P(rjsh) specifies theprobability of observing stimulus-dependent neural population responses r Bottom An oriented bar stimulus elicits noisy responses

from orientation-tuned neurons whose tuning curves are specified by parameters hfrac14 (Ar l1 lN)T Top Observed noisy single-

trial responses r frac14 (r1 r2 rN)T of each neuron (b) A behavioral decoding model takes as input the stimulus-evoked neural

responses r frac14 (r1 r2 rN)T and uses them to determine the probability of a behavior b In the deterministic model shown here

neural responses r are multiplied by weights xfrac14 (x1 xN)T and summed to form a decision variable (ufrac14Rixiri) which is compared

to a threshold (s) to predict a binary perceptual decision (c) One can define a biologically interpretable psychometric function by

using the output r of a neural encoding model as the input to a behavioral decoding model

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 3

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

A fairly straightforward derivation of a psychomet-ric function defined using the neural encoding modelshown in Figure 1 and with linear decoding (Fisherlinear discriminant) is given in Appendix B Thisanalysis is similar to those presented in several previousstudies (eg Ma 2010 Pestilli et al 2009) Ourderivation yields the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth4THORN

where w(c) denotes the contrast tuning (also calledcontrast gain) of neurons in the population and K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing population sensi-

tivity to changes in orientation around the verticalreference (0 frac14 p2) at 100 contrast

In this paper we consider three different functionalforms for the contrast gain function w(c) One formsuggested from neurophysiological findings (Albrechtamp Hamilton 1982) is the Naka-Rushton function

w1

cgeth1THORN

frac14 cn

cn thorn cn50

eth5THORN

having parameters g(1)frac14 (n c50)T This functional form

(Equation 5) is also sometimes referred to as thehyperbolic ratio function (Albrecht amp Hamilton 1982)Another form is the hyperbolic tangent (tanh) function

w2

cgeth2THORN

frac14 tanhethbcTHORN frac14 ebc ebc

ebc thorn ebc eth6THORN

commonly used in machine learning (Bishop 2006)having parameter g(2) frac14 (b)T Both of these functionalforms (Naka-Rushon Tanh) are shown in Figure 2Finally we consider a Gaussian form that allows forthe possibility of a nonmonotonic relationship betweencontrast and firing rate given by

w3

cgeth3THORN

frac14 exp 1

2r2ethc lTHORN2

eth7THORN

with parameters g(3) frac14 (l r)TOur interest in fitting multiple models to the same

data set is to test the efficacy of psychophysical data fordistinguishing between competing hypotheses of neuralencoding This approach follows previous work usingfits of multiple models to behavioral data to gaininsight into sensory or cognitive mechanisms (Qamar etal 2013 van den Berg Awh amp Ma 2014) Thecomparison between the Naka-Rushton model and theGaussian model is a coarse-grained qualitative com-parison because the two models are qualitatively verydifferent (monotonic vs nonmonotonic) whereas thecomparison between Naka-Rushton and Tanh is a fine-grained quantitative comparison because the twomodels are both monotonic and qualitatively verysimilar (Figure 2)

Recovering neural encoding model parameters

Fitting thresholds

Because we can write our psychometric function(Equation 4) in terms of d0 (25) we can use thresholdstaken at multiple contrasts to estimate the psychomet-ric function parameters using least-squares curvefitting Figure 3 shows the best fit of the model(Equation 4) with Naka-Rushton contrast gain(Equation 5) to the data from Skottun Bradley SclarOhzawa and Freeman (1987 their figure 1) We see inFigure 3 that this model provides an excellent fit totheir data (Supplementary Figure S1) We find that thevalues recovered for the Naka-Rushton contrastfunction parameters n c50 from their threshold data liewithin the range measured in previous neurophysio-logical work (Albrecht amp Hamilton 1982) as shown inFigure 4 (red circles)

Figure 2 Two competing hypotheses for the functional form of

contrast gain tuning Despite the qualitative similarity of the

Naka-Rushton (Equation 5) and Tanh (Equation 6) models we

observe a better quantitative fit to neurophysiological data by

the Naka-Rushton function particularly at lower contrasts (a)

Fits of both models (Equations 5 and 6) to contrast gain

responses of a representative V1 neuron Data points

graphically adapted from figure 3 of Albrecht and Hamilton

(1982) (b) Fits of both models (Equations 5 and 6) to contrast

gain responses of several V1 neurons Data points graphically

adapted from figure 1 of Albrecht and Hamilton (1982) (c)

Residual sum-of-squares error for the fits of both models in (b)

We see a better fit for the Naka-Rushton model (sign-rank test

n frac14 9 p frac14 00039 001)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 4

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Experiment 1 Direct estimation from psychophysicaltrials

The data in Skottun et al (1987) only providesthresholds and therefore our estimates of g(1) frac14 (nc50)

T were not obtained as in most psychophysicalexperiments in which one finds the maximum likeli-hood estimate of model parameters using stimulus-response data Dw frac14 fsi bignifrac141 (Kingdom amp Prins2010) In order to directly test the use of psychophysicaldata to recover the parameters of neural tuning curveswe ran an orientation discrimination experiment(Experiment 1) on nine subjects (seven naive) in whichwe covaried orientation and contrast Additionaldetails of Experiment 1 are described in theSupplementary Methods Contour plots of subjectperformance P(b frac14 1js frac14 (cd)TKg(1)) are shown inFigure 5 (and Supplementary Figure S3) with fits of themodel (Equation 4) with Naka-Rushton gain (Equa-

tion 5) to subject data in the middle column We foundin a subsequent experiment (Supplementary Material)that this model could also generalize reasonably wellfor most (but not all) subjects to predict responses to asmall validation set of novel stimuli (SupplementaryFigure S4)

We see in Figure 4 that the values of the Naka-Rushton parameters n c50 estimated from ourExperiment 1 data (black diamonds) lie within theneurophysiologically observed range Numerical val-ues of these parameters are given in SupplementaryTables S1 and S2 Interestingly we find that all of ourestimates of the half-saturation parameter c50 ob-tained in these experiments (along with five of sixestimates of c50 from Skottun et al 1987) lie towardthe lower end of the physiologically observed range(ie around 5 contrast see Albrecht amp Hamilton1982) This suggests the subjects may be using theneurons that are most sensitive to contrast when theyperform the task consistent with the lsquolsquolower enve-lopersquorsquo principle of sensory coding (Egger amp Britten2013 Mountcastle LaMotte amp Carli 1972 L Wanget al 2007)

Comparing competing models

Exploring model space

In Experiment 1 whose goal was to show that onecan estimate neural model parameters from psycho-physical data we assumed a known form (Equation 5)of the contrast gain function based on previous

Figure 3 Fits of the behavioral decoding model (Equation 4)

with Naka-Rushton contrast gain (Equation 5) to threshold data

(79 performance) graphically adapted from figure 1 of Skottun

et al (1987) Plot of residual sum-of-squares error for models

with Naka-Rushton (red) and Tanh (green) contrast gain

(Equation 6) are given in Supplementary Figure S1

Figure 4 Estimates of neural contrast gain function parameters

n and c50 (Naka-Rushton) from psychophysical data Red dots

denote estimates from threshold data (Skottun et al 1987)

black diamonds are estimates from fitting the model directly to

psychophysical trial data (Experiment 1) We see that all of the

estimates lie within the physiological range (blue lines frac14 l 6

196 r) (Albrecht amp Hamilton 1982)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

neurophysiological investigations (Albrecht amp Hamil-ton 1982) Supposing that the correct functional formof the contrast gain function w(c) was not knownbeforehand from physiological recordings we may beinterested in evaluating various possibilities by fittingthe model (Equation 4) to psychophysical data withdifferent choices for w(c) and seeing which bestaccounts for the observed results Such informationderived from relatively fast and inexpensive psycho-physical experiments could provide important clues toguide subsequent neurophysiology research

In order to test the ability of psychophysicalexperiments to compare competing models of neuralcontrast gain we will also consider two otherpossibilities for the contrast gain given by thehyperbolic tangent (Tanh) function (Equation 6) andthe familiar Gaussian tuning curve (Equation 7) Thesethree possible choices (Equations 5 6 and 7) ofcontrast gain function w(c) define a discrete space ofthree competing neural encoding models which weindex by i frac14 1 2 3 By fitting each model topsychophysical data we may evaluate their relativelikelihoods using the Akaike Information Criterion(AIC) which measures goodness-of-fit while penalizingmodel complexity (Akaike 1974 Burnham amp Ander-son 2003) Previous work has shown that it isimportant that any model comparison method takes

complexity into account because an overly complexmodel often fits training data well but fails to generalizeto novel observations (Bishop 2006 Pitt amp Myung2002)

We denote the value of the AIC for the i-th model byAICi with model i being preferred to model j if AICi AICj We define a model preference index

Pij frac14 AICi AICj eth8THORNwhere a positive value of Pindashj indicates model i ispreferred to model j and a negative value indicating j ispreferred to i The model preference index is definedimplicitly with respect to a fixed number of observa-tions ie Pindashjfrac14 Pindashj (n) where n is the number of trialsused to compute the AIC We define a change in modelpreference after k additional trials as

DPij frac14 Pijethnthorn kTHORN PijethnTHORN eth9THORNIn our analysis model 1 assumes Naka-Rushtoncontrast tuning (Equation 5) model 2 assumes Tanhtuning (Equation 6) and model 3 assumes Gaussiantuning (Equation 7)

Computing the AIC for fits of all three models tothe data collected in Experiment 1 allows us todetermine the model preferences P1ndash2 (Naka-Rush-tonndashTanh) and P1ndash3 (Naka-RushtonndashGaussian) Wesee in Figure 6a that the Naka-Rushton model ispreferred over the Gaussian model for all nine subjectsand over the Tanh model for seven of nine subjectswith the preference being quite strong for manysubjects Statistical tests show that over these ninesubjects both model preferences are significantlydifferent from zero (sign-rank test nfrac14 9 P1ndash2 0 pfrac14002 P1ndash3 0 p frac14 0004) Figure 6b shows how thismodel preference P1ndash2 evolves with the number ofexperimental trials We see that as more trials arecollected the model preference (for most subjects)seems to change in favor of the Naka-Rushton modelwhose better ability to fit the data overcomes thecomplexity penalty imposed by the AIC We also seefrom Figure 6b that the final model preferences areestablished after about 1000ndash1200 trials Similarresults were obtained using the Bayes InformationCriterion which more severely penalizes modelcomplexity (Bishop 2006) changing the final modelpreference for only one subject (SupplementaryFigures S5 and S6)

Experiment 2 Optimizing stimuli for model comparison

In Experiment 1 data was collected using themethod of constant stimuli which previous work hassuggested may be suboptimal for purposes of modelestimation and comparison (Watson amp Fitzhugh1990) Therefore we conducted a second experiment(Experiment 2) in order to determine if stimuli

Figure 5 Contour plots of the psychometric function P(bfrac14 1jsfrac14(c d)T) as a function of orientation (d) and contrast (c) for

three subjects in Experiment 1 Other subjects shown in

Supplementary Figure S3 Left Raw data Middle Fits of model

(Equation 4) with Naka-Rushton (Equation 5) contrast gain

(model 1) to data Right Fits of (Equation 4) with Tanh

(Equation 6) contrast gain (model 2) to data

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 3: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

equations are of little practical use without specific

assumptions about the neural encoding ie P(rjsh) orbehavioral decoding ie P(bjrx) models Even with

such assumptions one must be aware that there are

practical limitations on the number of neuronal

parameters h that can be accurately estimated during

the course of a psychophysical experiment As studies

with classification images show (Ahumada 1996

Eckstein amp Ahumada 2002 Mineault Barthelme amp

Pack 2009 Murray 2011) binomial (eg yesno)

responses provide relatively little information per trial

necessitating a large number of trials to attain accurate

estimates of the perceptual filter However we dem-

onstrate here that it is very realistic to use psycho-

physical data to estimate and compare low-dimensional

analytical models (eg May and Solomon 2015a

2015b Pestilli et al 2011 Pestilli et al 2009) in a

process of focused hypothesis testing

Orientation discrimination model

We now consider the application of our modeling

framework to a simple orientation discrimination task

in which a subject has to determine in which of two

directions (clockwise counterclockwisethorn) a sinu-

soidal grating stimulus having contrast c (0 c 100) has been tilted (by d8) with respect to vertical

In order to do this we must specify concretely the

hypothesized neural code r the observable behaviors b

the hypothesized neural encoding model P(rjsh) andthe hypothesized behavioral decoding model P(bjrx)

Figure 1 Schematic illustration of neural encoding and behavioral decoding models (a) A neural encoding model P(rjsh) specifies theprobability of observing stimulus-dependent neural population responses r Bottom An oriented bar stimulus elicits noisy responses

from orientation-tuned neurons whose tuning curves are specified by parameters hfrac14 (Ar l1 lN)T Top Observed noisy single-

trial responses r frac14 (r1 r2 rN)T of each neuron (b) A behavioral decoding model takes as input the stimulus-evoked neural

responses r frac14 (r1 r2 rN)T and uses them to determine the probability of a behavior b In the deterministic model shown here

neural responses r are multiplied by weights xfrac14 (x1 xN)T and summed to form a decision variable (ufrac14Rixiri) which is compared

to a threshold (s) to predict a binary perceptual decision (c) One can define a biologically interpretable psychometric function by

using the output r of a neural encoding model as the input to a behavioral decoding model

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 3

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

A fairly straightforward derivation of a psychomet-ric function defined using the neural encoding modelshown in Figure 1 and with linear decoding (Fisherlinear discriminant) is given in Appendix B Thisanalysis is similar to those presented in several previousstudies (eg Ma 2010 Pestilli et al 2009) Ourderivation yields the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth4THORN

where w(c) denotes the contrast tuning (also calledcontrast gain) of neurons in the population and K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing population sensi-

tivity to changes in orientation around the verticalreference (0 frac14 p2) at 100 contrast

In this paper we consider three different functionalforms for the contrast gain function w(c) One formsuggested from neurophysiological findings (Albrechtamp Hamilton 1982) is the Naka-Rushton function

w1

cgeth1THORN

frac14 cn

cn thorn cn50

eth5THORN

having parameters g(1)frac14 (n c50)T This functional form

(Equation 5) is also sometimes referred to as thehyperbolic ratio function (Albrecht amp Hamilton 1982)Another form is the hyperbolic tangent (tanh) function

w2

cgeth2THORN

frac14 tanhethbcTHORN frac14 ebc ebc

ebc thorn ebc eth6THORN

commonly used in machine learning (Bishop 2006)having parameter g(2) frac14 (b)T Both of these functionalforms (Naka-Rushon Tanh) are shown in Figure 2Finally we consider a Gaussian form that allows forthe possibility of a nonmonotonic relationship betweencontrast and firing rate given by

w3

cgeth3THORN

frac14 exp 1

2r2ethc lTHORN2

eth7THORN

with parameters g(3) frac14 (l r)TOur interest in fitting multiple models to the same

data set is to test the efficacy of psychophysical data fordistinguishing between competing hypotheses of neuralencoding This approach follows previous work usingfits of multiple models to behavioral data to gaininsight into sensory or cognitive mechanisms (Qamar etal 2013 van den Berg Awh amp Ma 2014) Thecomparison between the Naka-Rushton model and theGaussian model is a coarse-grained qualitative com-parison because the two models are qualitatively verydifferent (monotonic vs nonmonotonic) whereas thecomparison between Naka-Rushton and Tanh is a fine-grained quantitative comparison because the twomodels are both monotonic and qualitatively verysimilar (Figure 2)

Recovering neural encoding model parameters

Fitting thresholds

Because we can write our psychometric function(Equation 4) in terms of d0 (25) we can use thresholdstaken at multiple contrasts to estimate the psychomet-ric function parameters using least-squares curvefitting Figure 3 shows the best fit of the model(Equation 4) with Naka-Rushton contrast gain(Equation 5) to the data from Skottun Bradley SclarOhzawa and Freeman (1987 their figure 1) We see inFigure 3 that this model provides an excellent fit totheir data (Supplementary Figure S1) We find that thevalues recovered for the Naka-Rushton contrastfunction parameters n c50 from their threshold data liewithin the range measured in previous neurophysio-logical work (Albrecht amp Hamilton 1982) as shown inFigure 4 (red circles)

Figure 2 Two competing hypotheses for the functional form of

contrast gain tuning Despite the qualitative similarity of the

Naka-Rushton (Equation 5) and Tanh (Equation 6) models we

observe a better quantitative fit to neurophysiological data by

the Naka-Rushton function particularly at lower contrasts (a)

Fits of both models (Equations 5 and 6) to contrast gain

responses of a representative V1 neuron Data points

graphically adapted from figure 3 of Albrecht and Hamilton

(1982) (b) Fits of both models (Equations 5 and 6) to contrast

gain responses of several V1 neurons Data points graphically

adapted from figure 1 of Albrecht and Hamilton (1982) (c)

Residual sum-of-squares error for the fits of both models in (b)

We see a better fit for the Naka-Rushton model (sign-rank test

n frac14 9 p frac14 00039 001)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 4

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Experiment 1 Direct estimation from psychophysicaltrials

The data in Skottun et al (1987) only providesthresholds and therefore our estimates of g(1) frac14 (nc50)

T were not obtained as in most psychophysicalexperiments in which one finds the maximum likeli-hood estimate of model parameters using stimulus-response data Dw frac14 fsi bignifrac141 (Kingdom amp Prins2010) In order to directly test the use of psychophysicaldata to recover the parameters of neural tuning curveswe ran an orientation discrimination experiment(Experiment 1) on nine subjects (seven naive) in whichwe covaried orientation and contrast Additionaldetails of Experiment 1 are described in theSupplementary Methods Contour plots of subjectperformance P(b frac14 1js frac14 (cd)TKg(1)) are shown inFigure 5 (and Supplementary Figure S3) with fits of themodel (Equation 4) with Naka-Rushton gain (Equa-

tion 5) to subject data in the middle column We foundin a subsequent experiment (Supplementary Material)that this model could also generalize reasonably wellfor most (but not all) subjects to predict responses to asmall validation set of novel stimuli (SupplementaryFigure S4)

We see in Figure 4 that the values of the Naka-Rushton parameters n c50 estimated from ourExperiment 1 data (black diamonds) lie within theneurophysiologically observed range Numerical val-ues of these parameters are given in SupplementaryTables S1 and S2 Interestingly we find that all of ourestimates of the half-saturation parameter c50 ob-tained in these experiments (along with five of sixestimates of c50 from Skottun et al 1987) lie towardthe lower end of the physiologically observed range(ie around 5 contrast see Albrecht amp Hamilton1982) This suggests the subjects may be using theneurons that are most sensitive to contrast when theyperform the task consistent with the lsquolsquolower enve-lopersquorsquo principle of sensory coding (Egger amp Britten2013 Mountcastle LaMotte amp Carli 1972 L Wanget al 2007)

Comparing competing models

Exploring model space

In Experiment 1 whose goal was to show that onecan estimate neural model parameters from psycho-physical data we assumed a known form (Equation 5)of the contrast gain function based on previous

Figure 3 Fits of the behavioral decoding model (Equation 4)

with Naka-Rushton contrast gain (Equation 5) to threshold data

(79 performance) graphically adapted from figure 1 of Skottun

et al (1987) Plot of residual sum-of-squares error for models

with Naka-Rushton (red) and Tanh (green) contrast gain

(Equation 6) are given in Supplementary Figure S1

Figure 4 Estimates of neural contrast gain function parameters

n and c50 (Naka-Rushton) from psychophysical data Red dots

denote estimates from threshold data (Skottun et al 1987)

black diamonds are estimates from fitting the model directly to

psychophysical trial data (Experiment 1) We see that all of the

estimates lie within the physiological range (blue lines frac14 l 6

196 r) (Albrecht amp Hamilton 1982)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

neurophysiological investigations (Albrecht amp Hamil-ton 1982) Supposing that the correct functional formof the contrast gain function w(c) was not knownbeforehand from physiological recordings we may beinterested in evaluating various possibilities by fittingthe model (Equation 4) to psychophysical data withdifferent choices for w(c) and seeing which bestaccounts for the observed results Such informationderived from relatively fast and inexpensive psycho-physical experiments could provide important clues toguide subsequent neurophysiology research

In order to test the ability of psychophysicalexperiments to compare competing models of neuralcontrast gain we will also consider two otherpossibilities for the contrast gain given by thehyperbolic tangent (Tanh) function (Equation 6) andthe familiar Gaussian tuning curve (Equation 7) Thesethree possible choices (Equations 5 6 and 7) ofcontrast gain function w(c) define a discrete space ofthree competing neural encoding models which weindex by i frac14 1 2 3 By fitting each model topsychophysical data we may evaluate their relativelikelihoods using the Akaike Information Criterion(AIC) which measures goodness-of-fit while penalizingmodel complexity (Akaike 1974 Burnham amp Ander-son 2003) Previous work has shown that it isimportant that any model comparison method takes

complexity into account because an overly complexmodel often fits training data well but fails to generalizeto novel observations (Bishop 2006 Pitt amp Myung2002)

We denote the value of the AIC for the i-th model byAICi with model i being preferred to model j if AICi AICj We define a model preference index

Pij frac14 AICi AICj eth8THORNwhere a positive value of Pindashj indicates model i ispreferred to model j and a negative value indicating j ispreferred to i The model preference index is definedimplicitly with respect to a fixed number of observa-tions ie Pindashjfrac14 Pindashj (n) where n is the number of trialsused to compute the AIC We define a change in modelpreference after k additional trials as

DPij frac14 Pijethnthorn kTHORN PijethnTHORN eth9THORNIn our analysis model 1 assumes Naka-Rushtoncontrast tuning (Equation 5) model 2 assumes Tanhtuning (Equation 6) and model 3 assumes Gaussiantuning (Equation 7)

Computing the AIC for fits of all three models tothe data collected in Experiment 1 allows us todetermine the model preferences P1ndash2 (Naka-Rush-tonndashTanh) and P1ndash3 (Naka-RushtonndashGaussian) Wesee in Figure 6a that the Naka-Rushton model ispreferred over the Gaussian model for all nine subjectsand over the Tanh model for seven of nine subjectswith the preference being quite strong for manysubjects Statistical tests show that over these ninesubjects both model preferences are significantlydifferent from zero (sign-rank test nfrac14 9 P1ndash2 0 pfrac14002 P1ndash3 0 p frac14 0004) Figure 6b shows how thismodel preference P1ndash2 evolves with the number ofexperimental trials We see that as more trials arecollected the model preference (for most subjects)seems to change in favor of the Naka-Rushton modelwhose better ability to fit the data overcomes thecomplexity penalty imposed by the AIC We also seefrom Figure 6b that the final model preferences areestablished after about 1000ndash1200 trials Similarresults were obtained using the Bayes InformationCriterion which more severely penalizes modelcomplexity (Bishop 2006) changing the final modelpreference for only one subject (SupplementaryFigures S5 and S6)

Experiment 2 Optimizing stimuli for model comparison

In Experiment 1 data was collected using themethod of constant stimuli which previous work hassuggested may be suboptimal for purposes of modelestimation and comparison (Watson amp Fitzhugh1990) Therefore we conducted a second experiment(Experiment 2) in order to determine if stimuli

Figure 5 Contour plots of the psychometric function P(bfrac14 1jsfrac14(c d)T) as a function of orientation (d) and contrast (c) for

three subjects in Experiment 1 Other subjects shown in

Supplementary Figure S3 Left Raw data Middle Fits of model

(Equation 4) with Naka-Rushton (Equation 5) contrast gain

(model 1) to data Right Fits of (Equation 4) with Tanh

(Equation 6) contrast gain (model 2) to data

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 4: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

A fairly straightforward derivation of a psychomet-ric function defined using the neural encoding modelshown in Figure 1 and with linear decoding (Fisherlinear discriminant) is given in Appendix B Thisanalysis is similar to those presented in several previousstudies (eg Ma 2010 Pestilli et al 2009) Ourderivation yields the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth4THORN

where w(c) denotes the contrast tuning (also calledcontrast gain) of neurons in the population and K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing population sensi-

tivity to changes in orientation around the verticalreference (0 frac14 p2) at 100 contrast

In this paper we consider three different functionalforms for the contrast gain function w(c) One formsuggested from neurophysiological findings (Albrechtamp Hamilton 1982) is the Naka-Rushton function

w1

cgeth1THORN

frac14 cn

cn thorn cn50

eth5THORN

having parameters g(1)frac14 (n c50)T This functional form

(Equation 5) is also sometimes referred to as thehyperbolic ratio function (Albrecht amp Hamilton 1982)Another form is the hyperbolic tangent (tanh) function

w2

cgeth2THORN

frac14 tanhethbcTHORN frac14 ebc ebc

ebc thorn ebc eth6THORN

commonly used in machine learning (Bishop 2006)having parameter g(2) frac14 (b)T Both of these functionalforms (Naka-Rushon Tanh) are shown in Figure 2Finally we consider a Gaussian form that allows forthe possibility of a nonmonotonic relationship betweencontrast and firing rate given by

w3

cgeth3THORN

frac14 exp 1

2r2ethc lTHORN2

eth7THORN

with parameters g(3) frac14 (l r)TOur interest in fitting multiple models to the same

data set is to test the efficacy of psychophysical data fordistinguishing between competing hypotheses of neuralencoding This approach follows previous work usingfits of multiple models to behavioral data to gaininsight into sensory or cognitive mechanisms (Qamar etal 2013 van den Berg Awh amp Ma 2014) Thecomparison between the Naka-Rushton model and theGaussian model is a coarse-grained qualitative com-parison because the two models are qualitatively verydifferent (monotonic vs nonmonotonic) whereas thecomparison between Naka-Rushton and Tanh is a fine-grained quantitative comparison because the twomodels are both monotonic and qualitatively verysimilar (Figure 2)

Recovering neural encoding model parameters

Fitting thresholds

Because we can write our psychometric function(Equation 4) in terms of d0 (25) we can use thresholdstaken at multiple contrasts to estimate the psychomet-ric function parameters using least-squares curvefitting Figure 3 shows the best fit of the model(Equation 4) with Naka-Rushton contrast gain(Equation 5) to the data from Skottun Bradley SclarOhzawa and Freeman (1987 their figure 1) We see inFigure 3 that this model provides an excellent fit totheir data (Supplementary Figure S1) We find that thevalues recovered for the Naka-Rushton contrastfunction parameters n c50 from their threshold data liewithin the range measured in previous neurophysio-logical work (Albrecht amp Hamilton 1982) as shown inFigure 4 (red circles)

Figure 2 Two competing hypotheses for the functional form of

contrast gain tuning Despite the qualitative similarity of the

Naka-Rushton (Equation 5) and Tanh (Equation 6) models we

observe a better quantitative fit to neurophysiological data by

the Naka-Rushton function particularly at lower contrasts (a)

Fits of both models (Equations 5 and 6) to contrast gain

responses of a representative V1 neuron Data points

graphically adapted from figure 3 of Albrecht and Hamilton

(1982) (b) Fits of both models (Equations 5 and 6) to contrast

gain responses of several V1 neurons Data points graphically

adapted from figure 1 of Albrecht and Hamilton (1982) (c)

Residual sum-of-squares error for the fits of both models in (b)

We see a better fit for the Naka-Rushton model (sign-rank test

n frac14 9 p frac14 00039 001)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 4

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Experiment 1 Direct estimation from psychophysicaltrials

The data in Skottun et al (1987) only providesthresholds and therefore our estimates of g(1) frac14 (nc50)

T were not obtained as in most psychophysicalexperiments in which one finds the maximum likeli-hood estimate of model parameters using stimulus-response data Dw frac14 fsi bignifrac141 (Kingdom amp Prins2010) In order to directly test the use of psychophysicaldata to recover the parameters of neural tuning curveswe ran an orientation discrimination experiment(Experiment 1) on nine subjects (seven naive) in whichwe covaried orientation and contrast Additionaldetails of Experiment 1 are described in theSupplementary Methods Contour plots of subjectperformance P(b frac14 1js frac14 (cd)TKg(1)) are shown inFigure 5 (and Supplementary Figure S3) with fits of themodel (Equation 4) with Naka-Rushton gain (Equa-

tion 5) to subject data in the middle column We foundin a subsequent experiment (Supplementary Material)that this model could also generalize reasonably wellfor most (but not all) subjects to predict responses to asmall validation set of novel stimuli (SupplementaryFigure S4)

We see in Figure 4 that the values of the Naka-Rushton parameters n c50 estimated from ourExperiment 1 data (black diamonds) lie within theneurophysiologically observed range Numerical val-ues of these parameters are given in SupplementaryTables S1 and S2 Interestingly we find that all of ourestimates of the half-saturation parameter c50 ob-tained in these experiments (along with five of sixestimates of c50 from Skottun et al 1987) lie towardthe lower end of the physiologically observed range(ie around 5 contrast see Albrecht amp Hamilton1982) This suggests the subjects may be using theneurons that are most sensitive to contrast when theyperform the task consistent with the lsquolsquolower enve-lopersquorsquo principle of sensory coding (Egger amp Britten2013 Mountcastle LaMotte amp Carli 1972 L Wanget al 2007)

Comparing competing models

Exploring model space

In Experiment 1 whose goal was to show that onecan estimate neural model parameters from psycho-physical data we assumed a known form (Equation 5)of the contrast gain function based on previous

Figure 3 Fits of the behavioral decoding model (Equation 4)

with Naka-Rushton contrast gain (Equation 5) to threshold data

(79 performance) graphically adapted from figure 1 of Skottun

et al (1987) Plot of residual sum-of-squares error for models

with Naka-Rushton (red) and Tanh (green) contrast gain

(Equation 6) are given in Supplementary Figure S1

Figure 4 Estimates of neural contrast gain function parameters

n and c50 (Naka-Rushton) from psychophysical data Red dots

denote estimates from threshold data (Skottun et al 1987)

black diamonds are estimates from fitting the model directly to

psychophysical trial data (Experiment 1) We see that all of the

estimates lie within the physiological range (blue lines frac14 l 6

196 r) (Albrecht amp Hamilton 1982)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

neurophysiological investigations (Albrecht amp Hamil-ton 1982) Supposing that the correct functional formof the contrast gain function w(c) was not knownbeforehand from physiological recordings we may beinterested in evaluating various possibilities by fittingthe model (Equation 4) to psychophysical data withdifferent choices for w(c) and seeing which bestaccounts for the observed results Such informationderived from relatively fast and inexpensive psycho-physical experiments could provide important clues toguide subsequent neurophysiology research

In order to test the ability of psychophysicalexperiments to compare competing models of neuralcontrast gain we will also consider two otherpossibilities for the contrast gain given by thehyperbolic tangent (Tanh) function (Equation 6) andthe familiar Gaussian tuning curve (Equation 7) Thesethree possible choices (Equations 5 6 and 7) ofcontrast gain function w(c) define a discrete space ofthree competing neural encoding models which weindex by i frac14 1 2 3 By fitting each model topsychophysical data we may evaluate their relativelikelihoods using the Akaike Information Criterion(AIC) which measures goodness-of-fit while penalizingmodel complexity (Akaike 1974 Burnham amp Ander-son 2003) Previous work has shown that it isimportant that any model comparison method takes

complexity into account because an overly complexmodel often fits training data well but fails to generalizeto novel observations (Bishop 2006 Pitt amp Myung2002)

We denote the value of the AIC for the i-th model byAICi with model i being preferred to model j if AICi AICj We define a model preference index

Pij frac14 AICi AICj eth8THORNwhere a positive value of Pindashj indicates model i ispreferred to model j and a negative value indicating j ispreferred to i The model preference index is definedimplicitly with respect to a fixed number of observa-tions ie Pindashjfrac14 Pindashj (n) where n is the number of trialsused to compute the AIC We define a change in modelpreference after k additional trials as

DPij frac14 Pijethnthorn kTHORN PijethnTHORN eth9THORNIn our analysis model 1 assumes Naka-Rushtoncontrast tuning (Equation 5) model 2 assumes Tanhtuning (Equation 6) and model 3 assumes Gaussiantuning (Equation 7)

Computing the AIC for fits of all three models tothe data collected in Experiment 1 allows us todetermine the model preferences P1ndash2 (Naka-Rush-tonndashTanh) and P1ndash3 (Naka-RushtonndashGaussian) Wesee in Figure 6a that the Naka-Rushton model ispreferred over the Gaussian model for all nine subjectsand over the Tanh model for seven of nine subjectswith the preference being quite strong for manysubjects Statistical tests show that over these ninesubjects both model preferences are significantlydifferent from zero (sign-rank test nfrac14 9 P1ndash2 0 pfrac14002 P1ndash3 0 p frac14 0004) Figure 6b shows how thismodel preference P1ndash2 evolves with the number ofexperimental trials We see that as more trials arecollected the model preference (for most subjects)seems to change in favor of the Naka-Rushton modelwhose better ability to fit the data overcomes thecomplexity penalty imposed by the AIC We also seefrom Figure 6b that the final model preferences areestablished after about 1000ndash1200 trials Similarresults were obtained using the Bayes InformationCriterion which more severely penalizes modelcomplexity (Bishop 2006) changing the final modelpreference for only one subject (SupplementaryFigures S5 and S6)

Experiment 2 Optimizing stimuli for model comparison

In Experiment 1 data was collected using themethod of constant stimuli which previous work hassuggested may be suboptimal for purposes of modelestimation and comparison (Watson amp Fitzhugh1990) Therefore we conducted a second experiment(Experiment 2) in order to determine if stimuli

Figure 5 Contour plots of the psychometric function P(bfrac14 1jsfrac14(c d)T) as a function of orientation (d) and contrast (c) for

three subjects in Experiment 1 Other subjects shown in

Supplementary Figure S3 Left Raw data Middle Fits of model

(Equation 4) with Naka-Rushton (Equation 5) contrast gain

(model 1) to data Right Fits of (Equation 4) with Tanh

(Equation 6) contrast gain (model 2) to data

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 5: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Experiment 1 Direct estimation from psychophysicaltrials

The data in Skottun et al (1987) only providesthresholds and therefore our estimates of g(1) frac14 (nc50)

T were not obtained as in most psychophysicalexperiments in which one finds the maximum likeli-hood estimate of model parameters using stimulus-response data Dw frac14 fsi bignifrac141 (Kingdom amp Prins2010) In order to directly test the use of psychophysicaldata to recover the parameters of neural tuning curveswe ran an orientation discrimination experiment(Experiment 1) on nine subjects (seven naive) in whichwe covaried orientation and contrast Additionaldetails of Experiment 1 are described in theSupplementary Methods Contour plots of subjectperformance P(b frac14 1js frac14 (cd)TKg(1)) are shown inFigure 5 (and Supplementary Figure S3) with fits of themodel (Equation 4) with Naka-Rushton gain (Equa-

tion 5) to subject data in the middle column We foundin a subsequent experiment (Supplementary Material)that this model could also generalize reasonably wellfor most (but not all) subjects to predict responses to asmall validation set of novel stimuli (SupplementaryFigure S4)

We see in Figure 4 that the values of the Naka-Rushton parameters n c50 estimated from ourExperiment 1 data (black diamonds) lie within theneurophysiologically observed range Numerical val-ues of these parameters are given in SupplementaryTables S1 and S2 Interestingly we find that all of ourestimates of the half-saturation parameter c50 ob-tained in these experiments (along with five of sixestimates of c50 from Skottun et al 1987) lie towardthe lower end of the physiologically observed range(ie around 5 contrast see Albrecht amp Hamilton1982) This suggests the subjects may be using theneurons that are most sensitive to contrast when theyperform the task consistent with the lsquolsquolower enve-lopersquorsquo principle of sensory coding (Egger amp Britten2013 Mountcastle LaMotte amp Carli 1972 L Wanget al 2007)

Comparing competing models

Exploring model space

In Experiment 1 whose goal was to show that onecan estimate neural model parameters from psycho-physical data we assumed a known form (Equation 5)of the contrast gain function based on previous

Figure 3 Fits of the behavioral decoding model (Equation 4)

with Naka-Rushton contrast gain (Equation 5) to threshold data

(79 performance) graphically adapted from figure 1 of Skottun

et al (1987) Plot of residual sum-of-squares error for models

with Naka-Rushton (red) and Tanh (green) contrast gain

(Equation 6) are given in Supplementary Figure S1

Figure 4 Estimates of neural contrast gain function parameters

n and c50 (Naka-Rushton) from psychophysical data Red dots

denote estimates from threshold data (Skottun et al 1987)

black diamonds are estimates from fitting the model directly to

psychophysical trial data (Experiment 1) We see that all of the

estimates lie within the physiological range (blue lines frac14 l 6

196 r) (Albrecht amp Hamilton 1982)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 5

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

neurophysiological investigations (Albrecht amp Hamil-ton 1982) Supposing that the correct functional formof the contrast gain function w(c) was not knownbeforehand from physiological recordings we may beinterested in evaluating various possibilities by fittingthe model (Equation 4) to psychophysical data withdifferent choices for w(c) and seeing which bestaccounts for the observed results Such informationderived from relatively fast and inexpensive psycho-physical experiments could provide important clues toguide subsequent neurophysiology research

In order to test the ability of psychophysicalexperiments to compare competing models of neuralcontrast gain we will also consider two otherpossibilities for the contrast gain given by thehyperbolic tangent (Tanh) function (Equation 6) andthe familiar Gaussian tuning curve (Equation 7) Thesethree possible choices (Equations 5 6 and 7) ofcontrast gain function w(c) define a discrete space ofthree competing neural encoding models which weindex by i frac14 1 2 3 By fitting each model topsychophysical data we may evaluate their relativelikelihoods using the Akaike Information Criterion(AIC) which measures goodness-of-fit while penalizingmodel complexity (Akaike 1974 Burnham amp Ander-son 2003) Previous work has shown that it isimportant that any model comparison method takes

complexity into account because an overly complexmodel often fits training data well but fails to generalizeto novel observations (Bishop 2006 Pitt amp Myung2002)

We denote the value of the AIC for the i-th model byAICi with model i being preferred to model j if AICi AICj We define a model preference index

Pij frac14 AICi AICj eth8THORNwhere a positive value of Pindashj indicates model i ispreferred to model j and a negative value indicating j ispreferred to i The model preference index is definedimplicitly with respect to a fixed number of observa-tions ie Pindashjfrac14 Pindashj (n) where n is the number of trialsused to compute the AIC We define a change in modelpreference after k additional trials as

DPij frac14 Pijethnthorn kTHORN PijethnTHORN eth9THORNIn our analysis model 1 assumes Naka-Rushtoncontrast tuning (Equation 5) model 2 assumes Tanhtuning (Equation 6) and model 3 assumes Gaussiantuning (Equation 7)

Computing the AIC for fits of all three models tothe data collected in Experiment 1 allows us todetermine the model preferences P1ndash2 (Naka-Rush-tonndashTanh) and P1ndash3 (Naka-RushtonndashGaussian) Wesee in Figure 6a that the Naka-Rushton model ispreferred over the Gaussian model for all nine subjectsand over the Tanh model for seven of nine subjectswith the preference being quite strong for manysubjects Statistical tests show that over these ninesubjects both model preferences are significantlydifferent from zero (sign-rank test nfrac14 9 P1ndash2 0 pfrac14002 P1ndash3 0 p frac14 0004) Figure 6b shows how thismodel preference P1ndash2 evolves with the number ofexperimental trials We see that as more trials arecollected the model preference (for most subjects)seems to change in favor of the Naka-Rushton modelwhose better ability to fit the data overcomes thecomplexity penalty imposed by the AIC We also seefrom Figure 6b that the final model preferences areestablished after about 1000ndash1200 trials Similarresults were obtained using the Bayes InformationCriterion which more severely penalizes modelcomplexity (Bishop 2006) changing the final modelpreference for only one subject (SupplementaryFigures S5 and S6)

Experiment 2 Optimizing stimuli for model comparison

In Experiment 1 data was collected using themethod of constant stimuli which previous work hassuggested may be suboptimal for purposes of modelestimation and comparison (Watson amp Fitzhugh1990) Therefore we conducted a second experiment(Experiment 2) in order to determine if stimuli

Figure 5 Contour plots of the psychometric function P(bfrac14 1jsfrac14(c d)T) as a function of orientation (d) and contrast (c) for

three subjects in Experiment 1 Other subjects shown in

Supplementary Figure S3 Left Raw data Middle Fits of model

(Equation 4) with Naka-Rushton (Equation 5) contrast gain

(model 1) to data Right Fits of (Equation 4) with Tanh

(Equation 6) contrast gain (model 2) to data

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 6: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

neurophysiological investigations (Albrecht amp Hamil-ton 1982) Supposing that the correct functional formof the contrast gain function w(c) was not knownbeforehand from physiological recordings we may beinterested in evaluating various possibilities by fittingthe model (Equation 4) to psychophysical data withdifferent choices for w(c) and seeing which bestaccounts for the observed results Such informationderived from relatively fast and inexpensive psycho-physical experiments could provide important clues toguide subsequent neurophysiology research

In order to test the ability of psychophysicalexperiments to compare competing models of neuralcontrast gain we will also consider two otherpossibilities for the contrast gain given by thehyperbolic tangent (Tanh) function (Equation 6) andthe familiar Gaussian tuning curve (Equation 7) Thesethree possible choices (Equations 5 6 and 7) ofcontrast gain function w(c) define a discrete space ofthree competing neural encoding models which weindex by i frac14 1 2 3 By fitting each model topsychophysical data we may evaluate their relativelikelihoods using the Akaike Information Criterion(AIC) which measures goodness-of-fit while penalizingmodel complexity (Akaike 1974 Burnham amp Ander-son 2003) Previous work has shown that it isimportant that any model comparison method takes

complexity into account because an overly complexmodel often fits training data well but fails to generalizeto novel observations (Bishop 2006 Pitt amp Myung2002)

We denote the value of the AIC for the i-th model byAICi with model i being preferred to model j if AICi AICj We define a model preference index

Pij frac14 AICi AICj eth8THORNwhere a positive value of Pindashj indicates model i ispreferred to model j and a negative value indicating j ispreferred to i The model preference index is definedimplicitly with respect to a fixed number of observa-tions ie Pindashjfrac14 Pindashj (n) where n is the number of trialsused to compute the AIC We define a change in modelpreference after k additional trials as

DPij frac14 Pijethnthorn kTHORN PijethnTHORN eth9THORNIn our analysis model 1 assumes Naka-Rushtoncontrast tuning (Equation 5) model 2 assumes Tanhtuning (Equation 6) and model 3 assumes Gaussiantuning (Equation 7)

Computing the AIC for fits of all three models tothe data collected in Experiment 1 allows us todetermine the model preferences P1ndash2 (Naka-Rush-tonndashTanh) and P1ndash3 (Naka-RushtonndashGaussian) Wesee in Figure 6a that the Naka-Rushton model ispreferred over the Gaussian model for all nine subjectsand over the Tanh model for seven of nine subjectswith the preference being quite strong for manysubjects Statistical tests show that over these ninesubjects both model preferences are significantlydifferent from zero (sign-rank test nfrac14 9 P1ndash2 0 pfrac14002 P1ndash3 0 p frac14 0004) Figure 6b shows how thismodel preference P1ndash2 evolves with the number ofexperimental trials We see that as more trials arecollected the model preference (for most subjects)seems to change in favor of the Naka-Rushton modelwhose better ability to fit the data overcomes thecomplexity penalty imposed by the AIC We also seefrom Figure 6b that the final model preferences areestablished after about 1000ndash1200 trials Similarresults were obtained using the Bayes InformationCriterion which more severely penalizes modelcomplexity (Bishop 2006) changing the final modelpreference for only one subject (SupplementaryFigures S5 and S6)

Experiment 2 Optimizing stimuli for model comparison

In Experiment 1 data was collected using themethod of constant stimuli which previous work hassuggested may be suboptimal for purposes of modelestimation and comparison (Watson amp Fitzhugh1990) Therefore we conducted a second experiment(Experiment 2) in order to determine if stimuli

Figure 5 Contour plots of the psychometric function P(bfrac14 1jsfrac14(c d)T) as a function of orientation (d) and contrast (c) for

three subjects in Experiment 1 Other subjects shown in

Supplementary Figure S3 Left Raw data Middle Fits of model

(Equation 4) with Naka-Rushton (Equation 5) contrast gain

(model 1) to data Right Fits of (Equation 4) with Tanh

(Equation 6) contrast gain (model 2) to data

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 6

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 7: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

explicitly optimized for purposes of model comparisonwere more effective for this goal than the stimuli usedin Experiment 1 (Supplementary Figure S2)

There are several ways to define the optimalcomparison stimulus (OCS) in neurophysiology andpsychophysics experiments (Cavagnaro et al 2010DiMattina amp Zhang 2011 Z Wang amp Simoncelli2008) and in the current study we used an informa-tionndashtheoretic criterion that finds the stimulus thatminimizes the expected entropy of the posteriordensity over model space (Cavagnaro et al 2010)This stimulus sfrac14 (c d)T may be found by maximizingthe expression

UethCTHORNethsTHORN frac14Xmifrac141

P0ethiTHORNDKL pethbjs iTHORN pethbjsTHORNfrac12 eth10THORN

where P0(i) is the prior probability of each model DKL

the Kullbeck-Lieber divergence (Cover amp Thomas2006) p(bjsi) is the response probability conditionedon the stimulus and model and p(bjs) is the overallresponse probability averaged across models Intui-tively this method minimizes uncertainty about whichmodel is true by presenting stimuli that are expected toyield a posterior density with most of the probabilitymass on one or a few models ie a density withminimum entropy (Cover amp Thomas 2006) This

informationndashtheoretic criterion has been used incognitive science to choose stimuli optimized fortesting competing hypotheses of memory decay anddecision making under risk (Cavagnaro GonzalezMyung amp Pitt 2013 Cavagnaro Pitt amp Myung2011)

Data was obtained during a two-phase experimentconducted on a single testing day an estimation phase(E-phase Experiment 1) in which data is collected formodel-fitting purposes followed by a comparisonphase (C-phase Experiment 2) in which stimulioptimized for model discrimination were presented(DiMattina amp Zhang 2011) Immediately after theconclusion of Experiment 1 (E-phase NE frac14 1200trials) a single OCS was found by optimizing(Equation 10) based on fits of model 1 (Naka-Rushton) and model 2 (Tanh) to Experiment 1 dataSearch for the OCS was restricted to contrasts greaterthan 1 and orientations from 08 to 208 based onobservation of at what point the two models seemedto differ the most as well as the fact that stimulipresented at values less than 1 contrast are oftenbarely visible (Campbell amp Robson 1968) The OCSfor each subject are illustrated in Figure 7 (leftpanels) Note that many of these stimuli have contrastc rsquo 1 and orientation d 58 and hence lie outsidethe range of stimuli (contrasts and orientations) usedto estimate the models (Supplementary Figures S2and S4)

In Experiment 2 the OCS was repeatedly presentedto the subject for NCfrac14 200 trials during the Experiment2 C-phase interleaved with 200 stimuli chosen atrandom with uniform probability from the stimulusgrid used during the Experiment 1 (SupplementaryFigure S2) for 400 trials total We will heretofore referto these randomly chosen Experiment 1 (E-phase)stimuli as IID stimuli We see from Figure 7 (rightpanels) that for many (but not all) subjects the OCS(blue curves) does a much better job than the IIDstimuli (green curves) of shifting the model preferenceP1ndash2 in the direction of the Naka-Rushton modelDP1ndash2 frac14 P1ndash2(NE thorn NC) ndash P1ndash2(NE) 0 Statisticalanalysis demonstrates that over all subjects the medianvalue of DP1ndash2 is significantly larger for the OCS(median DP1ndash2frac14541) than IID (median DP1ndash2frac14004)trials (sign-rank test nfrac14 9 pfrac14 00117)

Our goal in Experiment 2 was not to do an in-depthinvestigation of adaptive stimulus optimization meth-ods for model comparison (a very important problemneeding more research) but rather to demonstrate thepotential utility of such an approach Our resultssuggest that utilizing stimuli optimized for neuralencoding model comparison is certainly no worse andin many cases much better than continued presentationof the (IID) stimuli used in Experiment 1

Figure 6 Model preferences Pindashj based on fits of three

competing neural encoding models to data from Experiment 1

Model 1 assumes Naka-Rushton (Equation 5) contrast gain

model 2 assumes Tanh (Equation 6) contrast gain and model 3

assumes Gaussian (Equation 7) contrast gain (a) Final model

preferences P1ndash2 and P1ndash3 based on fits to all Experiment 1

trials For most subjects we see a final preference (P1ndash2 0) for

model 1 (Naka-Rushton) over model 2 and for all subjects we

see a preference (P1ndash3 0) for model 1 over model 3 (b)

Dynamics of model preference P1ndash2 for the two qualitatively

similar models (Naka-RushtonndashTanh) for the nfrac14 8 subjects

completing 2000thorn trials Final model preferences are estab-

lished by 1000 trials

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 7

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 8: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Numerical simulations of model comparisonexperiments

In order to more rigorously examine the potentialutility of adaptive OCS in the ideal case in which oneof the candidate models is actually the true processgenerating the data we performed a simulation ofExperiment 2 (C-phase) for all subjects In thesesimulations we took as the ground truth the Naka-Rushton model (model 1) and used the fit of thismodel to actual E-phase (Experiment 1) data togenerate synthetic C-phase (Experiment 2) data Wequantified the C-phase change in model preferenceindex DP1ndash2 for both IID and OCS data collectionstrategies in which the Naka-Rushton model wasassumed true In the actual experiments at the end ofthe E-phase there was already a model preference(P1ndash2 6frac14 0 see Figure 6a) so in order to determine howoften the two data collection strategies (OCS IID)would result in a correct choice given no initialpreference we set the initial model preference to zeroso that DP1ndash2 frac14 P1ndash2

Results of Nmc frac14 100 Monte Carlo simulations ofExperiment 2 are shown in Figure 8 In each panelwe plot the median value of P1ndash2 (thick lines blue frac14OCS green frac14 IID) the range containing 95 ofsimulations (thin lines) and the trajectory of P1ndash2

observed experimentally (red lines) For many (butnot all) subjects we see a reasonably good agreementbetween the simulation predictions and the observedchange in model preferences during the C-phase Wefind that over the group of subjects there is acorrelation (Pearson nfrac149 rfrac14071 pfrac14003) betweenthe predictions of DP1ndash2 predicted by the simulationsand those observed experimentally (SupplementaryFigure S7) The simulations tend to predict a largervalue of DP1ndash2 than observed experimentally (medi-an experiments frac14 541 simulations frac14 1359) al-though just like the experiments the median DP1ndash2

obtained is larger for simulations using OCS thanIID (median frac14 067) data collection strategies Wealso find that one is more likely to make a correctmodel choice using the OCS data collection method(Supplementary Table S3) with IID yielding a correctchoice after NC frac14 200 trials (given no initialpreference) in 80 of simulations but OCS in about99 Additional simulations also reveal that OCSstimuli can also be more effective for modelcomparison in cases in which model 2 is the groundtruth (Supplementary Figure S8) These simulationssuggest the potential usefulness of this adaptivestimulus optimization method for comparing com-peting models of neural encoding

Figure 7 Left panels OCS sfrac14 (c d)T for discriminating models 1 and 2 (black circles) superimposed on a contour plot of the model

comparison utility function (Equation 10) Color bars shown for only two subjects to minimize clutter Right panels Evolution of the

model preference P1ndash2 during Experiment 2 for both OCS (blue curves) and stimuli chosen at random from the grid used in

Experiment 1 (IID green curves) Top right panel graphically illustrates the change in model preference (DPindashj) defined in the text

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 8

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 9: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Discussion

Neural codes from behavior

For more than 40 years there has been a rich two-waytraffic of ideas between sensory neurophysiology andpsychophysics with computational modeling oftenforming the bridge between these levels of analysis (Goldamp Shadlen 2007 Nienborg et al 2012 Parker ampNewsome 1998 Romo amp de Lafuente 2013) Mostoften computational modeling has been applied toneural data in order to predict or explain behavior (egKiani et al 2014 Purushothaman amp Bradley 2005Shadlen amp Newsome 2001) rather than being applied tobehavioral data to gain insight about neural mechanismsHowever in a number of recent studies a growingnumber of investigators have taken the complementaryapproach of using behavioral experiments or neuralmodeling of optimal behavior to inform and connectwith neural encoding models Here we briefly reviewsome of this work before relating it to the present study

One example of deriving neural codes from behav-ioral considerations is accuracy maximization analysiswhich finds optimal neural encoding models for specificnatural perception tasks (Burge Fowlkes amp Banks

2010 Burge amp Geisler 2011 2014 2015 W GeislerPerry Super amp Gallogly 2001 W S Geisler 2008 WS Geisler et al 2009) This methodology has beenapplied to determine the neural receptive fields thatwould be optimal for performing natural vision taskssuch as separating figure from ground (Burge et al2010 W S Geisler et al 2009) estimating retinaldisparity (Burge amp Geisler 2014) and estimating thespeed of visual motion (Burge amp Geisler 2015) Theneural encoding models derived account for experi-mentally observed neural tuning properties andalthough these models were not estimated by fittingpsychophysical data (as done here) a Bayesian idealobserver reading out these optimal neural codesmanages to accurately account for human psycho-physical performance (eg Burge amp Geisler 2015)

Another line of research which fits theoreticallyoptimal performance has employed neural implemen-tations of Bayesian ideal observers to understand howoptimal or near-optimal behavioral performance can beexplained in terms of probabilistic population coding(Beck et al 2008 Ma 2010 Ma Beck Latham ampPouget 2006 Ma et al 2011 Qamar et al 2013) Onerecent study of this kind has demonstrated that one canaccount for near-optimal visual search behavior seen inhuman observers using a neural model implementingprobabilistic population codes that represent stimulus

Figure 8 Results from Nmcfrac14 100 Monte Carlo simulations of Experiment 2 in which synthetic data is generated by fits of the Naka-

Rushton model to Experiment 1 data Simulation results are shown for OCS (blue curves) and IID (green curves) stimuli with thick

lines denoting median values of P1ndash2 and thin lines denoting the middle 95 of values Superimposed on these plots are the dynamic

model preferences (red curves) actually observed during the real Experiment 2 performed on subjects (Figure 7)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 9

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 10: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

reliability (Ma et al 2011) Another recent study(Qamar et al 2013) demonstrated that a neural modelthat accounts for the ability of subjects to make trial-by-trial adjustments of decision boundaries in acategorization task automatically learns to performdivisive computations like those seen in visual neurons(Carandini amp Heeger 2012)

Another recent example of predicting receptive fieldproperties by modeling behavioral performance comesfrom a recent study (Yamins et al 2014) that exploreda large number of computational models of the ventralvisual stream using a high-throughput modelingtechnique (Pinto Doukhan DiCarlo amp Cox 2009)This work revealed that models that could account forhuman behavioral performance on a challenging objectrecognition task (but not fit to neural data) hadintermediate and output-layer units whose responsesclosely matched neural tuning observed in visual areasV4 and IT (Yamins et al 2014) Another neuralmodeling study (Salinas 2006) showed that one canexplain the shape of tuning curves used by differentsensory systems by taking into account the downstreammotor behavior that decodes these sensory representa-tions using examples as diverse as binocular disparityin vision and echo delay in bats These studies suggestthat behavior can provide strong constraints on thenature of neural computation in the sensory systems

Other efforts to use behavior to inform theories ofneural mechanism come from the perceptual learningliterature in which investigators have proposed neuralmodels that account for improvements in performancewith experience despite relatively stable early-stagesensory encoding (Dosher Jeter Liu amp Lu 2013Dosher amp Lu 1998 1999 Petrov et al 2005) Onerecent model demonstrates that Hebbian modificationsto the task-specific readout of a stable neural populationis sufficient to explain perceptual learning and explainsthe empirically observed lsquolsquoswitch costrsquorsquo when thebackground noise context changes (Petrov et al 2005)Other work (Bejjanki Beck Lu amp Pouget 2011) hassuggested that perceptual learning can be construed asimproved probabilistic inference in which altering onlyfeed-forward weights input weights to a recurrent neuralnetwork can yield a modest sharpening of tuning curvesas observed experimentally (Yang amp Maunsell 2004)

A number of studies have used classification images(Ahumada 1996 Murray 2011) to make directcomparisons between the properties of perceptualfilters and neural response properties (see review byNeri amp Levi 2006) For instance one such studydemonstrated that performance on a bar detection taskcould be explained using a combination of linearmatched filtering and contrast energy detection similarto mechanisms known to exist in V1 simple andcomplex cells (Neri amp Heeger 2002) Other studies haverevealed striking relationships between the optimal

perceptual filter for orientation discrimination and thereceptive fields of V1 neurons (Ringach 1998 Solo-mon 2002) or have demonstrated multiplicativeperceptual combination of visual cues similar to thatobserved physiologically (Neri 2004) Classificationimage studies (Eckstein Shimozaki amp Abbey 2002Murray et al 2003 Neri 2004) have demonstratedthat consistent with physiological studies of attentionaleffects on neurons the shape of perceptive fields do notchange with attention Taken as a whole this body ofwork suggests that the classification image techniquecan potentially shed light on neural mechanisms

Several recent studies have considered the optimaldistribution of neuronal tuning curves for efficientlyencoding sensory variables and the implications ofanisotropic neural populations for perceptual behavior(Ganguli amp Simoncelli 2010 2014 Girshick Landy ampSimoncelli 2011 Wei amp Stocker 2015) In addition tocomparing the predictions of theoretical models to thedistribution of neural tuning curves observed experi-mentally models of population decoding with suchanisotropic populations have also been shown to explainpsychophysical data such as orientation and spatialfrequency discrimination thresholds (Ganguli amp Simon-celli 2010) and perceptual biases (Girshick et al 2011Wei amp Stocker 2015) Although these studies do notdirectly infer physiological properties from fits ofpsychometric models to behavioral data they dodemonstrate that behaviorally relevant considerations(ie optimal representation of the world and perceptualdecisions) can explain some features of neural encoding

In the pattern vision literature a number ofinvestigators have utilized numerical simulations ofearly visual processing aimed at explaining psycho-physical performance on contrast detection tasks(Chirimuuta amp Tolhurst 2005 Clatworthy Chirimuu-ta Lauritzen amp Tolhurst 2003 Goris et al 2013Goris Wichmann amp Henning 2009) One study of thiskind demonstrated that a large number of well-knownresults in the contrast detection literature could beaccounted for by a neural population model of theearly visual system that takes into account knownbiological nonlinearities (Goris et al 2013) Anothernotable study modeling spatial pooling in humancontrast detection (Morgenstern amp Elder 2012) wasable to define analytical models specified in terms oflocal Gabor receptive field parameters The authorsfound that the best model to account for theirpsychophysical data had local receptive fields approx-imately the size of those seen in V1 whose outputs werepassed through an energy filter and summed similar toknown mechanisms in visual cortex As in the presentstudy (Figure 4) these authors presented a directcomparison with their estimated parameter values andpreviously published physiological data (Morgensternamp Elder 2012 figure 14)

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 10

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 11: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

One of the limitations of numerical models is that itcan often be difficult to directly relate the neural modelparameters to behavioral performance Thereforeanother recent study (May amp Solomon 2015a 2015b)took the approach of deriving analytical models ofpsychophysical performance on contrast detection anddiscrimination tasks using the natural link betweenpsychophysical performance and the Fisher informa-tion of a neural population (Dayan amp Abbott 2001)The authors managed to demonstrate a surprisinglysimple and intuitive relationship between the parame-ters of the neural code and perceptual performance andwere able to account for the results of previousnumerical simulation studies (Chirimuuta amp Tolhurst2005 Clatworthy et al 2003) Along these same linestwo recent studies (perhaps most closely related to thepresent work) used fits of low-dimensional analyticallydefined neural models to psychophysical data in orderto predict how contrast gain encoding in orientation-tuned visual neurons may be modulated by attentionand understand mechanisms of attentional pooling(Pestilli et al 2011 Pestilli et al 2009) This workdemonstrates very elegantly how one can use dataobtained from behavioral experiments to make precisequantitative predictions about neural encoding

Relationship to previous work

As detailed above a number of previous studies haveused fits of neural models to behavioral data in order togain insight about neural encoding and decodingmechanisms The current work is complementary tothese studies and makes a number of novel contribu-tions to extend this general approach further

First we present for didactic purposes a simplemathematical derivation that frames the psychometricfunction explicitly in terms of the neural encodingmodel showing that one can in principle use psycho-physical data to estimate neural encoding modelparameters (Appendix A) Although the main result(Equation 3) is not directly useful without specificassumptions about the neural encoding and behavioraldecoding models it serves to make explicit the generalapproach taken here and in related work

Second unlike many previous studies that either fitor compare neural models to previously collectedpsychophysical data (eg Goris et al 2013 May ampSolomon 2015a 2015b) or to theoretically optimalideal observer performance (eg Burge amp Geisler2014 2015 Geisler et al 2009) we performed our ownpsychophysical experiments on human subjects andestimated model parameters directly by fitting topsychophysical trial data

Third because our model was a simple low-dimensional analytical model (eg May amp Solomon

2015a 2015b Morgenstern amp Elder 2012 Pestilli et al2009) as opposed to being defined by a complexnumerical simulation (Chirimuuta amp Tolhurst 2005Clatworthy et al 2003 Goris et al 2013 Goris et al2009) or a high-dimensional perceptual filter (Ahuma-da 1996 Murray 2011 Neri amp Levi 2006) it waspossible to do fast model fitting online during thecourse of the experimental session rather than doing sopost hoc as in previous work As illustrated previously(DiMattina amp Zhang 2011 2013 Tam 2012) theability to estimate models in real time during the courseof the experiment is essential if one wishes to generatenovel stimuli to compare competing models

Fourth in contrast to many previous studies wepresent direct comparisons (Figure 4) between the valuesof model parameters estimated from fitting psycho-physical trial data and those independently measured(Albrecht amp Hamilton 1982) in physiological studies(but see Morgenstern amp Elder 2012 Neri amp Levi 2006for exceptions) This direct comparison made byourselves and others (eg Morgenstern amp Elder 2012)with previously published physiology data makes astrong case that one can get accurate estimates of neuralsystem parameters by fitting psychophysical trial data

Finally and perhaps most importantly althoughother studies have demonstrated how one can usepsychophysical data for post hoc model comparison(Morgenstern amp Elder 2012 Pestilli et al 2009 Qamaret al 2013 van den Berg et al 2014) we extend thisidea further by considering how one can adaptivelyoptimize stimuli explicitly during the experimentalsession for purposes of model comparison We showthat adaptively optimized stimuli are far more effectivefor model comparison than post hoc analyses usingboth experiments (Figure 7) and numerical simulations(Figure 8) Although qualitatively very different models(Naka-Rushton and Gaussian contrast gain) can bewell discriminated without this technique (Figure 6) itcan be very helpful to distinguish between qualitativelysimilar models (Naka-Rushton and Tanh see Figure2) We feel that this general approach of adaptivestimulus generation offers great promise for psycho-physical and physiological experiments (Cavagnaro etal 2013 Cavagnaro et al 2011 DiMattina amp Zhang2013 Myung et al 2013 Z Wang amp Simoncelli 2008)and is of great interest for future work

Limitations

In the example analyzed in this study we onlyestimated a relatively modest number of biologicallyinterpretable parameters from psychophysical dataHowever although the example we use is fairly modestthe theoretical results we present here are fully generaland can be applied in a variety of contexts subject only

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 11

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 12: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

to the practical limitations imposed by the amount ofpsychophysical data that is needed to reliably estimatehigh-dimensional models (Mineault et al 2009 Mur-ray 2011) It is of great interest for us to extend thismethodology to higher dimensional examples forinstance estimating a parametric model of orientation-tuning anisotropy in area V1 (see below)

In both the present example as well as many previousefforts to fit neural models to behavioral data theneural encoding models were fairly simple early-stageneural encoders for instance a population of V1 cellstuned to orientation andor contrast (Goris et al 2013May amp Solomon 2015a 2015b Pestilli et al 2011Pestilli et al 2009) In our opinion this method islikely to be most useful for recovering physiologicalproperties of low-level neural encoders that can bespecified by a few parameters Although previousstudies have fit neural models to behavioral data arisingfrom higher level cognitive tasks such as visual searchor working memory (Bays 2014 Ma et al 2011) it isless likely that the method presented here will be able toprovide much direct physiological insight in these cases

Another limitation of the present study is that werelied on fairly simple assumptions about the neuronalnoise (independent responses) and a linear decodingstrategy However even with these simplifications weattained excellent fits of our derived model to thepsychophysical data (Figure 5 Supplementary FigureS3) and the model we derived had reasonably goodpredictive validity for novel stimuli (SupplementaryFigure S4) Previous work has demonstrated thatsimple linear decoders are adequate for estimatingstimulus parameters from neural data (Berens et al2012) and has suggested that naive decoding strategiesthat do not take noise correlations into account can benearly as effective as decoding that assumes suchknowledge (May amp Solomon 2015a)

Finally a further limitation is that by design of theexperiment (which focused on low-contrast gratings)we were only concerned with estimating the parametersof the contrast gain functions for a subpopulation ofthe most sensitive neurons which we assumed to beidentically tuned In reality there is diversity in thecontrast thresholds (c50) and shapes (n) of contrast gainfunctions (Albrecht amp Hamilton 1982) Therefore ourresults only demonstrate that our subpopulation ofinterest is sufficient to explain the observed psycho-physical behavior and does not rule out the possibilitythat other neurons not considered by our model maycontribute as well

Future directions

We feel that there is a lot of potential for this generalmethodological approach to be applied to test hy-

potheses of the large-scale organization of heteroge-neous neural population codes using psychophysicalexperiments One well-studied example is the popula-tion of orientation-tuned neurons in V1 which aremore densely located and more narrowly tuned nearthe cardinal (horizontal vertical) than near obliqueorientations (Li Peterson amp Freeman 2003) Thisorientation-tuning anisotropy matches the statistics ofnatural images (Girshick et al 2011) and when suchan anisotropic neural population is combined with aBayesian decoder it can explain a number of biasesobserved in orientation discrimination tasks (Wei ampStocker 2015) One can in principle apply ourmethodology to estimate a parametric model describingheterogeneity in V1 tuning curve parameters (egvariations in density and tuning width as a function ofpreferred grating orientation) This could be accom-plished by defining a parametric model of the stimulus-dependent Fisher information IF() frac14 F( v) as afunction of reference orientation which would serveas the link between the neural population code andpsychophysical performance (May amp Solomon 2015aWei amp Stocker 2015) After estimating the parametersv from psychophysical experiments one can thenoptimize neural population code parameters h tominimize

R[0p)(F(v) IF(h))

2 d where IF(h)denotes the Fisher information predicted by a neuralencoding model having parameters h Because manydifferent neural population codes are capable of givingrise to very similar Fisher information profiles (Wei ampStocker 2015) additional constraints such as codingefficiency (Ganguli amp Simoncelli 2014) may benecessary in order to get a unique solution for neuralpopulation code parameters Conducting such techni-cally challenging psychophysics experiments aimed atunderstanding the large-scale organization of neuralpopulation codes is an interesting direction of futureresearch

Conclusions

Although psychophysics can certainly never sup-plant physiological studies several recent modelingstudies suggest that modeling behavioral data canprovide insights into neural encoding mechanismsPerhaps this ability of behavior to provide guidance toneurophysiology is not too surprising given the longhistory of psychophysical observations accuratelypredicting physiological mechanisms many years beforetheir discovery for instance the neural encoding ofcolor (Helmholtz 1925 Read 2015 Wald 1964Young 1802) We believe that behavioral studies willcontinue to play an important role in guidingneurophysiological research making the developmentof better computational methodology for integrating

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 12

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 13: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

behavioral and neurophysiological studies an impor-tant and worthwhile goal

Keywords computational modeling neural encodingpsychometric functions psychophysics

Acknowledgments

The author would like to thank FGCU studentSteven Davis for help with the experiments and thanksCurtis L Baker Jr Nick Prins and Wei-Ji Ma forcomments on the manuscript The author declares nocompeting financial interests

Commercial relationships noneCorresponding author Christopher DiMattinaEmail cdimattinafgcueduAddress Computational Perception Laboratory De-partment of Psychology Florida Gulf Coast Universi-ty Fort Myers FL USA

References

Ahumada A J (1996) Perceptual classification imagesfrom vernier acuity masked by noise Perception25 ECVP abstract supplement

Akaike H (1974) A new look at the statistical modelidentification Automatic Control IEEE Transac-tions on 19(6) 716ndash723

Albrecht D G amp Hamilton D B (1982) Striatecortex of monkey and cat Contrast responsefunction Journal of Neurophysiology 48(1) 217ndash237

Barbour D L amp Wang X (Feb 14 2003) Contrasttuning in auditory cortex Science 299(5609) 1073ndash1075

Bays P M (2014) Noise in neural populationsaccounts for errors in working memory TheJournal of Neuroscience 34(10) 3632ndash3645

Beck J M Ma W J Kiani R Hanks TChurchland A K Roitman J Pouget A(2008) Probabilistic population codes for Bayesiandecision making Neuron 60(6) 1142ndash1152

Bejjanki V R Beck J M Lu Z-L amp Pouget A(2011) Perceptual learning as improved probabi-listic inference in early sensory areas NatureNeuroscience 14(5) 642ndash648

Bensmaia S J Denchev P V Dammann J FCraig J C amp Hsiao S S (2008) The represen-tation of stimulus orientation in the early stages of

somatosensory processing The Journal of Neuro-science 28(3) 776ndash786

Berens P Ecker A S Cotton R J Ma W JBethge M amp Tolias A S (2012) A fast andsimple population code for orientation in primateV1 The Journal of Neuroscience 32(31) 10618ndash10626

Bishop C M (2006) Pattern recognition and machinelearning New York Springer

Bollimunta A Totten D amp Ditterich J (2012)Neural dynamics of choice Single-trial analysis ofdecision-related activity in parietal cortex TheJournal of Neuroscience 32(37) 12684ndash12701

Borst A amp Theunissen F E (1999) Informationtheory and neural coding Nature Neuroscience2(11) 947ndash957

Britten K H Shadlen M N Newsome W T ampMovshon J A (1992) The analysis of visualmotion A comparison of neuronal and psycho-physical performance The Journal of Neuroscience12(12) 4745ndash4765

Burge J Fowlkes C C amp Banks M S (2010)Natural-scene statistics predict how the figurendashground cue of convexity affects human depthperception The Journal of Neuroscience 30(21)7269ndash7280

Burge J amp Geisler W S (2011) Optimal defocusestimation in individual natural images Proceed-ings of the National Academy of Sciences USA108(40) 16849ndash16854

Burge J amp Geisler W S (2014) Optimal disparityestimation in natural stereo images Journal ofVision 14(2)1 1ndash18 doi1011671421 [PubMed][Article]

Burge J amp Geisler W S (2015) Optimal speedestimation in natural image movies predicts humanperformance Nature Communications 6 7900

Burnham K P amp Anderson D R (2003) Modelselection and multimodel inference A practicalinformation-theoretic approach New York Spring-er Science amp Business Media

Campbell F W amp Robson J (1968) Application ofFourier analysis to the visibility of gratings TheJournal of Physiology 197(3) 551ndash566

Carandini M amp Heeger D J (2012) Normalizationas a canonical neural computation Nature ReviewsNeuroscience 13(1) 51ndash62

Cavagnaro D R Gonzalez R Myung J I amp PittM A (2013) Optimal decision stimuli for riskychoice experiments An adaptive approach Man-agement Science 59(2) 358ndash375

Cavagnaro D R Myung J I Pitt M A amp Kujala

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 13

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 14: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

J V (2010) Adaptive design optimization Amutual information-based approach to modeldiscrimination in cognitive science Neural Compu-tation 22(4) 887ndash905

Cavagnaro D R Pitt M A amp Myung J I (2011)Model discrimination through adaptive experi-mentation Psychonomic Bulletin amp Review 18(1)204ndash210

Chirimuuta M amp Tolhurst D J (2005) Does aBayesian model of V1 contrast coding offer aneurophysiological account of human contrastdiscrimination Vision Research 45(23) 2943ndash2959

Clatworthy P Chirimuuta M Lauritzen J ampTolhurst D (2003) Coding of the contrasts innatural images by populations of neurons inprimary visual cortex (V1) Vision Research 43(18)1983ndash2001

Cohen M R amp Newsome W T (2009) Estimates ofthe contribution of single neurons to perceptiondepend on timescale and noise correlation TheJournal of Neuroscience 29(20) 6635ndash6648

Cover T M amp Thomas J A (2006) Elements ofinformation theory Hoboken NJ John Wiley ampSons

Dayan P amp Abbott L F (2001) Theoreticalneuroscience volume 806 Cambridge MA MITPress

DiMattina C (2015) Fast adaptive estimation ofmultidimensional psychometric functions Journalof Vision 15(9)5 1ndash20 doi1011671595[PubMed] [Article]

DiMattina C amp Zhang K (2011) Active datacollection for efficient estimation and comparisonof nonlinear neural models Neural Computation23(9) 2242ndash2288

DiMattina C amp Zhang K (2013) Adaptive stimulusoptimization for sensory systems neuroscienceFrontiers in Neural Circuits 7 101

Dosher B A Jeter P Liu J amp Lu Z-L (2013) Anintegrated reweighting theory of perceptual learn-ing Proceedings of the National Academy ofSciences USA 110(33) 13678ndash13683

Dosher B A amp Lu Z-L (1998) Perceptual learningreflects external noise filtering and internal noisereduction through channel reweighting Proceed-ings of the National Academy of Sciences USA95(23) 13988ndash13993

Dosher B A amp Lu Z-L (1999) Mechanisms ofperceptual learning Vision Research 39(19) 3197ndash3221

Eckstein M P amp Ahumada A J (2002) Classifica-tion images A tool to analyze visual strategies

Journal of Vision 2(1)i doi10116721i[PubMed] [Article]

Eckstein M P Shimozaki S S amp Abbey C K(2002) The footprints of visual attention in thePosner cueing paradigm revealed by classificationimages Journal of Vision 2(1)3 25ndash45 doi101167213 [PubMed] [Article]

Egger S W amp Britten K H (2013) Linking sensoryneurons to visually guided behavior Relating MSTactivity to steering in a virtual environment VisualNeuroscience 30(5ndash6) 315ndash330

Ganguli D amp Simoncelli E P (2010) Implicitencoding of prior probabilities in optimal neuralpopulations In Advances in neural informationprocessing systems (pp 658ndash666) Cambridge MAMIT Press

Ganguli D amp Simoncelli E P (2014) Efficientsensory encoding and Bayesian inference withheterogeneous neural populations Neural Compu-tation 26 2103ndash2134

Geisler W Perry J Super B amp Gallogly D (2001)Edge co-occurrence in natural images predictscontour grouping performance Vision Research41(6) 711ndash724

Geisler W S (2008) Visual perception and thestatistical properties of natural scenes AnnualReview of Psychology 59 167ndash192

Geisler W S Najemnik J amp Ing A D (2009)Optimal stimulus encoders for natural tasksJournal of Vision 9(13)17 1ndash16 doi10116791317 [PubMed] [Article]

Girshick A R Landy M S amp Simoncelli E P(2011) Cardinal rules Visual orientation percep-tion reflects knowledge of environmental statisticsNature Neuroscience 14(7) 926ndash932

Gold J I amp Shadlen M N (2007) The neural basisof decision making Annual Review of Neuroscience30 535ndash574

Goris R L Putzeys T Wagemans J amp WichmannF A (2013) A neural population model for visualpattern detection Psychological Review 120(3)472ndash496

Goris R L Wichmann F A amp Henning G B(2009) A neurophysiologically plausible popula-tion code model for human contrast discriminationJournal of Vision 9(7)15 1ndash22 doi1011679715[PubMed] [Article]

Graf A B Kohn A Jazayeri M amp Movshon J A(2011) Decoding the activity of neuronal popula-tions in macaque primary visual cortex NatureNeuroscience 14(2) 239ndash245

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 14

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 15: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Helmholtz H V (1925) Handbook of physiologicaloptics Mineola NY Dover

Kiang N Y (1965) Discharge patterns of single fibersin the catrsquos auditory nerve Cambridge MA MIT

Kiani R Cueva C J Reppas J B amp Newsome WT (2014) Dynamics of neural population responsesin prefrontal cortex indicate changes of mind onsingle trials Current Biology 24(13) 1542ndash1547

Kingdom F amp Prins N (2010) Psychophysics Apractical introduction London Academic Press

Lewi J Butera R amp Paninski L (2009) Sequentialoptimal design of neurophysiology experimentsNeural Computation 21(3) 619ndash687

Li B Peterson M R amp Freeman R D (2003)Oblique effect A neural basis in the visual cortexJournal of Neurophysiology 90(1) 204ndash217

Ma W J (2010) Signal detection theory uncertaintyand Poisson-like population codes Vision Re-search 50(22) 2308ndash2319

Ma W J Beck J M Latham P E amp Pouget A(2006) Bayesian inference with probabilistic pop-ulation codes Nature Neuroscience 9(11) 1432ndash1438

Ma W J Navalpakkam V Beck J M van denBerg R amp Pouget A (2011) Behavior and neuralbasis of near-optimal visual search Nature Neuro-science 14(6) 783ndash790

May K A amp Solomon J A (2015a) Connectingpsychophysical performance to neuronal responseproperties I Discrimination of suprathresholdstimuli Journal of Vision 15(6)8 1ndash26 doi1011671568 [PubMed] [Article]

May K A amp Solomon J A (2015b) Connectingpsychophysical performance to neuronal responseproperties II Contrast decoding and detectionJournal of Vision 15(6)9 1ndash21 doi1011671569[PubMed] [Article]

Mineault P J Barthelme S amp Pack C C (2009)Improved classification images with sparse priors ina smooth basis Journal of Vision 9(10)17 1ndash24doi10116791017 [PubMed] [Article]

Morgenstern Y amp Elder J H (2012) Local visualenergy mechanisms revealed by detection of globalpatterns The Journal of Neuroscience 32(11)3679ndash3696

Mountcastle V B LaMotte R H amp Carli G(1972) Detection thresholds for stimuli in humansand monkeys Comparison with threshold events inmechanoreceptive afferent nerve fibers innervatingthe monkey hand Journal of Neurophysiology35(1) 122ndash136

Muniak M A Ray S Hsiao S S Dammann J F

amp Bensmaia S J (2007) The neural coding ofstimulus intensity Linking the population responseof mechanoreceptive afferents with psychophysicalbehavior The Journal of Neuroscience 27(43)11687ndash11699

Murray R F (2011) Classification images A reviewJournal of Vision 11(5)2 1ndash25 doi1011671152[PubMed] [Article]

Murray R F Sekuler A B amp Bennett P J (2003)A linear cue combination framework for under-standing selective attention Journal of Vision 3(2)2 116ndash145 doi101167322 [PubMed] [Article]

Myung J I Cavagnaro D R amp Pitt M A (2013) Atutorial on adaptive design optimization Journal ofMathematical Psychology 57(3) 53ndash67

Neri P (2004) Attentional effects on sensory tuningfor single-feature detection and double-featureconjunction Vision Research 44(26) 3053ndash3064

Neri P amp Heeger D J (2002) Spatiotemporalmechanisms for detecting and identifying imagefeatures in human vision Nature Neuroscience5(8) 812ndash816

Neri P amp Levi D M (2006) Receptive versusperceptive fields from the reverse-correlation view-point Vision Research 46(16) 2465ndash2474

Nienborg H Cohen R amp Cumming B G (2012)Decision-related activity in sensory neurons Cor-relations among neurons and with behavior AnnualReview of Neuroscience 35 463ndash483

Paninski L Pillow J amp Lewi J (2007) Statisticalmodels for neural encoding decoding and optimalstimulus design Progress in Brain Research 165493ndash507

Parker A J amp Newsome W T (1998) Sense and thesingle neuron Probing the physiology of percep-tion Annual Review of Neuroscience 21(1) 227ndash277

Pestilli F Carrasco M Heeger D J amp Gardner JL (2011) Attentional enhancement via selectionand pooling of early sensory responses in humanvisual cortex Neuron 72(5) 832ndash846

Pestilli F Ling S amp Carrasco M (2009) Apopulation-coding model of attentions influence oncontrast response Estimating neural effects frompsychophysical data Vision Research 49(10) 1144ndash1153

Petrov A A Dosher B A amp Lu Z-L (2005) Thedynamics of perceptual learning An incrementalreweighting model Psychological Review 112(4)715ndash743

Pinto N Doukhan D DiCarlo J J amp Cox D D(2009) A high-throughput screening approach to

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 15

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 16: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

discovering good forms of biologically inspiredvisual representation PLoS Computational Biology5(11) e1000579

Pitt M A amp Myung I J (2002) When a good fit canbe bad Trends in Cognitive Sciences 6(10) 421ndash425

Purushothaman G amp Bradley D C (2005) Neuralpopulation code for fine perceptual decisions inarea MT Nature Neuroscience 8(1) 99ndash106

Qamar A T Cotton R J George R G Beck JM Prezhdo E Laudano A Ma W J (2013)Trial-to-trial uncertainty-based adjustment of de-cision boundaries in visual categorization Pro-ceedings of the National Academy of Sciences USA110(50) 20332ndash20337

Read J (2015) The place of human psychophysics inmodern neuroscience Neuroscience 296 116ndash129

Ringach D L (1998) Tuning of orientation detectorsin human vision Vision Research 38(7) 963ndash972

Romo R amp de Lafuente V (2013) Conversion ofsensory signals into perceptual decisions Progressin Neurobiology 103 41ndash75

Sachs M B amp Abbas P J (1974) Rate versus levelfunctions for auditory-nerve fibers in cats Tone-burst stimuli The Journal of the Acoustical Societyof America 56(6) 1835ndash1847

Sadagopan S amp Wang X (2008) Level invariantrepresentation of sounds by populations of neuronsin primary auditory cortex The Journal of Neuro-science 28(13) 3415ndash3426

Salinas E (2006) How behavioral constraints maydetermine optimal sensory representations PLoSBiology 4(12) e387

Shadlen M N Britten K H Newsome W T ampMovshon J A (1996) A computational analysis ofthe relationship between neuronal and behavioralresponses to visual motion The Journal of Neuro-science 16(4) 1486ndash1510

Shadlen M N amp Newsome W T (2001) Neuralbasis of a perceptual decision in the parietal cortex(area lip) of the rhesus monkey Journal ofNeurophysiology 86(4) 1916ndash1936

Skottun B C Bradley A Sclar G Ohzawa I ampFreeman R D (1987) The effects of contrast onvisual orientation and spatial frequency discrimi-nation A comparison of single cells and behaviorJournal of Neurophysiology 57(3) 773ndash786

Solomon J A (2002) Noise reveals visual mechanismsof detection and discrimination Journal of Vision2(1)7 105ndash120 doi101167217 [PubMed][Article]

Tam W (2012) Adaptive modeling of marmoset inferior

colliculus neurons in vivo PhD thesis JohnsHopkins University Baltimore MD

van den Berg R Awh E amp Ma W J (2014)Factorial comparison of working memory modelsPsychological Review 121(1) 124ndash149

Vogels R amp Orban G (1990) How well do responsechanges of striate neurons signal differences inorientation A study in the discriminating monkeyThe Journal of Neuroscience 10(11) 3543ndash3558

Wald G (Sept 4 1964) The receptors of human colorvision Science 145 1007ndash1016

Wang L Narayan R Grana G Shamir M amp SenK (2007) Cortical discrimination of complexnatural stimuli Can single neurons match behav-ior The Journal of Neuroscience 27(3) 582ndash589

Wang Z amp Simoncelli E P (2008) Maximumdifferentiation (MAD) competition A methodolo-gy for comparing computational models of per-ceptual quantities Journal of Vision 8(12)8 1ndash13doi1011678128 [PubMed] [Article]

Watson A B amp Fitzhugh A (1990) The method ofconstant stimuli is inefficient Perception amp Psy-chophysics 47(1) 87ndash91

Wei X-X amp Stocker A A (2015) A Bayesianobserver model constrained by efficient coding canexplain lsquolsquoanti-Bayesianrsquorsquo percepts Nature Neurosci-ence 18(10) 1509ndash1517

Yamins D L Hong H Cadieu C F Solomon EA Seibert D amp DiCarlo J J (2014) Perfor-mance-optimized hierarchical models predict neuralresponses in higher visual cortex Proceedings of theNational Academy of Sciences USA 111(23) 8619ndash8624

Yang T amp Maunsell J H (2004) The effect ofperceptual learning on neuronal responses inmonkey visual area V4 The Journal of Neurosci-ence 24(7) 1617ndash1626

Young T (1802) The Bakerian lecture On the theoryof light and colours Philosophical Transactions ofthe Royal Society of London 92 12ndash48

Appendix A

A psychophysical experiment will yield stimulusndashresponse data Dw frac14 fsi bignifrac141 Assuming that subjectresponses are independent across trials we can writethe data likelihood

PethDwjhxTHORN frac14Pn

ifrac141Pethbijsi hxTHORN eth11THORN

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 16

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 17: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Bayesrsquo rule expresses the posterior probability of h x interms of the data likelihood (Equation 11) yielding

PethhxjDwTHORN frac141

ZPethDwjhxTHORNPethhxTHORN eth12THORN

where Z is a normalizing constant and P(hx) reflectsany prior beliefs about the neural code and thedecoding parameters Using Equations 11 and 2 wemay rewrite Equation 12 as

PethhxjDwTHORN frac141

ZPn

ifrac141

RPethbijrxTHORNPethrjsi hTHORN dr

PethhxTHORN

eth13THORNAssuming that h and x are independent so P(hx) frac14P(h)P(x) marginalizing Equation 13 over x yields

PethhjDwTHORN frac141

ZPn

ifrac141

R RPethbijrxTHORNPethrjsi hTHORNPethxTHORN d r dx

PethhTHORN

eth14THORNIn the case of fixed decoding model parameters x sothat P(x)frac14 d(x x) (where d denotes the Dirac deltafunction) we obtain

PethhjDwTHORN frac141

ZPn

ifrac141

RPethbijr xTHORNPethrjsi hTHORN dr

PethhTHORN

eth15THORNfrom Equation 14 A symmetrical argument of thesame form as that presented above can be used to showthat we can use psychophysical data to estimate theparameters of a behavioral decoding model given aneural encoding model with known parameters h usingthe equation

PethxjDwTHORN frac141

ZPn

ifrac141

RPethbijr hxTHORNPethrjsi hTHORN dr

PethxTHORN

eth16THORNThis allows a similar process of model fitting and

comparison to be used in order to test competinghypotheses of neural decoding for instance whethersimple linear decoding (Berens et al 2012) or morecomplicated decoding mechanisms (Graf Kohn Ja-zayeri amp Movshon 2011) are needed to accuratelyrecover stimulus parameters or explain behavior

Appendix B

Let stimulus s_frac14 (0 ndash d c)T denote the clockwisestimulus and sthorn denote the counterclockwise stimuluswith parameters sthornfrac14 (0thorn d c)T We will assume thatorientation and contrast are coded by a population of Nindependent neurons whose expected noisy (Poisson)

response ri to a stimulus sfrac14 ( c)T is given by the 2-Dcontrast-modulated tuning curve fi ( c)frac14 w(c) fi ()where fi () describes the orientation tuning of the ithunit and w(c) describes the contrast gain with 0 w(c) 1 for contrast (in percentage) 0 c 100 We will alsoassume that all of the units decoded for the behavioraldecision have the same contrast gain function w(c) Fortractability we approximate the Poisson noise responseby Gaussian noise with mean and variance lfrac14 r2frac14 fiThis fully specifies our neural encoding model P(rjsh) asa factorial Gaussian distribution

To specify the behavioral decoding model P(bjr x)where b frac14 1 indicates a correct response (bfrac14 0incorrect) we assume that the responses of all units arepooled linearly to form a new decision variable

u frac14XNifrac141

xiri frac14 xTr eth17THORN

where the xi are dependent on the perceptual taskBecause the weighted sum of Gaussian variables is alsoGaussian this new decision variable (Equation 17) isGaussian and the expected value for stimulus s0frac14 (0c)T is given by

l0 frac14 wethcTHORNXNifrac141

xi fieth0THORN eth18THORN

with variance

r2 frac14XNifrac141

x2i Var rifrac12 frac14 wethcTHORN

XNifrac141

x2i fieth0THORN eth19THORN

Because our perturbed stimuli s_ and sthorn are assumed tobe very close to the reference s0 we will assume thevariance of the response to these stimuli is also equal tothe same r2 in Equation 19 The expected value ofresponses to stimulus s6 frac14 (0 6 d c)T is given by

l6 frac14 wethcTHORNXNifrac141

xi fieth06dTHORN eth20THORN

and from Equations 18 through 20 we obtain anexpression for the well-known psychophysical quantity

d0eth0 d cTHORN frac14lthorn l

reth21THORN

for a perturbation of size d

d0eth0 d cTHORN

frac14ffiffiffiffiffiffiffiffiffiwethcTHORN

p RNifrac141xi fieth0 thorn dTHORN fieth0 dTHORNfrac12 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

RNifrac141x

2i fieth0THORN

q

eth22THORNwhere Equation 22 is obtained readily by pluggingEquations 19 and 20 into Equation 21

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 17

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27
Page 18: Comparing models of contrast gain using psychophysical ...By marginalizing the joint probability P(b,rjs,x,h) over r, we can express the probability of a behavior entirely as a function

Introducing the notation f frac14 ( f1() fN ())T

and Rfrac14 diag [f] and suppressing arguments we canrewrite Equation 22 as

ethd0THORN2 frac14 2wethcTHORNxTethfthorn fTHORN 2

xT2R0x

eth23THORN

and we recognize the term in brackets as the ratio ofvariability between groups to that within groups whenobservations are projected onto the vector x Thevector maximizing this ratio is known as the Fisherlinear discriminant and is given by

xF R10ethfthorn fTHORN eth24THORN

For small perturbations d the direction of the vectorxF does not depend on d because we may approxi-mate ethfthorn fTHORNrsquo f

0

02d because thorn ndash _ frac14 2d

Substituting Equation 24 into Equation 23 and usingthis approximation yields

d0eth0 d cTHORN frac14 2ffiffiffiffiffiffiffiffiffiwethcTHORN

pd

ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

p eth25THORN

where

IFeth0THORN frac14XNifrac141

f 0i eth0THORN

2

fieth0THORNeth26THORN

is the population Fisher information about the stimulus

orientation around the reference stimulus 0 a well-

known result from population coding theory (Dayan amp

Abbott 2001) Given our expression (Equation 25) for

d0 and using the fact that the probability of correct

response (b frac14 1) in the two-alternative forced choice

task is U(d02) (single interval) or U(d0ffiffiffi2p

) (two-

interval) we obtain the final model

Pethb frac14 1js hxTHORN frac14 UK

ffiffiffiffiffiffiffiffiffiwethcTHORN

pd eth27THORN

where K ffiffiffiffiffiffiffiffiffiffiffiffiffiIFeth0THORN

pis a parameter describing

population sensitivity to changes in orientation around

0 at 100 contrast

Journal of Vision (2016) 16(9)1 1ndash18 DiMattina 18

Downloaded From httpjovarvojournalsorgpdfaccessashxurl=dataJournalsJOV935414 on 07052016

  • Introduction
  • Methods and results
  • e01
  • e02
  • e03
  • f01
  • e04
  • e05
  • e06
  • e07
  • f02
  • f03
  • f04
  • e08
  • e09
  • f05
  • e10
  • f06
  • f07
  • Discussion
  • f08
  • Ahumada1
  • Akaike1
  • Albrecht1
  • Barbour1
  • Bays1
  • Beck1
  • Bejjanki1
  • Bensmaia1
  • Berens1
  • Bishop1
  • Bollimunta1
  • Borst1
  • Britten1
  • Burge1
  • Burge2
  • Burge3
  • Burge4
  • Burnham1
  • Campbell1
  • Carandini1
  • Cavagnaro1
  • Cavagnaro2
  • Cavagnaro3
  • Chirimuuta1
  • Clatworthy1
  • Cohen1
  • Cover1
  • Dayan1
  • DiMattina1
  • DiMattina2
  • DiMattina3
  • Dosher1
  • Dosher2
  • Dosher3
  • Eckstein1
  • Eckstein2
  • Egger1
  • Ganguli1
  • Ganguli2
  • Geisler1
  • Geisler2
  • Geisler3
  • Girshick1
  • Gold1
  • Goris1
  • Goris2
  • Graf1
  • Helmholtz1
  • Kiang1
  • Kiani1
  • Kingdom1
  • Lewi1
  • Li1
  • Ma1
  • Ma2
  • Ma3
  • May1
  • May2
  • Mineault1
  • Morgenstern1
  • Mountcastle1
  • Muniak1
  • Murray1
  • Murray2
  • Myung1
  • Neri1
  • Neri2
  • Neri3
  • Nienborg1
  • Paninski1
  • Parker1
  • Pestilli1
  • Pestilli2
  • Petrov1
  • Pinto1
  • Pitt1
  • Purushothaman1
  • Qamar1
  • Read1
  • Ringach1
  • Romo1
  • Sachs1
  • Sadagopan1
  • Salinas1
  • Shadlen1
  • Shadlen2
  • Skottun1
  • Solomon1
  • Tam1
  • vandenBerg1
  • Vogels1
  • Wald1
  • Wang1
  • Wang2
  • Watson1
  • Wei1
  • Yamins1
  • Yang1
  • Young1
  • Appendix A
  • e11
  • e12
  • e13
  • e14
  • e15
  • e16
  • Appendix B
  • e17
  • e18
  • e19
  • e20
  • e21
  • e22
  • e23
  • e24
  • e25
  • e26
  • e27

Recommended