2488 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
A spectral theory of color perception
James J. Clark1,* and Sandra Skaff1,2
1Centre for Intelligent Machines, McGill University, 3480 University Street, Montreal, Quebec, Canada2Currently with Xerox Research Centre Europe, 6, chemin de Maupertuis, F-38240 Meylan, France
Received February 17, 2009; revised September 17, 2009; accepted September 29, 2009;posted September 30, 2009 (Doc. ID 107314); published November 4, 2009
. INTRODUCTION: A PHYSICAL THEORYF COLOR PERCEPTIONlong-standing philosophical question is whether color isphysical quantity or is purely a mental construct [1]. In
ther words, is color “out in the world” or “in the brain”?n one side of the debate stand the Objectivists, or Real-
sts, who hold that colors have objective essences and areroperties of physical objects. On the other side stand theroponents of the Illusion theory of color, which holdshat colors are virtual properties and our perceptions ofolor are illusory, in that physical objects are not colored,hey just appear to be.
In this paper we propose taking the philosophy of colorealism at face value and using it to develop a scientificheory of color perception. Our approach is based on thedeas of a prominent group of color realist philosophersho identify color with surface spectral reflectances. In-uential representatives of this community are Byrnend Hilbert [2] and Tye [3]. We modify their ideas slightlyo posit that the physiological, and thence psychological,uantity known as color is the particular model of spectrarovided by the sensory apparatus of the human body. Welaim, then, that all statements about color can be derivedrom the physical laws underlying spectra and of thetructure of the human sensory apparatus.
Noted philosopher of color Christopher Hardin com-ented in [4] that“It is a curious sociological fact that many philosophers,
ut very few visual scientists, are color realists.” He sug-ests that the reason for this fact is that philosophers areenerally loath to “banish colors from the physical world”or fear that they “will take up refuge in what Gilbertyle once called the dust-bin of the mind, never to be dis-
odged.” [4]. He does not, however, speculate as to why vi-ual scientists generally take the opposite view, that colors in the mind. This may be a result of the intense focus ofision scientists on physiological and psychological stud-
es of the color vision system. With such a strong focus onhe brain it is perhaps natural that the bias of color sci-ntists is to consider color an internal quantity. This biass not universal, however. Recently cognitive scientistsave been considering perceptual systems in terms of the
nteraction of the observer with its environment. Suchonsiderations lead to treating perception as a process ofnducing sensorimotor relationships that reveal invarianttructures of the world, independent of the specific detailsf the perceptual apparatus (see O’Regan and Noë [5] fordefinitive statement of this viewpoint). In the case of
olor perception the relevant invariant physical quanti-ies are the spectral energy distribution of lights and thepectral reflection functions of surfaces.
The closest that the current science of color comes to aocus on spectral modeling in understanding color visionrocesses is in the study of color constancy. Color con-tancy is the ability of the human visual system to per-eive a surface as a given color independently of the colorf the light illuminating the surface. While there areany successful color constancy theories and associated
lgorithms that do not consider the spectra of the light,he most flexible and robust methods are based on spec-ral models [6]. Most of these methods follow the seminalaper of Maloney and Wandell [7], who suggested utiliz-ng a linear model of the spectrum of the illuminant andf the surface reflectance spectrum. This is not surprising,ince the confounding of the illuminant and surface oc-urs at the level of the spectra, and not at the neuralevel. If one considers the role of color perception to behat of surface spectral modeling, then the problem ofolor constancy disappears, to be replaced by the funda-ental problem of finding the invariant surface spectrum.hus discounting the illuminant spectrum becomes partf the determination of the surface spectrum rather thanart of a color constancy or color correction process.In this paper our goal is to break with the mainstream
009 Optical Society of America
irmepttisp
ss
2Icqatmdtstf
olhltTtlrw=ippsttmttta
atpcbhtptcg
shwo
AHccTfa
ctmtttsbco
mFtccoolif
cpeofdif
rcmrshuptblldctwmT
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2489
nternal theories of color that are based solely on photo-eceptor channel signals. We describe a spectral-odeling-based theory of color vision that qualitatively
xplains various aspects of empirical color appearancehenomena. The approach we take is to begin with a sta-istical representation of the spectra of light. This allowshe application of the mathematical framework known asnformation geometry to deduce the structure of colorpace, which in turn answers the question of why the em-irical color appearance data takes the form it does.In this paper we focus our efforts on the modeling of the
pectra of lights and do not consider here modeling thepectral reflectance functions of surfaces.
. STATISTICAL SPECTRAL MODELINGn order to develop physically meaningful descriptions ofolor properties, one must identify these with physicaluantities. As mentioned in the introduction, we proposessociating with every spectrum a particular color percep-ion. Thus, to determine the structure of color space weust determine the structure of the space of spectra. To
o this we formulate the details of the theory in a statis-ical setting and associate spectra with probability den-ity functions. In this way, we will form an association be-ween color space and the space of probability densities,or which there exists a rich mathematical structure.
We begin by considering a stream of photons, detectedne at a time by a mechanism that can measure the wave-ength of each photon. Each photon emitted by the sourceas a certain probability, Pr��1����2�, that its wave-
ength, �, lies in the range ��1 ,�2�. This quantity obeyshe laws of probability derived from Cox’s postulates [8].hat is, taking A to be the condition where ��1����2�,
hen, since a photon must have some particular wave-ength, Pr�A�=1 represents certain truth, while Pr�A�=0epresents certain falsehood. Also, Pr�A�+Pr�AC�=1,here AC is the complement of A, and Pr�A ,B�Pr�A�Pr�B �A�=Pr�B�P�A �B�. If �2−�1=d� becomes van-
shingly small, then Pr��1����1+d��=p��1�d�. Thus��� is seen to be a probability density function. In thisaper this probability density will be refereed to as thepectral profile of the photon source. Note that the spec-ral profile carries no information about the intensity ofhe photon source. The intensity of the photon sourceust be determined or given separately. The product of
he spectral profile and the intensity will be referred to ashe spectrum. The separation of the source intensity andhe spectral profile is a key aspect of the theory and willffect the computational structure.Let S denote the space of all spectral profiles (or prob-
bility density functions). In the theory being proposed inhis paper, each element of S corresponds to a differentossible spectral profile and hence to a distinct color per-eption. Only some of these possibilities can be obtainedy the human visual system, however. The data that theuman visual system has to work with to model the spec-ral profile of incoming light is the collection of signalsrovided by the retinal photoreceptors. These photorecep-or signals divide the space S into a group of equivalencelasses, called metamer sets [9]. The spectral profiles in aiven metamer set all produce the same photoreceptor
ignals. The metamer sets are exclusive (i.e., they do notave any spectral profiles in common) and cover thehole of S. Therefore the metamer sets form a partitionf S.
. Spectral Modeling as Metamer Selectionuman color vision is marked by a singleness of the per-
ept. When humans see a colored patch, they report per-eiving a single color, rather than a whole set of colors.his suggests some mechanism for selecting one element
rom the metamer set. We refer to this selection processs spectral modeling.One could argue that there is no need to select a spe-
ific element of the metamer set, and instead work withhe more general equivalence classes defined by theetamers. If one considers only a single observer, then
he (equivalence) classes of metamers are well defined byhe characteristics of the photosensors. Inasmuch ashese equivalence classes form a partition of the stimuluspace (here taken to be the space of spectral power distri-utions), one could conceivably associate colors with theselasses. This approach was taken by Krantz [10] in devel-ping the geometry of color from very general principles.
Difficulties arise, however, when dealing with two orore observers, each having different sensing systems.or two observers one is faced with the problem that
here are now two different definitions of the equivalencelasses of metamers. How do we now define the notion ofolor? One approach is to say that each observer has theirwn, private, notion of color, defined by its own specific setf metameric equivalence classes. This amounts to the il-usion theory of color mentioned in Section 1, where colors considered purely a mental concept and can be differentrom observer to observer.
We are proposing to take a different approach, that ofolor realism, in which colors are associated with physicalroperties and as such should be the same for all observ-rs. This means that the notion of color cannot be basedn the metamer equivalence classes (since these changerom observer to observer), but is based rather on the un-erlying space of spectral energy distributions, which be-ng a physical space outside of the observers is the sameor all observers.
There is actually experimental data that supports theealist view, suggesting that different observers can per-eive the same colors in spite of having quite differentetameric equivalence classes. This support comes from
eports of people that have unilateral color deficient vi-ion, e.g., that have red or green deficiency in one eye yetave normal color vision in the other. For example, thenilateral dichromat reported by Graham and Hsia [11]erceives yellow or blue in his dichromatic eye much inhe same way as in his trichromatic eye. Color matchingy this observer showed that monochromatic blue (or yel-ow) stimuli, with a wavelength of 470 nm �570 nm�,ooked similar in both eyes. Similarly, the unilateralichromat reported by MacLeod and Lennie [12] per-eived stimuli with a wavelength of 610 nm (orange) to behe same in both eyes. These studies imply that observersith differing sensory systems, and hence differingetamer sets, can nonetheless perceive similar colors.his not to say that all stimuli are perceived in the same
wtccwla
vsf(stbfde
d[stcctdabb
woptpsfipsstettfss
tsfficpmutoseo
tsbsbmsgdt
BIteoatot[cwd=ttt
I
Fpstpbsps
2490 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
ay by different observers, only that some may be—thosehat induce the same spectral model. For example, in thease of the dichromat and trichromat they may both per-eive the color of light of 570 nm wavelength in the sameay (both see yellow) but will differ in their perception of
ight of 650 nm wavelength (the trichomat perceiving rednd the dichromat perceiving a dark yellow).Note that we are not saying that different observers
iewing the same physical surface will always have theame color percept. In general, the colors perceived by dif-erent observers will be different, reflecting differencesusually minor for normal color vision) in their metamerets. Our example of the unilateral monochromat is in-ended to show that some color percepts can be the sameetween observers (e.g., yellows and blues) in spite of dif-erences in the metamer set structures. This is becauseifferent metamer sets can contain the same maximumntropy spectrum.
The realist view can therefore handle the individualifferences in the wavelengths chosen as “unique hues”13]. In this case each observer would have a particulartimulus wavelength that they would deem to correspondo a unique hue, and this stimulus would result in a spe-ific spectral model and hence, in our view, to a particularolor perception. What must be kept in mind, however, ishat the spectral model obtained may, and probably will,iffer from observer to observer, implying that the colorscribed to what an observer calls a unique hue will varyetween observers. Thus, my shade of unique green wille different that yours.Proceeding with the thesis that colors are associated
ith spectral models, we will now examine the geometryf the space of spectral models. We must first quantify therocess for obtaining the model. We propose that the spec-ral model produced in response to light falling on thehotoreceptors is determined by two factors—the spectralensitivity profiles of the photoreceptors, f���, which de-ne the partitioning of S into the metamer sets, and thearticular selection process used, which determines thepecific element of the metamer set that is chosen as thepectral model. This is illustrated in Fig. 1. In the figurehree different photoreceptors define metamer sets. Inach of these sets, one particular element is singled out byhe spectral modeling approach. The locus of all such dis-inguished elements over all metamer sets will be re-erred to as the spectral color manifold and captures allpectral models that can be generated by the perceptualystem.
There have been many ways proposed in the literatureo select a single element of the metamer set. Metamerets are generally difficult to compute and to represent, soor reasons of computational simplicity many approachesrst identify a subset of the metamer set, and then a spe-ific element is picked from this subset. One common ap-roach to defining such metamer subsets is to use linearodels [7] to describe spectra. The linear model approach
ses the linear superposition of a small set of basis func-ions to approximate the spectral profile, with the weightsf the basis functions chosen so as to make the resultingpectral profile be in the metamer set. Note that, in gen-ral, the linear model approach can only describe a subsetf the metamers, so not all metamers can be obtained in
hese approaches. For example, monochromatic (impulse)pectra cannot be modeled, unless a very large number ofasis functions are used. Morovic and Finlayson [9] de-cribe three different ways of singling out a single mem-er of a linear model metamer set: the centroid or meanetamer (in which the weight vectors in the set are as-
umed to be equally probable), the most likely metameriven an assumption that the weight vectors are normallyistributed, and the metamer with the smoothest spec-rum.
. Maximum Entropy Metamer Selectionn this paper we perform spectral modeling by selectinghe element of the metamer set that has the maximumntropy [14]. This approach is motivated by the writingsf Jaynes [15], who held that, given a set of constraints onprobability distribution, the best choice of the distribu-
ion out of all those consistent with the constraints is thene with maximum entropy. The principal justification ofhis view is the so-called entropy concentration theorem16,17]. In the context of spectral profiles the entropy con-entration theorem states that the number of differentays that N detected photons can be placed into niscrete wavelength bins is W�p1 ,p2 . . .pn�N! / ��Np1�! . . . �Npn�!�, where pi is the proportion of pho-
ons placed into bin i. For large N the Stirling approxima-ion gives W�exp�NH�p1 ,p2 , . . . ,pn��, where H is the en-ropy of the bin distribution:
H�p1,p2, . . . pn� = − �i=1
n
pi log�pi�. �1�
t can be seen that higher-entropy distributions have a
metamer set 3
metamer set 2
metamer set 1
space S of all possible spectral profiles
spectral color manifold
distinguished elementof the metamer set= spectral model
ig. 1. Diagram illustrating the spectral modeling process. Thehotoreceptor signals define a metamer set, which is the set of allpectral profiles that can result in these signals. One element ofhe metamer set is distinguished by the spectral modeling ap-roach and is taken as the spectral model. The space traced outy these distinguished elements of the metamer sets is called thepectral color manifold. It is the space of spectral models that areossible given the particular photoreceptor characteristics andpectral modeling approach.
mdteshcbrtcwmmtt
3SFswgitmdmwsc
sbrcicpmtncc
ATmMps
wat�sa
iols
nfiswacosnuz
is
stel
w�g
So
w
npo
Itpmnn
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2491
uch larger number of possible ways to fill the bins thano low-entropy distributions. The entropy concentrationheorem is based only on a counting of possibilities, how-ver, and it does not consider the probability of these pos-ibilities. It may be, for physical reasons, that some of theigh-entropy distributions are unlikely. But even so, theombinatorics is working against the lower-entropy possi-ilities even if they may have higher probability of occur-ing, as there are just fewer of them. The entropy concen-ration theorem suggests that the photon bin countsonsistent with the measured photoreceptor signals thatill most frequently be encountered are those with theaximum entropy of all possibilities. Thus, in the re-ainder of this paper, we will select as our spectral model
he metameric spectral profile that has the maximum en-ropy over all metamers.
. INFORMATION GEOMETRY OFPECTRAL PROFILESormulating spectral models in terms of probability den-ity functions permits the application of the methods ofhat is known as information geometry [18]. Informationeometry is based on the foundational work of Fisher [19]nvestigating the geometry of probability spaces. Informa-ion geometry considers such spaces as differentiableanifolds. This allows the use of tools from the field of
ifferential geometry, in particular the concepts of Rie-annian metrics and affine connections. In this paper, weill use these tools to determine the structure of color
pace, such as the similarity or distance between differentolors.
In keeping with the position that color perceptions areynonymous with spectral profiles, the manifold definedy all possible maximum entropy spectral models will beeferred to as the spectral color manifold. The spectralolor manifold is a very restricted submanifold of thenfinite-dimensional spectral manifold S. The form andharacteristics of the spectral color space depend on thehotoreceptor sensitivity functions and on the spectralodeling technique. If either of these changes, so too will
he spectral color manifold. But the underlying space S isot changed—changing the photoreceptors merelyhanges the subset of spectra (or colors) that can be per-eived.
. Spectral Models and Measurementso begin the study of the structure of the spectral coloranifold, we first examine the measurement process.easurements are assumed to be obtained by a linear
rojection of the spectral profile onto the photoreceptorensitivity functions:
r = ���
f���p���d�, �2�
here p����S is the spectral profile of the incident lightnd f��� is the vector of projection kernels, correspondingo the spectral sensitivity profiles of the photoreceptors.= ��min,�max� is the support of the photoreceptor spectral
ensitivities, which is the wavelength interval over whicht least one of the photoreceptors has a nonzero sensitiv-
ty (i.e., the visible wavelength range). � is the intensityf the light incident on the photoreceptors over wave-engths in the support of the photoreceptor spectral sen-itivities,
Spectral components outside the visible range (by defi-ition) lie in the null space of the projection operation de-ned by the measurement equation. Any variation in thepectral power in wavelengths outside the visible rangeill cause no change in the measurements. Thus, in thebsence of any other sources of information or additionalonstraints, nothing can be said about the spectral powerutside the visible range. When we look at a rainbow andee darkness at either side of a band, our vision system isot telling us that there is zero energy in the infrared andltraviolet—on the contrary, it is telling us that there isero energy in the visible wavelengths in these areas.
For the time being, we will assume that the intensity �s known, and focus on a normalized version of the mea-urement vector, �=r /�:
� =��
f���p���d�. �3�
It is a well-known result [15] that the probability den-ity p̂��� having the maximum entropy consistent withhe measurements defined by the equation above is an el-ment of an exponential family of densities with the fol-owing form:
p̂��;�� = exp�� · f��� − �����, �4�
here � is the vector of natural parameters. The quantity��� normalizes the model spectral profile so that it inte-rates to 1, as is required for it to be aprobability density:
��
p̂��;��d� =��
exp�� · f����exp�− �����d� = 1. �5�
ince exp�−����� does not depend on �, it can be takenutside of the integral:
exp�− �������
exp�� · f����d� = 1, �6�
hich then yields the following expression for ����:
���� = log��
exp�� · f����d�. �7�
The natural parameters � can serve as a set of coordi-ates for the exponential family of distributions. The ex-ectations of f��� with respect to p̂�� ;�� yield another setf coordinates:
�̂ = Ep̂f��� =��
f���p̂��;��d�. �8�
f we take f��� to be the photoreceptor spectral sensitivi-ies, it is immediately evident that, since the true spectralrofile p��� and the model spectral profile p̂�� ;�� areetamers, the expectations in Eq. (8) are equal to theormalized responses of the photoreceptors. Thus we willot distinguish between � and �̂ in what follows.
pmtmspsdmp(afggcdfAatm�lcpasatt
tttt
T
al
T
Ttcsp
Ic(
Ttm
pmtptvswmsspf
BTstgjt
cdtmFmmiocaoessitpAsirbvc
2492 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
Suppose we are given two different spectral profiles,P��� ,pQ����S. These give rise to normalized measure-ents �P and �Q, which in turn give rise to model spec-
ral profiles p̂P��� and p̂Q���, which lie in the spectral coloranifold. Because of the assumed linearity of the mea-
urement process a linear combination of the two spectralrofiles apP���+ �1−a�pQ���, a� �0,1�, results in a mea-urement vector that is the linear combination of the in-ividual measurements, �P+Q= �1−a��P+a�Q. As thiseasurement arises from a physically realizable spectral
rofile, there exists a corresponding model spectral profilei.e., the maximum entropy element of the metamer setssociated with �P+Q) that lies on the spectral color mani-old. The line formed in the spectral color manifold as aoes from 0→1 is referred to as the m-geodesic. This is aeodesic in the sense of being the curve that minimizes aertain distance measure, the canonical divergence (to beescribed later). As this line lies completely in the mani-old, the manifold is considered to be, in the parlance ofmari [18], m-flat (where “m” stands for “mixture”). Inddition, given the � coordinates of any two model spec-ral profiles, p̂P and p̂Q, that lie on the spectral coloranifold, a linear combination of these two coordinates,
1−a��P+a�Q, defines a model spectral profile that alsoies on the manifold. In this case the resulting spectralolor manifold is said to be e-flat (where “e” stands for “ex-onential”). The line formed in the spectral color manifolds a goes from 0→1 is referred to as the e-geodesic. Thepectral color manifold is therefore said to be dually flat,s it is both m-flat and e-flat. Note that this flatness is inhe sense of the e-and m-geodesic, and does not mean thathe manifold is Euclidean.
As pointed out by Amari [18] the dually flat nature ofhe spectral color manifold implies a strong link betweenhe � and � coordinates. In particular they are relatedhrough a Legendre transformation between two poten-ial functions � and �:
���� = max�
�� · � − �����. �9�
he transformation is its own inverse, so that
���� = max�
�� · � − �����, �10�
nd the two transformations are related through the fol-owing identity:
���� + ���� = � · �. �11�
he coordinates � and � are therefore related by
�����
��i= �i, �12�
�����
��i= �i. �13�
he potential ���� can be identified with the normalizingerm in Eq. (4). An interpretation of the ���� potentialan be obtained by considering the entropy of the modelpectral profile, p̂. This is given by the negative of the ex-ectation of the logarithm of p̂�� ;��:
H�p̂� = − Elog�p̂��;��� = −��
p̂��;��log�p̂��;���d�.
�14�
f we take p̂ to be a member of an exponential family, wean substitute the expression for p̂ from Eq. (4) into Eq.14) and get
hus the potential ���� is equal to the negative of the en-ropy of the model spectral profile associated with a nor-alized measurement vector �.Note that, depending on the form of f, there will be
oints in the � space that admit no corresponding ele-ent of the exponential family. A trivial example of this is
hat the elements of � cannot be negative, because of theositivity of spectra p��� and the photoreceptor sensitivi-ies f���. Also, it can be seen that the maximum possiblealue of an individual component �i of � is induced by apectrum p���= fi��� /��fi���d�. For any measurementith �i higher than this there is no metameric spectralodel that gives this measurement. We will refer to the
et of points � that admit spectral models as the admis-ible normalized measurement space, M. In � space alloints have corresponding elements of the exponentialamily. M is therefore the image of the entire � space.
. Fisher Information Metriche information-geometric treatment of the manifold ofpectral models permits specification of a metric andherefore a distance measure. Distance measures are ofreat importance in color science, as they provide a way toudge the relative similarity between colors and to quan-ify discriminibility of colors.
There are an infinite number of metrics that could behosen, and each could predict different color similarity–iscriminibility phenomena. There is, however, one par-icular metric that has unique properties that make itore appropriate than any other, and that is the so-calledisher metric (sometimes referred to as the Fisher–Raoetric). As Maybank [20] explains, it is desirable that aeasure of distance between any two points in a probabil-
ty space (i.e., the space of spectral power distributions inur application) be invariant under reparametrizations oroordinate transformations (e.g., in wavelength). Amarind Nagaoka [18] showed that this invariance propertynly holds for the Fisher metric (up to a scale factor). Anven stronger statement was proven by Cencov [21], whohowed that the Fisher metric is the only metric on thepace of discrete probability distributions that is compat-ble with all Markov morphisms (or stochastic maps), inhe sense that application of a stochastic map to twooints does not increase the distance between the points.stochastic map in this context is a linear map from the
pace of probability distributions into itself, or a subset oftself. Examples include permutation of wavelengths, andeducing the dimensionality of the probability space (e.g.,y having wider histogram bins). In the context of colorision, this compatibility means that a stochastic mapannot increase the discriminibility between two colors,
aF
h
RRttvl
Tttfm
mtld
Frp
Tot
wptinfIp[
Fp
p
wp
Lt
b
Fwi
Agg
T“ec
4PTpn
T
Ts
Nnm
scl
ea
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2493
s defined by the metric. Cencov’s result shows that theisher metric is the only metric that has this property.The Fisher metric (or the Fisher information matrix)
as the following form:
gij��� =��
p̂��;��� �
��ilog p̂��;��
�
��jlog p̂��;�� d�.
�16�
ao [22] showed that the Fisher information matrix is aiemannian metric on the space of probabilities. The ma-
rix G= �gij� formed by the elements of the metric is posi-ive definite, as is its inverse G−1. This means thatTGv0 for any nonzero vector v. For exponential fami-
ies the Fisher information metric can be written as
gij��� =��j
��i=�
�
p̂��;���fi��� − �i��fj��� − �j�d�
=��
p̂��;��fi���fj���d� − �i�j. �17�
he integral in the rightmost equation above can behought of as the response to the model spectrum of a syn-hetic photoreceptor with a spectral sensitivity profile ofij���= fi���fj���. Let us denote this response �ij. Then theetric is given by
gij��� = �ij��� − �i����j���. �18�
The metric allows us to quantify the structure of theanifold, i.e., the relative distances between points on
he manifold. The line element, or the square of the arcength between two points separated by an infinitesimalistance, is given by
ds2 = d�TG���d� = d� · d� �19�
or a dually flat space the metric relative to the dual pa-ameters ��� is the inverse of the metric relative to therimary parameters ���, so that
ds2 = d�TG−1���d� = d� · d�. �20�
he Riemannian distance between two points, P and Q,n the manifold is the length of the geodesic joining thesewo points:
L�P,Q� =�s�P�
s�Q�
ds =�0
1 �d�
dtTG
d�
dt 1/2
dt, �21�
here t is a parameter along the geodesic. In general com-utation of the Riemannian distance is difficult owing tohe required integration and to the difficulty in determin-ng the geodesic curve. The integration in most caseseeds to be done numerically, which becomes problematic
or large � values, where the elements of G become small.nstead, we work with a pseudodistance that is much sim-ler to compute. This is the so-called canonical divergence18]:
D�P�Q� = ���P� + ���Q� − �P · �Q. �22�
or small differences in the � coordinates of the twooints, the canonical divergence is a second-order ap-
roximation to one half of the square arc length distance:
D�P�P + �� = �TG�/2 + o����2� � ds2/2, �23�
here �i is the difference in the �i coordinate betweenoints Q and P.The canonical divergence is related to the Kullback–
eibler divergence, which is a commonly used measure ofhe difference between probability distributions [23]:
D�P�Q� = DKL�Q�P� =��
p̂Q���log� p̂Q���
p̂P��� d�. �24�
From Eq. (11) we see that the canonical divergence cane rewritten as
D�P�Q� = ��Q� − ��P� + �P · ��P − �Q�. �25�
rom this equation and Eq. (15) relating � to the entropy,e can see that the canonical divergence can be expressed
n terms of the entropy as follows:
D�P�Q� = H�P� − H�Q� + �P · ��P − �Q�. �26�
n important special case is that of the canonical diver-ence between an arbitrary spectral model Q and the ori-in in � coordinates:
D0�Q� = D�0�Q� = H�0� − H�Q�. �27�
he origin in � coordinates corresponds to the uniform, orwhite,” spectral profile, and is the profile with the high-st possible entropy. Thus the origin has a special physi-al significance.
. SPECTRAL MODELING IN THERESENCE OF NOISEhe measurement provided by the photosensors will, inractice, be corrupted by noise. If we assume an additiveoise process the measurement equation becomes
r = ���
f���p���d� + �. �28�
he normalized measurement � is given by
� =��
f���p���d� + �/�. �29�
he expectation coordinates �̂ associated with a modelpectrum are
�̂ =��
f���p̂���d�. �30�
ote that �̂ is no longer equal to � in general, so we willow have to distinguish these whenever there is nonzeroeasurement noise.The metamer set is broadened in the presence of noise,
ince there is a greater number of possible spectra thatould give rise to the observed measurement. As the noiseevel increases, so does the volume of the metamer set.
We can still apply our maximum entropy spectral mod-ling approach. Let us assume that the noise is zero meannd bounded such that
wppweottm
Tt
Nfitoscws
wmm
s
TAttb
bunwwwmstwpt�cm
watfmmes
emsc−c−hteowcttiscmzms
attltstn
5IIsh=ssotci
f
2494 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
�TQ� � 1, �31�
here Q is a positive definite matrix. This constraint im-lies that the noise is bounded within an ellipsoid whoserincipal axes are given by the eigenvectors of Q, andith lengths given by the eigenvalues of Q. Let �̂ be the
stimate of the normalized measurement that would bebtained if there were no noise (we will informally refer tohis as the “noise-free normalized measurement”). The ac-ual measurement is related to the noise-free measure-ent by
��� − �̂� = �. �32�
hus the bounded-noise constraint can be expressed inerms of the measurements as follows:
��̂ − ��TQ��̂ − �� � 1/�2. �33�
ow we can formulate the spectral modeling problem asnding the spectrum that has maximum entropy subjecto the constraint of Eq. (33). Let the set of �̂ in the spacef admissible measurements, M, that satisfy this con-traint be referred to as Z���. Then, for each �̂�Z���, wean find a maximum entropy spectral model �̂ consistentith �̂. We can then find the �̂ that has the corresponding
pectral model with highest entropy of all Z���:
�̂ = arg max�̂�Z���
H��̂��̂��, �34�
here H��̂��̂�� is the entropy of the maximum entropyodel associated with the estimate of the noise-free nor-alized measurement �̂.Let us define �0 to be the normalized measurement as-
ociated with the uniform, or white, spectral model:
�0 =1
��
�
f���d�. �35�
his spectrum is the one with globally maximum entropy.simple test of whether ��−�0�TQ��−�0��1/�2 suffices
o determine whether �0�Z���. If this test returns true,hen �̂=�0. If this is not the case, then �̂ must lie on theoundary of Z���.The solution detailed above assumes that the noise is
ounded. Noise produced by practical noise sources can benbounded, however (such as in the case of Gaussianoise). For such unbounded-noise processes this methodill always yield �0 as the solution, since there will al-ays be a possible (however improbable) noise value that,hen added to �0, will result in the observed measure-ent. This is clearly unacceptable, as it means that every
urface would be perceived as a shade of gray. To avoidhis problem one can instead employ an Bayesian method,here the solution is that which maximizes a posteriorrobability incorporating a likelihood measure capturinghe consistency of the model with the given measurement
and a prior based on the entropy of the model. In thisase the estimate of the noise-free normalized measure-ent can be obtained as
�̂ = arg min�̂
��̂ − ��TA��̂ − �� − �H��̂��̂��, �36�
here A is positive definite and � is a constant trading offdherence to the prior as compared with consistency withhe measurement. If �=0, then the estimate of the noise-ree normalized measurement will simply be the giveneasurement. As � becomes larger, the estimate willove toward �0. This approach is identical to maximum
ntropy methods used in quantum physics to estimatepectra [24].
As pointed out by Jaynes [16] the Bayesian method isquivalent to the bounded-noise constrained optimizationethod. To see this, consider the solution, �̂1, of the Baye-
ian method in a particular case for some 0��� . Onean always find a scalar q1 such that ��̂1−��T�q1A���̂1��=1/�2. The data consistency term ��̂−��TA��̂−�� isonstant �=1/q1�2� for all points �̂ for which ��̂��T�q1A���̂−��=1/�2. Thus the solution �̂1 to Eq. (36)as the maximum entropy for all such points (otherwisehe optimization would have found a point with higherntropy). This means that �̂1 is the same as the solutionf the constrained minimization problem [eq. (34)], wheree interpret the noise as being bounded with Q=q1A. We
an consider q1A as setting an effective noise level. Notehat the effective noise level depends on the value of therade-off factor � that is used. Tikhonov and Arsenin [25],n their presentation of the regularization approach to theolution of ill-posed problems, described the converse pro-edure, whereby they show that the unconstrained opti-ization solution is equivalent to the constrained optimi-
ation solution. In their treatment � is a Lagrangeultiplier whose value is determined so as to make the
olution satisfy the constraint ��̂−��T�q1A���̂−��=1/�2.Higher effective noise levels imply larger values of �
nd lead to solutions with higher entropy. In the limit ashe noise level goes to infinity, the model spectrum goes tohe uniform, or white, spectrum. For any nonzero noiseevel the entropy of the model spectrum will be finite, sohat one will never obtain an impulse monochromaticpectrum as a solution. This also means that the magni-ude of � will be bounded, with lower bounds for higheroise levels.
. DETERMINATION OF STIMULUSNTENSITYn the preceding text we had left the intensity value � un-pecified (or took it to be equal to 1). Now we will discussow the value of � can be determined. Recall that �r /�, where r is the measurement vector and � is thetimulus intensity. It should be noted that � is the inten-ity of the incident light only over the visible range (i.e.,ver the range of wavelengths for which the photorecep-ors have nonzero sensitivity). The algorithm does notonsider at all the intensity of the light outside of the vis-ble range.
A photoreceptor with a constant sensitivity profile���=1 can be seen to yield the value of �
0
Teptdet(mlptt
tevrlwmtemitittmmcs
vwcawMoits
n
Nnim
ad
emtc
zm
paa
t(E
tnmiw5ce
ao
Fipinawtmm
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2495
r0 = ���
f0���p���d� = ���
p���d� = �. �37�
he human visual system, at least, does not appear to bequipped with a photoreceptor with such a sensitivityrofile, however; so we are left with the question of howhe intensity can be determined. One possible approach toetermination of the intensity is to extend the maximumntropy paradigm. To see how this might work, considerhat a given measurement vector r defines a metamer setpossibly expanded by measurement noise) in S. Theaximum entropy spectral modeling approach then se-
ects a particular element of this metamer set. But thearticular element that is chosen depends on the value ofhe intensity, �. We could, therefore, determine which ofhese spectral models has the highest entropy.
For each value of � finding the maximum entropy solu-ion amounts to finding the point �̂ on the surface of anllipsoid with axes defined by the eigenvectors and eigen-alues of �2Q, centered at the normalized measurement/�. As � varies, these ellipsoids will change in size, with
arger ellipsoids corresponding to smaller values of �, andill shift their centers to different points along the-geodesic linking the origin and r. As long as rTQr1
hese ellipsoids will trace out a truncated solid cone withlliptical cross section, with the centerline given by the-geodesic passing through the points �=r (correspond-
ng to �=1) and �=0 (corresponding to �= ), and withhe apex at the origin. Let us call this truncated cone thentensity cone. Each spectral model defined by points inhe intensity cone has an associated entropy value, andhe maximum entropy spectral modeling approach deter-ines the model from the �̂ in the cone that has the maxi-um entropy. Figure 2 shows graphically the intensity
one intersecting the set of allowable normalized mea-urements.
Note that if rTQr�1 then the ellipsoids for different �alues will all be contained within one another and eachill contain the origin. This means that in this case the
ones open up and the solution space becomes the entiredmissible space M, and the solution is therefore thehite solution �0 (since this has maximum entropy over). It is interesting to speculate that the reason that col-
rs are perceived as shades of gray in low-light situationss not because the color channels are turned off but ratherhat the signal-to-noise ratio �r � / ��� is so low that thepectral model is always chosen to be the white spectrum.
The corresponding Bayesian, unconstrained, solution isow given by the following modification of Eq. (36):
�̂,� = arg min�,����̂��� −
r
� T
A��̂��� −r
� − �H��̂�� .
�38�
ote that here we perform the search in the � space. Theormalized measurement �̂ corresponding to the result-
ng �̂ is given by Eq. (8). The maximum entropy spectralodel p̂��� is given by Eq. (4).Implementation. Unlike most empirical color appear-
nce models, except for a few special cases, our approachoes not provide analytic formulas for the model param-
ters in terms of the photoreceptor signals. Instead, nu-erical solution processes must be employed to compute
he model parameters. In this section we outline the pro-ess for computing the model.
• Given the measurement vector r, solve the minimi-ation problem of Eq. (38), which provides the spectralodel coordinates �̂ and an estimate of the intensity �.
In carrying out the minimization the quantity �̂ is com-uted from the candidate model �̂ by using Eq. (8). It isssumed that the photoreceptor sensitivity functions f���re given.The entropy H��̂� is computed by using Eq. (14).
• Once the spectral model parameter vector �̂ is found,he canonical divergence can be computed by using Eq.26) [or in the special case where one point is the origin,q. (27)].
In all of the computations described in subsequent sec-ions we take A=I, the identity matrix, implying isotropicoise. The Matlab function fminsearch is used to do theinimization. Because of the presence of local minima it
s necessary to run the fminsearch function many timesith different initial starting points. It is found that using0 different starting points randomly selected was suffi-ient to handle the most difficult cases (cases in which thentropy was low, e.g., nearly monochromatic spectra).
The photoreceptor sensitivity profiles used in the ex-mples are the L,M,S cone 2° data on the human retinabtained from Stockman et al. [26].
Intensity Cone
r
r/β
η1
η2
η3
η0
η̂
(white point)
(estimate of noise-freenormalized measurement)
(normalized measurement)
(photoreceptor measurement)
set of possible normalizedmeasurements, M
line of possible normalizedmeasurements (scaled by β)
ig. 2. Diagram illustrating the determination of the normal-zed measurement estimate given noisy measurements. Given ahotoreceptor measurement vector r, a normalized measurements obtained by dividing by the intensity �. The estimate of theoise-free normalized measurement will be located on the bound-ry of the ellipsoid centered on the normalized measurementith size given by the noise level scaled by the intensity. The in-
ensity is chosen so as to maximize the entropy of the spectralodel associated with the estimate of the noise-free normalizedeasurement.
6Naeidto
ecClefissmve
sfh
T
wstdacsptr
nttsctthjnsodS
nantot
spswtvs
Ifcotismwaahtd
AAaefpac
slecodofcmtd
dntaitmp
2496 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
. COLORFULNESS AND SATURATIONow that we have the spectral modeling theory definednd the information-geometric tools at hand, we canvaluate how well the maximum entropy spectral model-ng theory accounts for the empirical color appearanceata. In this section we will first investigate the ability ofhe approach to predict the color appearance values of col-rfulness and saturation.
Modern color appearance models have relied on nonlin-ar transformations of the cone responses to specify theoordinates of perceptual color space (e.g., Luv, Lab, andIECAM02 [27,28]). Unfortunately, the form of these non-
inear transformations has been determined in a mainlympirical fashion. There is, as yet, no derivation fromrst physical principles that produces the empirical colorpaces most closely describing human color vision. In thisection we will show that the maximum entropy spectralodeling view of color can predict the color appearance
alues of colorfulness quite well without the need for anympirical curve fitting.
We begin by considering how the entropy of the modelpectra varies in � space. Along an e-geodesic emanatingrom the origin passing through any other point �0, weave that �= t�0. The entropy along the geodesic is
H�t� = ��t�0� − t�0 · ��t�. �39�
he derivative of the entropy with respect to t is given by
dH�t�
dt=
���t�0�
�t−
��t�0�
�t· ��t� − �t�0� ·
���t�
�t, �40�
dH�t�
dt=
���t�0�
��t�0�·
��t�0�
�t−
��t�0�
�t· ��t� − �t�0�T
���t�
���t��0
= − t�0TG�t��0, �41�
here G�t� is the Fisher information matrix (metric ten-or). Since the matrix G�t� is positive definite, we havehat dH�t� /dt�0 for all t0. Thus the entropy is strictlyecreasing along an e-geodesic emanating from the origin,nd the canonical divergence with the origin is strictly in-reasing. The isoentropy surfaces (or the isodivergenceurfaces) are nested, with surfaces of higher entropy com-letely enclosed by surfaces of lower entropy. Thus eitherhe entropy or the canonical divergence can be used as aadial coordinate in � space.
From a physical perspective the square root of the ca-onical divergence between the model spectral profile andhe white spectral profile is a measure of the distance be-ween these two profiles. Thus the divergence can be con-idered a measure of the purity of the stimulus color. Inolor science color purity is quantitatively captured byhree different values—colorfulness, chroma, and satura-ion [29]. Colorfulness refers to the perceived amount of aue in the color of a stimulus. Chroma is the colorfulness
udged relative to the brightness of a similarly illumi-ated white surface. Saturation is the colorfulness of atimulus relative to the brightness of a stimulus. The col-rfulness of a stimulus source depends on its brightness;arker surfaces appear less colorful than bright surfaces.aturation, on the other hand, is invariant to the bright-
ess of the surface. The CIE definition of saturation is asfunction of the ratio of the colorfulness to the bright-
ess. In the CIECAM02 appearance model [28] this func-ion is a square root. Hunt [29] suggests that in the casef viewing surfaces under constant illumination chroma ishe appropriate appearance quantity.
We saw earlier that as the stimulus intensity � falls thepectral model moves closer to the white point. This im-lies that the canonical divergence distance D0
1/2 for thepectral model decreases as the intensity decreases. Thuse could hypothesize that this distance corresponds to
he colorfulness of the stimulus. That is, the canonical di-ergence distance defines the following colorfulness mea-ure:
CD��� = D01/2��� = �H�0� − H����1/2. �42�
t should be reiterated here that, in our theory, the reasonor the decrease in colorfulness as stimulus intensity de-reases has nothing to do with the color channels shuttingff or becoming inoperative. Instead, what is happening ishat the noise (assuming that the noise level is constant,ndependent of intensity) is becoming large relative to theignal being estimated. This expands the size of theetamer set, bringing in elements that are closer to thehite point. Eventually this set will expand sufficiently sos to include the white point, at which stage all colors willppear as shades of gray. Lights that have a yellowishue have spectra that closest to the white point and willherefore become perceived gray first as intensity is re-uced, followed by the other hues.
. Comparisons of the Theory with Experimental Datany theory of color vision must be in at least qualitativegreement with experimental observations. There existxperimental data on color appearance, specifically color-ulness, that could potentially be used to test our modelredictions. The two most extensive experimental appear-nce datasets are from the LUTCHI [30] and the more re-ent UCL [31] studies.
However, it must be understood that our model as ittands is limited in that it does not consider receptor non-inearities, chromatic and luminance adaptation, or theffects of surround. It can be thought of as modeling aolor appearance computation taking place downstreamf any adaptation processes. The LUTCHI and UCLatasets were (purposefully) compiled under a wide rangef luminance and surround conditions. Therefore the ef-ects of receptor nonlinearities and adaptation are signifi-ant and must be taken into account when doing anyodel comparisons. As our model at this point does not
ake these effects into account, its predictions cannot beirectly compared with the LUTCHI and UCL data.In the absence of the ability to compare our model pre-
ictions to the LUTCHI and UCL data, we employ theext-best thing, and that is to compare our predictions tohe predictions of another model—one that is generallyccepted as providing good matching to human data. Thedea is that the other model will stand in as a proxy forhe human data. We choose to use the CIECAM02 [28]odel, as it is considered by some as one of the best em-
irical human appearance models.
(MnC[amMtt
mse
pnrt
smmntiCtoecmwmf0stv
t
Fu
Fpgsc
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2497
We computed the maximum entropy spectral modelusing the process outlined in Section 5) for the matte
unsell patches and compared our theoretical colorful-ess measure to that provided by the empiricalIECAM02 appearance model. The Munsell Book of Color
32] is a collection of color patches that uniformly occupylarge gamut in perceptual space. Parkkinen et al. [33]easured the spectral reflectance of the patches in theunsell book. These reflectances can be considered spec-
ral power distributions through the artifice of assuminghat the surfaces are illuminated by an equal-energy illu-
ig. 3. (Color online) Maximum entropy spectral models compasing �=0.0005.
0 20 40 60 80 100 120 1400
0.2
0.4
0.6
0.8
1
CIECAM02 Chroma
Squ
are
Roo
tofC
anon
ical
Div
erge
nce
ig. 4. (Color online) Comparison between the CIECAM02 em-irical chroma (colorfulness) measure and the canonical diver-ence colorfulness measure of Eq. (42). The dashed line repre-ents the best linear fit to the data. The canonical divergence wasomputed by using �=0.0005.
inant. We use this collection of measured spectra as aet of light power spectra representative of those experi-nced by human observers in everyday life.
Figure 3 shows the spectral models produced by ourrocedure for a representative sample from the Parkki-en Munsell spectra database. The models are seen to beather smooth, but fairly close approximations to the ac-ual spectra.
Figure 4 shows the canonical divergence distance mea-ure CD of Eq. (42) plotted against the empirical chromaeasure provided by the CIECAM02 color appearanceodel for all 1250 Munsell patch spectra in the Parkki-en database. The canonical divergences and intensity es-imates were determined from the Munsell spectra by us-ng the optimization process described in Section 5. TheIECAM02 chroma values were derived from CIE tris-
imulus values obtained by projecting the Munsell spectranto the CIE standard observer sensitivity curves. Thequations described in [28] were used to compute thehroma values, under the assumption of equienergy illu-ination, with complete adaptation to the white point. Itas also assumed that the background has the same lu-inance as the patch. Under these conditions the color-
ulness and the chroma are equivalent. A � value of.0005 was used in computing the maximum entropypectral model. Note the very linear relationship betweenhe theoretical distance values and the empirical chromaalues.
The intensity estimates provided by the maximum en-ropy approach for the Munsell patch spectra are shown
th true spectra for four different Munsell patches. Computed by
red wi
istatssss
pptsatnmscsefiisIasl(saoomic2
BMWdcasllp
T
Sw
F
ItaTma
Fpmb
Fdwc(cFmpUo
2498 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
n Fig. 5, plotted against the true intensities. It can beeen that the relation is very linear, with the intensity es-imate being higher than the true intensity by a factor ofbout 1.1. This overestimation does not imply a failing ofhe algorithm. Rather, it is an expected by-product of thepectral modeling approach, which selects a metamericpectral profile different than the true Munsell patchpectrum. The increased intensity is needed for the modelpectra to match the given photoreceptor measurements.
There are some psychophysical observations related toerceived saturation for which we can do a direct com-arison with our theory. For example, it is well knownhat some pure monochromatic stimuli (very narrowbandpectra) appear more saturated than others. Blue and redppear more saturated than yellow. To see whether ourheory can replicate this observation, we computed the ca-onical divergence colorfulness measure as a function ofonochromatic stimulus wavelength. The results are
hown in Fig. 6. Four curves are shown, two theoreticalurves giving the canonical divergence colorfulness corre-ponding to two different � values representing differentffective noise levels, and two experimental curves takenrom the study of Uchikawa et al. [34]. It is evident thatncreasing � (equivalent to reducing intensity or increas-ng the noise level) increases the entropy of the modelpectrum and therefore reduces the saturation measure.t is also clear from the figure that certain wavelengthsppear more saturated than others. In particular theaturation is highest around 420, 540, and 670 nm (or vio-et, green, and red) and lowest around 480 and 600 nmcyan and yellow). These theoretical results are quiteimilar to the experimental data reported by Uchikawa etl., as well as that of Kulp and Fuld (Figs. 8a, 9a, 10a, 11af [35]). It must be pointed out that, although the shape ofur theoretical curves are similar to those of the experi-ental data, to get the shapes to line up requires a shift
n wavelength of about 20 nm (in Fig. 6 the experimentalurves have been shifted to longer wavelengths by0 nm).
0 10 20 30 40 50 600
10
20
30
40
50
60
70
Munsell spectrum intensity
max
imum
entr
opy
inte
nsity
estim
ate
ig. 5. (Color online) Comparison between the actual Munsellatch intensity and the intensity estimated by using the maxi-um entropy approach. The canonical divergence was computed
y using �=0.0005.
. Analytic Expression for the Colorfulness ofonochromatic Stimuli as a Function of Wavelengthe can derive an analytical expression for the canonical
ivergence in the noise-free case for very narrow mono-hromatic pulses. This analysis will give us some insights to why the saturation versus wavelength curve has thehape it does. In the monochromatic case when the noiseevel is very low the magnitude of the � vector will be veryarge. Let �0 be the wavelength of the monochromatic im-ulse. The canonical divergence is then
D�0��0� = H�0� − H����0��. �43�
his can be rewritten as
D�0��0� = ��0� − �����0�� + ���0� · �. �44�
ince the stimulus is assumed to be an impulse at �=�0,e have that �=f��0�. Thus
D�0��0� = ��0� − �����0�� + ���0� · f��0�. �45�
rom Eq. (7) we have the functional definition of �:
�����0�� = log��
exp����0� · f�d�. �46�
f � has a large enough magnitude, we can approximatehe integral in the above equation by using a saddle-pointpproximation, wherein the integrand is expanded in aaylors series about the value of �, where ���0� ·f��� isaximum. If we call this value �m, then the saddle-point
pproximation gives
400 450 500 550 600 650 7000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
wavelength in nm
satu
ratio
n(a
rbitr
ary
scal
e)
ig. 6. (Color online) Square root of the canonical divergenceistance measure D0 for monochromatic stimuli as a function ofavelength. Shown are curves (solid) for two different � values
orresponding to two different effective signal to noise ratioslarger � corresponds to lower intensity). The upper solid curveorresponds to �=0.00056, and the lower solid curve to �=0.56.or comparison are shown two curves derived from the experi-ental data from Table I of Uchikawa et al. (1984) [subject HU,
urity 0.3 (dotted curve) and 0.7 (dashed curve)]. The data fromchikawa et al. have been shifted by 20 nm to align with the the-
retical curve.
Ipa
Fni
Wtorltf�rit�ipctsa+usm
7MTpItb
AMrwcygt
otatpctbi[moos
wltbstlHmtm
c
Tohchftit
lLwtsif
lglJe
BIte
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2499
�����0�� = ���0� · f��m� + 1/2 log�− 2�
���0� · � d2f
d�2��=�m
� .
�47�
f the spectral model approaches a monochromatic im-ulse, then the location of the maximum of ���0� ·f��� willpproach �m→�0. Thus we can write
D�0��0� � ��0� − 1/2 log�− 2�
���0� · � d2f
d�2��=�0
� . �48�
or large � the canonical divergence measure is domi-ated by the term in the denominator of the log function
n the saddle-point approximation of ����, so that we have
D�0��0� � 1/2 log�− ���0� · � d2f
d�2��=�0
. �49�
e can freely perform linear transformations of the f vec-ors and the � coordinates without changing the structuref the spectral color space and without altering distanceelationships. We can obtain opponent channel–uminance channel versions of the f sensitivities by theransformations f1����= f1���− f2���, f2�= f3− �f1+ f2� /2, and
3�= �f1+ f2� /2 (corresponding to the �R−G�, �B−Y�, andR+G� channels). We can immediately determine theelative saturation levels of the unique hues correspond-ng to the peak (maximum and minimum) sensitivities ofhese opponent and luminance channels. For example, if�= ��� � �1,0,0�, where ���� is very large, the correspond-ng spectral model is a sharp peak located at the positiveeak of the R−G channel sensitivity curve (i.e., red). Theanonical divergence is seen to be high when the magni-ude of the second derivative of the relevant channel sen-itivity curve is high. The blue (S-cone), green (M-cone),nd red (L-cone) peaks are narrower than the yellow �LM� peak and hence have higher second-derivative val-es. Thus the colorfulness for the blue, green, and redpectral models will be higher than that of the yellowodel.
. STRUCTURE OF THE SPECTRAL COLORANIFOLD
he previous section looked at the distance between aoint on the spectral color manifold and the white point.n this section we look at the local structure of the spec-ral color manifold, expressed by the incremental distanceetween neighboring points on the manifold.
. Line Elementsost readers will have observed that when looking at a
ainbow, or at the spectral band produced by passinghite light through a prism, it is apparent that different
olors take up different amounts of space in the band. Theellowish region is relatively thin compared with thereenish or reddish regions. Color scientists have oftenackled the problem of understanding the nonuniformity
f the CIE chromaticity diagram in terms of the just no-iceable differences (JND) in color. The JND is themount of wavelength shift that is needed for an observero detect a change in the color of a stimulus. The startingoint for most attempts to elucidate the structure of per-eptual color space is the specification of a line element forhe space. The line element is the incremental distanceetween neighboring points in the space as a function ofncremental changes in the coordinates of the space. Vos36,37] provides excellent reviews of the various line ele-ents that have been put forward. The first specification
f a line element was made by Helmholtz [38], as the sumf the squares of the small changes in the photoreceptorignals weighted by the JNDs of these signals:
dsH2 = � dL
JND�L� 2
+ � dM
JND�M� 2
+ � dS
JND�S� 2
, �50�
here L ,M ,S indicate the long, medium, and short wave-ength cone photoreceptor signals, respectively. He fur-her assumed that the photoreceptor JNDs obeyed We-er’s law and were proportional to the photoreceptorignal level (e.g., JND �L�=�L). This assumption meanshat the Helmholtz line element is actually the Euclideanine element, with coordinates log�L� , log�M� , log�S�. Theelmholtz line element has a serious problem in thataking the log space isophotes to be lines orthogonal to
he isohue lines resulted in a luminance that did notatch the observed result.Schrödinger [39] proposed, without theoretical justifi-
ation, a modified version of the Helmholtz line element:
dsS2 =
1
�L + M + S��� dL
�L 2
+ � dM
�M 2
+ � dS
�S 2� . �51�
his line element can be seen to be Euclidean in the co-rdinates �L ,�M ,�S. In this space the isophote and iso-ue curves are orthogonal resulting in a luminance verylose to the observed. This line element found little use,owever, because of its empirical choice of the square rootunction. Later, Bouman and Walraven [40] derived effec-ively the same line element [without the �L+M+S� termn the denominator] by applying photon signal detectionheory to set JND �X�=�X.
Stiles [41] proposed a modification of the Helmholtzine element that consisted of a relative weighting of the, M, and S components in the ratio of 1:2:16. Thiseighting was motivated by the observed difference in
he relative Weber fractions � of the cone classes. Thislight but important change greatly improved the match-ng of the predicted luminance function with the observedunction.
Vos and Walraven [42] modified the Bouma–Walravenine element by using an extended photon noise model,iving a polynomial functional form for the JND. This al-owed a better modeling of the intensity variation of theND and also provided a connection to the Stiles line el-ment at normal illuminant levels.
. Maximum Entropy Spectral Modeling Line Elementn our spectral modeling approach the differential struc-ure of the spectral color manifold is captured by the linelement associated with the Fisher metric:
TaLc
wGefibtmtiv
lthmtittF
o
wli
wsy
ttsss
(0ttst
mfTtla5(Mlcfimpe4aopatolmtt
8Ic
FsltcTccl
2500 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
ds2 = d�TG−1d�. �52�
he line elements described in the previous section werell expressed in terms of the photoreceptor signals,M ,S. Taking our measurement as ��=r= �L ,M ,S�, we
an write the Fisher line element as
ds2 =1
�2 ��g−1�11dL2 + �g−1�22dM2 + �g−1�33dS2
+ 2�g−1�12dLdM + 2�g−1�23dMdS + 2�g−1�13dSdL�,
�53�
here the �g−1�ij are the elements of the inverse metric−1. Unlike the previously defined line elements this
quation does not assume that the metric is diagonal. Inact the metric provided by the Fisher information matrixs, in general, not diagonal. Thus there are interactionsetween the photoreceptor signals. Since the metric ma-rix is positive definite we can find a coordinate transfor-ation ��=UT� that diagonalizes the metric, where U is
he matrix of the normalized eigenvectors of G−1, remov-ng the interactions between the transformed coordinatealues.
The form of the maximum entropy spectral manifoldine element is rather opaque, and the relation, if any, tohe previously proposed line elements is unclear. We can,owever, compute the line element for the special case ofonochromatic impulse spectra and use this to examine
he wavelength discrimination predicted by our approachn comparison with empirical observations. To do this weake the spectral profile of the light incident on the pho-oreceptors to have the form of an impulse p���=���−�0�.rom Eq. (20) we have that
ds = �d�T
d�0G−1���
d�
d�0 1/2
d�0 �54�
r
d�0
ds= �d�T
d�0G−1���
d�
d�0 −1/2
, �55�
here �0 is the wavelength of the monochromatic stimu-us. It can be seen that, since the stimulus spectral profiles an impulse,
d�
d�0= �df���
d��
�=�0
, �56�
hich depends only on the shape of the photoreceptorensitivity curves. Combining this equation with Eq. (55)ields
d�0
ds= ��dfT���
d��
�=�0
G−1����df���
d��
�=�0
−1/2
. �57�
The quantity d�0 /ds can be taken to be proportional tohe JND, �, in wavelength, under the assumption thathe minimum perceptible distance s is constant over thepectral color manifold. The variation of d�0 /ds withtimulus wavelength, computed by using Eq. (57), ishown in Fig. 7 for three different intensity values
signal-to-noise ratios), computed by using a � value of.0005. The metric is computed by using Eq. (18), wherehe �i are taken from the normalized measurements andhe �ij are computed by projecting the model spectrum as-ociated with the measurements onto the synthetic pho-oreceptor sensitivity curves fi���fj���.
Compare the theoretical curves of Fig. 7 to the experi-ental wavelength discrimination data from McCree [43]
or two different light levels, shown replotted in Fig. 7.he theoretical model provides a wavelength discrimina-
ion curve that lies somewhere between the low and highight intensity curves of McCree. In both our theoreticalnd the experimental curves there is a peak around20–560 nm (green) and two minima near 470–500 nmcyan) and 580 nm (yellow). The higher intensity curve of
cCree shows significant shifting of these peaks and val-eys as compared with the lower-intensity curve. The lo-ations of the peaks and valleys of the theoretical curveall somewhere between the empirical low- and high-ntensity curves. The theory predicts an additional mini-
um around 670 nm, a wavelength that the McCree ex-eriments did not cover. The higher-intensityxperimental data exhibit a significant peak around60 nm. This latter peak is the so-called König–Dietericinomaly [37] and may be due to nonlinear photoreceptorr opponent channel behavior, such as saturation. The pa-ers of Vos [36,37] provide good explanations of the effectsnd modeling of the photoreceptor response nonlineari-ies. We have not incorporated these nonlinear effects intour results, in order to focus on the intensity-related non-inearities inherent in the maximum entropy spectral
odeling approach. A more complete model will includehe photoreceptor nonlinearities as well as a more de-ailed noise model.
. CONCLUSIONSn this paper we have presented a new theory of color per-eption, one that is based on color realism, the philosophi-
350 400 450 500 550 600 650 7000
1
2
3
4
5
6
7
8
9
stimulus wavelength − nm
∆λ−
nm
theoreticalexp − 150 Trolandexp − 0.85 Troland
ig. 7. (Color online) Quantity d� /ds for the maximum entropypectral model as a function of monochromatic stimulus wave-ength, computed by using �=0.0005 and �=1. Also shown arewo curves replotted from McCree’s [43] Fig. 4, showing empiri-al wavelength discrimination data for two different light levels.he scale for the theoretical curve is arbitrary, set for ease ofomparison. It is evident in both the empirical and the theoreti-al curves that wavelength discrimination is greatest in the yel-ow and cyan regions of the spectrum.
cistaswptwmmetprsbct
tammmtvnpeosigtpomsdppemlwmcmsTc
pfwtnstc
dcfstiaysSuc
abids
ATgetTK
R
1
1
1
1
1
1
J. J. Clark and S. Skaff Vol. 26, No. 12 /December 2009 /J. Opt. Soc. Am. A 2501
al view that that color is spectra. Taking this view, wedentify the space of color perceptions with the space ofpectra. To facilitate the analysis of color space we takehe further step of associating spectral profiles with prob-bility density functions. This chain of development re-ults in an identification of the space of color perceptionsith the space of probability density functions. We thenropose that a particular color perceiving system, such ashe human eye and brain, performs spectral modeling,hich is the selection of a distinct element of theetameric spectra consistent with the spectral measure-ents provided by the photoreceptors. The spectral mod-
ling process defines a sub-manifold in the space of spec-ra, equivalent to a statistical model in the space ofrobability densities. This submanifold determines theange of color perceptions that are possible for a specificystem. The structure of the spectral color manifold is sety the Fisher information metric associated with the spe-ific statistical model implied by the photoreceptor sensi-ivity curves and the spectral modeling approach.
In this paper we propose a maximum entropy approacho spectral modeling. Other spectral modeling approachesre possible, such as selecting the average metamer, anday result in similar characteristics. We extend the maxi-um entropy method to handle situations in which theeasurements are noisy and also use it to provide an es-
imate for the intensity of the spectrum. The principal ad-antages of the maximum entropy spectral modeling tech-ique are its wide applicability, as many physicalrocesses generate high-entropy stimuli, and the math-matical convenience of the resulting exponential modelsf the spectra. The latter means that many quantitiesuch as distances are straightforward to compute by us-ng the mathematical machinery provided by informationeometry. One of the main results is the formula for dis-ances between points on the spectral color manifold. Thisrovides a theoretical basis for the computation of the col-rfulness of a stimuli as the distance between the spectralodel corresponding to the measurement induced by the
timulus and the uniform, or white, spectral model. Thisistance measure is shown in the paper to give theoreticalredictions of colorfulness that qualitatively match em-irical observations. In addition, the theory provides anxplicit formula for the line element of the spectral coloranifold. This provides theoretical predictions of wave-
ength discrimination curves that also are in agreementith empirical observations. In our theory the presence ofeasurement noise causes the metamer set to expand. In
ombination with the maximum entropy formalism, thiseans that, as the signal-to-noise level decreases, the re-
ulting spectral model moves closer to the white point.his provides an explanation of why colorfulness de-reases when stimulus intensity decreases.
It should be noted that the mathematical formalismresented in this paper is general in that it can be usedor any set of photoreceptors, whether it be the threeavelength-selective channels of the human trichromat,
he two channels of the human dichromat, the six chan-els of birds, or even the sixteen channels of the mantishrimp [44]. Our approach implies that the color percep-ions obtained in these diverse systems can be directlyompared. This is, in fact, a by-product of the physical un-
erpinnings of our theory—assuming that perception ofolor arises from a physical quantity, namely, spectra, dif-erent sensory systems provide different models of thisingle physical quantity. It is possible that different sys-ems may produce the same model in certain situations,n which case we could truly say that these two observersre perceiving the same color. For example, the blue orellow perceived by a dichromat could be said to be theame as the blue or yellow perceived by a trichromat.uch a statement is not possible, or is even meaninglessnder current theories of color perception, as there is noommon ground to serve as a basis for comparison.
In conclusion, this paper shows that a mathematicallynd computationally sound theory of color perception cane constructed from the color realist premise that color isdentified with spectra, and it demonstrates that the pre-ictions of the theory are not at odds with empiricaltudies.
CKNOWLEDGMENTShis work was supported by the Natural Sciences and En-ineering Research Council of Canada through a Discov-ry Grant awarded to the first author. The authors thankhe reviewers for comments that strengthened the paper.hey also thank Tal Arbel, Mike Langer, Erik Myin, andevin O’Regan for discussions on color vision.
EFERENCES1. B. Maund, “Color,” in The Stanford Encyclopedia of
Philosophy, Winter 2008 ed., Edward N. Zalta, ed., http://plato.stanford.edu/archives/win2008/entries/color/.
2. A. Byrne and D. Hilbert, “Color realism and color science,”Behav. Brain Sci. 26, 3–64 (2003).
3. M. Tye, “The puzzle of true blue,” Analysis 66, 173–178(2006).
4. C. L. Hardin, “A spectral reflectance doth not a color make,”J. Philos. 100, 191–202 (2003).
5. J. K. O’Regan and A. Noë, “A sensorimotor account of visionand visual consciousness,” Behav. Brain Sci. 24, 939–1011(2001).
6. D. H. Brainard and W. T. Freeman, “Bayesian colorconstancy,” J. Opt. Soc. Am. A 14, 1393–1411 (1997).
7. L. T. Maloney and B. A. Wandell, “Color constancy: amethod for recovering surface spectral reflectance,” J. Opt.Soc. Am. A 3, 29–33 (1986).
8. R. T. Cox, The Algebra of Probable Inference (JohnsHopkins Univ. Press, 1961).
9. P. Morovic, and G. D. Finlayson, “Metamer-set-basedapproach to estimating surface reflectance from cameraRGB,” J. Opt. Soc. Am. A 23, 1814–1822 (2006).
0. D. H. Krantz, “Color measurement and color theory: I.Representation theorem for Grassmann structures,” J.Meteorol. Soc. Jpn. 12, 283–303 (1975).
1. C. H. Graham and Y. Hsia, “Studies of color blindness: aunilaterally dichromatic subject,” Proc. Natl. Acad. Sci.U.S.A. 45, 96–99 (1959).
2. D. I. A. MacLeod and P. Lennie, “Red-green blindnessconfined to one eye,” Vision Res. 16, 691–702 (1976).
3. M. A. Webster, E. Miyahara, G. Malkoc, and V. E. Raker,“Variations in normal color vision. II. Unique hues,” J. Opt.Soc. Am. A 17, 1545–1555 (2000).
4. J. J. Clark and S. Skaff, “Maximum entropy models ofsurface reflectance spectra,” in Proceedings of the IEEEInstrumentation and Measurement Technology Conference,IMTC 2005 (IEEE, 2005), Vol. 2, pp. 1557–1560.
5. E. T. Jaynes, “Prior probabilities,” IEEE Trans. Syst. Sci.Cybern. SSC-4, 227–241 (1968).
2502 J. Opt. Soc. Am. A/Vol. 26, No. 12 /December 2009 J. J. Clark and S. Skaff
6. E. T. Jaynes, “On the rationale of maximum-entropymethods,” Proc. IEEE 70, 939–952 (1982).
7. E. T. Jaynes, “Concentration of distributions at entropymaxima,” in E. T. Jaynes: Papers on Probability, Statisticsand Statistical Physics, R. D. Rosenkrantz, ed. (D. Reidel,1979), pp. 315–334.
8. S.-I. Amari and H. Nagaoka, Methods of InformationGeometry, Vol. 191 of Translations of MathematicalMonographs (American Mathematical Society, 2000).
9. R. A. Fisher, “Theory of statistical estimation,” Proc.Cambridge Philos. Soc. 22, 700–725 (1925).
0. S. J. Maybank, “The Fisher-Rao metric,” Math. Today44(6), 255–257 (2008).
1. N. N. Cencov, Statistical Decisions Rules and OptimalInference, Vol. 53 of Transactions on MathematicalMonographs (American Mathematical Society, 1982).
2. C. R. Rao, “Information and accuracy attainable in theestimation of statistical parameters,” Bull. Calcutta Math.Soc. 37, 81–91 (1945).
3. S. Kullback and R. A. Leibler, “On information andsufficiency,” Ann. Math. Stat. 22, 79–86 (1951).
4. N. Silver, D. S. Sivia, and J. E. Gubernatis, “Maximum-entropy method for analytic continuation of quantumMonte Carlo data,” Phys. Rev. B 41, 2380–2389 (1990).
5. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill-posedProblems (Winston, 1977).
6. A. Stockman, L. T. Sharpe, and C. C. Fach, “The spectralsensitivity of the human short-wavelength cones,” VisionRes. 39, 2901–2927 (1999).
7. D. B. Judd and G. Wyszecki, Color in Business, Science,and Industry, 3rd ed. (Wiley, 1975).
8. N. Moroney, M. D. Fairchild, R. W. G. Hunt, C. J. Li, M. R.Luo, and T. Newman, “The CIECAM02 color appearancemodel,” in 10th IS&T/SID Color Imaging Conference(2002) pp. 23–27.
9. R. W. G. Hunt, “Saturation, superfluous or superior?” 9thIS&T/SID Color Imaging Conference (2001), pp. 1–5.
0. M. R. Luo, A. A. Clarke, P. A. Rhodes, A. Schappo, S. A. R.Scrivner, and C. J. Tait, “Quantifying colour appearance.Part I. LUTCHI colour appearance data,” Color Res. Appl.
16, 166–180 (1991).
1. M. H. Kim, T. Weyrich, and J. Kautz, “Modeling humancolor perception under extended luminance levels,” inInternational Conference on Computer Graphics andInteractive Techniniques (ACM Digital Library, 2009),article 27.
2. A. H. Munsell, Munsell Book of Color: Matte FinishCollection (Munsell Color, 1979).
3. J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen,“Characteristic spectra of Munsell colors,” J. Opt. Soc. Am.A 6, 318–322 (1989).
4. K. Uchikawa, H. Uchikawa and P. K. Kaiser, “Luminanceand saturation of equality bright colors,” Color Res. Appl. 9,5–14 (1984).
5. T. D. Kulp and K. Fuld, “The prediction of hue andsaturation for non-spectral lights,” Vision Res. 35,2967–2983 (1995).
6. J. J. Vos, “Line elements and physiological models of colorvision,” Color Res. Appl. 4, 208–216 (1979).
7. J. J. Vos, “From lower to higher colour metrics: a historicalaccount,” Clin. Exp. Optom. 86, 348–360 (2006).
8. H. Helmholtz, Handbuch der physiologischen Optik, 2nded. (Voss, 1896).
9. E. Schrödinger, “Grundlinien einer Theorie derFarbenmetrik im Tagessehen,” Ann. Phys. 368, 481–584(1920).
0. M. A. Bouman and P. L. Walraven, “Quantum theory ofcolour discrimination of dichromats,” Vision Res. 2,177–187 (1962).
1. W. S. Stiles, “A modified Helmholtz line element inbrightness-colour space,” Proc. Phys. Soc. London 58,41–65 (1946).
2. J. J. Vos and P. L. Walraven, “An analytical description ofthe line element in the zone fluctuation model of colorvision, I and II,” Vision Res. 12, 1327–1365 (1972).
3. K. J. McCree, “Small field tritanopia and the effects ofvoluntary fixation,” J. Mod. Opt. 7, 4, 317–323 (1960).
4. T. W. Cronin and J. Marshall, “Parallel processing andimage analysis in the eyes of mantis shrimps,” Biol. Bull.200, 177–183 (2001).
