2198 J. Opt. Soc. Am. A/Vol. 27, No. 10 /October 2010 Ratnasingam et al.
Optimum sensors for color constancy in scenesilluminated by daylight
Sivalogeswaran Ratnasingam,1,* Steve Collins,1 and Javier Hernández-Andrés2
1Department of Engineering Science, University of Oxford, OX1 3PJ, Oxford, UK2Department of Optics, Sciences Faculty, University of Granada, Granada 18071, Spain
. INTRODUCTIONn naturally illuminated scenes direct sunlight andhadow can create a scene with a wide dynamic rangehat can then lead to saturation and underexposure ofarts of a scene. These large variations in intensity to-ether with changes in the spectral power distribution ofaylight can also cause unwanted variations in the appar-nt color of the surfaces in a scene. It is these variationsn the recorded colors of surfaces that make it difficult tose color as a reliable source of information when creatingachine vision systems. In contrast reliable chromaticity
nformation can be obtained from the sunlight or skylightlluminated scenes using an algorithm based on thelackbody model of the spectrum of the illuminant [1].archant and Onyango [2] proposed an algorithm for
olving color constancy under daylight by taking ratios ofensor responses. This algorithm is based on the assump-ions that the power spectrum of daylight can be approxi-ated by the blackbody model, and that the spectralidth of the image sensors is infinitely narrow. In addi-
ion, the large variation of intensity and power spectrumf illuminant in a daylight scene can be easily separatedrom the reflectance by taking the logarithm of the sensoresponses. In solving color constancy the advantage of us-ng logarithmic responses was proposed in the Retinex al-orithm [3] and in Horn’s algorithm [4]. Based on thelackbody assumption Finlayson and Hordley [5] pro-osed an algorithm based on the logarithm of three sensoresponses to find a one-dimensional (1D) solution to theolor constancy problem. However, it was shown thatnding a 1D solution to the color constancy problem leadso confusion of perceptually different colors [6]. Finlayson
nd Drew [7] applied this 1D color constancy algorithm toour sensor responses to form an illuminant-independentwo-dimensional (2D) space [7]. Based on the blackbodyodel Romero et al. [8] proposed an algorithm for color
onstancy in scenes illuminated by natural light. Recent-y, Ratnasingam and Collins [1] proposed a model-basedlgorithm that extracts two illuminant-independent fea-ures that represent the surface reflectance using datarom four sensors. For mathematical convenience Ratnas-ngam and Collins [1] assumed that the sensors respondo only a single wavelength, and that the illuminant spec-rum can be modeled by a blackbody spectrum. Thisodel-based algorithm estimates the illuminant effect on
ne sensor response using the responses of two other sen-ors to create an illumination-independent feature [1].owever, sensors with such extremely narrow spectral re-
ponses are both difficult to manufacture and would re-uire long exposure times. Ratnasingam and Collins [1]ave shown that narrow spectral responses are not essen-ial to the algorithm.
In this paper two methods of improving the quality oflluminant-independent reflectance descriptors (referredo as features) are investigated. In Section 2 the perfor-ance of the model-based algorithm proposed by Ratnas-
ngam and Collins [1] is investigated for sensors with dif-erent spectral bandwidths and levels of both sensor noisend quantization noise. A projection-based approach tobtaining features that are independent of the illuminantpectrum is then investigated as a method of improvinghe quality of the features that can be obtained from theesponses of four sensors with different spectral re-ponses. These spectral responses are modeled using a
010 Optical Society of America
Gs[dtblttlo[tstmtfc5
2AlwIsissat
wanwtpc
ftfttmep
fd(m[std(d4bstCd[C
fsn5ufsfssostuHnsibtd
FbGss
Ratnasingam et al. Vol. 27, No. 10 /October 2010 /J. Opt. Soc. Am. A 2199
aussian function, and the response of each sensor isimulated by numerically integrating the image equation1]. The resulting sensor responses are then used as theata from which illuminant-independent features are ob-ained, and the quality of these features is then assessedy using them to distinguish between perceptually simi-ar colors. Another approach to obtaining better results iso optimize the spectral responses of the four differentypes of sensors for this particular application. In particu-ar it may be possible to extend the developing interest inrganic photodetectors integrated onto silicon substrates9] to use families of different organic chromophores [10]o create cameras whose pixels have application-specificpectral responses. In Section 3 the effect of optimizinghe wavelength at which the response of each sensor isaximal is investigated using gradient descent with par-
icular sets of reflectances and daylight spectra. The per-ormance of the optimized algorithms is investigated byhanging reflectances and illuminants in Sections 4 and, respectively.
. PERFORMANCE EVALUATIONsimple method of extracting chromaticity features from
ogarithmic sensors with four different spectral responsesas described recently by Ratnasingam and Collins [1].
n the derivation of this method it was assumed that thepectral width of each sensor is infinitely narrow, and thelluminant spectrum can be approximated by a blackbodypectrum. If these assumptions are valid then it is pos-ible to create two illuminant independent features (F1nd F2) from the responses of four sensors using the equa-ions
F1 = log�R2� − �� log�R1� + �1 − ��log�R3��, �1�
F2 = log�R3� − �� log�R2� + �1 − ��log�R4��, �2�
here R1, R2, R3, and R4 are the sensor responses, and �nd � are two coefficients that will be referred to as chan-el coefficients. If �1, �2, �3, and �4 are the wavelengths athich the four sensors have their maximum responses
hen the variations in both the illuminant intensity andower spectrum can be removed if the two channel coeffi-ients satisfy the following two equations [1]:
1
�2=
�
�1+
1 − �
�3, �3�
1
�3=
�
�2+
1 − �
�4. �4�
In the previous investigation of the feature spaceormed by F1 and F2 [1], the reflectance data used werehose Munsell reflectances that correspond to colors takenrom a thin plane in the CIELab color space, and the rela-ive spread of each of the reflectance samples on the fea-ure space was assessed using a Mahalanobis distanceetric. The sensitivity functions of the sensors were mod-
led using Gaussian functions with a sensible choice ofarameters to cover the entire visible spectrum evenly.
Previously Ratnasingam and Collins investigated theeature space formed by F1 and F2 using CIE standardaylight spectra with correlated color temperaturesCCTs) between 5000 K and 9000 K. However, measure-ents of actual daylight spectral power distributions
11,12] show that the CCT of measured daylight can occa-ionally fall outside this range. Therefore, in this studyhe algorithm’s performance with the entire CCT rangeefined by the International Commission on IlluminationCIE) was used. In particular a set of spectra of CIE stan-ard daylight was chosen with CCTs varying between000 K and 25000 K [13]. The particular CCTs used coulde chosen so that they are spaced evenly along the miredcale (given by 106 K−1) [14]. However our overall aim iso differentiate surfaces illuminated by daylight. TheCTs used have therefore been chosen to have a similaristribution of CCTs as the actual measured daylight11,12]. This set of 20 daylight spectra is referred to as theIE standard test daylights in the rest of this paper.In the initial study of the model-based algorithm [1] the
eatures were obtained from the responses of evenlypread Gaussian sensors in the visible spectrum �400m to 700 nm� with spectral peak positions 437.5 nm,12.5 nm, 587.5 nm, and 662.5 nm (FWHM 80 nm). Fig-re 1 shows the illuminant-independent feature space
ormed with the features (F1 and F2) obtained from thisensor combination. In generating the sensor responsesor this feature space 204 Munsell reflectances [15] withimilar relative lightness were illuminated by the 20pectra of CIE standard test daylights. In Fig. 1 the colorf each reflectance is used to represent the points in thepace that correspond to the surface. The figure showshat, in typical spaces such as the one shown in this fig-re, color in the space varies smoothly across the space.owever, there is a small gap in the upper right hand cor-er of the feature space, and on one side of this gap dis-imilar colors appear as neighbors. These features occurn all the feature spaces that we have observed and ariseecause of the metamer problem that occurs wheneverhe responses of a small number of detectors are used toistinguish between different reflectance spectra.
ig. 1. (Color online) Chromaticity space formed by the model-ased algorithm with unquantized responses of evenly spreadaussian sensors of FWHM of 80 nm. In this space 204 Munsell
amples are projected when illuminated by 20 spectra of CIEtandard test daylights.
rtsprasbat
rd(ustTmcrtt
wpbdebfiMcuttfcflp
cRmdtbsistwgasinnoT
ttndatdcvutt13
ssidtsfl
wsvn
wnecrmfl�Aecvsbsbwtp
sttpslrns
l
2200 J. Opt. Soc. Am. A/Vol. 27, No. 10 /October 2010 Ratnasingam et al.
A closer inspection of the feature space shows that theesidual dependence on the illuminant means that each ofhe Munsell reflectances creates a small cluster of re-ponses in the feature space. The size of these clusters de-ends on several factors including the width of the sensoresponses, the amount of noise in the sensor responses,nd the difference between the spectrum of the lightource and that of a blackbody. To determine the spectralandwidth of the sensors needed to obtain useful featuresmethod was proposed to determine the significance of
he area occupied by each cluster [1].The cluster of points formed by each of the Munsell test
eflectances when illuminating with CIE standard testaylights form a non-uniform distribution of pointsshown in Fig. 1). To account for this observed non-niform distribution it is appropriate to characterize theize of each cluster of points using a distance metric thatakes into account this non-uniform spread of points.herefore, the Mahalanobis distance was applied to deter-ine a boundary that ideally encloses all points in each
luster of responses corresponding to the same Munselleflectance. For n-dimensional normally distributed data,he Mahalanobis distance between the center of the dis-ribution C and a point in the distribution P is defined as
DM2 = �P − C���−1�P − C�, �5�
here � is the covariance matrix of the distribution. For aair of surface reflectances representing colors separatedy a known distance in CIELab space the Mahalanobisistance can be used to determine a boundary aroundach cluster. To determine the Mahalanobis distanceoundary of a particular reflectance pair the first step isnding the center of each cluster of responses. Then theahalanobis distance from the center of the respective
lusters to the boundary was increased from a small valuentil the boundaries formed by both members of a pairouched each other. To assess the dependency of the fea-ure space on the illuminant the number of responses thatell inside the respective boundary in the pair was thenounted. This test was repeated on all the 100 pairs of re-ectances in the test data set, and the percentage ofoints falling within the correct boundary was recorded.The lightness component of a color and the brightness
omponent of an illuminant are inseparable [5,16]. Inatnasingam and Collins’ model-based algorithm [1] re-oving the brightness component of the illuminant to
eal with potential variations in brightness also removeshe lightness component of a color. This is an advantageecause when changing the viewing illuminant of a colorurface the variation in luminance is large compared tots chrominance [17]. The model-based algorithm pre-erves only the chromaticity descriptors of a surface. Inhe previous paper [1] a thin plane of Munsell samplesith CIELab L values between 47.8 and 50.2 was used inenerating the test sets. However, in this paper, to moreccurately assess the algorithm for chromaticity con-tancy, the 1269 Munsell reflectance spectra were normal-zed in such a way that all these samples have a lumi-ance L value in CIELab color space of 50 units. Thisormalization ensured that the only differences in the col-rs of all the reflectance spectra was in their chromaticity.he particular value of L �L=50� was chosen because it is
he mid range of the ‘L’ axis, and it is also used in defininghe CIE standard color difference model (E94) [18]. In theew test sets applied in this paper the reflectance pairsiffer only by their chromaticity. In CIELab space therere several qualitative descriptors defined depending onhe application [11]. One example of a set of qualitativeescriptors is defined by Abrardo et al. [19], who describeolors that differ by between 1.0 to 3.0 CIELab units asery good matches to each other and 3.0 to 6.0 CIELabnits as good color matches to each other [19,20]. Fromhis normalized Munsell data set two sets of test reflec-ances were chosen. Each of these test reflectance sets has00 pairs of reflectances with pairwise distances of 2.99 to.01 and 5.995 to 6.005 CIELab units, respectively.The reflectance data and the daylight spectra were
ampled at 1 nm intervals, and the response of each sen-or to the different reflectances was obtained by integrat-ng the product of Munsell reflectance, the CIE standardaylight spectra, and a Gaussian function representinghe spectral sensitivity of the sensors. For an image sen-or with spectral sensitivity F��� imaging a scene with re-ectance S��� the noisy response R is given by
R = N�1,�2��400 nm
700 nm
S���E���F���d�, �6�
here E��� is the power spectral distribution of the lightource. N�1,�2� is a normal distribution with a meanalue of one and a variance that determines the signal-to-oise ratio (SNR) of the response.For each sensor response the sensor noise �N�1,�2��
as simulated using 100 normally distributed randomumbers. The final step in the model was to represent theffects of using an analog-to-digital converter (ADC) toonvert the sensor responses to digital quantities. In rep-esenting the quantizer effect the first stage is to deter-ine the maximum sensor response. A white standard re-ectance and the CIE standard daylight illuminant6500 K� were used to determine this maximum response.s the optimization process shifts the peak position ofach of the sensors the maximum sensor response wasalculated by shifting the sensor’s peak position in theisible spectrum in 1 nm steps. This way the maximumensor response corresponding to different sensor spectralandwidths was calculated. This maximum sensor re-ponse was then divided by 2n, where n is the number ofits applied in the quantizer, and each sensor responseas then approximated to the nearest one of these quan-
ized levels. In this investigation 8 and 10 bits were ap-lied to quantize the sensor responses.When capturing an image of a scene with an image sen-
or some parts of the scene are well exposed and generatehe maximum sensor response, and some other parts ofhe scene will be underexposed. To avoid either underex-osure or overexposure of any of the modeled sensor re-ponse the spectral power distribution of the daylight il-uminant was scaled in such a way that the sensoresponses are all near the middle of the ADC range. Fi-ally, the features were obtained from the noisy sensor re-ponses using the method described in Eqs. (1) and (2).
The model-based algorithm was tested with differentevels of sensor noise. The SNR of data available from any
csrFdtatspiopSgstms
bshsawnwrt3wstntniwthoo
rtstld
mrfrtuflddbstdetsaddnr2c
bdw4SFmmm
FMdr
Fe4rs
Ratnasingam et al. Vol. 27, No. 10 /October 2010 /J. Opt. Soc. Am. A 2201
amera depends on several factors including the chargetorage capacity of the pixel, the noise introduced by theeadout electronics, and the photon shot noise [21].owler [21] modeled the expected variations of the SNR ofigital cameras and showed that good quality camerashat have pixels with a large charge storage capacity given SNR of larger than 30 dB for all the photocurrentshat can be detected when a 10 bit ADC is used to repre-ent the response from each pixel. Based on their visualsychophysical experiments Xiao et al. [22] report that anmaging device should be able to achieve a SNR of 30 dBr above across the whole dynamic range to render thehoton noise invisible [22]. Imagers are available withNRs larger than 40 dB [23]. We have therefore investi-ated the performance of the model-based algorithm withensor noise of 30 dB and 40 dB. The sensor noise withhese two SNR values was simulated by generating nor-ally distributed random numbers (100 samples) with
tandard deviations of 3% and 1%, respectively.The Mahalanobis distance boundary was drawn for
oth members of each test reflectance pair on the featurepace using the method described above. Two typical Ma-alanobis distance boundaries drawn around the re-ponses from one pair of Munsell reflectances when theyre illuminated by 20 CIE standard daylight spectra andith 100 samples of noise that represent 40 dB Gaussianoise are shown in Fig. 2. The number of points fallingithin the correct boundary was counted for all pairs of
eflectances in a test set, and the percentage of pointshat fell within the correct boundary was recorded. Figure
shows the test results of the model-based algorithmhen applying the responses generated by evenly spread
ensors with different FWHM �20 nm to 200 nm�. In thisest the 3- and 6-unit Munsell test data sets were illumi-ated with CIE standard daylight spectra. As expectedhe performance of the algorithm degrades when theoise level is increased. This is because as the noise level
ncreases the variability in the responses increases asell, and this variability leads to increases in the size of
he clusters, and more points fall outside the correct Ma-alanobis distance boundary. Therefore the performancef the algorithm drops with decreasing SNR. The otherbservation is that the overall performance of the algo-
ig. 2. Typical Mahalanobis distance boundaries for a pair ofunsell samples when illuminated with 20 spectra of CIE stan-
ard daylights. Noise was simulated by generating 100 values ofandom numbers that represent Gaussian noise of 40 dB.
ithm drops with the sensor width. The reason is that ashe width increases the overlap between the adjacent sen-itivity function increases. This increase in overlap be-ween spectral sensitivities leads to an increase in corre-ation between the sensor responses and results in aegradation in performance.The model-based algorithm relies on the blackbodyodel of the illuminant spectrum in calculating the cor-
ect channel coefficients to discount the illuminant effectrom the sensor responses. The performance of this algo-ithm might be improved by adapting the algorithm to es-imate illuminant effects with real illuminants instead ofsing the blackbody model. To estimate the illuminant ef-ect on the sensor responses an approach proposed by Fin-ayson and Drew [7] was applied. This algorithm was alsoeveloped using the assumptions that the power spectralistribution of the illuminant can be approximated by alackbody spectrum, and the image sensors respond to aingle wavelength. In Finlayson and Drew’s [7] approachhe ratios of sensor responses were taken to remove anyependency on the illuminant intensity and scene geom-try. These normalized responses are then projected inhe direction of illuminant-induced variation on the sen-or responses. This projection results in a 2D space that ispproximately independent of illuminant. In finding theirection of illuminant-induced variations the CIE stan-ard illuminants were used instead of blackbody illumi-ants as used by Finlayson and Drew [7]. As this algo-ithm normalizes the lightness component of a color theD space formed by this approach also represents thehromaticity of a color.
Finlayson and Drew’s algorithm [7] and the model-ased algorithm [1] were tested with Munsell reflectanceata and CIE standard daylight. The results obtainedith Gaussian sensors with peak spectral responses at37.5 nm, 512.5 nm, 587.5 nm, and 662.5 nm when theNR of the data from the sensors is 40 dB are shown inig. 4. Applying Finlayson and Drew’s [7] algorithm to re-ove the illuminant effect gives only a slight improve-ent compared to the model-based algorithm. The perfor-ance of both of these algorithms might be improved by
ig. 3. Results of the model-based algorithm when testing withvenly spread sensor responses. Gaussian noise of 30 dB and0 dB was applied to the sensor responses. The resulting linearesponses were quantized to 10 bits. Munsell 3- and 6-units testets were illuminated with the CIE standard test daylights.
ot
3Tlrad[FbswwM1std
tttpmwpfn
wrtcumtu
pti(fimvtnpwznaw2sesw6s
FMaq
FDolu
2202 J. Opt. Soc. Am. A/Vol. 27, No. 10 /October 2010 Ratnasingam et al.
ptimizing the sensor characteristics applied to capturehe data required for the algorithms.
. OPTIMIZATIONhe sensor parameters of both the model-based and Fin-
ayson and Drew’s [7] algorithms were optimized sepa-ately in such a way that the ability to identify perceptu-lly similar colors is improved. In the rest of theiscussion the optimized version of Finlayson and Drew’s7] approach is referred as a projection-based algorithm.or this initial study the number of variables was limitedy assuming that the spectral bandwidth of all the sen-ors was the same and the parameters of the algorithmsere optimized with different sensor spectral band-idths. In optimizing the sensor parameters 100 pairs ofunsell reflectance samples with members separated byCIELab unit were illuminated by 20 spectra of CIE
tandard daylight. Again this set of spectra was chosen sohat it had a distribution of CCTs similar to the measuredaylight. However, the individual CCTs of this set of spec-
ig. 5. Initial and optimized performance of the model-based aunsell data (1 unit) and the CIE standard training daylights.
pplied in testing both algorithms. Gaussian noise of 40 dB was auantized to 10 bits. (a) Model-based algorithm. (b) Projection-ba
ig. 4. Test results of model-based algorithm and Finlayson andrew’s [7] algorithm with evenly spread sensors. Gaussian noisef 40 dB was applied to the sensor responses and the resultinginear responses were quantized to 10 bits. Munsell data (3 and 6nits) were illuminated by the CIE standard test daylights.
ra (CIE standard training daylights) are different fromhe CCTs of the test spectra. This set of CIE standardraining daylights was used to determine the number ofoints that fall inside the correct boundaries using theethod described in Section 2. The inverse of this numberas then used as the error measure in the optimizationrocess. In particular if the gradient of this error measureor a set of independent parameters p is �G�p� then theew parameters
pn+1 = pn − � � G�pn�, �7�
here � is the parameter step size. If this change in pa-ameters was found to increase the error measure thenhe change in parameters must be too large. In these cir-umstances the step size was reduced by a factor of 0.9ntil a set of parameters was found that reduced the erroreasure. The gradient at this new set of parameters was
hen calculated and a new set of parameters determinedntil it was impossible to reduce the error measure.For the model-based algorithm the two outer sensors’
eak positions and the channel coefficients were taken ashe independent parameters in this optimization. The twonner sensor positions can be calculated from Eqs. (3) and4). The outer sensor positions and the two channel coef-cients were taken as the independent parameters toake sure that the sensitivity functions cover the entire
isible region and that both the neighboring sensors con-ribute approximately equally in estimating the illumi-ant effect on the inner sensors, respectively. For therojection-based algorithm the four peak sensor positionsere taken as the independent parameters in the optimi-
ation. The optimization was performed separately byormalizing the sensor responses by the responses gener-ted by sensors 1 to 4 and for different spectral band-idths of the Gaussian sensors interested between0 nm to 200 nm. Two sets of sensible starting sensor po-itions were chosen for the investigation. The first is thevenly spread sensors used to generate the chromaticitypace in Fig. 1 ; the second set is the equal-weight sensorsith peak positions at 437.5 nm, 493.5 nm, 565.5 nm, and62.5 nm. For the model–based algorithm, for a givenensor position the initial channel coefficients can be cal-
jection-based algorithm. Gaussian sensors were optimized withll data (3 and 6 units) and the CIE standard test daylights areto the sensor responses and the resulting linear responses were
gorithm.
nd proMunseppliedsed al
cgetslspsasplpshsflttoswtqe
phamFTaloplscasvpc
4TImst[vfltnimtBsttm
drco
T
Fm
Ratnasingam et al. Vol. 27, No. 10 /October 2010 /J. Opt. Soc. Am. A 2203
ulated from Eqs. (3) and (4). For the projection-based al-orithm the unit vectors can be calculated by applyingigen vector decomposition on the space formed by the ra-io of sensor responses (this was done in logarithmiccale) generated by the 20 spectra of CIE standard day-ight and the 1-unit Munsell reflectance data set. Test re-ults with initial and final parameters of model-based androjection-based algorithm for 40 dB sensor noise arehown in Fig. 5. It can be seen that the performance of thelgorithms has improved slightly when testing with Mun-ell and CIE standard test daylights. The improvement inerformance when testing with 3-unit reflectance data isarger than that obtained with the 6-unit data. This isarticularly true of the model-based algorithm. The rea-on for this is probably that although both algorithmsave been derived assuming sensors that respond at aingle wavelength the model-based algorithm lacks theexibility to deal with the effects of breaking this assump-ion that arise from the data-based projection operation inhe alternative algorithm. Consequently the performancef the model-based algorithm is more dependent on sen-ors with narrow spectral responses. However, sensorsith narrow spectral responses are difficult to manufac-
ure and, since they are starved of photons, they would re-uire long exposure times. A possible limit to the narrow-st spectral responses that might be both possible and
able 1. Optimized Parameters of the Model-BasedAlgorithm
Sensor ID 1 2 3 4Peak position (nm) 434.4 513.6 593.3 676.9Channel coefficient �=0.425, �=0.444
Table 2. Optimized Parameters of the Projection-Based Algorithm When Normalizing the Sensor
Responses by the Response of Sensor 3
Sensor ID 1 2 3 4Peak position (nm) 400.0 487.6 556.0 675.1
ig. 6. Sensitivity functions of optimized Gaussian sensors forizing the Gaussian sensors Munsell data (1 unit) and the CIE
ractical is suggested by the Sony DXC930 camera, whichas three types of pixels sensors with spectral widths ofpproximately 80 nm. Of all the different sets of opti-ized sensors that have been obtained those with aWHM of 80 nm are therefore particularly interesting.heir parameters are therefore listed in Tables 1 and 2,nd their responses are shown in Fig. 6. The parametersisted in Tables 1 and 2 were obtained when starting theptimization with evenly spread sensors. For therojection-based algorithm typical optimized sensorsisted in Table 2 were obtained when normalizing the sen-or responses by the response of sensor 3. An importantonclusion from the parameters such as those in Tables 1nd 2 is that the spectral responses of optimum sensorets can be quite different. In view of the considerable in-estment needed to develop a sensor with a particulareak spectral response it is important to ensure that theorrect spectral responses are specified.
. ROBUSTNESS OF THE CONCLUSIONS TOHE CHANGE IN REFLECTANCE DATA
n Section 3 the performance of the projection-based andodel-based algorithms was investigated with the Mun-
ell reflectance data set. However, this data set is believedo be generated with a limited number of basis functions24]. Therefore the performance of the algorithms was in-estigated with real-world measured reflectances. The re-ectances used are flower reflectances measured aroundhe world [25]. These 2211 measured reflectances wereormalized as described in Section 2. From these normal-
zed reflectances two sets of 100 pairs of reflectances withember separations from 2.998 to 3.002 and from 5.998
o 6.001 units in the CIELab color space were obtained.oth of the algorithms were compared with evenly spreadensors, and the results are shown in Fig. 7. Similar tohe results presented in Section 3 these results also showhat the improvement achieved by applying the projectionethod is small.As both the algorithms were optimized with Munsell
ata and CIE standard daylight illuminant the algo-ithms were tested with floral data to ensure that the pro-ess used to find the optimum sensor combination has notverfit the sensor responses to the Munsell data. The re-
del-based algorithm and (b) projection-based algorithm. In opti-rd training daylights are applied.
(a) mostanda
sstforfpprtfw
5IIagttu
(swddscstswttpasopatwr
Frwnsb
Fsaq
FmMTrr1
2204 J. Opt. Soc. Am. A/Vol. 27, No. 10 /October 2010 Ratnasingam et al.
ults in Fig. 8 show that the optimization has found a sen-or combination that also improves the performance ofhe algorithm with measured reflectances, and the resultsor both algorithms suggest that the optimization has notverfit to the Munsell data set. More importantly, theseesults suggest that using projection to extract featuresrom the responses of optimized sensors leads to an im-rovement in the quality of the features. However, a com-arison between the results obtained with the differenteflectance data sets, Fig. 9, shows that the similarity be-ween the Munsell and floral data set is not good enoughor the Munsell to accurately predict the performanceith floral data.
. ROBUSTNESS TO THE CHANGE INLLUMINANTn Sections 3 and 4 the performances of the model-basednd the projection-based algorithms have been investi-ated with CIE standard daylight illuminants. However,he CIE standard daylight model was developed withhree basis vectors. These basis vectors were generatedsing daylight measurements taken in three places
ig. 7. Performance of model-based and projection-based algo-ithms with evenly spread sensors. In this test floral reflectancesere illuminated by the CIE standard test daylights. Gaussianoise of 40 dB was applied to the sensor responses and the re-ulting linear responses were quantized to 10 bits. (a) Model-ased algorithm. (b) Projection-based algorithm.
ig. 8. Initial and optimized performance of the model-based andell data (1 unit) and the CIE standard training daylights are apre applied in testing both algorithms. Gaussian noise of 40 dB wuantized to 10 bits.
Ottawa, Canada; Rochester, New York, USA, and Middle-ex, England). It is known that the daylight spectra varyith location, time of day, time of year, and weather con-itions. Figure 10 shows the spectrum of CIE standardaylight �6500 K� and three of the measured daylightpectra with CCTs close to 6500 K [11]. From this figure itan be seen that even though the CIE standard daylightpectrum models the overall shape of the measured spec-ra, it does not represent the fine details caused by atmo-pheric absorption. This atmospheric absorption variesith weather conditions and the elevation of the Sun, and
his elevation of the Sun varies with time and location onhe Earth [18]. To ensure that any conclusions are inde-endent of the data, the performance of the model-basednd the projection-based algorithms was tested with mea-ured daylight. In the test 146 daylight spectra measuredn a day in the first week of each month in 1997 was ap-lied [11]. The measurements were taken between 5:30m and 7:30 pm under different types of weather condi-ions in Granada, Spain. Test results of both algorithmsith evenly spread sensors are shown in Fig. 11. These
esults show that both of the algorithms are comparable
tion-based algorithms. In optimizing the Gaussian sensors Mun-Floral data (3 and 6 units) and the CIE standard test daylightslied to the sensor responses. The resulting linear responses were
ig. 9. Test results of the projection-based algorithm with opti-ized sensor responses. Gaussian sensors were optimized withunsell data (1 unit) and the CIE standard training daylights.he CIE standard test daylights were applied in illuminating theeflectances. Gaussian noise of 40 dB was applied to the sensoresponses and the resulting linear responses were quantized to0 bits.
projecplied.as app
iapp
wrlarswtsmd
clrdttsdmi
idmtwrlldtrsc
6TftiDsr
Frssts
Fstr
Fc
Ratnasingam et al. Vol. 27, No. 10 /October 2010 /J. Opt. Soc. Am. A 2205
n performance with evenly spread sensors. These resultsnd the previous results suggest that the performance im-rovement achieved by applying the projection-based ap-roach is small.As the model-based and projection-based algorithms
ere optimized with CIE standard daylight both algo-ithms were tested with measured daylight. Figure 12 il-ustrates the performance of both algorithms with initialnd optimized sensor responses when applying Munselleflectance and measured daylight spectra. These resultshow that the optimization of both algorithms has not al-ays improved the performance of the algorithms when
ested with Munsell reflectances and measured daylightpectra. This suggests that the CIE standard daylightight not be a good representative data set for measured
aylight at a particular location.The performance of the projection-based algorithm was
ompared with CIE standard daylight and measured day-ight. Figure 13 illustrates the performance of the algo-ithm with Munsell reflectances illuminated by CIE stan-ard daylight and measured daylight. It can be seen thathere is a significant performance drop when changinghe illuminant from CIE standard daylight to the mea-ured daylight. The reason could be that the CIE stan-ard daylight model was developed using the daylighteasurements taken in three places, and the measured
lluminants applied in our investigation were measured
ig. 12. Initial and optimized performance of (a) model-based apectra. Gaussian sensors were optimized with Munsell data (1 uested with Munsell reflectance data (3, 6 units) and 146 spectra oesponses and the resulting linear responses were quantized to 1
ig. 10. Spectra of CIE �6500 K� and measured daylight withorrelated color temperature 6508 K, 6481 K, and 6519 K.
n Granada, Spain, under different types of weather con-itions over a longer period of time (two years). As theeasurements used in the CIE standard model and our
est illuminants are taken from different parts of theorld the CIE standard daylight might not adequately
epresent the measured daylight used in this paper. Thiseads to the performance difference when changing the il-uminants from CIE standard daylight to the measuredaylight. The results presented in Figs. 12 and 13 suggesthat the CIE standard daylight model is not a good rep-esentative spectra that could be used to optimize the sen-or characteristics or to predict the exact performance of aolor constancy algorithm.
. CONCLUSIONShe performance of a blackbody-model-based algorithm
or color constancy under daylight illuminant was inves-igated and compared with the performance obtained us-ng an alternative algorithm proposed by Finlayson andrew [7]. Both of these algorithms are based on the as-
umptions that four types of sensors are used that eachespond to a different single wavelength, and that the il-
ig. 11. Test results of model-based and projection-based algo-ithms when testing with responses generated by evenly spreadensor responses. Gaussian noise of 40 dB was applied to theensor responses and the resulting linear responses were quan-ized to 10 bits. Munsell reflectances were illuminated by mea-ured daylight.
projection-based algorithm with Munsell and measured daylightd the CIE standard training daylights. Both the algorithms wereured daylight. Gaussian noise of 40 dB was applied to the sensor
.
nd (b)nit) anf meas0 bits
ltsustTbfitpsnceattamt
ffmstiswpzaosdreips
psldmtmssbpcfrt
R
1
1
1
1
1
1
1
1
1
Ftsstq
2206 J. Opt. Soc. Am. A/Vol. 27, No. 10 /October 2010 Ratnasingam et al.
uminant spectral power density can be approximated byhat of a blackbody. Results that have been presentedhow that either of the algorithms can be used to obtainseful information even when each of the sensors re-ponds to a relatively wide range of wavelengths andhere is significant overlap between the sensor responses.he difference between the algorithms is that the model-ased algorithm only requires the calculation of two coef-cients, while the alternative algorithm requires a projec-ion. While the coefficient calculation is very easy the bestrojection can only be calculated using the measured sen-or responses to several colors under different illumi-ants. Although this makes the projection algorithm lessonvenient for the user, the projection algorithm might bexpected to give better results for real illuminant spectrand sensor responses. Somewhat surprisingly the resultshat have been obtained suggest that the performance ofhe two algorithms when used to process noisy data fromsensible choice of sensors is almost identical. Using theore sophisticated algorithm to process the sensor data is
herefore not a reliable way to achieve better results.An alternative approach to obtaining better results
rom a set of sensors is to exploit the flexibility offered byamilies of different organic chromophores [10] to opti-ize the spectral responses of the four different types of
ensors. The effect of optimizing the spectral responses ofhe four sensors on the results that could be obtained us-ng the two algorithms has been investigated using ateepest descent algorithm. The optimization metric thatas chosen for this investigation was the separability oferceptually similar colors. The data used during optimi-ation were the freely available Munsell reflectance datand the CIE standard daylight spectra. As expected thisptimization improved the separability of perceptuallyimilar Munsell samples illuminated by CIE standardaylight spectra, and the results obtained were compa-able to those achieved by the human visual system. How-ver, it was observed that the optimization did not alwaysmprove the performance of the model-based androjection-based algorithms when measured illuminantpectra were used. This shows that optimizing the sensor
ig. 13. Test results of the projection-based algorithm with op-imized sensors when illuminating the Munsell reflectanceamples with the CIE standard test daylight spectra and mea-ured daylight spectra. Gaussian noise of 40 dB was applied tohe sensor responses and the resulting linear responses wereuantized to 10 bits.
arameters with CIE standard daylight does not neces-arily result in an optimum set of sensors for real day-ight. In addition, the results obtained with CIE standardaylight are significantly better than those obtained witheasured daylight. These two observations suggest that
he CIE standard daylight spectra do not represent theeasured daylight spectra in enough detail for the CIE
tandard daylight spectra to be used to reliably optimize aystem designed to distinguish a quite subtle differenceetween two colors. Although the optimization could beerformed using the data measured in Granada it is notlear that the resulting system would perform well in dif-erent locations. Before a set of optimum sensors can beeliably defined, daylight spectra from different parts ofhe world are required.
EFERENCES1. S. Ratnasingam and S. Collins, “Study of the photodetector
characteristics of a camera for color constancy in naturalscene,” J. Opt. Soc. Am. A 27, 286–294 (2010).
2. J. A. Marchant. and C. M. Onyango, “Shadow-invariantclassification for scenes illuminated by daylight,” J. Opt.Soc. Am. A 17, 1952–1961 (2000).
3. E. H. Land and J. J. McCann, “Lightness and retinextheory,” J. Opt. Soc. Am. 61, 1–11 (1971).
4. B. K. P. Horn, “Determining lightness from an image,” Com-put. Graph. Image Process. 3, 277–299 (1974).
5. G. D. Finlayson and S. D. Hordley, “Color constancy at apixel,” J. Opt. Soc. Am. A 18, 253–264 (2001).
6. M. Ebner, Color Constancy, Wiley Series in Imaging Sci-ence and Technology (Wiley, 2007).
7. G. D. Finlayson and M. S. Drew, “4-sensor camera calibra-tion for image representation invariant to shading, shad-ows, lighting, and specularities,” in Proceedings of IEEE In-ternational Conference on Computer Vision (IEEE, 2001),pp. 473–480.
8. J. Romero, J. Hernández-Andrés, J. L. Nieves, and E. M.Valero, “Spectral sensitivity of sensors for a color-image de-scriptor invariant to changes in daylight conditions,” ColorRes. Appl. 31, 391–398 (2006).
9. S. Takada, M. Ihama, M. Inuiya, T. Komatsu, and T. Saito,“CMOS color image sensor with overlaid organic photocon-ductive layers having narrow absorption band,” Proc. SPIE6502, 650207 (2007).
0. R. Jansen van Vuuren, K. D. Johnstone, S. Ratnasingam,H. Barcena, P. C. Deakin, A. K. Pandey, P. L. Burn, S. Col-lins, and I. D. W. Samuel, “Determining the absorption tol-erance of single chromophore photodiodes for machine vi-sion,” Appl. Phys. Lett. 96, 253303 (2010).
1. J. Hernández-Andrés, J. Romero, J. L. Nieves, and R. L.Lee, Jr, “Color and spectral analysis of daylight in southernEurope,” J. Opt. Soc. Am. A 18, 1325–1335 (2001).
2. S. T. Hendersons and D. Hodgkiss, “The spectral energydistribution of daylight,” Br. J. Appl. Phys. 14, 125–133(1963).
3. Munsell Color Science Laboratory, “Daylight spectra,”http://mcsl.rit.edu/.
4. S. Tominaga and B. A. Wandell, “Natural scene-illuminantestimation using the sensor correlation,” in Proc. IEEE 90,42–56 (2002).
Ratnasingam et al. Vol. 27, No. 10 /October 2010 /J. Opt. Soc. Am. A 2207
9. A. Abrardo, V. Cappellini, M. Cappellini, and A. Mecocci,“Art-works color calibration using the VASARI scanner,” inProceedings of IS&T and SID’s 4th Color Imaging Confer-ence: Color Science, Systems and Applications (IS&T, 1996),pp. 94–97.
0. J. Y. Hardeberg, “Acquisition and reproduction of color im-ages: colorimetric and multispectral approaches,” Ph.D. dis-sertation (Ecole Nationale Supérieure des Télécommunica-tions, 1999).
1. B. Fowler, “High dynamic range image sensor architec-tures,” in High Dynamic Range Imaging Symposium andWorkshop, Stanford University, California, 2009,http://scien.stanford.edu/HDR/HDR_files/
2. F. Xiao, J. E. Farrell, and B. Wandell, “Psychophysicalthresholds and digital camera sensitivity: The thousandphoton limit,” Proc. SPIE 5678, 75–84 (2005).
3. S. Winkler and S. Susstrunk, “Visibility of noise in naturalimages,” Proc. SPIE 5292, pages 121–129 (2004).
4. J. P. S. Parkkinen, J. Hallikainen, and T. Jaaskelainen,“Characteristic spectra of Munsell colors,” J. Opt. Soc. Am.A 6, 318–322 (1989).
5. S. E. J. Arnold, V. Savolainen, and L. Chittka , “FReD: Thefloral reflectance spectra database,” Nature, http://dx.doi.org/10.1038/npre.2008.1846.1.
