Development of a UV to IR extension to the standard colorimetry, based on a seven band modified DSLR camera to better characterize
surfaces, tissues and fabrics
Marcello Melis*a, Alice Babbib, Matteo Miccolic
aProfilocolore Srl, 22 Via Spluga, 00141 – Roma, Italy; bUniversity of Rome “Tor Vergata”, Faculty of Electronics; cUniversity of Rome “La Sapienza”, Faculty of Biomedical Engineering
ABSTRACT
Starting from the standard colorimetry, as defined by the International Commission on Illumination – CIE, we propose an enhanced approach to add near UV and near IR sides to the usual visible electromagnetic spectrum, extending standard XYZ colorimetric coordinate to 7 values, calculated with a revision of the Color Matching Functions (CMF) that we called Hypercolor Matching Functions (HMF). We used a modified digital reflex camera and a set of optical filters to realize such a system and we measured 7 band based colorimetric distances among several pigments, that, compared to distances calculated in the XYZ space, resulted into a better and more reliable separation among reflectance spectra.
Keywords: multispectral imaging, modified DSLR, pigment classification, color measurement, UV and IR camera.
1.INTRODUCTION In a number of scientific fields researchers and professionals face the problem of getting information about the characteristics of a surface without even touching it. This is the wide field of the non invasive analysis that ranges from ancient painting diagnosis to medicine (especially dermatology) and forensic. In all those situations the first step is a visual inspection, better if done by a specific domain expert, but quite often this is non enough and further analysis is required. Visual inspection, and better, photographic analysis lay in the range of visible light and, if well performed, could give extra information to guess the nature of the surface from its reflectance. Further investigation can be done widening the spectral range to include the near UV (about 300 to 400nm) and the near IR (700 to 1000/1100nm). The most accurate analysis that can be done is based on a spectral scan of the surface that gives point by point the spectral reflectance of the surface. This is an already used yet quite diffused investigation method that implies a number of pros and cons. It is really very precise, and the only possible further step is to use X-ray analysis or chemical test to find the physical nature of the pigments or tissues. In this case, however, we are leaving the non-invasiveness nature of the analysis. The cons of full spectral analysis are the need of a quite expensive equipment, long acquisition time, considerable amount of resources to manage, store and process the data, and some constraints both about the nature of the geometry of the surface to analyze and about the optical lenses that can be used.
With all this in mind we followed a double line of research, developing, on one side, a sort of extension of the standard colorimetry, developing a modification to the 3 CIE's Color Matching Functions (CMF) that we called the 7 Hypercolor Matching Functions (HMF) covering the UV to IR spectrum, and modifying, on the other side, a commercial digital reflex camera to best fit, with the help of optical filters, the theoretical HMF.
2.HYPERCOLOR MATCHING FUNCTIONS 2.1CIE's Color Matching Functions
Since the 1931 the CIE defined what they called the Color Matching Functions (CMF) that were directly derived from the sensitivity curves of the average human eye with some modification. Given the portion of a spectrum in the visible range, when it is weighted with the 3 CMFs (figure 1) it is transformed in three values, YXZ, called the colorimetric coordinates of that spectrum. This is the way we can assign at any spectra its unique colorimetric measure.
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
Using as a source a monochromatic light and ranging it from red to blue, scanning the whole visible spectrum, and weighting it with the CMF, we obtain a set of points in the XYZ orthogonal space. Imaging a plane crossing the axes at the values X=1, Y=1 and Z=1, and the intersection of the vectors from the origin to these points with this plane, we obtain a special shape. If we furthermore project this shape on the XY plane, the we get the very well know Chromaticity Diagram (figure 2) in the plane xy where x=X/(X+Y+Z) and y=Y/(X+Y+Z). This represents all the colours the human eye is able to see, and each color in the diagram can be obtained with additive synthesis between the appropriate percentage of two colors at the extremes of a segment that includes the color to be synthesized.
One of the criteria the CMF x(λ), y(λ), and z(λ) has been built is to give equal values X=Y=Z as a response to an equal-energy stimulus, i.e. to a stimulus of the same value along all the visible spectrum. In other words the three curves bound the same amount of area. Another of the riteria is that the y(λ) curve corresponds exactly the photopic eye efficiency curve V(λ). This has been choosed to directly derive photometric measures from the y(λ).
It has to be noted that the Chromaticity Diagram never reaches coordinates {x=1, y=0, z=0} or {x=0, y=1, z=0} or {x=0, y=0, z=1}, and there are always at least two non-zero coordinates for whatever spectra we weight with the CMF. This reflects in the rounded shape of the diagram.
2.2Proposed ideal Hypercolor Matching Functions
Let's imagine piecewise linear functions ranging 0 to 1 and back to 0 in the span of 200 nm and overlapping each other by their 25% of area and crossing at the 50% of the value. Well, these are the ideal Hypercolor Matching Functions (HMFi)that we took as reference for our work, and that are shown in figure 3
A spectrum weighted with these HMFt is translated into 7 hypercolor coordinates {Ht1 .. Ht7}.
figure 1: CIE's Color Matching Functions
figure 2: Chromaticity Diagram
figure 3: Ideal Hypercolor Matching Functions
Proc. of SPIE Vol. 8084 80840C-2
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
We call them ideal not only because of their impossible physical feasibility, but because they show a number of interesting characteristics:
if the CMF were like the second, third and fourth HMF, the chromaticity diagram would be totally filled, getting to the maximum saturation of each color
the equienergy criterion is achieved when all the functions have the same gain
the change in the hypercolorimetric coordinates is linear with the change of a monochromatic source along the spectrum
the “minimum distinguishable delta lamba”, i.e. the minimum distance of two monochromatic sources that is measurable, is independent of the absolute wavelength
In the following we will see how close we can get to these functions, using physical (real) devices, such as a modified digital reflex camera and optical filters.
3.THE IMAGING SYSTEM 3.1The camera
The market offers a high number of digital reflex cameras, with a variety of resolutions (megapixels) and all sort of functions. We wanted a camera with a very high quality/price rate, enough resolution, very good signal to noise ratio even at high sensitivity, a camera of the last generation, some movie capability to eventually use the camera even for dynamic phenomena, and of course a band of sensitivity, after modification, wide enough to cover the spectra of our interest. Those consideration let us choice the Nikon D7000, just launched on the market that complies with all our requirements. It rated at the high end of overall quality by the DxO Mark indipendent laboratories, as shown in figure 4.
The high resolution of 16 megapixels, along with very low noise even at high sensitivities as 3200 ISO has been of real help in this study and related research and applications.
The first operation performed on the camera has been the measurement of its spectral sensitivity. To perform the measures we set up a test bench composed of arc/Xenon light source, a regulated current/voltage Halogen light source, a collimator, a monochromator with 6nm minimum band-pass, an integration sphere with five 1-inch ports and one 4-inches port (where we attached the camera), a spectroradiometer with 1.5nm resolution in the range of 300-1050nm, and all the software to control, command and read the data from the instruments and from the camera.
figure 5: DxO Mark laboratories camera ranking of quality/price rate figure 4: Nikon D7000
DSLR
Proc. of SPIE Vol. 8084 80840C-3
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
The diagram in figure 5 shows the sensitivity of the camera. The three graphs correspond to the specific spectral transmittance of the three Bayer matrix (or Color Filter Array, CFA) filters that cover the photo sites of the sensor. It is interesting to see how the red channel spans up to the blue zone as it happens for the X channel of the CMS.
To limit the sensitivity to the visible spectrum the sensor is covered with an IRcut band pass filter (actually two filters, one is the cleaning window). We removed it and measured the spectral transmittance as shown in figure 6.
The filter has been substituted with an UV to IR clear optical window. The thickness of this window has been accurately computed (and then verified) to restore the original optical path and allow the camera to focus to infinity at the most critical wavelength, that was in the IR. The removal of the IRcut filter leads to the spectral sensitivity shown in figure 7.
figure 6: Nikon D7000 standard spectral sensitivity
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
As it can be seen, even if it is not possible to remove the CFA, yet the sensitivity without the IRcut filter widens to the range 350 to above 1000nm. This is the spectral range of our interest. 3.2 The filters
To keep the whole system economically affordable we didn't want to ask for custom band pass filters, each one of them, otherwise, could have been as expensive as the whole camera. The market of machine vision offers very good filters, available in several diameter and ready to be mounted on the lenses. Combining them we were able to obtain a good slicing of the 350-1000nm spectrum as shown in figure 8.
Some comments: the UV band doesn't show the overlap with the visible band as we would have had. This will be fixed in the near future with a new filter. The visible spectrum doesn't require to be segmented because of the Bayer filters that are already optimized to acquire the colors. The last IR filters is a long pass, but it doesn't really matter because of the sensitivity of the camera that after 1000nm quickly goes to very low values. A part of the UV band, all the other show the desired crossing of the sensitivity curves at about the 50% of the value.
3.3The overall system sensitivity and the real HMF
Combining the modified camera sensitivity with the filter transmittance we obtain is the real and final overall system sensitivity before any further trimming.
figure 9: Band pass filters
figure 10: Overall system spectral sensitivity, before trimming
Proc. of SPIE Vol. 8084 80840C-5
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
To take a multispectral image it needs to take five different picture, each of which using one of the 5 band-pass filter, and then the 5 images have to be registered to give each pixel the right set of multispectral values.
Because of the presence of the Bayer matrix in each band, outside the visible one, there are three sensitivity curves that doesn't contribute to a further subdivision of the band itself. After demosaicing the images taken in that bands, i.e. after each pixel has been assigned all the three rgb values coming from the Bayer matrix, to represent the reflectance of pixels in those bands we selected the channel with the best signal to noise ratio. In the UV the strongest signal is the continuous blue one, in the first IR (700-800nm) it is the dotted red one and the the other two infrared (800-900 and 900-1000nm) the dashed green ones.
We wanted the final HMF to be compliant to a couple criteria that have been used to derive also the CMF from the eye's sensitivity curves. This in the direction of getting the HMF the closer possible to the CMF in the range of visible light.
First we wanted the HMF to respond giving H1=H2=H3=H4=H5=H6=H7 coordinates to an equal-energy stimulus. Then we wanted to have the third band (the green) with the same gain of the Y channel of the CMF.
Setting specific sensitivity of the camera for each band (especially in the UV where we rose the sensitivity of the camera up to 3200ISO) and with some post processing, we got the final and actual HMF as a result the camera/filter/setup, that we called HMFf .
This is the spectral sensitivity of the whole system whose curves are shown in figure 10.
4.TEST DATA AND RESULTS 4.1Pigments
To test the system we used the reflectance of 48 different surfaces.
We measured the spectral reflectance of the standard Gretag 24 color checker;
figure 11: final HMF: the HMFf
figure 12: Color checker
Proc. of SPIE Vol. 8084 80840C-6
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
the spectral reflectance of 20 samples from a fine art oil color catalog (from Maimeri);
and the spectral reflectance of 4 special pigments (courtesy of Prof Aldrovandi from OPD) prepared at the Opificio delle Pietre Dure of Florence and measured in collaboration with Istituto IFAC (Istituto di Fisica Applicata “Nello Carrara”) of the CNR (National Council of Research).
Table 1. List of all the tested pigments
1 Black 13 Dark Skin 25 Blu Cobalto OPD/IFAC 37 Rosso di cadmio arancio
7 Terra verde 19 Neutral 6,5 31 Red 43 Superbianco rapido
8 Terra verde antica 20 Blu di cobalto chiaro 32 Lacca di garanza chiara 44 Superbianco
9 Verde permanente chiaro
21 Blu Ceruleo OPD/IFAC
33 Orange Yellow 45 Bianco di titanio
10 Verde ossido di cromo
22 Purplish Blue 34 Rosa quinacridone 46 Bianco d'argento
11 Viridian OPD/IFAC
23 Purple 35 Neutral 8 47 Bianco di zinco
12 Blue 24 Bluish Green 36 Orange 48 White
The spectral reflectance of the pigments have been also used to calculate a 7x3 matrix to convert the HMFf coordinates into the CMF's XYZ. We called these the X'Y'Z' coordinates.
figure 13: Maimer oil color pigments
Proc. of SPIE Vol. 8084 80840C-7
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
Each pigment has been evaluated using:
the standard CMF's : XYZ coordinates
the HMFf derived X'Y'Z' coordinates
the theoretical HMFt : Ht1 .. Ht7 coordinates
the HMFf : Hf1 .. Hf7 coordinates
only three of the HMFf Hf2, Hf3, Hf4 coordinates (the closest to the XYZ)
Then, for each set of coordinates, we measured the euclidean distance among each pair of pigments and put them into triangular matrices.
Finally we represented the triangular matrices graphically (as images) with a scale of false colors, proportional to the pigments' distances.
figure 14: Tested pigments spectral reflectances
figure 15: Pigments distances: in the CMF system (lower) and in the HMFt system (upper)
Proc. of SPIE Vol. 8084 80840C-8
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
figure 17: Pigments distances: with XYZ (lower) and with the derived X'Y'Z' (upper)
Proc. of SPIE Vol. 8084 80840C-9
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
4.2Results
Analyzing figure 14 we see that in average the distances from the ideal HMF are higher than the distances from the CMF. This is not surprising because the ideal HMF exploits a wider portion of the spectrum.
Moving to the finale (actual) HMF (figure 16) and comparing distances from only three coordinates (red, green blue) to distances from the whole spectrum, we have an increase of discrimination of pigments in the upper side (full 7 bands system). This results is quite important because says that the actual system is able to distinguish pigments better that a three channel (actual) system.
Now in figure 17 we compare performance of the standard XYZ system to the performance of the HMFf drived X'Y'Z' system. This is another quite important results because it says that the real system, with a conversion 7x3 matrix is able to show a quite accurate standard colorimetry of the pigments. In figure 17 we computed the absolute difference among upper and lower side of the previous matrix. As it can be seen the maximum difference is in the order of 6 unit, that compared to the 0..140 dynamic of the individual triangular matrices means that the X'Y'Z' derived coordinates are exceptionally close to the standard (yet theoretic) XYZ colorimetric coordinates.
5.CONCLUSIONS In this study we have shown how it is possible to realize a multispectral system starting from commercial devices and filters and some software, achieving interesting results when applied to real pigments and surfaces.
Even the derived X'Y'Z' coordinates fit quite well to the colorimetric XYZ coordinates.
Next step in our research will be to setup a classification system based on the HMFf and to train it on a set of known pigments, and to try to let the system to automatically identify the presence of known pigments in a given surface.
The results of such a system would give benefits not only in the field of painting restoration but also in a number of specializations in medicine, and in the forensic field.
figure 18: Difference between XYZ and X'Y'Z' measured pigments distances
Proc. of SPIE Vol. 8084 80840C-10
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
REFERENCES
[1]OLEARI, C., “MISURARE IL COLORE, SECONDA EDIZIONE”, HOEPLI, (2008).
[2]NAKAMURA, J., “IMAGE SENSORS AND SIGNAL PROCESSING FOR DIGITAL STILL CAMERAS”, CRC PRESS, TAYLOR & FRANCIS, (2006).
[3]PALMER, J., GRANT, B., “THE ART OF RADIOMETRY”, SPIE PRESS, (2010).
[4]GONZALEZ, R., WOODS, R., “DIGITAL IMAGE PROCESSING”. THIRD EDITION, PEARSON-PRENTICE HALL, (2008).
[5] SHARMA, G., “DIGITAL COLOR IMAGING HANDBOOK”, CRC PRESS, (2003).
[6] WYSZECKI, G., STILES, W., “COLOR SCIENCE”, SECOND EDITION, WILEY INTER-SCIENCE, (2000).
[7]ALDROVANDI, A., PICOLLO, M., “METODI DI DOCUMENTAZIONE E INDAGINI NON INVASIVE SUI DIPINTI”, IL PRATO, (2007).
Proc. of SPIE Vol. 8084 80840C-11
Downloaded From: http://proceedings.spiedigitallibrary.org/ on 09/18/2013 Terms of Use: http://spiedl.org/terms
![Page 1: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/1.jpg)
![Page 2: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/2.jpg)
![Page 3: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/3.jpg)
![Page 4: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/4.jpg)
![Page 5: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/5.jpg)
![Page 6: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/6.jpg)
![Page 7: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/7.jpg)
![Page 8: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/8.jpg)
![Page 9: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/9.jpg)
![Page 10: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/10.jpg)
![Page 11: SPIE Proceedings [SPIE SPIE Optical Metrology - Munich, Germany (Monday 23 May 2011)] O3A: Optics for Arts, Architecture, and Archaeology III - Development of a UV to IR extension](https://reader040.documents.pub/reader040/viewer/2022020616/575095891a28abbf6bc2b0a3/html5/page/11.jpg)
