Julian Bescos, Jorge H. Altamirano, Antonio Santisteban, and Javier Santamaria
Color images degraded by defocusing and chromatism are restored using different digital models and linearfiltering techniques. The restoration process for color images is properly performed through the filtersassociated with three transfer functions for tristimulus values. However, the use of three monochromaticOTFs corresponding to the maxima of the color receiver is also discussed for different cases of wavelength-dependent aberrations, since it enables a significant reduction in the computer processing time.
1. Introduction
Color images are at present widely used in manyareas to increase the available information for humanobservers or to encode by color different types of infor-mation. In the first case, color images are formedthrough imaging systems which are designed in generalto give the required image quality and to preserve thecolor fidelity. In the second case, great sensitivity ofthe visual human system to small color changes is usedto display in false color information collected in severalspectral bands that does not correspond to the usualcolor channels.
Nevertheless, the number of studies on formationand processing of color images is small in comparisonwith the wide use of such images. This fact is partiallydue to the lack, until very recently, of accurate colorim-eters as well as of a consolidated uniform color theory.As a consequence most efforts in image processinghave been addressed for image enhancement purposesor more attractive display, and only a limited numberof papers include quantitative studies. However, thesituation is slowly changing since both drawbacks arebeing overcome, especially the first. In this way, thepossibility to correct in a quantitative way point non-linearities, imperfect color responses, etc. can now beimplemented more accurately.
Previous work on digital color restoration has beendevoted to correct color errors introduced by an imper-
A. Santisteban is with IBM Scientific Center, Paseo Castellana 4,28001 Madrid, Spain; the other authors are with Institute of Optics,Serrano 121, 28006 Madrid, Spain.
fect color recording system and to control with maxi-mum colorimetric fidelity the recording of digital im-ages from a monitor onto film.2 Diverse digital filtershave been used to restore in 1-D the chromatic effectsintroduced on the image of a wide slit by a single lens.3
Also, optical filtering has been used to enhance colorsaturation by attenuation of the zero order in poly-chromatic light. 4
The compensation of defects introduced on the colorimages during the imaging process requires the use ofappropriate color models. The different behavior ofthe systems for each wavelength must be taken intoaccount in the model, although it may cost a significantincrease in computer time. In this sense, the introduc-tion of efficient restoration schemes should be consid-ered with regard to the resulting image quality.
In this paper, the restoration of color images degrad-ed by defocusing and chromatism is discussed in con-nection with different color imaging models. Colorimages are first digitally formed in the presence ofthose aberrations using three transfer functions for thetristimulus values56 expressed in terms of the 1931CIE Color System.7 In this case, the transfer of thepolychromatic contrast by an optical system is associ-ated with the Ytristimulus value OTF, while the trans-fer of chromaticity depends on the relative differencesamong the three polychromatic OTFs. Later, the de-graded images have been restored with two models; thefirst through digital filters associated with the poly-chromatic transfer functions for the tristimulus val-ues, and the second through filters corresponding tothe monochromatic transfer functions of the maximaof the color receiver.
In the following sections image formation and resto-ration theory for color imaging is briefly outlined, andthe results obtained with the two models in the pres-ence of defocusing and chromatism are discussed.
The formation of color images through optical sys-tems in the presence of spatially incoherent light canalso be expressed by Fourier analysis, extended topolychromatic light. In this case, the linearity re-quirement is satisfied if the spatial color distribution isexpressed by the tristimulus values, and space-invari-ant patches are obtained, in a similar way to the mono-chromatic case, for optical systems already correctedto some degree. In this way, the tristimulus values inthe image (XI, YI, ZI), expressed in terms of the 1931CIE System, are derived from those of the object (Xo,Yo, Zo) by three convolution integrals 5 :
Xl(x',y') = JJXo(xy) Dx(x'-x,y'-y) dx dy,
Y1(x',y') = JJ Y 0(xy) D(x-xy'-y) dx dy, (1)
Zl(x',y') = JJ Zo(x,y) Dz(x' - xy' - y) dx dy,
where Dx, Dy, and Dz are the tristimulus values of thepolychromatic point spread function (PSF). Alterna-tively, in the Fourier domain we obtain
9 1 (u,v) = 9 0 (u,v)Hx(u,v),
?/(u,v) = O(uv)Hy(uv),
21(u,v) = Zo(u,v)Hz(u,v),
where S(X) is the spectral distribution of the source,and x(X), y(X), and z(X) are the three sensitivity func-tions of the human eye.7 Therefore the spectrum ofthe final image, Eqs. (2), can be obtained by the prod-uct of the Fourier transformation of the distribution ofthe tristimulus values in the object Xo, Yo, and Zo andthe tristimulus values' transfer functions Hx, Hy, andHz. The presence of degradations in the optical sys-tem modifies Dx, Dy, and Dz, and the modulus andphase of the OTFs Hx, Hy, and Hz. The originalimage spectrum, degraded according to Eq. (2), can berecovered using inverse filters
{1 1 1 VHX Hy Hz
or other filters on the degraded image spectrum. Inthe case of additive noise, the most accepted restora-tion technique is Wiener filtering, which minimizes themean-square difference between the original and re-stored images. The filters are defined as
where the ̂ represents Fourier transformation, and Hx,Hy, and Hz are the tristimulus value transfer functions(OTFs). These latter functions can be obtained byFourier transformation of the tristimulus values in thePSF, Dx, Dy, and Dz, or from the monochromatictransfer functions, by the following integrations:
Hx(uv) =
HY(u,v) =
J S(X) * Hx(u,v) * x. * dX
J S(X) * x;dX
f S(X) H(u,v) * y, * dX
J S(A) . ydX
fS(A H,(u,v) dA
nfz(UV) =J S(A) z~dX
where k,(u,v) and f(u,v) are the spectral densities ofnoise and original image, respectively. To achieveoptimum restoration 0,(u,v) and of(u,v) must beknown a priori. However, for linear image restorationthe quality of the restoration judged by its visual ap-pearance is relatively insensitive to the spectral char-acteristics of 0,b and pf. We assume that they arespectrally white, so that kn(u,v)/f(u,v) can be taken tobe an adjustable constant parameter. This assump-tion is good when the aberrations are severe comparedwith the variation of q5n and pf.
Ill. Application of Restoration Models
A. Original Image
A test image was obtained by digitizing on a Perkin-Elmer 1010A flatbed microdensitometer a well-fo-cused slide. The image was digitized at 512 X 512pixels for the red, green, and blue spectral bands se-
Fig. 1. Original digital image: red, green, and blue components, from the left.
Fig. 2. Tristimulus values of the monitor color channels for differ-ent gray levels.
lected by the 94, 93, and 92 Kodak Wratten filters ofthe microdensitometer. The image was displayed on acalibrated monitor of a Ramtek 9351 terminal andtaken as the original image. Figure 1 shows the red(R), green (G), and blue (B) components of the originaldigital image recorded from the monitor, and Fig. 6 atthe top, the color digital image.
Since the color degradation and restoration processis made in terms of the tristimulus values defined inthe 1931 CIE System, the original color image wasexpressed in the same System. For that the gain andcontrast of the color monitor were first adjusted foreach color channel to obtain, for the different dis-played gray levels, the chromaticity coordinates of theD65 source. Then, the contribution of the three colorchannels of the monitor to the X, Y and Z values wasexperimentally obtained. The color measurementsfor both cases were made with a Spectra-Pritchardphotometer-colorimeter. Figure 2 gives, for each col-or channel and white, the contribution of the differentgray levels to the X, Y and Z values. These resultsallow a linear adjustment of the gray level tristimulusvalues and therefore express the original color image inthe 1931 CIE System.
B. Degradation and Restoration Models
The digital image expressed in X, Y and Z valueswas digitally degraded, via Eq. (3), through the threetransfer functions for the tristimulus values. TheseOTFs provide a realistic model of the effects of aberra-tions in actual systems.6 The transfer functionsHxHyHz were obtained as DFTs of the polchromaticPSF, which have been calculated with a numericalmethod for computing diffraction patterns of opticalsystems in the presence of aberrations.8'9 The PSFwas sampled (512 points) in such a way that the cutoff
a)I
Ob
Fig. 3. Color image components blurred by defocusing through the tristimulus values OTFs: (a) 60W 20 = 0.5 tm; (b) 6oW 2 = 1 im.
frequency of the optical system for the shortest wave-length (0.38 ,m) coincides with the Nyquist frequencyof the digitized image. This would correspond to anoptical system of reduced aperture, since the space-bandwidth product of optical systems is usually muchgreater at normal apertures.
The first restoration model uses three Wiener filterscorresponding to the degraded transfer functions forthe tristimulus values. The ratio of the noise spectraldensity q5) to signal spectral density of is assumedconstant and very small. So, Wiener filters with verysmall ¢onhf values are nearly inverse filters where onlyinformation very close to OTF zeros is not recovered.The ratio is adjusted experimentally assuming that theaberrations are more influential than the variations ofd0n and of on the spatial frequency. This ratio allows usto control noise amplification or smoothing.
The second restoration model is a simplified ap-proach in which the restoration filters of Eq. (4) areconstructed from three monochromatic transfer func-tions corresponding to the wavelengths of the pixels ofthe color sensitivity functions of the human eye. Themonochromatic OTFs have been obtained as the auto-correlation (computed via DFT) of the pupil functionthat accounts for defocusing and other aberrations.This procedure requires much less computer time thanthe method used to obtain the OTFs in the first model.
C. Results
The previous models have been used for color imagesdegraded by defocusing and chromatism. Figure 3shows the components of the degraded images ob-tained in the presence of defocusing, 5 0W20 = 0.5 and 1,um. The parameter 5OW 20 represents the optical pathdifference introduced by the shift of focus AZ at theedge of the aperture and is related to AZ by6 5oW20 =
- 1/2n'sin 2a'AZ, where a' is the semiaperture angle andn' is the refractive index.
The OTFs for tristimulus values computed for defo-cusing, 6OW 2 0 = 1 Am, are given in Fig. 4, top.
Figure 5 shows the red, green, and blue restoredcomponents for a 1-Mm defocused image, using the firstmodel based on the Wiener filters corresponding to thetristimulus values OTFs with a value nl¢hf = 0.0001that gives the best results.10 The filters, shown in Fig.4, bottom, are similar for the more significant frequen-cy range. Finally, the degraded and restored color
images are shown in the center and bottom of Fig. 6,respectively.
The second model, based on the filters of the mono-chromatic OTFs, has also been used to restore theimages of Fig. 3. The results are not included here,since the differences between the images restored bythe two models cannot be detected visually.
Later, the models were applied to images degradedby aberrations varying with X, such as longitudinalchromatism. Figure 7 shows the degraded color com-ponents obtained with the OTFs for the tristimulusvalues of Fig. 8 that correspond to the uncorrected
HO
.S4
O..;
O.2
. c
C..
0-2
W
5 50 10. 00 050 000 1 0 050 0R
0 00 50 00 100. fl 150 100 000 001 000Zt sll as s; ! !; Ac us 4 s
R
Fig. 4. Top, OTFs for the tristimulus values in the presence ofdefocusing: 50W20 = 1um. Bottom, Wiener filters corresponding to
OTFs for tristimulus values (top).
Fig. 5. Restored image of the defocused image of Fig. 3(b) using OTFs for tristimulus values.
Fig. 7. Color image components blurred by the longitudinal chromatism of a single lens.
- - - H,
- H.
....... .Hz
0 es 50 11 000. 020
R
- - - H 55
H 6
..... H.4 5
N
N
NN
Zs 50 10 4 0. 10 . 0 2 o 0 5 33 4
tom) obtained with a kn/,bf ratio of 1 X 10-3. Thisresult is not surprising since the monochromatic OTFsfor the maxima of the sensitivity functions (bottom ofFig. 8), are different from the OTFs for the tristimulusvalues (top of Fig. 8). Therefore, the usefulness of themuch simpler and efficient second method is ad-dressed for aberrations with small wavelength depen-dence and cannot be applied in the case of chromaticaberrations.
IV. Conclusions
Color images have been digitally degraded by thetransfer functions of the tristimulus values, computedin the presence of defocusing and chromatism. Thedegraded images have been restored by the Wienerfilters associated with those polychromatic transferfunctions and give good results for 1-mm defocusingvalues and typical chromatism of a single lens. Asimplified restoration model, based on the filters cor-responding to monochromatic OTFs of the maxima ofthe color receiver, has also been used and represents animportant reduction in computer time required for thepolychromatic OTF calculations. This simple modelbehaves adequately for nondispersive aberrations, butit gives unsatisfactory results for aberrations varyingwith wavelength, such as chromatism.
When this work was done J. H. Altamirano was onleave from the Escuela Superior de Fisica y Matemati-cas of the Instituto Politecnico, Mexico.
Fig. 8. Top, OTFs for tristimulus values in the presence of longitu-dinal chromatism of a single lens. Bottom, monochromatic OTFscorresponding to the maxima of the color receiver for the same
chromatism.
linear chromatism of a single lens working at 1:12aperture. The degraded color image (not included)shows a yellowish tone in the white object points, dueto the blurring of the blue component. Nevertheless,the restored color components shown in Fig. 9, top, andthe composite color image using the first model, give aquality similar to that obtained in the case of defocus-ing. However use of the second model fails, as can beseen from the best restored components (Fig. 9, bot-
References1. C. E. Mancill, "Digital Color Image Restoration," USCIPI Re-
port 630 (1975).2. R. A. Wallis, "Film Recording of Digital Color Images," USCIPI
Report 570 (1975).3. J. Bescos, I. Glaser, and A. A. Sawchuk, "Restoration of Color
Images Degraded by Chromatic Aberration," Appl. Opt. 19,3869 (1980).
4. J. Santamaria, A. Plaza, and J. Bescos, "Color Image Enhance-ment by White Light Spatial Filtering," Opt. Commun. 45, 244(1983).
5. J. Bescos and J. Santamaria, "Formation of Colour Images.Optical Transfer Functions for the Tristimulus Values,"Photogr. Sci. Eng. 6, 355 (1977).
6. J. Bescos and J. Santamaria, "Colour Based Quality Parametersfor White Light Imagery," Opt. Acta 28, 43 (1981).
Fig. 9. Top, restored components of the image of Fig. 7 through the filters of the OTFs for the tristimulus values shown in Fig. 8 (top).
tom, restored components of the image of Fig. 7 through the filters of the monochromatic OTFs shown in Fig. 8 (bottom).
7. G. Wyszecki and W. S. Stiles, Colour Science (Wiley, New York,1967).
8. M. J. Yzuel and J. Santamaria, "Polychromatic Optical Image.Diffraction Limited System and Influence of the LongitudinalChromatic Aberration," Opt. Acta 22, 673 (1975).
9. M. J. Yzuel and J. Bescos, "Polychromatic Off-Axis OpticalImage. Influence of the Transverse Chromatic Aberration,"Opt. Acta 22, 913 (1976).
10. L. R. Berriel, J. Bescos, and A. Santisteban, "Image Restorationfor a Defocused Optical System," Appl. Opt. 22, 2772 (1983).
