Adaptation of spectral constraints to electronicallyhalftoned pictures
Thomas Scheermesser, Manfred Broja, and Olaf Bryngdahl
Department of Physics, University of Essen, 4300 Essen 1, Germany
Received April 30, 1992; revised manuscript received September 3, 1992; accepted September 9, 1992
An iterative halftoning algorithm that involves Fourier transforms permits the realization of control of the bi-nary image in the spectral domain. An approach is presented for adapting the spectral constraints to the char-acteristics of the graytone image and to the characteristics expected from the binary image. We proposeoversampling of the spectrum to take into account the continuous spectrum of the final image.
1. INTRODUCTION
The widespread use of printing and display devices thatproduce only two different output levels, i.e., black andwhite, has led to an increased interest in halftoning tech-niques. The halftoning process transforms a graytoneimage into a binary one, which can be printed or displayedwith the use of a two-level device. Various halftoningtechniques exist; carrier and error-diffusion methods aretwo well-known examples. 3 These methods have differ-ent properties and lead to images with characteristics thatare typical of the method used.
Some image properties can be described adequately inthe spectral domain.4' 5 While most halftoning methodspermit modification of the image spectrum only indirectly,the iterative Fourier transform algorithm permits directcontrol of and access to the image spectrum during thebinarization process.6 In this way it is possible to influ-ence the characteristics of the resulting image and toadapt the image to specific requirements. The same typeof algorithm has been used to design halftone screens withspecific spectral properties.7
The various expectations regarding halftoned imagesdepend on the purpose of the binary image. For example,a visually observed image should in general have charac-teristics different from those of an image processed by anelectronic device.
If a linear system is used to process the binary image,the properties of the system can be treated adequately inthe spectral domain. An imaging system, for example,possesses the characteristic of a low-pass filter, whichmeans that only the spectral components in a low-pass re-gion DR of the spectrum pass through the system. Then anatural constraint to the binary image is6 that the spectraof the binary and the graytone images should be identicalin an appropriate low-pass region L, such that
O C L. (1)
In other words, if the difference between the two spectra,i.e., the spectrum of the quantization noise that is causedby the halftoning process, exists only at sufficiently highfrequencies, the output of the imaging system is identicalfor both the graytone and the binary image.
Such a low-pass constraint requires two assumptionsthat are not always valid. First, the spatial resolution ofthe binary image must be high enough to ensure the possi-bility of low-pass filtering with a given system. If theresolution of the processing system is higher than or of theorder of the resolution of the image, it follows that
(2)
and no low-pass region satisfying the desired constraintcan be found. Second, the assumption that the process-ing system acts as a low-pass filter is true for imaging sys-tems but not necessarily for every processing system.
If the assumptions mentioned are not valid, there is noreason to persist in controlling a low-pass region, as hasbeen proposed in Ref. 6. Instead, in this paper we proposethe adaptation of the spectral constraints to the character-istics of the original graytone image and to the character-istics expected of the formed binary image. In addition,we combine this procedure with the introduction of phase-control areas, which allows the enhancement of image de-tails, and with oversampling in the spectral domain inorder to obtain a representation that conforms to the con-tinuous spectrum of the resulting image.
2. ADAPTATION OF CONTROL AREAS TOTHE IMAGE SPECTRUM
The region in which the spectra of the binary and thegraytone image should be identical can be chosen in manydifferent ways, which depend on the characteristics ex-pected of the final image. In this paper we propose toadapt the control areas to the characteristics of the imageinstead of to the processing system. This procedure ispromising especially when the constraints resulting fromthe system are impossible to satisfy. In this way most ofthe information encoded in the graytone image can bepreserved during the halftoning process, while the mostirrelevant parts of the spectrum are discarded to the de-gree necessary for introduction of the quantization noise.
As a criterion of relevance we take the amplitude at agiven point in the spectrum of the graytone image. If themodulus at a given frequency is high, that frequency isimportant and should be reproduced well in the halftone
Fig. 1. Amplitude A of the spectrum (spatial frequency v) of thegraytone image is used to define control areas. In fl3, both theamplitude and the phase are controlled; in f12 only the phase iscontrolled; in flI, neither is controlled.
image. At points in the spectral domain where the ampli-tude is greater than a threshold tA, the spectra of the half-tone and the graytone image should be identical, whilequantization noise may be introduced where the ampli-tude is lower than tA (see Fig. 1).
It should be mentioned that, although high amplitudesare an adequate criterion for significance, it may be char-acteristic of an image that at certain frequencies of itsspectrum the amplitude is zero. If these specific frequen-cies should be absent in the halftone image, the choice ofcontrol ranges can be modified by the inclusion of frequen-cies with amplitudes below a given level.
It is well known that the phase in the spectrum of animage contains most of the information of the image,' par-ticularly the information about details and edges. Henceit should be possible for one to introduce spectral quan-tization noise in a given area in the amplitude only and toadapt the phase in this area to that of the graytone imagein order to keep the phase information. In this way thenecessity that there be a difference between the halftoneand the graytone spectra may be fulfilled in a way thattakes the image properties into account.
The control of the phase alone in an area of the spec-trum leads to an enhancement of details in the resultingimage. Thus phase control can be used to make detailsvisible that would otherwise be hidden by quantizationnoise; further, phase control can be used even to overen-hance details and edges, e.g., to facilitate their detection.
Thus, as a generalization of the criterion above, a thirdspectral range can be defined in which the phase isadapted to the phase in the spectrum of the graytoneimage. A second threshold amplitude t is defined: ifthe amplitude in the spectrum of the graytone image liesbetween tp and tA, the phase at this point is controlled(see Fig. 1).
To determine the control areas mathematically, considera discrete graytone image fa, consisting of N2 pixels, withm, n E {-N/2, . . ., N/2 - 1} and fnn E [0,1]. Let Fn bethe Fourier transform of An, with
Fnn = 9I(fn)
= Ama exp(iprn.0.
In the part Q3 of the spectrum, which contains the mostrelevant information (in terms of amplitude), the spec-trum of the halftone image should be as close as possibleto that of the original image; in Q2 the halftone phaseshould be equal to the original phase, while the halftonemodulus may differ from the original modulus; in part fI1of the spectrum, quantization noise may be introducedwithout restrictions. The situation in which no area ofphase-only control is desired is described by Eqs. (4) bysetting tA = tp.
For two reasons it is not advisable to set t to zero andthus control the phase in the whole spectrum. First, asmentioned above, large areas of phase control lead to over-enhanced details, which may result in undesirable imagecharacteristics. Second, as we have observed in ongoingresearch, control of the phase during the quantization in-evitably leads to an implicit control of the amplitude. Thisrigidness can prevent introduction of sufficient quantiza-tion noise and may cause a stagnation of the algorithm.Thus the threshold levels must be chosen in a way thatleads to the desired image properties but that also ensuresthe realization of the spectral constraints.
When the quantization is carried out with a computer,both image and spectrum are sampled and periodically re-peated distributions. When the final image is printed, itis not repeated and thus possesses a continuous spectrum.If a sampled spectrum is used during the quantization, asit is when quantization proceeds by application of a dis-crete Fourier transform to the image, the noise spectrumthat arises between the sampling points is not controlled.
The use of a sampled or a continuous spectrum alsomakes a difference for the control ranges defined inEqs. (4). The amplitudes of the spectrum between thesampling points can be higher than the chosen thresholdeven if the amplitudes at the surrounding sampling pointsare lower. This means that those parts of the spectrumthat are of interest according to the definition above re-main uncontrolled, because the spectrum is not evaluatedat these points.
Thus we propose to use an oversampled version of thespectrum during the quantization, taking at least one in-termediate point into account, to approximate the con-tinuous spectrum of the final image. For an oversampledspectrum the coupling between amplitude and phase isstronger than in the nonoversampled case.0 " This is aconsequence of the sampling theorem, which describes theconnection between the sampling points and should betaken into account to ensure that the stipulated spectralconstraints are feasible.
f _ g(N)
(3)
According to the above discussion the spectral ranges f1,f12 , and f13 are defined as
fl = {(m, n) Amn t},
f2 = {(m, n) t < Am s tA},
Q3 = {(m, n) Amn > tA}-Fig. 2. Schematic diagram of the iterative Fourier transform
(4) algorithm.
A�_/7\ ...- -...........
-
l ^ l/r [+ l l L :
Scheermesser et al.
F Qj
414 J. Opt. Soc. Am. A/Vol. 10, No. 3/March 1993
3. THE ITERATIVE HALFTONINGALGORITHM
To realize the constraints discussed in Section 2, we canuse an iterative Fourier transform algorithm (see Fig. 2).Starting from the graytone image fmon = fnn, we carry outa binarization by applying the binarization operator toobtain the binary image gm:
(k+1) = f(k)gina M
1
step(fkn - Zmn)
if fmk) 1 - A
if fmkn s A
otherwise
where Zmn is a pseudorandom number with Zmn Eis a free parameter with A E [0,1/2], and
step(x) := O if x 0
the amplitude and the phase are controlled in f13, and onlythe phase is controlled in i2. In hI, the spectrum remainsunchanged. This may be expressed by the operator 0P:
Fmn = OPGmn
Fmn
= Bmn exp(i9mn)
Gmkn
if (m, n) E 3if (m, n) E f2
if (m,n) E fli
Now FM`)n is transformed back into the spatial domain,which leads to fmn. A new cycle starts by the applicationof 91 to f mn, which results in the binary image gm.
One cycle of the binarization algorithm can be expressedin terms of an operator 9, which transforms a binary
[0,1], A image gmn into another binary image g°n
(k+l) = W9mn _mn
(6)
When oversampled spectra are used, the operator 9 ismodified to ensure that gmkn"' is zero outside the extent ofthe original image.
The choice of A is important if we are to ensure an opti-mal result of the algorithm, i.e., to avoid stagnation andminimize the deviation of the spectrum of the halftoneimage from the original one in the control areas. If A ischosen too large, the algorithm stagnates after a few itera-tion cycles. If A is chosen too small, the algorithm doesnot converge properly, and the difference between theimages of two consecutive cycles is relatively large, evenafter a considerable number of cycles. The optimal valueof A turns out to be a function of the extent of the controlareas. Let coj, withj E {1, 2, 3}, be the number of samplingpoints in the area fIj and N2 be the total number ofsampling points in the spectrum. N2 and wj are re-lated by
N 2 = (1 + C02 + &)3. (7)
We found empirically that when the spectrum isnonoversampled
° 2N2 (2/3X02 + W3) (8)2N 2
is the optimum choice for A, which means that an itera-tion with A = AO leads to a minimal least-square devia-tion in the control areas. For oversampled spectra thecoupling between amplitude and phase is much stronger,and the effect of f12 on A is nearly the same as that of fI 3.
Then the optimal A is found to be
AO0- (wO2 + (03) (9)2N 2
These results are consistent with those found for the con-trol of a single low-pass area.' 2
After the binarization has been carried out, g m(1n is trans-formed into the spectral domain, leading to
(1) = Bmn exp(i fm ) , (10)
and the spectral constraints are fulfilled by the replace-ment of G (1 by Fnn in S23 and qL by (Omn in hI2; i.e., both
= @ 919;g (12)
The iteration consists of a repeated application of a to thestart distribution gmln (see Fig. 2). After N cycles,
(N+1) = g3)9mnl mng~ (13)
is obtained.At best, the differences introduced in each cycle get
smaller until no more changes in the image occur and itsspectrum fulfills the required constraints. However, usu-ally no binary image exists that has exactly the requiredspectral properties; in such a case the iteration is termi-nated after a number of cycles, and the approximate solu-tion is accepted.
4. EXPERIMENTAL RESULTS
Binary images were produced by using the algorithm de-scribed in Section 3, to illustrate the characteristics of theresulting distributions. A graytone image of 2562 pixelswas halftoned by the application of 80 cycles of the algo-rithm. Two quantizations were performed with tA = tp,
which means that an area of phase-only control did notexist. The resulting images are shown in Figs. 3(a) and3(b). The parameters for the two quantizations wereidentical. In Fig. 3(a) a nonoversampled spectrum wasused, while in Fig. 3(b) the quantization was performedwith a two-times oversampled spectrum.
In these examples the threshold was set to 0.08% of thedc peak in the spectrum of the graytone image. In thefollowing the threshold values are given in percent ofthe dc peak, so that the parameters remain independentof a specific normalization. Choosing tA = tp = 0.08%results in an extent of f1 3 of approximately 21% of thespectrum.
The resulting control areas are shown in Figs. 4(a) and4(b). The white area belongs to f2 3 and the black area toflj. The image spectrum has comparatively high valueson the vertical axis, which would be neglected if a low-passregion were controlled. The influence of oversampling onthe controlled regions is clearly visible. For example, in asmall region near the horizontal axis all intermediatesampling points are controlled. At high frequencies inthis region the original sampling points, which wouldbe obtained by Nyquist sampling, are not controlled.
Scheermesser et al.
Vol. 10, No. 3/March 1993/J. Opt. Soc. Am. A 415
(a) (C)
(b) (d)Fig. 3. Halftoned images that were obtained with the use of the control areas shown in Fig. 4. (a) No oversampling; parameters:tA = tp = 0.08%, 2 = 0%, 3- 21%. (b) Oversampling; parameters: same as in (a). (c) No oversampling; parameters:tA = 0.25%, tp = 0.08%, f12 16%, 3- 5%. (d) Oversampling; parameters: same as in (c).
Through the coupling between the original and the inter-mediate points given by the sampling theorem, the spec-tral quantization noise is reduced at the original samplingpoints, as well. On the other hand, without oversampling,the horizontal axis is mostly uncontrolled above a certainfrequency.
Both images possess the essential characteristics of theoriginal image. Hence the proposed procedure leads tocontrol areas, adapted to the graytone image, that contain
the most relevant spectral information. Nevertheless, thedifference between Figs. 3(a) and 3(b) is clearly visible.The image in Fig. 3(b) resulting from the control of anoversampled spectrum is less noisy than the one inFig. 3(a). The consideration of intermediate points hasled to a better adaptation of the control areas to the imageproperties.
As a quantitative measure for the deviation of the spec-trum of the binary image from the original spectrum,
Scheermesser et al.
416 J. Opt. Soc. Am. A/Vol. 10, No. 3/March 1993
we consider
=2 IG.. - F1 2, (14)
where fkj is the area of interest, and
Gmn = Bmn exp(ii#.n) (15)
denotes the spectrum of the binary image. Analogously,
A(P = > 1iymn - cP°nj (16)(m n) E j
is a measure of the phase deviation. These quantities areshown in Table 1 for the images in Fig. 3. To compare the
(a)
results, we evaluated the oversampled spectra only at theoriginal sampling points. The values of o-2 and Aq in f13are slightly higher when oversampled spectra are used.The control of the values at intermediate sampling pointsintroduces additional restrictions on the noise, because theintermediate and the original points are coupled by thesampling theorem. Therefore o-2 and Ap are not reducedso much as in the nonoversampled case. Nevertheless, theimages produced with oversampling seem to be of highervisual quality than the others, because the noise is adaptedbetter to the image characteristics.
The results of binarizations with a nonzero range ofphase control are shown in Figs. 3(c) and 3(d), with tA
raised to 0.25%, resulting in a range f13 that covers ap-
(c)
(b) (d)
Fig. 4. (a)-(d) Control areas for the corresponding images in Fig. 3. Black indicates control range Q. 'I gray indicates f12, and whiteindicates (13.
Scheermesser et al.
Vol. 10, No. 3/March 1993/J. Opt. Soc. Am. A 417
Table 1. Values of u2 and ASp in the ControlRanges 12 and Q3 for the Images Shown in
proximately 5% of the spectrum. tp remained at 0.08%,leading to a range (12 of approximately 16% extent, whileboth ranges together cover the same part of the spectrumas in the previous examples. The corresponding controlareas are shown in Figs. 4(c) and 4(d). The white areas be-long to (13, the gray areas to (12, and the black areas to (1l .
The most obvious difference between the image inFig. 3(a) and the image in 3(b) lies in the edge enhance-ment caused by the phase-control area. The effect ismost pronounced in the areas of the teeth, hair, and eyes(eyelashes). Consequently the images look sharper thanthe corresponding images without phase control. This ef-fect could be enhanced even more by enlargement of therange of phase control.
However, in addition, the local arrangement of the pixelsis changed. The fact that the amplitude in (12 is no longercontrolled makes it possible to introduce quantizationnoise in this area. Therefore more low-frequency noise ispresent, and the texture of the images is more grainy.
The coupling that exists between amplitude and phaseprevents an unlimited introduction of noise in the area ofphase control. Thus during the quantization process thenoise is reduced in (12, as well. This coupling is muchstronger in an oversampled spectrum, and less noise is in-troduced here. Consequently the texture of the image inFig. 3(d) is finer than that in Fig. 3(c).
The difference between Figs. 3(c) and 3(d) is of thesame kind as that between Figs. 3(a) and 3(b). The imagethat is binarized with intermediate sampling points in thespectrum considered is less noisy than the other image.Again the control areas are adapted better to the imagecharacteristics as a consequence of the oversampling.
Considering the values of a-2 and Aqp that are shown inTable 1, the same tendencies occur in the area (1 inFigs. 3(c) and 3(d) as for the images in Figs. 3(a) and 3(b).In (12, where the phase alone is controlled, a-2 is lowerwhen oversampled spectra are used. This is a result ofthe strong coupling between amplitude and phase that ispresent in this case. The control of the phase in (12 leadsto an implicit control of the amplitude in this area. Sincethe coupling between amplitude and phase is muchstronger with oversampling, a-2 in (12 is reduced below thatin the nonoversampled case, although the phase deviationis larger.
5. CONCLUSIONS
We have proposed a method in which the quantizationnoise spectrum that evolves in a halftoning process isadapted to the characteristics of the graytone image.The amplitude of the spectrum is used as a criterion forthe relevance of spectral components. This procedureleads to images in which the essential characteristics ofthe original image can be found. In addition, introduc-tion of an area of phase control can enhance details in theresulting image.
That the final, nonrepeated image has a continuousspectrum may be taken into consideration by controllingvalues between the sampling points in the spectral domain.If the values between the sampling points are taken intoaccount, the result is a better adaptation of the spectralconstraints to the image properties. In this respect it isadvantageous to use oversampled spectrum during thequantization process.
ACKNOWLEDGMENTS
We thank Michael Tluk and Ingo Kummutat for their helpwith the figures. This project was supported by theDeutsche Forschungsgemeinschaft.
REFERENCES
1. B. E. Bayer, 'An optimum method for two-level rendition ofcontinuous-tone pictures," in Proceedings of the IEEE Inter-national Conference on Communication (Institute of Electri-cal and Electronics Engineers, New York, 1973).
2. R. W Floyd and L. Steinberg, 'An adaptive algorithm for spa-tial greyscale," Proc. Soc. Inf. Disp. 17/2, 75-77 (1976); Di-gest of Technical Papers (Society for Information Display,Playa Del Ray, Calif., 1975), pp. 36-37.
3. R. Ulichney, Digital Halftoning (MIT, Cambridge, Mass.,1987).
4. D. Kermisch and P. G. Roetling, "Fourier spectrum of half-tone images," J. Opt. Soc. Am. 65, 716-723 (1975).
5. R. Ulichney, "Dithering with blue noise," Proc. IEEE 76,56-79 (1988).
6. M. Broja, F Wyrowski, and 0. Bryngdahl, "Digital halftoningby iterative procedure," Opt. Commun. 69, 205-210 (1989).
7. R. J. Rolleston and S. J. Cohen, "Halftoning with random cor-related noise," J. Elect. Imag. 1, 209-217 (1992).
8. A. V. Oppenheim and J. S. Lim, "The importance of phase insignals," Proc. IEEE 69, 529-541 (1981).
9. M. Broja, F Wyrowski, and 0. Bryngdahl, "Significance ofspectral phase in synthesis of binarized images," Opt. Com-mun. (to be published).
10. A. V. Oppenheim, M. H. Hayes, and J. S. Lim, "Iterative pro-cedures for signal reconstruction from Fourier transformphase," Opt. Eng. 21, 122-127 (1982).
11. V T. Tom, T. F Quatieri, M. H. Hayes, and J. H. McClellan,"Convergence of iterative nonexpansive signal reconstructionalgorithms," Lincoln Lab. Tech. Note 1980-21 (MIT LincolnLaboratory, Lexington, Mass., 1980).
12. R. Mrusek, M. Broja, and 0. Bryngdahl, "Halftoning by car-rier and spectrum control," Opt. Commun. 75, 375-380(1990).
