Dynamic range compression of images bytwo-wave mixing in photorefractive materials
Michael Snowbell and Baruch Fischer
We propose and demonstrate a method for the dynamic-range compression of images using two-wave
mixing in photorefractive materials. This method, in which an image is carried on the signal beam,
permits a certain amount of control over the shape of the input and output curves by the variation of the
pump intensity, the coupling constant, and the time at which the output image is examined. We also
discuss the possible option of adding a feedback loop from the output image so that the range compression
can be adapted to suit the intensity profile in the input image.Key words: Dynamic image processing, nonlinear photorefractive wave mixing.
Introduction
When photographing images, one often comes acrossthe situation in which various parts of the picture areat intensities that vary by a few orders of magnitude.Upon photographing such images, one sees that thebright parts of the picture saturate the picture andthe darker parts do not receive enough exposure.Thus for example, an input image with an especiallybright background will flood the photographic imageso that details of a particular item in the foregroundcannot be seen. The human eye and the eyes ofother advanced organisms provide a partial solutionto this problem by a mechanism known as darknessadaptation." 2 When the eye is subject to a particularbackground intensity, the response curve (the amountof response as a function of input intensity) of thecones is shifted accordingly so that the sensitivity isoptimal over a particular range near each backgroundlevel; for intensities over this range the eye reachessaturation. This shift in response curves is shown inFig. 1. The adaptation of the cones and of otherelements in the human visual system (rods andbipolar and ganglion cells) is a complex process thatdepends on a number of factors in the image enteringthe eye and is not yet totally understood. Yet theaspect of tailoring the response curve so that theoptimal response is over a certain range, above whichsaturation occurs, is central.
The authors are with the Department of Electrical Engineering,Advanced Opto-Electronics Research Center, Technion-Israel In-stitute of Technology, Haifa 32000, Israel.
A basic device utilizing the fanning effect in photore-fractive crystals for image range compression hasbeen demonstrated.3 This device utilizes the variedtime dependence of the fanning buildup on the inten-sity of the pump beam. A monochromatic picture isimaged into a photorefractive crystal, and this beamexperiences a loss of intensity because of fanning.Since the time constant involved in two-wave mixingdecreases with intensity [with an experimental depen-dence of roughly r C I-0.7 (Refs. 4-6)], different partsof the picture with varying intensities experiencedifference time constants. Thus the parts of thepicture that have higher intensities develop theirfanning at a faster rate than do the areas in thepicture with the lower intensity. Thus if we con-sider the picture at the output of the crystal in time ata time before steady state, the bright parts of thepicture will lose a higher percentage of their intensitythan do the darker parts. This pre-steady-state im-age will thus experience a reduction of the contrastbetween the brighter and the darker parts of thepicture, i.e., a dynamic-range compression. The ex-tent of this compression depends on the specific timeat which the image is photographed, and at steadystate, compression does not take place as the lower-intensity fanning catches up with the higher-inten-sity fanning.
This solution is a partial one at best. Since theI-0.7 dependence of the time constant is not a verystrong one for higher intensities, the compressionwill not be strong unless the image is first attenuateduniformly. Another problem is that this effect in-volves the weakening of the signal in order to observethe compression (at the expense of the fanning),which may be an undesirable quality. Additionally,
Fig. 1. Adaptation of response curves of cones at various back-ground intensities (after Ref. 1).
Method
For a basic understanding of how this method workswe consider the two-wave-mixing dynamic equations.For the basic configuration of Fig. 2, in which the Aiare the electric-field amplitudes of the beams and Oi isthe angle between beam i and the optic (c) axis, wehave 4'8
Cos 01 aA(z, t) -aA1(z, t) - g*(z, tA 2(z, t),
COS 02 az ) = -A 2 (Z, t) + g(z, t)Al(z, t), (1)
where a is the attenuation of the material and g(z, t)is proportional to the amplitude of the index-of-refraction grating written in the photorefractive crys-tal. The time-dependent differential equation forg(z, t) is given by
the compression does not remain at steady state andis time dependent. This may not be a critical limita-tion, but it requires that the image be photographedat a particular time. Additionally, the degree ofcontrol over this process is limited: a change oforientation of the beam with respect to the crystalaxis does result in a change in the coupling constant,which changes the dynamics of the development ofthe fanning, but otherwise the ability to create differ-ent response curves tailored to specific needs ofdynamic-range compression and contrast reduction islimited.
The double phase-conjugate mirror (DPCM), whosereflectivity is dependent on the ratio of the pumpbeams, has been suggested and demonstrated as apossible tool for thresholding.7 Since the DPCMoperates only for a certain range of pump ratios, thenif we fix one pump, the DPCM will create a phase-conjugate reflection for the second pump if its inten-sity is within a certain range of intensities permittedby the DPCM. If the second pump beam intensity iseither more or less than this range, then no phaseconjugate at all will be generated. This is then adifferent type of processing based on the distributionof intensity in an image, but it is not the desireddynamic-range compression since higher intensitiesare eliminated rather than just saturated. Also, thedynamic range of the DPCM is narrow, so it wouldnot be able to process images with too wide a differ-ence in its intensity profile without completely elimi-nating some of the intensities in the picture.
We propose and demonstrate a new all-opticalmethod for dynamic-range compression using two-wave mixing in photorefractive crystals with theimage being carried by the signal. This methodpermits a much greater control over the form ofresponse curves by varying parameters of the two-wave mixing, and it is shown to effectively reducecontrast in simple and complex images.
dg(z, t) A1(z, t)A2*(z, t)' , + g(Z t)= - y (2)
where T is the time response of the crystal, Io is thetotal intensity (Io = A1 12 + A 212) in the interactionregion of the beams, and y is the photorefractivecoupling coefficient, which depends on the orientationof the beams, their polarization, and other materialparameters.
The general dynamic solution has been studiednumerically4 (and in the undepleted pumps approxi-mation, analytically8). The steady-state solution fortwo-wave mixing is readily derived as
where = 2 Re(y) and Ii = IAi 2 represents theintensities of the beams. For a positive y, beam 2(the signal) experiences a gain at the expense of beam1 (the pump beam). For a given pump input inten-
PR
A
A 2 -
z
Fig. 2. Two-wave-mixing configuration in photorefractive (PR)materials.
sity, I1(O) = Ip, and a given gain constant, i, the gainat the crystal output (z = ) for a signal that entersthe crystal with incident intensity of I, is given (in thezero-loss case) as
I _ 12(l) _ I2(1) Is + IpG = I2(0) I I + Ip exp(-Fl)
(4) 0
For very small signal intensities [I, << Ip exp(-Fl)]the gain becomes G = exp(R), whereas for high signalintensity (I, >> Ip) it reduces to unity. Physicallythe small-signal case corresponds to the undepletedpump case, whereas the large-signal case correspondsto the saturation of the pump beam. Between thesetwo extremes the gain decreases monotonically.
This gain characteristic is precisely what is desiredfor dynamic-range compression. Low intensities re-ceive a large gain, whereas high intensities do notreceive any gain at all; thus the intensity contrastbetween different parts of an image is reduced. Inaddition, the signal receives an overall gain so thatthe contrast reduction does not involve a weakeningof the signal.
This method permits a great deal of control overthe form of the response curve (the output-signalintensity, Iout, versus the input intensity of the signal,Is). Figure 3 illustrates some possible response curvesresulting from changing the pump intensity. We seethat increasing the pump intensity causes saturationfor a higher-intensity signal, although for low-intensity signals the gain remains the same. This issimilar to the shifting of the response curve in the eyewhereby the response curve can saturate at variousintensities while the shape of the curve remainsessentially the same.
A different type of response-curve shaping can alsobe achieved by changing the coupling constant.Figure 4 shows various response curves resultingfrom changing XI. It can be seen from this figurethat changing -yl changes the essential shape of thecurve: increasing the coupling constant increasesthe gain for low intensities, resulting in a highercontrast reduction. Additionally, the transition to
0.9
0.8
0.7 ------------
0.6- X.-- p=0.6
0~~~~~~~~~~~~~~.4 ---04.... ----------- --0.6
0.5 0.1 -~~~~~~~~~~~~~~~~~~I i
Fig. 3. Stcady-stato output-signal intensity versus input-signalintensity for various values of Ip(yl = 4).
.1
I in
Fig. 4. Same as Fig. 4 but for various values of yl(Ip = 1).
saturation occurs over a much narrower range ofintensities when yl is higher.
,yl may be adjusted by changing the orientation ofthe beams and of the crystal. By examining Eq. (2),one can see that another method of effectively lower-ing the coupling coefficient is possible. By additionof an incoherent background beam of intensity Ib,
which makes total intensity Io = I, + I2 + Ib, thegrating writing mechanism is weakened. Effec-tively, this is equivalent to reducing y directly. Thusone may position the crystal to permit maximum -yand then vary y effectively by adding a backgroundbeam. This has the advantage of eliminating theneed to reorient the crystal to change y.
These curves are based on the steady-state inten-sity expressions. However, the transition to steadystate is a smooth and monotonic one, and examiningthe picture before steady state is fully achieved stillgives the same approximate behavior as that in steadystate. However, if we examine the picture at anearly stage of the two-wave-mixing process, the dy-namic-range compression will not be as strong as inthe steady state. This can be understood in light ofthe dependence of the signal buildup time on thesignal-to-pump ratio (SPR). Since low SPR signalstake longer to buildup to their steady-state valuesthan the high SPR signals,4 the stronger elementsexperience their gain faster, and only later do theweaker elements experience enough gain to approachthe steady-state compression. A set of responsecurves calculated for various times before steadystate with the method of Ref. 4 is shown in Fig. 5 foryI = 4 andIp = 1.
Experimental Results
In order to check the basic operation of such a device,we set up the simple configuration shown in Fig. 6.The lens (lens 1) situated between the slide and thecrystal is used to accomplish two goals. First, itdemagnifies the image so that the original image of
1 cm can fit in the 0.2-cm crystal and also so thatthe crystal is in the image plane of the side so that thedynamic-range compression in the crystal takes placeon the image and not on a transformation of that
Fig. 5. Output-signalintensityversusinput-signal intensity. Thecurves correspond to output at different times in units of r. = 4andI, = 1.
image. Since the full two-wave-mixing process oc-curs over the length of the entire crystal (typically 0.7cm), it is important to ensure that the depth of theimage is larger than or at least of the order of thelength of the crystal; i.e., moving 0.35 cm in eitherdirection from the image plane would more or lessleave the image intact. Therefore we used smallinput images and a relatively large-focal-length lens(f = 135 mm with distances of 80 cm between theslide carrying the image and the lens and 16.24 cmbetween the lens and the crystal, permitting a demag-nification of 1/5). An additional lens is used afterthe crystal to magnify the output image. Fromsimple measurements of steady-state gain (factoringout losses) at low signal intensities [I/ Ii- exp(2yl)],the yl in the system used was approximately 3.1, andthe pump beam was made to be wider in the crystalthan the signal to ensure that the entire signal beamprofile was overlapped in the crystal. Thus signal-to-pump ratios given here are factored to take intoaccount the different areas of the signal and the pumpbeams.
To check the basic operation of the system, weentered a simple image consisting of two half-circlesat different intensities. In the first case we entered a
picture whose contrast ratio (bright-side intensity todark-side intensity) was 62:1. Using a signal-to-pump ratio of 0.5 for the bright side of the imageand 9 x 10-3 for the dark side (or an average SPR of0.254), we reduced the contrast to 2.18:1 as thesystem approached steady state, giving a reduction incontrast by a factor of 28.4. This corresponds fairlywell with the steady-state calculations, which indi-cate a contrast reduction by a factor of 32.3 for thesevalues. A photograph of the image before and afterthe compression is shown in Fig. 7. By changing theintensity of the pump beam, reducing yl (with back-ground beam insertion), and observing at pre-steady-state times, we also achieved different contrast reduc-tions in the range from 1 (no change) to 28.4 (largecontrast reduction).
To give an idea of the usefulness of this type ofsystem, we imaged the picture into a CCD camerabefore and after the dynamic-range compression.Figure 8 shows the line profiles of intensities acrossthe two-semicircle image. In Fig. 8(a) (before thecompression) the CCD line profile shows that theCCD camera reached saturation for the bright half ofthe image, yet the darker part of the image was barelydiscernible from background noise in the CCD image.Figure 8(b) shows the line-profile measurement of theCCD camera after the compression. The bright partof the image was mostly taken out of the saturation ofthe CCD camera, but now the darker part has asignificant intensity level on the CCD. Thus whatwas mostly distinguishable from darkness for theCCD becomes quite visible after the dynamic-rangecompression first adjusts the relative intensities inthe image. This line profile also indicates that thetransition between the two halves of the image was
beam
(a)
lens IPR
outputimage
Fig. 6. Experimental configuration of dynamic-range compres-sion.
(b)
Fig. 7. Dynamic-range compression of image 1:before and (b) after compression.
circles image (a) before and (b) after compression.profile of se
enhanced by this processing. Apparently this is dueto the aforementioned image depth problem. Sincethe crystal was not far from the focus of the beam, theimage was present in one plane in the crystal but notthroughout the entire crystal; therefore processingdid not occur only in the image plane. In this way,other filtering effects also took place. To improvethis configuration, one should use larger-focal-lengthlenses and/or smaller original images (and hence lessdemagnification, which is necessary) to increase thedepth of the focal plane to overlap the entire crystal.
A second image was entered into the crystal, thisone a circle divided into quarters, each of a differentintensity. With an signal-to-pump ratio averagedover the entire signal of 0.1 the contrast was reducedfrom 1650:148:71:1 to 18.4:10.0:4.2:1.0. A photo-graph of the image before and after compression isshown in Fig. 9. Thus the order of the intensitymagnitude is still preserved, permitting intensityvariations to remain in the image but with a strongcompression. Once again, varying the intensity ofthe pump permitted different values of compressionin the image.
(b)
Fig. 9. Dynamic-range compression of image 2 (quarter cirices) (a)before and (b) after compression.
In order to test the usefulness of this method for areal image, we used a dark picture of a gorilla on abright background as an input picture. This isshown in Fig. 10(a), in which the bright backgroundsaturated the photographic image, and the face of the
120 gorilla is not discernible. Figure 10(b) shows thephotographic image after the compression has takenplace, indicating that this method can be effective for
mi- images carrying more complex information on it.
(a)
(b)
Fig. 10. Dynamic-range compression of image 3 (picture of agorilla head) (a) before and (b) after compression.
Filtering in the Fourier Plane with Two-Wave Mixing
This dynamic-range compression method discusseduntil now has dealt with the compression taking placein the image plane. Shifting the lens so that thecrystal lies in the Fourier plane permits this dynamic-range compression to act on spatial frequency compo-nents. This method causes the frequency compo-nents with the strongest components to receive less ofa gain than spatial Fourier components that wereweak to begin with. For most simple images the dccomponents are strong as compared with higher-frequency components. These higher-frequencycomponents, which are responsible for sharp changesthat occur in the image, thus receive the highest gain,resulting in edge enhancement. The reverse occursif the high-frequency components are stronger thanthe lower spatial frequency components, which wouldresult in an edge de-enhancement. This type of edgeenhancement has been proposed and demonstrated9 -'2as well as edge enhancement using the DPCM.7
Dynamic-Range Expansion
Considering the fact that the expression for thesteady-state intensity of a signal beam provided thepossibility of dynamic-range compression using animage carried on a signal beam, it seems possible thatthe opposite function, dynamic-range expansion of animage, might be possible with the pump beam as theimage carrier. Indeed the shape of the steady-stateresponse curve with the pump as the image carrier isexponential in shape, thus providing in theory amechanism for dynamic-range expansion. However,as opposed to the time behavior of the signal beam,which reaches its steady-state value smoothly andremains close to its steady-state behavior within a fewpercentage points, the pump behavior oscillatesstrongly before it settles on its steady-state value.The time-dependent behavior of the pump is illus-trated in Fig. 11 for various yl values. The intensityof the pump is plotted on a logarithmic scale so thatchanges in orders of magnitude close to zero can beseen more easily. Thus for example, the pump in atwo-wave-mixing process with yl = 3 undergoes an
100
10-1
' 10-2-
.- 10-3-
0.= 10
0.
10-6L
- 2 0 2 4 6 8 10 12 14 16time
Fig. 11. Time development of the pump for various '9 values (logscale versus time scale) in units of T.
undershoot to less than 10-6 before settling on itssteady-state value of slightly more than 10-3. Thusan undershoot of three orders of magnitude occurs inthe time development of the pump. The nature andthe number of these undershoots are dependent onthe coupling constant. Thus on its way to steadystate a dynamic-range expansion system would un-dergo strong oscillations that would vary with theintensity distribution of the image carried by thepump.
Theoretically, permitting the interaction to pro-ceed for a long enough time would let all componentsof the pump reach steady state, giving the desiredcompression. However, in the laboratory the systemwas not seen to reach complete steady state, and animage carried by the pump beam was seen to havemany different instable intensity regions. It seemedthat small disturbances or small vibrations wereenough to set the dynamics of the interaction farenough to not reach steady state. However, if onewere to affect the signal beam at the same time, theseoscillations would not be apparent. Perhaps a moresteady setup would permit the pump beam to pass theoscillation stage fully without getting set back beingaffected.
Discussion
We have demonstrated a dynamic-range compressiondevice using two-wave mixing in photorefractive me-dia. The controllability of the form of the responsecurve is similar in certain ways to the way theresponse curves of various cells in the eyes of ad-vanced organisms can be adapted. In humans themechanism responds to certain features in the imagebeing perceived, but another important factor inliving organisms is the mechanism of biofeedback.Thus for example, a person who is interested in thefeatures of a particular object in his field of view willcontinually use the information received about thepicture to further determine how to shift more pre-cisely the response curves so that the features of theobject can be best perceived. This leads one toconsider the possibility of using a servo loop in thedynamic-range compression application to serve as afeedback mechanism by which a finer tuning of theparameters can constantly update the response curve,thus making the performance self-adjusted. Thisservo loop could consist of, for example, a detector atthe output of the device that sends signals to adjustthe pump intensity or the intensity of a backgroundbeam. Of course one would have to come up withthe criteria by which any type of feedback decisionswould be made. This could vary by application.As an example, one could always place the object thatone is interested in observing at the center of theimage (as the human eye does). A detector placed inthe middle of the output image could then update thecontrol mechanism so that an ideal level of theintensity in the center could be established. In othercases, a measurement of the background intensitiescould be used as feedback for setting the various
parameters so that the background intensity may besaturated.
The nature of the resolution of the dynamic-rangecompression should also be investigated. For cer-tain types of input images one may not want thecontrast reduction to be overly localized. In such acase one may want the contrast reduction to takeplace only on the average intensity of larger regions inthe picture. In this way the localized detail revealedby differences in intensities would be preserved. Forsuch an application a low-resolution system is desired.The other resolution issue that must be addressed isthe inherent limit in the pixelization ability of thephotorefractive material. The photorefractive effectis a nonlocal one and depends on the diffusion anddrift of charge carriers to establish a grating pattern.Thus one cannot assume that dynamic-range compres-sion will occur down to scale zero (or even to thediffraction limit) if one desires to use such a resolution.A localized intensity has meaning only if this inten-sity is able to redistribute the charge to form agrating.
Thus one must take into account both the desiredresolution and the pixelization limit of the photorefrac-tive material. In a well-designed system the size ofthe image entering the crystal would be set so that theresolution limit of the crystal gives the desired maxi-mum resolution for the dynamic-range compression.
Conclusion
In this paper we have proposed and demonstrated amethod for controllable dynamic-range compressionof images using two-wave mixing in photorefractivematerials. A large amount of control over the shapeof the response (input and output) curves may takeplace by varying certain parameters of the two-wavemixing. Operation in the Fourier plane is discussedas is the difficulty of constructing a dynamic-rangeexpansion device using the same principle. Thepossible use of a feedback loop to update the param-eters is discussed.
We thank the American Technion Society, NewYork Metro region, and especially Norman Seiden,Mel Dubin, and Steve Shapiro for most vital supportof our research.
References1. M. D. Levine, Vision in Man and Machine, McGraw-Hill, New
York, 1985), pp. 100-150.2. F. S. Werbelin, "The control of sensitivity in the retina," Sci.
Am. 22(1), 71-77 (1973).3. H. K. Liu, "Photorefractive crystal compresses dynamic range
of images," NASA Tech. Briefs (October 1991).
4. M. Horowitz, D. Kligler, and B. Fischer, "Time-dependent
behavior of phototorefractive two- and four-wave mixing," J.Opt. Soc. Am. B 8, 2204-2217 (1991); M. Horowitz, R. Daisy,
and B. Fischer, "Signal-to-pump ratio dependence of buildupand decay rates in photorefractive nonlinear two-beam cou-pling," J. Opt. Soc. Am. B 9, 1685-1688 (1991).
5. D. Mahgerefteh and J. Feinberg, "Erasure rate and coating inphotorefractive barium titanate at high optical power," Opt.Lett. 13, 1111-1113 (1988).
6. S. Ducharme and J. Feinberg, "Spread of the photorefractiveeffect in a BaTiO3 single crystal," J. Appl. Phys. 56, 838-842(1984).
7. S. Sternklar, S. Weiss, and B. Fischer, "Optical information
processing with the double phase conjugate mirror," Opt. Eng.26, 423-427 (1987).
8. M. Cronin-Golomb, "Analytic solution for photorefractive two
beam coupling with time varying signal," in PhotorefractiveMaterials, Effects, and Devices, Vol. 17 of 1987 OSA TechnicalDigest Series (Optical Society of America, Washington, D.C.,
1987), pp. 142-144.9. Y. Fainman, E. Klancnik, and S. H. Lee, "Optimal coherent
image amplification by two-wave coupling in photorefractive
BaTiO3," Opt. Eng. 25, 228-234 (1986).10. J. P. Huignard and J. P. Herriau, "Real-time coherent object
edge construction with Bi12 SiO20 cyrstals," Appl. Opt. 17,
2671-2672 (1978).11. J. Feinberg, "Real-time edge enhancement using the photore-
fractive effect," Opt. Lett. 5, 330-332 (1980).12. E. Ochoa, J. W. Goodman, and L. Hesselink, "Real-time
enhancement of defects in a periodic mask using photorefrac-tive Bi12SiO20,` Opt. Lett. 10,430-432 (1985).
