Reconstruction of smeared and out-of-focus images by regularizationV. S. Sizikov and I. A. Belov
St. Petersburg State Institute of Precision Mechanics and Optics (Technical University), St. Petersburg,Russia~Submitted September 27, 1999!Opticheski� Zhurnal67, 60–63~April 2000!
One of the inverse problems of optics is the reconstrtion of distorted images.1 The term image can refer to aphotograph of a person, text, or an object in nature~includ-ing a photograph taken from space!, a television or motion-picture image, a telescopic photograph, an optoelectronicproduction of a celestial body, etc. However, to fix ideas,shall henceforth use the term image to refer to a photograWe assume that preliminarily treatment of the imagebeen carried out and that, specifically, the scratches onphotograph have been removed, its contrast has beenjusted, and other operations~not requiring mathematicatreatment! have been performed. We shall dwell on the mdifficult problem, i.e., the mathematical reconstruction of iages distorted as a result of smearing~displacement, movement! or defocusing.
RECONSTRUCTION OF SMEARED IMAGES
Let us examine this problem in the case of a smea~displaced, moved! photograph. Let the photographed obje~which is assumed to flat because of the large distance t!and the photographic film in the camera be oriented parato the camera lens aperture at the distancesf 1 and f 2 , re-spectively, from the lens. In this case 1/f 111/f 251/f andf 1> f 2 , where f is the focal distance of the lens~Fig. 1!.
We assume that the film underwent a linear and unifodisplacement~movement! by D or that the object~for ex-ample, a fast-moving target! underwent a displacement b2D f 1 / f 2 during the exposure time. As a result, the imagethe photographic film is smeared. The problem is descrimathematically by the relation2–4
1
D Ex
x1D
w~j,y!dj5g~x,y!, ~1!
whereg(x,y) is the intensity on the film~the smeared image!as a function of the rectangular coordinatesx and y, the xaxis being directed along the displacement~smear!, andw(j,y) is the true image~the image which would have beerecorded on the film in the absence of the shift, i.e., ifD50!.
The relation~1! is a one-dimensional integral equatiorelative tox(j,y) at each fixed value ofy, which plays the
351 J. Opt. Technol. 67 (4), April 2000 1070-9762/2000/0403
-
e-eh.shead-
t-
dtitel
nd
role of a parameter. It can be written in the form ofFredholm-type integral equation of the first kind in convoltion form3,4
E2`
`
k~x2j!w~j,y!dj5g~x,y!, 2`,x,y,`, ~2!
where
k~x!5H 1/D for xP@2D,0#,
0 for x¹@2D,0#.~3!
If the x axis is antiparallel to the displacement, then
k~x!5H 1/D for xP@0,D#,
0 for x¹@0,D#.~4!
We note that this formulation of the problem canextended to the case of nonparallel object and film planeswell as the case of nonuniform and/or nonlinear displament of the film~or object!, as was done, for example, iRef. 5.
Finding the solution of Eq.~2! is an ill-posed problem.Fourier transformation and Tikhonov regularization2,6,7 areused to solve it in this paper. The regularized solution hasform
wa~j,y!51
2p E2`
`
Wa~v,y!e2 ivjdv. ~5!
Here the regularized Fourier-transform spectrum of the sotion is
anda.0 is the regularization parameter.There are several methods, for example, the discrepa
method, for choosinga.2,6,7 However, as tests have showthe trial-and-error method is most effective for imareconstruction.3,4 According to this method,wa(j,y) is cal-culated for a series of values ofa using formula~5! @alongwith ~6! and ~7!#, the solution wa(j,y) is displayed ingraphical form, and the value ofa which gives the best reconstructed image from the standpoint of physiologi~rather than mathematical! perception criteria is chosen. Thmethod is similar to tuning the contrast of a television ima~in that casea is inversely proportional to the contrast!.
We note that the valueD is usually not known in prac-tice, and it should also be determined by trial-and-error.for the smear direction~along which thex must be oriented!,it is determined from marks on the photograph~see Fig. 3bbelow!.
Thus, after correctly selecting the direction of thex axis~along the smear! and the value ofD, solving Eq.~2! ~moreprecisely, set of equations!, using formulas~5!–~7!, and se-lecting a by trial and error, we can reconstruct the undtorted photographw(j,y) from the distorted~smeared! pho-tographg(x,y).
RECONSTRUCTION OF OUT-OF-FOCUS IMAGES
Let the object photographed~which is assumed to beflat! and the photographic film be oriented parallel to the leat the distancesf 1 and f 21d from it, respectively, whered isthe image focusing error~Fig. 2!. In this case the followingrelation holds:2,8
FIG. 2. Reconstruction of an out-of-focus image.
352 J. Opt. Technol. 67 (4), April 2000
cy
l
e
s
-
s
E EA~x2j!21~y2h!2<p
w~j,h!
pr2 djdh5g~x,y!. ~8!
Hereg(x,y) is the intensity on the out-of-focus photograpw(j,y) is the intensity sought on the undistorted photogra~on the photograph which would have been obtained id50!, r5ad/ f 2 is the radius of the diffraction circles on thfilm ~circles into which each pointA8 or A9 is transformed;see Fig. 2!, anda is the lens aperture radius.
The relation~8! can be brought into a standard form, omore specifically, transformed into a two-dimensionFredholm-type equation of the first type in convolutioform:2,8
E2`
` E2`
`
k~x2j,y2h!w~j,h!djdh5g~x,y!,
2`,x,y,`, ~9!
where
k~x,y!5H 1/pr2 for Ax21y2<r,
0 for Ax21y2.r.~10!
In Eq. ~9! g(x,y) is the measured right-hand side,w(j,y) isthe function sought, and the kernelk(x,y) is called the pointscattering function.
We note that the problem of defocusing was also csidered in Ref. 5 for the case of nonparallelism betweenplane of the object and the plane of the film.
Finding the solution of Eq.~9!, as in the case of Eq.~2!,is an ill-posed problem. The solution obtained by Tikhonregularization and two-dimensional Fourier transformathas the form2,6,7
wa~j,h!51
4p2 E2`
` E2`
Wa~v1 ,v2!
3e2 i ~v1j1v2h!dv1dv2 . ~11!
Here
Wa~v1 ,v2!5K~2v1 ,2v2!G~v1 ,v2!
L~v1 ,v2!1aM ~v1 ,v2!, ~12!
where
L~v1 ,v2!5uK~v1 ,v2!u25K~v1 ,v2!K~2v1 ,2v2!,
K~v1 ,v2!5E2`
` E2`
`
k~x,y!ei ~v1x1v2y!dxdy,
G~v1 ,v2!5E2`
` E2`
`
g~x,y!ei ~v1x1v2y!dxdy,
M ~v1 ,v2!5~v121v2
2!2, ~13!
anda.0 is the regularization parameter, which is chosenin the reconstruction of smeared images, by trial and erWe note that the value ofd ~or r! is not knowna priori inpractice and that it is also usually determined by trial aerror.2,8
352V. S. Sizikov and I. A. Belov
teh
giza
ogef
dn
isingm
acerd
eac-ts
.itlont
ws
o6
rus
m-s ofal
ova-
Thus, after choosing the values ofd ~or r!, the stablerelations~11!–~13! can be used to reconstruct the undistorphotograph w(j,y) from the out-of-focus photograpg(x,y).
NUMERICAL REALIZATION
The IMAGE software~for Windows 95! was written inVisual C11 for solving the problem of reconstructinsmeared and/or out-of-focus images by Tikhonov regulartion and Fourier transformation according to Eqs.~1!–~7!and~8!–~13! with trial-and-error selection ofa ~as well asDand d! and display of the processing results on a monitBoth the ordinary problem of simulating the distorted imag(x,y) according to~1! or ~8! and the inverse problem oreconstructed the undistorted imagew(j,y) according to~5!or w(j,h) according to~11! are solved. The ordinary aninverse one-dimensional and two-dimensional Fourier traformations@see Eqs.~5!, ~7!, ~11!, and~13!# are performed inthe form of discrete Fourier transformation, which, in turn,realized in the form of fast Fourier transformation accordto programs that are modifications of the Fortran prograFTFIC ~see pp. 183–184 and 190–192 in Ref. 7! and FTFTC
~see p. 190 in Ref. 7!.Black-and-white image are processed using gray~a mix-
ture of red, green, and blue in equal proportions! to obtain alarge number of brightness gradations, and color imagesprocessed using separate processing in the three primaryors followed by superposition of the three images. Represtation of the colors in the Windows operating system accoing to the RGB~red, green, and blue! scheme is used morextensively in processing black-and-white images. Ecomponent is varied from 0 to 255~there are thus 256 brightness gradations!. Therefore, by mixing the three componenin equal proportions we can obtain: 0,0,0 for black;n,n,n,wheren51,...,254, for gray; and 255, 255, 255 for whiteThe ordinary problem is solved in the following manner. Ifis a ~distorted! photography, it is translated into a digitarepresentation by scanning. For simplicity, the commBMP ~bitmap! format for a graphic file is used in the presework. If it is a model example, an undistorted image~Fig.3a! is first formed using a graphics editor~Word, Paintbrush,etc.!, and then a smeared or out-of-focus image is formedthe computer according to~1! or ~8! ~Figs. 3b and 4a!.
Figure 3a shows an original image of text, Fig. 3b shothe imaged smeared at a 30° angle relative to the linetext, and Figs. 3c–3e show the regularized solutionwa(j,y)for the optimal value of the regularization parametera52.531023, for a50 ~i.e., without regularization!, and forthe overestimated valuea50.1, respectively. A total of 256discrete values ofy were used, and 256 discrete readingsg were taken at eachy alongx. If there were less than 25readings, zeros were added.
Figure 4 shows the analogous results for the reconsttion of an out-of-focus image~2563256 discrete reachewere taken!.
353 J. Opt. Technol. 67 (4), April 2000
d
-
r.
s-
s
reol-n--
h
n
in
sof
f
c-
CONCLUSION
1. The problems of the smearing and defocusing of iages are described by Fredholm-type integral equationthe first kind~for the one-dimensional and two-dimensioncases!.
2. These equations are effectively solved by Tikhonregularization with trial-and-error selection of the regulariztion parametera ~as well asD, d, andr!.
3. For its practical realization, the method is suppmented by the use of gray in processing black-and-wimages and by separate processing of color images inthree primary colors.
4. Everything taken together permits the efficient recostruction of smeared and out-of-focus images.
5. The method described can be used to process oldquality photographs, for reconstructing images of famoving targets, telescopic images of celestial bodies, or ptographs of terrestrial objects taken from space,improving the quality of tomography, etc.
1R. H. T. Bates and M. J. McDonnell,Image Restoration and Reconstruction @Clarendon Press, Oxford~1986!; Mir, Moscow ~1989!, 336 pp.#.
2A. B. Bakushinski� and A. V. Goncharski�, Ill-Posed Problems. NumericaMethods and Applications@in Russian#, Izd. MGU, Moscow ~1989!,199 pp.
3V. S. Sizikov, M. V. Rossi�skaya, and A. V. Kozachenko, ‘‘Processingsmeared image by differentiation, Hartley transformation, and Tikho
4V. S. Sizikov and I. A. Belov, ‘‘Modelling problem of distorted imagreconstruction by regularization method,’’ inAbstracts of 2nd Interna-tional Conference ‘‘Tools for Mathematical Modelling,’’St. Petersburg~1999!, pp. 127–128.
5A. N. Tikhonov, A. V. Goncharski�, V. V. Stepanov, and A. G. Yagola‘‘Inverse problems for processings photographic images,’’Ill-posed Prob-lems in the Natural Sciences, A. N. Tikhonov and A. V. Goncharsky~Eds.! @Mir Publishers, Moscow; Imported Publications, Chicago~1987!;Izd. MGU, Moscow~1987!, pp. 185–195#.
6A. F. Verlan’ and V. S. Sizikov,Integral Equations: Methods, Algorithmsand Programs@in Russian#, Naukova Dumka, Kiev~1986!, 544 pp.
7A. N. Tikhonov, A. V. Goncharski�, V. V. Stepanov, and A. G. YagolaNumerical Methods for the Solution of Ill-Posed Problems@Kluwer Aca-demic Publishers, Dordrecht–Boston~1995!; Nauka, Moscow~1990!, 232pp.#.
8V. S. Sizikov, A. V. Kuz’min, and A. V. Kozachenko, ‘‘Treatment oout-of-focused images by two-dimensional Hartley transformation aTikhonov regularization,’’ Izv. Vyssh. Uchebn. Zaved., Priborostroe42~8! ~1999!.
