Image restoration in singular vector space by the methodof convex projections
Osami Sasaki and Taku Yamagami
In singular vector space produced by singular value decomposition, the effect of noise on the expansioncoefficients of an object is clear. A new image restoration method estimates the expansion coefficientscorrupted by noise from the expansion coefficients corresponding to large singular values by incorporating apriori information about the noise and the object. This information can be described as closed convex sets.Iterative projections onto the convex sets are performed to obtain a restored image. The effectiveness of themethod in singular vector space is made clear through computer simulations for 2-D bandlimited degradedimages.
1. Introduction
The problems of restoring linearly degraded imagesarise in many fields of applied physics. In general, themathematical formulation of the image restoration isdefined by a system of linear equations. If the noise inthe degraded images arbitrarily produces large errorsin the solution, the inverse problems are ill-posed. Toavoid instability of the problems, Rushforth et al.1 andSevercan2 used a weighted and truncated singular val-ue decomposition (SVD) or a regularized pseudoin-verse in the restoration of bandlimited images. Sincethese techniques cannot incorporate additional a prio-ri information, they are not sufficient to reduce theeffect of the noise. Maeda and Murata3'4proposed theiterative regularized pseudoinverse method which ef-fectively uses additional constraints of non-negativityof the object through adaptive regularization. In thismethod, however, it is not easy to determine a suitablevalue for the parameter that controls the degree ofregularization for achieving good restoration at a lowsignal-to-noise ratio. This situation seems to becaused by the fact that the effect of the noise on theestimated solutions cannot be well understood in theobject space where the original and degraded imagesexist. Trussell and Civanlar5 utilized the informationabout the noise statistics. The convex sets for variousstatistical amounts of the residual in the object space
The authors are with Niigata University, Faculty of Engineering,8050 Ikarashi 2, Niigata-shi, Japan.
Received 1 October 1986.0003-6935/87/071216-06$02.00/0.
were formed to use the method6 -8 of projections ontoconvex sets, which was also applied to image restora-tion in computerized tomography.9 10 In the objectspace, however, the effect of the noise on the residual iscomplicated and the projection onto the convex setsrequires a large amount of computation.
On the other hand, the effect of the noise is clearlyexpressed by the expansion coefficients of an object insingular vector space obtained from the SVD algo-rithm. We have already proposed an expansion coeffi-cient method in which the expansion coefficients cor-responding to small singular values are estimated byusing the expansion coefficients corresponding to largesingular values and a priori information." In thispaper we describe an image restoration in singularvector space by the method of projections onto convexsets. Statistical properties of the least-squares esti-mate of expansion coefficients in the presence of noiseproduce a reasonable constraint which forms a convexset. The effect of noise is well understood with thisconvex set in singular vector space. Another convexset is formed by constraining the non-negativity of theobject. For these two convex sets, iterative processingis done between two domains of the object and itsexpansion coefficients. The dimension of the expan-sion coefficients is smaller than that of the degradedimage, especially in the 2-D case. Therefore, the com-putation time in singular vector space is shorter thanthat in object space.
In Sec. II we review a least-squares solution in singu-lar vector space, and we present an image restorationmethod by iterative projections onto the two convex*sets in Sec. III. In Secs. II and III we consider the 1-Dcase for the sake of simplicity. In Sec. IV characteris-tics of the method are discussed by numerical analysisfor 2-D bandlimited images. Some comparisons are
made between the two methods in singular vectorspace and in object space. Other numerical results insome different conditions are demonstrated in Sec. V.
II. Least-Squares Solution in Singular Vector Space
The problem of restoring a linearly degraded imageis defined by the system of linear equations:
The variance of the noise is given by using the unbiasedestimator
(12)
Non-negativity of the object forms the convex set
CN = Jdl Va Ž:0O.
where y is the N-dimensional degraded image vector, xis the L-dimensional original object vector, n is the N-dimensional noise vector, H is the N X L distortionmatrix, and N > L.
Applying the SVD algorithm to matrix H, Eq. (1)can be written as
y = USVTx + n, (2)
where U is an N X N orthogonal matrix, V is an L X Lorthogonal matrix, Tdenotes the transpose, and S is anN X L diagonal matrix whose elements si are called thesingular values of H. Let x, y, and n be expanded intothe forms, respectively,
Image restoration is achieved by finding some commonpoint of the system of convex sets which is expressed as
a e CO n CN- (14)
For this, the following iterative process is repeatedalternately for the two convex sets of Co and CN:
a(k+l) = a(k) + X[a() - (k)] 0 < X < 2, (15)
where a(k) is the projection of a(k) onto a convex set atthe kth iteration. The initial value a(°) is defined byusing the conventional weighted SVD method as fol-lows:
A(0) = 2 [/s2ai.
a = VTx, b = UTy, c = UTn, (3)
which are referred to as the expansion coefficient vec-tors in singular vector space. Using these coefficientvectors in Eq. (2) yields
(16)
It has been reported that the convergence of thissuccessive projection onto the convex sets is strong.6
We consider the projection operators for the twoconvex sets. The projection of a(^) onto Co is given by
b = Sa + c.
The solution of Eq. (4) is given by
a = S+b,
(4)
(5)
where S+ is the pseudoinverse of the matrix S. Thusthe elements of a can be expressed as
&i = a + sci i1.L. (6)
It is assumed that the elements of noise n are Gaussianand independent of each other so that
Eini = , (7)
E{nin j {a i j (8)
where E denotes the expectation. Then the distribu-tion of the estimate &i is Gaussian with a mean of ai andvariance of 2/s? which is expressed as
i - ri/si, aI( < ai - ri/si,a (k), lai(k) - i rialsi,
(i + ri/si, l(k) > a, + rio/s.(17)
The X in Eq. (15) is denoted by X0 for Co. The parame-ters of r and X0 should be determined so that theexpansion coefficients corresponding to large singularvalues help greatly in estimating those correspondingto small singular values. Numerical analysis aboutthese parameters will be shown in Sec. IV.
The projection onto CN requires a simple clipper inthe object space as follows:
Z Zi 0,0i 0 zi<0, (18)
where z = Va(k). Then, the projection ofa(k) onto CN isgiven by
(9)
111. Image Restoration by Iterative Projections ontoConvex Sets
We apply the method of projections onto a convexset to image restoration in singular vector space. Wehave a priori information about the noise given by Eq.(9) and about the object which is expressed as
Va 2 0. (10)
The convex set based on the information of Eq. (9) isdefined by
C0 = ail lai - til • ria/sil, i = 1,2,...,L. (11)
ak) = VTX(k) (19)
The X in Eq. (15) is denoted by XN for CN. Since CNdenotes a closed half-space in the object space, thevalue of XN is taken to be 1, that is, a(k+1) = a(^).
IV. Numerical Analysis
A. Two-Dimensional Bandlimited Images
Let F be the discrete Fourier transform matrixwhose elements are given by
Fm = exp(-j27rmn/N) m,n = 0,1,...,N - 1. (20)
It is assumed that a low-pass filter allows the dc com-ponent, the M-lowest positive-frequency components,
where F-' is the inverse of F. A 2-D bandlimitedimage is given by
Y = HXHT + N, (23)
where H is an N X L matrix which consists of the Lcolumns of matrix G, X is the L X L original imagematrix, and N is the N X N noise matrix. Using H =USVT, Eq. (23) can be expressed as
B = SAST + C, (24)
where
A = VXV, B = U2'YU, C = UTNU.
The solution of Eq. (24) is given by
A = S+B(ST)+.
Thus the elements of A can be expressed as
(25)
Ai =Ai + s'sj'Cij ij = 1. ,L. (27)
To express the convex set Co for 2-D image restorationin the same way as 1-D image restoration, we arrangesisj in the order of decreasing value and rewrite it by sl(1 = 1,. . . ,L2). We also rewrite Aij corresponding to slby al. Then, Eq. (11) can be used in 2-D image restora-tion.
Figure 1 shows the original image of L = 10, and Fig.2 shows the degraded image of N = 32 at M = 6 and 20-dB signal-to-noise ratio. We will analyze the charac-teristics of the method for this degraded image in thefollowing.
B. Initial Value
The weighting factors of the initial value in Eq. (16)are shown in Fig. 3(a) for various values of a. Theabsolute values of IiI and the values of a/si (i = 1-100)are shown by solid and dotted lines, respectively, inFig. 3(b). When the number of i is greater than aboutforty, the value of &i deviates greatly from the realvalue of ai due to the noise. Figure 4 shows the imagesrestored at various values of a when the other parame-ters, such as r and X, are the suitable values which willbe determined later. The iteration number is denotedby k, and the estimated solution error is defined by
Ea = IA - all/lIall, (28)
(26) where [Iall is the norm of a. Similar good restoredimages are obtained over a wide range from a = 10-3 to10-13. At a = 10-7, the estimated expansion coeffi-
Fig. 5. Restored images for three different distributions of ri whichare illustrated in Figs. 6(a), (b), and (c), respectively.
(b)
Fig. 4. Restored images for (a) a = 10-3, (b) a = 10-9, and (c) a =10-17.
(a)
cients which contain a large amount of the noise com-ponent prevent achieving good restoration. We take a= 10-9 as a suitable value in the following.
C. Distribution of r,
The distribution of ri should be determined so thatthe expansion coefficients suffering from noise are es-timated by effectively utilizing the expansion coeffi-cients corresponding to the large singular values.Therefore, the determination of ri is the most impor-tant problem in this method. The bound ri/si repre-sents the confidence limit on the deviation from the &i.If the confidence limit is chosen to be 0.997, ri is equalto 3.0. First, we take r to be 3 for all i. A restoredimage in this case is shown in Fig. 5(a). A good re-stored image is not obtained because the expansioncoefficients corresponding to large singular values arenot effectively utilized. Next, since it is expected that&i is almost equal to the real value ai when Iail > &/si, thevalue of ri is taken to be smaller for these expansioncoefficients. So, we make the distribution
(b)
(c)
ri =3.0
r =0.6-1, I
r. IqnI iR=0J.6
I I
20 40 60 80 100
Fig. 6. Three different distributions of ri.
D. Values of R1 and X0
Table I gives iteration numbers and errors for vari-ous values of R1. The values of R, = 0.6-0.8 producegood restoration. Table II gives them for various val-ues of X0. The value of X0 is constant during theiterative procedure. The value of X0 = 1.9 producesthe best restoration with a small number of iterations.
___ JR1 log(&/si) < V1,
3i1.0- 10g(a/sj) > V-(29)
This distribution completely divides all expansion co-efficients into two classes. An image restored at R =0.6 and V = -1.5 is shown in Fig. 5(b); a good result isobtained. Finally, considering the fact that the noisecontained in the i increases as the number of i in-creases, the distribution of ri is determined as
An image restored at R, = 0.6, VI, = -1.5, and V2 =-0.5 is shown in Fig. 5(c); a better result is obtained.Three different distributions of ri are illustrated in Fig.6. We use the distribution of ri given by Eq. (30) in thefollowing.
Table 1. Image Restoration for Various Values of R,
Table 11. Image Restoration for Various Values of AX
Xo k Ea
1.9 236 0.1421.0 311 0.1560.5 371 0.200
10,
10,R = 3.0
Ea = 0.473
R = 2.57
Ea = 0.242
(c)
Fig. 8. Image restoration at M = 5 and 20-dB signal-to-noise ratio:(a) degraded image, (b) distributions of 16i I and /si, and (c) restored
image.
Fig. 7. Images restored in(b)
object space: (a) R = 3.0 and (b) R =2.57.
E. Comparison with the Method in Object SpaceWe compare this method with the method in object
space. The following two convex sets in object spaceare taken in the 1-D case according to Eqs. (11) and(13):
CX = {xl Iyi - [Hx]il < R, i = 1,2.N),
CXN = IXi > O,
(31)
(32)
where yi and [Hx]i are the ith elements of the vectors yand Hx, respectively. The iterative procedure usingprojections is the same as that in singular vector space.Images restored in object space are shown in Figs. 7(a)and (b) for R = 3.0 and 2.57, respectively. In themethod in object space, it is difficult to find a suitablevalue of R which is constant for all the residuals. Com-putation of the residuals and the projection onto CQ ismore complicated than that of the method in singularvector space. The computation time to obtain therestored image of Fig. 5(c) was about one-twentieth ofthat to obtain the restored image of Fig. 7(b).
V. Numerical Results in Some Different Conditions
We present the results of image restoration in otherconditions. The degraded image, distribution of iland &/si, and the restored image at M = 5 and 20-dBsignal-to-noise ratio are shown in Figs. 8(a), (b), and(c), respectively. The results for a 10-dB signal-to-noise ratio are shown in Fig. 9. Comparing Figs. 8(b)and 9(b), the effect of noise by SNR degradation be-comes clear. The results for M = 4 and 20-dB signal-to-noise ratio are shown in Fig. 10. Figure 10(b) indi-cates clearly that the system of M = 4 is worse than thesystems of M = 6 and 5. For the M = 4 system, animage restoration is not sufficient since two smallpeaks appear which do not exist in the original image.
(c)
Fig. 9. Image restoration at M = 5 and 10-dB signal-to-noise ratio:(a) degraded image, (b) distributions of ili and 5/si, and (c) restored
image.
lo'
(b) (a)
k =500Ea =0. 341
Fig. 10. Image restoration atM = 4 and 20-dB signal-to-noise ratio:(a) degraded image, (b) distributions of ldil and /si, and (c) restored
We have described an image restoration in singularvector space by the method of iterative projectionsonto convex sets. The effect of the noise on the expan-sion coefficients is clear in the distributions of Iil and&/si. Considering these distributions, we can make aconvex set for the information about the noise. Wecan also determine the initial value of the expansioncoefficient from these distributions. Thus, the expan-sion coefficients corresponding to large singular valuescontribute much toward estimation of those sufferedfrom the noise. Iterative projections onto the twoconvex sets, one of which is formed by the constraint ofnon-negativity of the object, produce good image res-toration.
The characteristics of the method were analyzedthrough computer simulations for 2-D bandlimitedimages. The effectiveness of the method in singularvector space was demonstrated by comparison withthe method in object space. The method in singularvector space will be useful for other image restorationin linear systems.
References
1. C. K. Rushforth, A. E. Crawford, and Y. Zhou, "Least-SquaresReconstruction of Objects with Missing High-Frequency Com-ponents," J. Opt. Soc. Am. 72, 204 (1982).
2. M. Severcan, "Restoration of Images of Finite Extent Objects bya Singular Value Decomposition Technique," Appl. Opt. 21,1073 (1982).
3. J. Maeda and K. Murata, "Restoration of Band-Limited Imagesby an Iterative Regularized Pseudoinverse Method," J. Opt.Soc. Am. Al, 28 (1984).
4. J. Maeda and K. Murata, "Image Restoration by an IterativeRegularized Pseudoinverse Method," Appl. Opt. 23, 857 (1984).
5. H. J. Trussell and M. R. Civanlar, "The Feasible Solution inSignal Restoration," IEEE Trans. Acoust. Speech Signal Pro-cess. ASSP-32 201 (1984).
6. L. M. Bregman, "Finding the Common Point of Convex Sets bythe Method of Successive Projections," Dokl. Akad. Nauk SSSR162, 487 (1965).
7. L. G. Gubin, B. T. Polyak, and E. V. Raik, "The Method ofProjections for Finding the Common Point of Convex Sets,"USSR Comput. Math. Math. Phys. 7, 1 (1967).
8. D. C. Youla and H. Webb, "Image Restoration by the Method ofProjections onto Convex Sets. Part I," IEEE Trans. Med. Imag-ing MI-i, 81 (1982).
9. M. I. Sezan and H. Stark, "Image Restoration by the Method ofProjections onto Convex Sets, Part II," IEEE Trans. Med. Imag-ing MI-i, 95 (1982).
10. M. I. Sezan and H. Stark, "Image Restoration by Convex Projec-tions in the Presence of Noise," Appl. Opt. 22, 2781 (1983).
11. 0. Sasaki and T. Yamagami, "Image Restoration by IterativeEstimation of the Expansion Coefficients of an Object in Singu-lar Vector Space," Opt. Lett. 10, 433 (1985).
Meetings continued from page 1156
1987
May
4-7 Advanced Industrial Laser Safety Officers course, SanFrancisco Laser Inst. of Amer., 5151 Monroe St.,Toledo, OH 43623
4-15 Optical Science & Engineering course, Tucson P.Slater, Optical Systems & Eng. Short Courses, Inc.,P.O. Box 18667, Tucson, AZ 85731
11 Intro. to Acoustic Emission for Nondestructive Testingcourse, Norman D. Egle, 865 Asp Ave., Rm. 212,Oklahoma U., Norman, OK 73019
11-13 4th European Conf. on Integrated Optics, GlasgowSPIE, P.O. Box 10, Bellingham, WA 98827
11-15 Applied Optics course, Las Cruces L. Radziemski,Physics Dept., NMSU, Las Cruces, NM 88003
11-15 SID'87, New Orleans M. Goldfarb, Palisades Inst. forRes. Services, Inc., 201 Varick St., New York, NY10014
12-15 Int. Symp. on Optical Particle Sizing: Theory and Prac-tice, Rouen G. Gouesbet, Dept. Chem. Eng., INSAde Rouen, BPO8, 76130 Mont-Saint-Aignan, France
13-15 7th Int. Workshop on Expert Systems & Their Applica-tions, Avignon Agence de l'Informatique, Tour Fiat,92084 Paris-La Defense CEDEX 16, France
17-22 40th Ann. SPSE Conf. & Symp. on Hybrid ImagingSystems, Rochester D. Wilkalis, P.O. Box 821, Fair-port, NY 14550
18-20 SIAM Conf. on Optimization, Houston Soc. for Indus-trial & Applied Mathematics, 14th Fl., 117 S. 17thSt., Philadelphia, PA 19103
18-21 Modern Experimental Spectroscopy course, Las CrucesLaser Inst. of Amer., 5151 Monroe St., Toledo, OH43623
22-26 9th Int. Exhibition for Environmental Pollution Con-trol, Measuring & Testing Equipment, Seoul SHKInt. Services Ltd., 22/F., Tian An Centre, 151Gloucester Rd., Wanchai, Hong Kong
