New algorithm for transient suppressionfor images due to incomplete or partial

boundary dataProf. G. Eichmann and Prof. L.M. Roytman

Indexing terms: Algorithms, Mathematical techniques, Signal processing

Abstract: Any n-dimensional signal processing problem involves manipulation with the input data set with theregion of support bounded at some (or all) edges. The resulting output sequence is the convolution of thegeometrically bounded input data set and an impulse response of some filter, which not necessarily has the sameregion of support as the input data set. Thus the ambiguity due to unknown values of the input data set beyondits region of support results in indeterminacy of all (or some) of the output sequence values. The paper presentsan approach for resolving the problem of indeterminacy by employing the technique of minimum responsevariation for reconstructing the values of the input data set beyond its original region of support.

1 Introduction

One of the results of the advent of computers has been theemergence of n-dimensional digital signal processing.Because it demands large amounts of memory, multidi-mensional signal processing has only recently been appliedfor many applications such as image processing, tom-ography, robotics, seismic signal processing etc. Theseapplications involve processing of input data sets by n-dimensional spatial domain digital filters. Such filters canbe classified as nonrecursive finite impulse response (FIR)filters which generate a weighted sum of present and pastinputs to compute the present output and recursive infiniteimpulse response (IIR) filters which generate a weightedsum of present and past inputs and past outputs tocompute the present output. It can be noted that, for adesired response characteristic, recursive filters are prob-ably more efficient from a hardware point of view andhave found the most applications. But the problem of inde-terminacy of all of the output values associated with latertypes of filters is of serious concern and has been recentlyaddressed in References 1, 2 and 3 for the 2-dimensionalcase. Some work on recursive smoothing for systems withuncertain observations was reported in References 4 and 5.

The approach in Reference 2 is based on the extrapo-lation of the given input values to provide the boundaryvalue and is conceptually similar to results in Reference 1.It was observed in Reference 1 that the existing method ofresolving indeterminacy by truncating the input data set(settings of all its values beyond the region of support tozero) would result in the transient effect for both FIR andIIR filters. For the FIR type of filters, with a finite dura-tion of the impulse response, the resulting transient effectdue to data truncation at the boundaries is localisedwithin a finite number of steps. However, transient effectsdue to data truncation can propagate throughout theoutput image with an IIR filter and may obscure the imagecompletely for some types of filters. This problem isaddressed in Reference 1 by extending the boundary valuesof the given input data set beyond its region of supportand computing initial conditions at the image boundariesbased upon this extension.

We note that an interesting approach presented in Ref-erence 1 resolves the problem of border transients and issuperior to the existing method of truncating the input

Paper 4339G, (E10), first received 6th June and in revised form 24th October 1985The authors are with the Department of Electrical Engineering, City College, CityUniversity of New York, New York, NY 10031, USA

data set beyond its region of support. Our only commentis that by extending boundary values beyond the originalregion of support to suppress any transient on the bound-ary, we actually generate a ripple effect within the region ofsupport. Based on that we feel that to obtain a generalpicture of how well the approach works we need somequantitative measure to evaluate the method performancenot just on the boundaries but within the entire region ofsupport.

To this end, this paper addresses the problem of thespatial boundedness of the original data set and resultingindeterminacy of the output sequence values in somequantitative terms by introducing a quantitative measureto evaluate a ripple effect within the entire region ofsupport. Within such quantitative frameworks we can gen-erate optimum reconstructed values of the input data setas values minimising in some sense the ripple effect withinthe entire region of support. Here we use the least meansquares estimate as the ripple effect minimisation criterion,which results in a linear scheme for the reconstruction ofthe input data set beyond the original region of support.The same approach is then extended to the boundary-value problem.

2 Derivation of main results

For the sake of brevity, we carry our discussion for thecase of 2-dimensional signals and then extend the derivedresult for the case of n-dimensional signals.

Let us first state the problem of indeterminacy arisingfrom the processing of digital signals with a geometricallybounded region of support by spatial domain linear shiftinvariant filters.

A 2-dimensional digital signal is represented by asequence u(m, n). Without loss of generality, we can assumethat its region of support is in the upper-right quarter-plane (i.e., m and n cannot be negative). The value u(m, n)represents the value of the output for column m and row n.The output of the filtering process is represented by thesequence y(m, n) from the following difference equation:

y(m, n) = Z Z fl(*> J)u(m ~ h n - j), = 0 j = 0

- Z YW,Mm-Un-j)i = 0 j = 0

(1)

where a(i, j) and b(i, j) are filter coefficients.

1EE PROCEEDINGS, Vol. 133, Pt. G, No. 1, FEBRUARY 1986 27

We note that to compute any value in eqn. 1 we need(Lt + 1)(L2 + 1) past and present inputs and (L3 + 1)(L4

+ 1) — 1 past outputs. Thus for the considered input dataset, with the region of support bounded by the axes m = 0and n = 0, we would have indeterminacy for two stripes ofthe output values; both stripes are adjacent to the axis andhave widths of Lx and L2, respectively, if L3 + L4 = 0(FIR filter) and for all output values if L3 + L4 # 0 (IIRfilter).

As has already been mentioned, the existing data trun-cation approach based on setting all unknown values ofthe input data set beyond its region of support to zero,would result in the transition effect into the upper-rightquarter-plane. An alternative approach presented in Refer-ence 1 is based on the following input data set extension:

/(m, n) =Am, 0)/(0, n)

n < 0

m < 0

Such an extension results, in the case of stable filters, forthe first values of the outputs at the boundaries (m = 0;n = 0) of the input data set to be the same as the steady-state values of the output prior to the first input. We notethat, in that approach, unknown values of the input dataset are replaced by values at the boundaries and the netresult is that there is a suppression of any transient at theboundary. But by the same token, such a replacementinevitably generates some ripple effect within the entireregion of support and we have no mechanism to evaluatethat ripple effect.

We propose to treat the problem of indeterminacy ofthe output values as the problem of resolving the ambi-guity due to lack of information on the input set valuesbeyond the region of support. Solution of such a mini-misation problem would give the replacement values of theunknown initial conditions.

There are many ways to define the direction of tran-sients propagation. For instance, we can define row-to-row, column-to-column, line-to-line etc. transientpropagation effects. We also note that the form of thequantitative measure of the transient is not unique. In thispaper, we will be using least-mean-squares measures oftransients, which seem to be more natural for signal pro-cessing applications. One additional advantage of usingquadratic measures is that such a problem results in alinear scheme for derivation of values, optimal in the least-mean squares sense, of input data.

To generate these quadratic forms we note that, for any2-dimensional point (i,j), a value of the output is given byy{i, j) and the value of the transient is given by

where (m, n) is the nearest point in the assumed directionof the transient propagation. In the interesting commentson the paper, one of the anonymous referees suggestedthat in some cases it would be advantageous that the cri-terion defined in eqn. 2 be redefined to include only thatpart of the transient that is related to the initial conditions.Such an alternative formulation can be given as

Del,-, = [{y{i,j) — y(m, n)} — <x{u{i,j) — u(m, n)}]2 (2a)

where | a | ^ 1For instance, if the assumed direction of the transient

propagation is horizontal (column-to-column) the value ofDel,,- is equal to

In the case of the vertical (row-to-row) direction of thetransient propagation the value of

Let us consider a 2-dimensional input data set in theupper-right quarter plane with the finite region of supportbounded by the vertical line i = Nt and the horizontal linej = N2. If we try to minimise the output transients for theentire region we have to consider the sum Del of the tran-sient for each point within the entire region of support.

Define S, the total ripple error, as

(5)

where i, j = region of support. An alternative definition forthe total ripple error, based on eqn. 2a is given by

ieL (5a)

We note that, depending on the assumed direction of thetransient propagation, we would obtain a different form of5. For example, if the assumed direction of propagation isvertical, the value of S would be as follows:

N\ JV2

Svert = V Y Del,,-L-i L-i ij

i = O j = 0

JVi N2

= Z Z ly(h y) - y(U j - i ) ] 2

i = 0 / = 0

(6)

(3)

28

From eqn. 1, observe that the value y(i, j) is a linearweighted sum of the input values. Thus the form of S forany assumed direction of transient propagation is a quad-ratic form with respect to the unknown input values of ourdata set beyond the original region of support.

The process of minimisation of the form S involves dif-ferentiation of S in unknown input values (taken asvariables), and results in the generation of the system oflinear equations. From that system the unknown values ofthe input data set are derived.

Let us illustrate this process by the following example.Let the input sequence be given by u(0, 0) = 1, u(0, 1) = 2,M(1, 0) = - 1, «(1, 1) = 1 and the filter is defined by

y(m, n) = u(m, n) - \y{m, n - 1)

Assume that we wish to minimise the vertical (row-to-row)transient propagation. From eqn. 6 we can generate thefollowing form:

Svert= t ZW'.J)-J'('.J-I)]2 (7)f = 0 j = 0

For our case, from eqn. 6, Svert can be reduced to the fol-lowing expression:

Svert = [1 - b(0, -1)]2 + [2 - b(l ~ I)]2

+ C - I + 4M0, - I ) ] 2 + [ - 2 + ^ ( 1 , - I ) ] 2

The quadratic form Svert is minimised if the unknownoutput values are

y(0, - l ) = - f ; y(l - 1) = f

and the values of the output mask are

j/(0,0) = ^ ; j ; ( l , O ) = - f ; y(0, 1) = tf;

Thus the minimum value of S is:

So = min Svert £ 3Ay

If, for the same problem, the algorithm of eqn. 5a is used

IEE PROCEEDINGS, Vol. 133, Pt. G, No. 1, FEBRUARY 1986

with a = 1, it would result in the following values of theoutput mask:

y(0, — 1) = ff;y(\,0)=-h

y(l, - l ) = f; y(0, 0) =

the value of S is Sx s 15.12Let us try now the truncation approach. If all the initial

conditions are set to zero with the same vertical propaga-tion, the value of S is

52 = 15.25

If, for the same filter, the technique of image extension,eqn. 1, is used, the computed value of S is

53 = 16.25

Comparing the values of S, using three different processesfor comparing the unknown boundary value, we note

S0<Sl<S2< S3

As was expected, the value of So is the smallest, being thevalue generated by the least mean squares minimisationprocedure. The values of the transients for the last threeapproaches are approximately the same.

The approach described here for the 2-dimensionalsignals can easily be extended to the case of n-dimensionalsignals. The main difference is that in this case the tran-sient propagates along (n — l)-dimensional polyplanes.Similarly to the 2-dimensional case, values of the initialcondition can be derived by minimising the appropriatequadratic form in the least-mean-squares sense. Such aminimisation procedure can be reduced to pseudo-inversion of the appropriate matrices. We also note that asimilar approach can be applied if boundary conditionsare given. In this case, using eqn. 1, we express the valuesof the output within the region of support as the linearweighted sum of the boundary conditions and unknownvalues of the input data set beyond the region of support.We then generate a form similar to eqn. 5 and minimise itwith respect to the output values within the region ofsupport.

3 Conclusion

In this paper, we have proposed a numerical approach forresolving the problem of the indeterminacy of the outputvalues by generating the values of the input data setbeyond its original region of support. The approach pre-sented introduces some quantitative performance charac-teristics to evaluate the resulting restoration and comparesit with the existing approaches.

4 Acknowledgment

We wish to acknowledge the very useful comments of theanonymous referees. This work was supported in part by agrant from the US Air Force Office of Scientific Research.

5 References

1 ALEXANDER, W.E.: initial condition transient suppression for 2-dimensional recursive filters'. Proceedings of IEEE International Con-ference on Acoustics, Speech and Signal Processing, 1984, pp.2011-2014

2 WOODS, F., BIEMOND, J., and TEKALP, M. 'Boundary valueproblem in image restoration'. Proceedings of IEEE International Con-ference on Acoustics, Speech and Signal Processing, 1985, pp. 692-695

3 BIEMOND, J., RIESKE, F., GERBRANDS, J.: 'A fast Kalman filterfor images degraded by both blur and noise', IEEE Trans., 1983,ASSP-31, pp. 1248-1256

4 MONZINGO, R.A.: 'Discrete linear recursive smoothing for systemswith uncertain observations' ibid., 1981, AC-26, (3), pp. 754-757

5 HADIDI, M.F., and SCHWARTZ, S.C.: 'Linear recursive state estima-tors under uncertain observations', ibid., 1979, AC-24, pp. 944-948

George Eichmann received the B.E.E. andthe M.E.E. degrees from the City College ofNew York in 1961 and 1963 and the Ph.D.degree from the City University of NewYork in 1968, respectively. At present, he isa Professor and the Chairman of theDepartment of Electrical Engineering atthe City College of the City Univerity ofNew York. Among the areas of researchinterest are image and signal bandwidthcompression, optical computing, transform

shape descriptors, associative memory processors, artificial intel-ligence processors, and machine and robot vision. He haspublished and presented a number of papers in the above areas.He is a member of the I.E.E.E., the Optical Society of America,the Society of Photo-Instrumentation Engineers, the Sigma Xiand the Eta Kappa Nu Societies.

Leonid M. Roytman was born in the USSR,in 1949. He received the M.S. degree fromthe Institute of Communication Engineer-ing, Leningrad, USSR and the Ph.D. degreein electrical engineering from the Poly-technical Institute, USSR, in 1974. In 1971,he joined the Communication ResearchInstitute, Moscow, USSR, working in com-munications. In 1973, he joined the facultyof the Department of Electrical Engineer-ing, Tomsk Polytechnical Institute, USSR,

teaching, supervising, and establishing research activities in faultdiagnostics and multidimentional digital filters. He joined theNova Electric Company, Inc., NJ, USA, an an Engineer in 1978,designing uninterruptible power supply devices. In 1979, hejoined the Department of Electrical Engineering, Concordia Uni-versity, Montreal, Quebec, Canada, teaching and conductingresearch in fault diagnostics and signal processing. In September1980, he joined the faculty of the Pennsylvania State University,University Park, as an Associate Professor. In September 1984,he joined the faculty of the City College of the City University ofNew York. He is currently engaged in teaching and research per-taining to signal processing and circuit theory. Prof. Roytman isa member of the New York Academy of Sciences.

IKE PROCEEDINGS, Vol. 133, Pt. G, No. 1, FEBRUARY 1986 29