Genetic algorithm implementation of stack filter design for image restoration

K.K.Delibasis PE. Undrill G. G. Cameron

Indexing terms: Siuclc filters, Genetic algorithms, Medical image processing

Abstract: Stack filters are a class of nonlinear spatial operators used for noise suppression. Their design is formulated as an optimisation problem and genetic algorithms (GAS) are used to perform the configuration. Applying the mean absolute error (MAE) as the basis of an objective function, the stack filter is used to restore magnetic resonance (MR) images corrupted with uncorrelated additive noise from 10% and 50%. The filter is trained on corresponding patches of the original and noisy image and then applied to the whole image. The outcomes are compared with the median filter and return a smaller MAE for all noise levels. The dependency of MAE on training window size and GA early termination is examined, showing that a reduction of 75% in computational complexity can be achieved by a 10% relaxation in MAE. The design is then extended from 9-point to 13-point filters and by training on Poisson noise, the filter is applied to nuclear medicine bone scans where no absolute truth exists. Surface topology, image profiles and the measurement of relative contrast show its value in reducing noise whilst preserving contrast. Because of its computational complexity the process has been implemented as a distributed GA using the parallel virtual machine (PVM) software.

1 Introduction

Stack filters are adaptive, nonlinear, spatial operators that can perform well in suppressing noise. Their out- put is a composition of maximum and minimum opera- tions on samples within a window of finite support. Popular filters such as the standard median [l] and rank order filters [2] can be expressed as stack filters, as can the more complex, adaptive approaches such as L- filters whose output is defined as a linear combination of order statistics (OS) [2, 31. The main advantages of stack filters are their ability to adapt to different types

0 IEE, 1996 IEE Pvorcwliugs online no. 19960513 Paper first received 6th September 1995 and in revised form 26th March 1996 The authors are with the Department of Biomedical Physics and Bioengi- neering, University of Aberdeen, Foresterhill, Aberdeen, AB9 2ZD, UK

of noise and their generality. Their main drawback is the computational complexity of their configuration.

I . 7 Definitions Initially we define relationships between two binary sig- nals, introduce the idea of a positive Boolean function (PBF) and describe the stacking property. If ? and y” are two binary sequences, then equality and inequality between them are defined as:

Z = ij’ H xi = y; for Vi

Z 5 g~ (xi = 1 + yi = 1 for Vz)

xz 2 y j U a > j

(14 ( I b )

(2)

A binary sequence 2 possesses the stacking property if:

A PBF f, possesses the stacking property if, when applied to an ordered sequence of binary signals that have the stacking property, it produces an output that also has the stacking property:

A stack filter is a window filter based on a PBlF [8]. Nodes [4] and Muroga [5] showed that a PBF can he uniquely expressed as the minimum sum of products (MSP) of the elements off(?) with:

22 ij’* f(.’) 2 f(?A ( 3 )

‘Ln -1

f (3) = c pzmz (2)

mi (2) = 11 xj

(4)

(5)

k l

where

sowLe j and pi is a Boolean variable indicating whether the ith Boolean product contributes to the result of applying functi0n.f. The symbols of summation and product in the above equations denote OR and AND operators respectively. The computational complexity stems from the combinatorial choice of j .

1.2 Stack filter applicafion The relationships between l?BFs and threshold logic has been studied extensively [6, 71, and although not every PBF is a threshold logic, such logic with nonneg- ative weights and a nonnegative threshold corresponds to a unique PBF. The stages of applying a stack filter operator are described [8--101 as: (i) An M-valued discrete signal (with values from a subset of integers Q = {0, I , ..., A4 - 1)) is mapped to the filter window R(i) of width n = 2r + 1, where r is an integer:

= (w - r ) , . -> W ) , - ‘ 7 R(j + r,) (6)

177 JEE Pro.-Vis. Image Signul Process., Vol. 143, No. 3, June 1996

(ii) The signal is then decomposed into M ~ 1 levels by a threshold operator Tb:

1 if R(i) 2 b 0 otherwise Tb(R(i)) = { (7)

where h E [ I , ..., M - 11 c I. Summing all the resulting decomposed binary sequences produces the original sig- nal. This threshold decomposition is formalised into:

M-1

g ( j ) = T b ( R ( j ) ) (8) b = l

(iii) Each of the n-bit long binary signals is independ- ently input to the PBFJ Eqn. 3, for a PBF defining a stack filter, can be written as:

(iv) Finally, all the outputs of the PBF are summed, under the convention that true equals 1 and false equals 0, and the result, using eqn. 8, is:

M - 1

F ( A ( j ) ) = f ( T b ( z ( j ) ) ) (10) b= 1

An example of the above algorithm is given for the very simple case of a 3 x 1 median filter (Fig. I), veri- fying that the PBF defining the median filter is of the form:

f zz 2 1 2 2 + 2 2 x 3 f 23x1 (11) where xi is the result of a threshold operation, products are ANDs and summations ORs. With the one dimen- sional signal and a 3-pixel window, the leftmost column represents thresholding values and the rightmost col- umn the result of applying the PBF to the threshold decomposed signals. The output of the filter is the sum of the individual outputs of the PBF, i.e., (1 + 1 + 0 = 2).

L action of filtei

/ window

1 p 1 1 - 2

\ thresholds \ decomposed

1 1 1 1 1 1 - 1 - action of Boolean 2 0 1 0 1 0 - 1 function 3 0 0 0 1 0 - 0

signals Fig. 1 Realisation ofthe median

It is helpful to consider the relationship of a stack fil- ter with OS operators, often called rank operators. The Boolean products in the stack filter correspond to min operations whereas the Boolean sums correspond to max operators [9]. From this an alternative realisation for the median would be f = max(min(X,, X,), min(X,, X3), min(X,, X I ) ) where XI are window values. This representation gives a better insight into the cornbina- torial complexity of stack filter design. For a 3 x 3 win- dow there are up to Z9 - 1 (= 511) sub-groups of pixels chosen from the region of support, for each of which we need to find a minimum, followed by a global max- imum. In general, the number of PBFs (and therefore stack filters) associated with a given number o f input variables is not known. Eqn. 4 suggests an upper limit of 22n-1 PBFs of n input variables, although smaller number is likely as more than one expression, from eqn. 4, can be simplified to the same final function.

178

Taking this into account, a lower limit of 22n’2 has been derived.

In addition to a 3 x 3 filter we wish to examine larger filters to determine whether these better preserve image detail. Because of its computational complexity the design of a 5 x 5 stack filter would he unmanageable, hence a 13-point mask was chosen, as symmetrical dia- mond shape within a 5 x 5 pixel matrix. For a simple two-dimensional 3 x 3 filter, the number of possible stack filters lies between 2256 and whereas for the 13-point filter the limits become 24096 and 28191.

2 The objective function

The task is to restore a corrupted image, given its noise-free equivalent, so that we can subsequently apply filters where no ‘ground-truth’ exists. We shall train the filter on a small part of the image then meas- ure how it performs on the whole image. An objective function returns a measure of how close the restoration is to the original and controls the GA optimisation. Several error functions that can be used as objective functions have been suggested by Kasturi [ l l] . Here we have used the mean absolute error (MAE) between the noise free Io and restored image I I defined as:

MAE = - 1 I&(z , j ) - I l ( z , j ) I (12) n.

image

where y2 is the number of pixels over which the calcula- tion is performed. An alternative is the mean square error (MSE) but our experiments suggest that changes in the MSE statistic overemphasise the reduction of individual high impulses relative to widespread lower intensity noise. For the preservation of image fine structure (often a weak point in existing techniques) the MAE is more appropriate.

3

3.7 op tim isation problem The generalised algorithm used for design is: (i) given an ideal image and a corresponding image cor- rupted with a specific type of noise, select a training window, normally a 50 x 50 region chosen from a 150 x 150 pixel image, (ii) configure a stack filter by minimising the objective function over the training window.

Hung and Chou [I21 and Chou [13] have reported the use of GAS for this type of task, but they were con- cerned with one-dimensional filters and Coyle [ 141 has used linear programming to optimise the design o f stack filters within the constraint of having a linear objective function. Although our MAE objective func- tion is also linear, we do not wish the method to be constrained by such a choice and will therefore outline the use of GAS in general stack filter design.

3.2 Coding a Boolean function into a chromosome The challenge of stack filter design is in determining which PBF to use. Eqn. 4 suggests how such a coding can be performed. We first construct every possible term of the function. Each of the 2n - 1 terms is a product of selected input variables (where n is the number of pixels in the window). The ith product con- tains only the input variables xi for which thejth bit of

Using GAS to configure stack filters

Configuration of stack filters as an

IEE Proc.-Vis. Image Signal Process., Vol. 143, No. 3, June 1996

the binary representation of the index i equals one. The above statement can be formalised by rewriting eqn. 5 as:

m, = IT 5.7 (13) j : b , = l

where hJ denotes the jth bit in the binary representation of the term’s index i. The factors p i of eqn. 4 can be 1 or 0 dependent on whether the ith term is present in the PBF. Storing these binary factors in a chromosome is a natural way to proceed.

3.3 Genetic operators The implementation of genetic operators is as described in Goldberg [ 151: (a) Selection was performed using the stochastic remainder method, employing fitness scaling. (b) Several different versions of crossover were tested, 1 -point, 2-point and uniform crossover (maximum niimber of breakpoints) with bit exchange probabilities of 0.1 and 0.5. The results showed a significant differ- ence in speed of convergence in favour of uniform crossover. (c) Mutation acts upon the chromosome by randomly changing the value of individual bits, with a probability chosen within a range from 0.01 to 0.001. The encoded mask is mutated by randomly selecting a pair of bits that have different values and exchanging their values.

(d) Bit-wise, gradient-based, local search, was applied to the best chromosome, every 10 generations, after the 30th generation. This delay is so that local search is ini- tiated only after the populaiion has converged around a good candidate minimum.

3.4 The overall algorithm Our implementation of a system for stack filter design using GAS is as shown in Fig. 2.

3.5 Computational complexity The GA configuration uses an average of lo2 genera- tions with a population of LO2 chromosomes, requiring IO4 function evaluations, hence significant computation may be needed if effective GJters are to be discovered.

4 Implementation

We apply ‘centralised implementation of GAS’ [16], arranged in a distributed parallel paradigm (Fig. 3). The parallel virtual machine (PVM) software [ 171 was used as a communication harness between distributed workstations. The overall performance will depend on the degree to which parallelisation can be achieved and the proportion of time spent in communicating data compared with computation.

4. I Algorithmic decornposition Selection (step 6 of the serial algorithm), and crossover

initialise first generation randomly while (not terminatioqcondi tion) do for each chromosome c

1. simplify c 2. construct the Boolean function that c defines 3. filter the window of the noisy image with the resulting stack

4 . calculate the objective function as an image quality measure

5. if (specific-condition) apply local search starting from c 6. apply selection algorithm 7. apply crossover and mutation to produce new-generation

filter S

between the ideal image and the filtered noisy image

I Fig.2 Outline GA algorithm

broadcast to all slaves: chromosome, number of strips n for bit=l to all-bits-in-chromosome do i

assign the first k tasks to the k slave programs i=k while (i<=n+k-l) do I

listen for a result from a slave s i

i : =i+l if (i<=n) then

assign task i to slave s, receive(s,bits,keepqrevious,result) fitness:=fitness+result

I } if fitness better-than previous-fitness then

keep_previous:=true save last mutation to the local copy

keep-previous:=false discard last mutation

else

}

Fig.3 Parallel GA algorithm

IEE Proc.-Vzs. Image Sifinal Process.. Vol. 143, No. 3, June 1996 179

and mutation (step 7) take negligible time compared with the evaluation of the fitness function. The latter involves simplifying the chromosome (step l), filtering the training image window (step 3) and calculating the MAE (step 4). These evaluations can be carried out independently for each chromosome in the population.

4.2 Data decomposition The above approach would be sufficient to accelerate a wide range of GAS. Step 5 of the overall algorithm. local search, requires a large number of function evalu- ations, but algorithmic decomposition cannot be applied as each step depends on the result of the previ- ous step. Data decomposition, dividing the image into equal sections, assigns each to a different slave, how- ever the computation performed by each slave is now smaller and communication overheads increase propor- tionally.

In our specific UNIX workstation environment we used 10 systems, having a integrated power of 5.3 times the most powerful. By using the distributed parallel approach a performance improvement of 3.5 over the single system was achieved, reducing the design stage of a 3 x 3 filter from 1.5 hours to less than 0.5 hours. Once the sequence of min and max functions are estab- lished the application of the filter takes a few minutes on a single system. The distributed approach is more valuable for 13-point filters where design times can be reduced from 30 to 8 hours.

5 Results and discussion

5. I a known image structure A GA designed filter was applied to additive, uni- formly distributed, uncorrelated impulse noise with probabilities from 10% to 50Y'o. Because of its random intensity variation this form of noise is more difficult to remove than salt-and-pepper noise. The original image was part of a transverse MRI scan of a normal human brain and the median filter was used as a reference. The training window was located towards the upper left of the image and the MAE was evaluated after one or two applications. Fig. 4 shows qualitative perform- ance for 30'% noise. Both median and stack filters are effective, the main difference being that strong edges are retained in the twice-applied stack filter. Fig. 4 also shows the effect of stack filter size, where for the 13- point filter, linear structures are better preserved (folds in cerebellum A, and ligament B).

Experiment 7 - Impulse noise applied to

d e f Fig. 4 a Original

Stuck,filteu application to uniform uncorreluted uoix h 0.3 noise c Medial1 d 3 x 3 stuck (once)

f 13-point stack (twice)

180

E 3 X 3 Stack (twice)

Fig. 5 shows MAE for median and stack filters. Stack filters outperform the median at levels less than 40% after a single application and above that after two. Data for the first application of filters of two sizes are presented in Fig. 6. The 13-point filters have an advan- tage over the 3 x 3 filters at high noise levels. At low probability (0.1 - 0.2) noise 13-point filters produce images visually more acceptable (fewer residual extreme values and better retained structures) than the 3 x 3 stack filter. although the MAE suggests the converse. The reason for the crossover just below 40% for the first and second application is not yet clear, nor is the observation that the repeated 13-point filters above 30'31 can have a equal or higher MAE yet return an image with superior fine-structure definition.

f

01 I

0.05 0.15 0.25 0.35 0.45 0.55 noise probability

Fig. 5 .MAE wiriution ivith reputed upplicution - e- 3x3 stack iiltcr ( 1 ) - -.- ~ -3,; stack filter (2) -A- 3x3 median filter

i

P aJ

5 -

4

'I 01 I I I I

0.05 0.15 0.25 0.35 0.45 noise probability

Fig. 6 - +- 3x3 stack filter -4- - 13-point stack filter -A- 3x3 median filtci-

MAE i.nricitron ii'ifh stuck jilter .sire

5.2 Varying the training window size and terminal generation of GAS Our technique uses a training window of li9th full image area and 100 GA generations. Execution time is proportional to the both. The training window size was reduced with the affect on local and global MAE shown in Fig. 7. Whilst the smallest window produces a filter with good performance on itself, its effect on the full image is poor. Accepting 95% of the final per- formance allows the use of windows of 30 x 30 pixels. When the GA convergence is examined (Fig. X), by applying a 2-point crossover strategy, 95% of the 100 generation level is reached after 70 generations,. Taken together, a filter with 10'3) reduced performance can

IEE Pi.oc.-Vi,r. In.tage Signal Process., Vol. 143, No. 3, Jzme 1996

therefore be designed in 25'% of the time originally reported in Section 4.2.

10

Our approach also designs the filter on typical noise, then applies it to an unknown image. Owing to their photon formation, nuclear medicine (NM) images exhibit Poisson statistics. lh the absence of ground- truth, images with known structure (the MR images used previously) and similar intensity to tentative regions of interest in the NM images were corrupted with artificial Poisson noise and used for training.

01 I I I I I I

0 10 20 30 40 50 60 training window edge size

M A E vuricition ivirlt trubzing i t h h v .size Fig. 7 -A- training windom

0 full irnagc

Fig. 10 Bone scan imuge ufier stark filtering

I I I

0 0 20 40 60 00 100

5.3 Experiment 2 - Impulse noise applied to a unknown image structure Where no ground-truth exists a suboptimal approach to OS filtering has been proposed [3], in which image subregions, filtered with a family of 5 x 5 trimmed mean OS filters [18], are used to estimate a global filter.

IEE Pro<..-Vi,$. Iinuge Signul Prot,e.r.s , Vu/. 143, h'o 3, June 1996

Fig. 11 3-0 surfuceplot,fbi. inqnge of Fig. 9

The alternative approach of using a phantom as the training object was rejected because of difficulty in achieving a precise match to geometric truth.

50 t 01 20 30 40 50 60

axial position Fig. 13 original image RpI = 1.91

Cross-section through upper 'hot spots' in marked box of Fig. 9:

200

50 t 01 I I

20 30 40 50 60 axial position

Fig. 14 mean-filtered image Rpt = 1.63

Cross-section through upper 'hot spots' in marked box of Fig. 9 -

250r 200c

50 t 20 30 40 50 60

axial position Fig. 15 median-filtered image R,,, = 1.74

Cross-section through upper 'hot spots' in marked box ojFig. 9:

200 ""r 50t 01 20 30 LO 50 60

Fig.16 Cross-section through upper 'hot spots' in marked box o j Fig. 10 Rp, = 1.88

axial position

We will not present results based on the MAE because of the absence of ground-truth, instead we examine a typical nuclear medicine bone scan, before and after filtering by a 13-point stack filter, Figs. 9 and 10, respectively. A region containing three sites of high

182

radioisotope uptake on the spine is selected (white rec- tangle) and in Figs. 11 and 12 the before and after image values are presented as a 3-D surface plots. The three sites are marked with arrows. After filtering the relative contrast of these sites is retained, despite the general smoothing effect of noise suppression. In Figs. 13-1 6, vertical intensity profiles through the two most significant of these 'hot-spots' are shown for the origi- nal and processed images and a simple statistic is Rpr derived from the mean contrast for the peakkrough pair PITi and P2T2. Whilst the mean and median val- ues are, as expected, less than the noisy original, the value for the stack filter shows contrast preserving characteristics while also retaining those of a noise- reducing operator.

6 Conclusions

The design of stack filters for noise suppression using GAS is described and its computational complexity established. Since GAS provide a robust method of fil- ter configuration, capable of incorporating arbitrarily complex objective functions, we have developed a sys- tem that allows a heterogeneous cluster of worksta- tions, operating in parallel, to achieve this in reasonable time. The major objective of this distributed approach is to allow the natural processes of experi- mentation to evolve as rapidly as possible.

Results of successive application of the filter are pre- sented for additive and embedded noise situations, drawn from medical imaging, which show that where the ground-truth is known the derived filters have superior performance, as measured by MAE, over con- ventional median filters. From the results, the following observations need further examination:

the superior performance over the median at levels of noise less than 20% is not maintained at 40-50% (Figs. 5 and 6,

above 20% noise, double application of the stack fil- ter becomes preferable both in terms of MAE and image quality (Figs. 4 and 5) ,

13-point filters can have a higher MAE yet return an image with superior fine-structure definition (Figs. 4 and 6).

The answer to at least some of the above must reside in the nonlinearity between MAE and visual percep- tion. New work should focus on how to incorporate visual perception or receiver operating characteristics (ROC) into the objective function.

Where ground-truth does not exist, a simulated train- ing method can realise a suitable filter which suppresses noise and retains contrast. In this latter case, qualita- tive assessment was by examination of profile and sur- face characteristics and quantitative assessment through contrast of selected image features, although a controlled set of observer trials using a panel of experts may be the only way the determine value-aspects of image quality.

An alternative to the multiple applications of the fil- ter would be to redesign a new filter after an initial application. This would be a progressive way to approach a minimum MAE and has similarities with many other adaptive optimisation techniques. As well as extending the computational task, the noise struc- ture itself would be changed at each iteration, whereas in this work we begin with a specific noise distribution, and design a filter to achieve optimal restoration.

IEE Proc.-Vis Image Signal Process., Vol. 143, No. 3, June 1996

7 References

I JUSTUSSON, B.I.: ‘Median filtering: statistical properties’, Top. Appl. Phys., 1981,43, pp. 161-196

2 BOVIK, A.C., HUANG, T.S., and DUNSON, D.C.: ‘A generali- sation of median filtering using linear combination of order statis- tics’, IEEE Trans., 1983, ASSP-31, pp. 1342-1350

3 NAAMAN, L., and BOVIK, A.C.: ‘Least squares order statistics filters for signal restoration’, IEEE Trans., 1988, CS-38, pp. 244- 257 NODES, T., and GALLAGHER, N.: ‘Median filters: some mod- ifications and their properties’, IEEE Trans., 1982, ASSP-30, pp. 719-746

4

5 MUROGA, S.: ‘Threshold logic and its applications’ (Wiley, New York, 1971)

6 GILBERT, E.: ‘Lattice-theoretic properties of frontal switching functions’, J. Math. Phys., 1954, 33, pp. 57-67

7 SHENG, C.L.: ‘Threshold logic’ (Academic Press, New York, 1969)

8 WENDT, P., COYLE, E., and GALLAGHER, N.: ‘Stack fil-

9 MARAGOS, P., and SCHAFER, W.: ‘Morphological filters: ters’, IEEE Trans., 1986, ASSP-34, pp. 898-911

Part I - Their set theoretic analysis and relation to-linear shift invariant filters’, IEEE Trans., 1987, ASSP-35, pp. 1153-1 169

10 GABBOUJ, M., and COYLE, E.: ‘Minimum mean absolute error stack filtering with structural constraints and goals’, IEEE Trans., 1990, ASSP-38, pp. 955-968

11 KASTURI, R., and WALKUP., J.: ‘Nonlinear image restoration in signal dependent noise’, in I-IUANG, J. (Ed.): ‘Advances in computer vision and image processing,’ Vol. 2, (1986), pp. 167- 212

12 HUNG, C., and CHU, H.: ‘A p,enetic algorithm approach to the configuration of stack filters’. l’roceedings of the third interna- tional conference on Genetic algorithms (Morgan-Kauffman, 1989), pp. 219-224

13 CHOU, C.: ‘The application of an adaptive plan to the configura- tion of nonlinear image processing algorithms’. Proceedings SPIE conference on Nonlinear image processing, 1990, Vol. 1247, pp, 248-257

14 COYLE, E., LIN, J., and GABBOUJ, M.: ‘Optimal stack filter- ing and the estimation and structural approaches to image processing’, IEEE Trans., 1989, ASSP-37, pp. 2037-2066

15 GOLDBERG, D.: ‘Genetic algorithms in optimization, search and machine learning’ (Addison--Wesley, 1989)

16 BIANCHINI, R., and BROWN, C.: ‘Parallel genetic algorithms on distributed memory architectures’. Technical Report 436, Uni- versity of Rochester, 1993

17 SUNDERAM, V., GEIST, G., DONGARRA, J., and MAN- CHEK, R.: ‘The PVM concurrent computing system’, Parallel Cornput., 1994, 20, (4), pp. 481496

18 RESTREPO, A., and BOVIIK, A.C.: ‘Adaptive trimmed mean fil- ters for image restoration’, IEEE Trans., 1988, ASSP-36, pp. 1326-1337

IEE Proc-Vi.s. Image Signal Process., Vol. 143, No. 3, June 1996 183