Contextual clustering for image segmentation

Downl

Contextual clustering for image segmentation

Andrea BaraldiISAO-CNRVia Gobetti 10140129 Bologna, ItalyE-mail: [email protected]

Palma BlondaIESI-CNRVia Amendola 166/570126 Bari, ItalyE-mail: [email protected]

Flavio ParmiggianiISAO-CNRVia Gobetti 10140129 Bologna, ItalyE-mail: [email protected]

Giuseppe SatalinoIESI-CNRVia Amendola 166/570126 Bari, ItalyE-mail: [email protected]

Abstract. The unsupervised Pappas adaptive clustering (PAC) algo-rithm is a well-known Bayesian and contextual procedure for pixel label-ing. It applies only to piecewise constant or slowly varying intensity im-ages that may be corrupted by an additive white Gaussian noise fieldindependent of the scene. Interesting features of PAC include multireso-lution implementation and adaptive estimation of spectral parameters inan iterative framework. Unfortunately, PAC removes from the scene anygenuine but small region whatever the user-defined smoothing param-eter may be. As a consequence, PAC’s application domain is limited toproviding sketches or caricatures of the original image. We present amodified PAC (MPAC) scheme centered on a novel class-conditionalmodel, which employs local and global spectral estimates simulta-neously. Results show that MPAC is superior to contextual PAC andstochastic expectation-maximization as well as to noncontextual (pixel-wise) clustering algorithms in detecting image details. © 2000 Society ofPhoto-Optical Instrumentation Engineers. [S0091-3286(00)02704-5]

Subject terms: supervised and unsupervised learning; contextual and noncontex-tual clustering; image segmentation; maximum likelihood and maximum a poste-riori classification; Markov random field.

Paper 990157 received Apr. 8, 1999; revised manuscript received Oct. 21, 1999;accepted for publication Oct. 21, 1999.

s o

ge

as-n,

inatu

et

sei.e

edon.

n-

ge

he

ing

o-

ile

ion

eldel atng

res-

eldod-

ifi-ob-

and

1 Taxonomy of Image Partition Algorithms

Picture segmentation and classification are two exampleimage labeling~partitioning! problems. We identify asyi

the data vector observed at pixeli , which belongs to a 2-Drandom field~image! S, such thati P$1,N%, whereN is thetotal number of pixels. The observed data sety5(y1 , . . . ,yN) is also called the incomplete data. In imasegmentation~classification!, each pixeli can take one la-bel or statusxiP$1,C%, where C is the total number ofunsupervised categories~supervised classes!. Statusxi is~not! provided with semantic meanings in the case of clsification~segmentation! tasks: e.g., in image classificatiopixels assigned to a specific land cover~e.g., roads! have aclear semantic meaning for human interpretation, whileimage segmentation a region has no land cover stlinked to it. An arbitrary labeling~partition! of imageS isdenoted byx5(x1 , . . . ,xN), where the image~category orclass! statusx is considered hidden, i.e., not observable. Spair z5(y,x) is the so-called complete data set.

Picture labeling~segmentation and classification! tasksare subjective and context-dependent cognitive procesIn mathematical terms, these processes are ill-posed,there is no single goal for picture partition algorithms.1,2 Tomake picture labeling a well-posed problem, it is regardas an optimization task featuring a firm statistic foundatiFor example,Bayesianimage labeling tries to maximizethe joint distribution

p~zuf!5p~y,xuf!5p~yux,f!p~xuf!}p~xuy,f!, ~1!

where, by Bayes’ theorem,f is a set of parameters~un-

Opt. Eng. 39(4) 907–923 (April 2000) 0091-3286/2000/$15.00

oaded From: http://opticalengineering.spiedigitallibrary.org/ on 09/24/2013 T

f

s

s..,

known, to be determined! that characterizes the distributioof z, p(yux,f) is the likelihood or class-conditional distribution of pixel values,p(xuf) is the label field or priordistribution of class labels representing the prior knowledabout the size and shape of regions, andp(xuy,f) is theposterior distribution. In the rest of this paper, we drop tconditioning onf for notational convenience.

Two major classes of models are used in formulatclass-conditional densitiesp(yux), leading to the develop-ment of two major classes of image classification algrithms: the class oftexture classification algorithmsem-ploys spatial correlations to model image textures, whthe class ofmultispectral classification algorithmsadopts aspectral class-conditional model when texture informatis negligible.3

In the first classification case, a continuous random fimodel describes the statistical dependency of a gray leva lattice point on that of its neighbors, given the underlyiclasses~interpixel feature correlation!. It may employ thecausal autoregressive model, the simultaneous autoregsive model, or the conditional Markov model.3–6 Althoughcomputationally expensive, the continuous random fimodel may be preferable to the spectral approach in meling images featuring distinct texture information.3,4

In the second classification case, multispectral classcation algorithms are based on the assumption thatserved pixel gray values are conditionally independentidentically distributed, given their~unknown! class labels,i.e.,

p~yux!5)i 51

N

p~yi uxi !. ~2!

907© 2000 Society of Photo-Optical Instrumentation Engineers

erms of Use: http://spiedl.org/terms

aysticss ins-in-

undxpo

asre-

unite

in

ui-

ll-

ms

ye-

ize

ms

m-

a

in

is

on

l

em

r

imal

-atonre-

belcan.o-

of

e-

ons-

n,

re

theon

cand orfs

forde-

esti-

Baraldi et al.: Contextual clustering for image segmentation

Downl

This assumption is adopted in the rest of this paper; it sthat no spatial texture, but only multispectral characterisof classes, are to be employed as discriminating featurethe labeling process.3,4,7 This hypothesis becomes increaingly acceptable as the dimensionality of observed datacreases, since within regions belonging to a single grocover spatial correlation has been found to decrease enentially with the dimensionality of images.3,7 A traditionalspectral model is that based on a multivariate-normalsumption for the distribution of independent spectralsponses, under the hypotheses that each category hasform intensity and that the image is corrupted by a whGaussian noise field independent of the scene.3,4,8–12Whenthese conditions hold, Equation~2! is such that

p~yi uxi !5expH 21

2s2 @yi2m~xi !#2J , ~3!

wheres is the white noise standard deviation expressedgray level units andm(xi) is the uniform intensity of cat-egoryxiP$1,C%.

In Equation~1!, when states are independent and eqprobable, then

p~x!5)i 51

N

p~xi !5)i 51

N

1/C, ~4!

such that maximization of Equation~1! is calledmaximumlikelihood (ML) estimation.4

An example of Bayesian ML procedure is the weknown hardc-means~HCM! clustering algorithm appliedto image segmentation tasks.13,14 Since HCM complieswith the hypotheses of Equations~2! to ~4!, it is equivalentto a crisp (hard-competitive, nonfuzzy, i.e., each pixel isassigned to only one class, such thatxiP$1,C%, because awinner-take-all assignment strategy is adopted! Bayesiannoncontextual (i.e., pixelwise, pixel-by-pixel)ML labelingprocedure. Bayesian noncontextual ML labeling algorithtend to generate noisy~salt-and-pepper! segmentedimages.9 Therefore, to obtain smooth segmentation, Basiancontextual maximum a posteriori~MAP! labeling pro-cedures have been developed. They attempt to minimEquation ~1! while employing contextual information tooptimize model parameters of either one or both terp(yux) andp(x), while Equation~4! does not hold.3,4,9 Forexample, a Markov random field~MRF!, where each pointis statistically dependent only on its neighbors, is often iposed on the spatial distribution of the statesp(x) to en-force spatial continuity in pixel labeling~interpixel classdependency, see Refs. 4, 5, 7 to 10, and 12, and forgeneral overview of MRF models, refer to Ref. 15!. Thisunderlying MRF model can be considered a ‘‘stabilizer’’the sense of the regularization theory,4 which helps in solv-ing otherwise ill-posed problems.16

An example of Bayesian contextual MAP procedurethe iterative conditional mode~ICM! algorithm,10 whichtries to minimize Equation~1! based on hypothesis~2! toreduce the computational complexity of the minimizatitask.3 If the 2-D random field~stochastic process! p(x) isany locally dependent MRF then

908 Optical Engineering, Vol. 39 No. 4, April 2000


-

-

i-

p~xi uxSi![p~xi uxNi

!, ~5!

wherexSiis the scene reconstruction anywhere, but pixei ,

andxNiis the scene reconstruction in neighborhood syst

Ni , centered on pixeli , to be defined according to anm’ thorder MRF~typically, second order; for more details refeto Refs. 8 and 17!. According to Equations~1!, ~2! and~5!we can write10

p~xuy!5p~xi ,xSiuy!

5p~xi uxSi,y!p~xSi

uy!

}p~yuxi ,xSi!p~xi uxSi

!p~xSiuy!

5p~yux!p~xi uxSi!p~xSi

uy!

5p~yux!p~xi uxNi!p~xSi

uy!

}p~yi uxi !p~xi uxNi!p~ xSi

uy!}p~yi uxi !p~xi uxNi!. ~6!

To maximize the left side of Equation~6! ICM pursuesmaximization of the right side of Equation~6!,p(yi uxi)pi(xi uxNi

), which guarantees that posteriorp(xuy)never decreases at any maximization step, i.e., suboptconvergence to a local maximum ofp(xuy) is ensured. Inother words, ICM estimates, at every pixeli in the image,the labelxi that maximizes the right side of Equation~6!where only pixel valueyi and the labels of the pixel neighbors, xNi

, are required. An optimization procedure thscans the image iteratively relates the ‘‘hats’’ in Equati~6! to the use of estimated label assignments from the pvious iteration in the current iteration, such that batch laupdating can be enforced at the end of each raster sThus, ICM is an iterative suboptimal Bayesian MAP prcedure that guarantees convergence to a local maximump(xuy) in just a few batch processing cycles~about six10!.Then, ICM must be started with a good initialization of thlabel scenex, which is often provided by a Bayesian noncontextual ML classifier such as HCM.10 When texturescan be discriminated over large regions, i.e., over regicontaining many pixels, as is likely in high resolution images, ICM is likely to be trapped3,6 in a local minimum ofthe cost function in Equation~1!. Moreover, ICM has beencriticized for its heavy dependence on initial classificatioand because it ignores the soft membership~degree of com-patibility! with which a pixel may be associated to mothan one class.10,18

With regard to Equation~5!, MRF parameters, i.e., theneighborhood size and the clique potentials, controlmodel capability of detecting image structures, basedstructures’ shape and size~e.g., linear elements19!. In theliterature, several MRF parameter estimation strategiesbe found, where MRF parameters are either user-definedata-driven on ana priori basis, i.e., by means ooff-line,8,10,20 or on-line10 parameter estimation techniqueemploying supervised data~i.e., known ground truth data!.In Ref. 20, the most appropriate neighborhood systemthe pixel under analysis is estimated to preserve imagetails based on supervised data. In Refs. 4 and 12, soft


the

ap-d ias

go-be

uid

beit-lsonoel.

-be-

sti-tin

s-

-c-

de-by

of

l,anu-

of

.,

--

del

-iteits

izext--

as

r-

n

l-

.

mtheon-d-

ofheg

by-alAC

fea-st-

p

ab-gea-

ngx-

ua--

foreAC

pa-ex-e

the

liartis

ts afor

4Cingdis-n 8.


Downl

mates of distribution parameters are computed viaexpectation-maximization~EM! algorithm.21,22 In Refs. 5and 23, coarse-to-fine multiresolution segmentationproaches are proposed while no adaptive neighborhooemployed, although in Ref. 6 clique potentials are fixeda function of scale. The multiresolution segmentation alrithm proposed in Ref. 6 is found to be less likely totrapped in local minima than the ICM algorithm,10 since, ateach resolution, regions are classified and used to gfiner resolutions. To summarize,adaptive and multiresolu-tion approachesto parameter estimation appear tohighly desirable in contextual labeling algorithms exploing MRF models. By analogy, these properties should ahold true in unsupervised learning frameworks, wheresupervised data are available for training the fitting mod

2 Algorithms for Image Segmentation

When only observablesyi , i 51, . . . ,N, are available, several ~unsupervised! image segmentation algorithms canfound in the literature.24–26In the context provided by Section 1, the Pappas adaptive clustering~PAC! algorithm forimage segmentation,9 which is a Bayesian contextual MAPprocedure exploiting an adaptive and multiresolution emate of system parameters, can be considered interesPAC’s functional features are summarized below.

1. To maximize Equation~1! while reducing computa-tion complexity, PAC is based on the ICM procesing scheme, see Equation~6!. As a consequence:

a. Like HCM and ICM, PAC provides hard pixel labeling. From an information processing perspetive, hard decisions are less effective than softcisions. By adapting system parameters, i.e.,varying the size of an estimation window~seepoint 2!, PAC is expected to mitigate the effectscrisp pixel labeling.4

b. Like ICM, PAC is a batch, iterative, suboptimacontextual, Bayesian MAP procedure whereMRF model is adopted to enforce spatial continity in pixel labeling, see Equation~6!. As a conse-quence, also PAC requires a good initializationthe label field~see Section 1!, which is pursuedwith a ~noncontextual! clustering procedure, e.gHCM. This also implies that the numberC of cat-egories~clusters! is an important user-defined parameter of PAC, affecting its initial label distribution.

2. PAC employs a spectral class-conditional mosimilar to Equation~3!, i.e., PAC applies only topiecewise constant or slowly varying intensity images that may be corrupted by an additive whGaussian noise field independent of the scene. Inspectral class-conditional model, by varying the sof an estimation window, PAC computes a contedependent~adaptive! estimate of the spectral parameters. In deeper detail, in Equation~3! PAC replacesnoncontextual spectral parametersm(xi), xiP$1,C%,i P$1,N%, with context-dependent estimatesm i(xi).In other words, unlike Equation~3!, PAC enables thesame region~label! type to feature different intensityaverages in different parts of the image, as long


s

e

g.

they are separated in space.9 Therefore, PAC is morerobust~insensitive! than HCM to changes in the usedefined number of input clusters.9

3. It employs a multiresolution framework where aMRF two-point clique potential~smoothing param-eter! is automatically adapted to the scale level. Mutiresolution analysis has been proved9 to speed up thecomputation and improve the performance of PAC

4. The main deficiency of PAC is that it removes frothe scene any genuine but small region whateveruser-defined smoothing parameter may be. As a csequence, its application domain is limited to proviing sketches or caricatures of the original image.9

The goal of this paper is to present a modified versionthe PAC~MPAC! segmentation scheme that improves tpattern-preserving capability of PAC without employinany MRF model supporting special features~e.g., thinlines19!. This objective is feasible because, as underlinedseveral authors,3,9 an MRF model by itself is not very useful unless we provide a good model for class-conditiondensity. On a test set of images we intend to compare Pand MPAC against existing segmentation techniquesturing a wide variety of functional properties. These exiing techniques are~1! well-known ~noncontextual! cluster-ing algorithms, such as HCM and the self-organizing ma27

~SOM!; ~2! the simulated annealing~SA! procedure,28

which is a general purpose approach for detecting thesolute minimum of a cost function, employed to converto the global minimum of a nonadaptive version of Equtions ~1! and ~3!, i.e., system parameters are fixed duriprocessing, while contextual information is employed eclusively to model the label fieldp(x), i.e., this informationis ignored in the spectral class-conditional model of Eqtion ~3!; and~3! a slightly modified version of the stochastic expectation maximization~SEM! algorithm,12 which is asoft labeling Bayesian contextual procedure and therecomplementary to hard labeling Bayesian contextual Pand MPAC processing schemes.

Since HCM and SOM are~noncontextual! clustering al-gorithms, their presentation is beyond the scope of ourper, which focuses on image segmentation algorithmsploiting contextual information. For more details on thstandard and well-known SA schedule we also referreader to the existing literature~e.g., see Ref. 28!. Withregard to SEM, since many readers may not be famiwith it, and since our version of SEM is slightly differenfrom the published one, a complete section of this workdedicated to its presentation.

This paper is organized as follows: Section 3 presennonadaptive Bayesian contextual MAP cost functionimage segmentation, to be minimized with SA; Sectionreviews the PAC algorithm; in Section 5, the MPAscheme is proposed; in Section 6 the SEM soft labelprocedure is summarized. Experimental results arecussed in Section 7; conclusions are presented in Sectio

3 Realization of a Nonadaptive BayesianContextual MAP Segmentation Procedure

When the 2-D random field~stochastic process! p(x) is anylocally dependent MRF, then Equation~5! holds. If pair-

909Optical Engineering, Vol. 39 No. 4, April 2000


-

s-ertof

he

npleura

on

e

. In

o

aln

-

theua-ionl-

tion

edthmws-edee

huf-dan

.e.,e,

Ac-nceretan

e

ingpti-

to.e.,ateld

ex-

that

ver-onsitive

el

a.

rort

toace.n-on.if-

y aesded

ardelr


Downl

wise interactions in a second-order MRFp(x) are consid-ered exclusively~i.e., three- and four-point cliques are ignored!, and a single smoothing parameter~two-point cliquepotential! b.0 is employed~i.e., all neighbor pairs aretreated equally!, then,9,10,15

pi~xi uxNi!}exp@a~xi !1bui~xi !#, ~7!

wherea(xi), related to one-point clique potentials, is theapriori knowledge of the relative likelihood of category asignmentxi ~Ref. 9!. In the rest of this paper, we consida(xi)50, ;xiP$1,c%, i.e., all pixel states are assumedbe equally likely. Counterui(xi) is the current number o8-adjacency neighbors ofi having labelxi and is termed‘‘self-aura measure.’’17 This measure increases when tseparability between cluster types~which can be related topure substances or fluids! increases, i.e, when the commoboundary between different clusters decreases. Commentary to the self-aura measure is the ‘‘cross-ameasure,’’17 v i(xi), such thatui(xi)1 v i(xi)58. By com-bining Equations~3! and ~7! with the log of Equation~6!we obtain the final form of the suboptimal iterative solutito the maximization of Equation~1!, which is

xi5arg minxiP$1,C%

$g@yi2m~xi !#21b• v i~xi !%, ; i PS, ~8!

where g51/2s2 is the free parameter controlling thamount of detail detected by the algorithm~which increasesass decreases!, while b is fixed. Vice versa, ifg is fixed to1 then b is considered the free parameter such thatb}s2. This is the case considered in the rest of this paperfact, coefficientsg andb are inversely related: reducingg~increasings! while b is kept constant is equivalent tincreasingb while g is kept constant, and vice versa.9 If sincreases, then Equation~8! finds a solution diminishinglyclose to the raw data. Ifs50 ~noiseless case!, thenb50,i.e., Equation~8! is equivalent to a Bayesian noncontextuML classifier. The first term on the right side of Equatio~8!, called theerror term,16 constrains intensity of the region ~label! typem(xi) to be close to observed datayi , i.e.,it represents the goodness of fit of the region type todata.10 The second weighted term on the right side of Eqtion ~8! estimates the degree of smoothness of the solutwhich increases with the spatial continuity in pixel labeing.

With regard to system parameter adapation in Equa~8!, in Ref. 10, it is noted that the exact value of weightbis usually unimportant with ICM if smaller values are uson earlier iterations, so that at an early stage the algorifollows the data, while at later stages the algorithm follothe region model.9 However, in many practical applications, b is kept constant,6,9,10 because the schedule needfor changing this parameter would require additional frparameters that are generally image dependent.6 If the 2-Dstochastic processp(x) is to support special features sucas thin lines, then pairwise interaction models do not sfice, and smoothing parametersbs, as well as neighborhoosize, must be adapted depending on the type of scenepossibly, on local image properties.10,19,20



-

,

d,

If SA is applied to minimize Equation~8! whereb iskept constant, then SA converges to a global minimum, iit provides an optimal, nonadaptive, hard-competitivBayesian, contextual, MAP segmentation procedure.cording to the principles of simulated annealing, the chaof random label assignments decreases as temperatuTdecreases.28 At each iteration,T is lowered by a constancooling rated. Pixels are visited according to a raster scand updated in batch~synchronous! mode at the end ofeach cycle.10 Unfortunately, SA is expected to convergmuch too slowly to be practically useful.6

SA optimal minimization of Equation~8! when param-eterb is fixed, i.e., system parameterb is nonadaptive, willbe included in the set of existing techniques, featurcomplementary functional features, against which subomal and adaptive MPAC must be compared.

4 Pappas Contextual Clustering for ImageSegmentation

As already mentioned in Section 2, PAC applies onlypiecewise constant or slowly varying intensity images, ismooth images with little useful texture information, thmay be corrupted by an additive white Gaussian noise fiindependent of the scene.9 In PAC, Equation~8! becomes

xi5arg minxiP$1,C%

$g@yi2mWi~xi !#

21b• v i~xi !%, ; i PS, ~9!

whereg51/2s2 is the free parameter, whileb is fixed to0.5. Vice versa, ifg is fixed to 1 thenb becomes the freeparameter such thatb5s2. In Equation~9! the spectralmodel for class-conditional density is adaptive and conttual. It employs a slowly varying intensity functionmWi

(xi)estimated as the average of the gray levels of all pixelscurrently belong to region typexi and fall inside a windowWi which is centered on pixeli . The window size de-creases monotonically as the algorithm approaches congence to guarantee robust estimation of intensity functias the segmentation becomes progressively more sensto local image properties.9 Exploitation of a decreasingwindow size should mitigate deficiencies in crisp labassignments.4 When the number of pixels of typexi withinwindow Wi is less than window widthWi ,w , then estimatemWi

(xi) is not considered reliable and pixeli cannot be

assigned to region typexi . Thus, isolated regions with aresmaller thanWi ,w are removed by the clustering algorithmAs shown in Ref. 9, the new spectral model of the erterm proposed in Equation~9! makes PAC more robusthan HCM in the choice of the number of clusters,29 be-cause regions of entirely different intensities can belongthe same category as long as they are separated in sp

In Ref. 9, a PAC hierarchical multiresolution implemetation is proposed to reduce the amount of computatiThis implementation constructs a pyramid of images at dferent resolutions by low pass filtering and decimating bfactor of 2. At each level in the pyramid, the algorithm usthe segmentation generated at the previous level, expanby a factor of 2, as a starting point. Additive noise standdeviation s is reduced by half when the resolution levdecreases, i.e., parameterb reduces by a factor of fou~while g is kept fixed to 1!, thenthe algorithm follows the


ve

isnges,

Cailsion

e,

e-nesof

itye-

la-


Downl

data at low resolution stages. This multiresolution ap-proach, besides reducing computation time, may improfeature-preserving capability~for details, refer to Ref. 9!.The PAC iterative scheme at a given resolution levelshown in Fig. 1. For implementation details on reducicomputation time in estimating the local intensity averagrefer to Ref. 9.

5 Modification of Pappas’ Algorithm

Our main objective is to improve the ability of the PAprocessing scheme in preserving small but genuine detBased on heuristics, we propose the following cost functas an adaptation of Equation~9!,

xi5arg minxiP$1,C%

$n~xi !1b• v i~xi !%, ; i PS, ~10!

such that

Fig. 1 PAC algorithm for image segmentation at a given resolutionlevel.


.

D~xi !55min$@yi2mWi

~xi !#2,@yi2mW~xi !#

2%,

if mWi~xi ! exists and is considered reliable;~11!

@yi2mW~xi !#2, if mWi

~xi ! does not

exist or is considered unreliable; ~12!

whereWi andmWi(xi) are defined as in Equation~9!, while

W identifies the fixed window that covers the entire imagsuch thatmW(xi) is the global~imagewide! estimate of theaverage gray value of all pixels currently belonging to rgion typexi . A scheme of the MPAC algorithm at a giveresolution level is shown in Fig. 2. The main differencbetween MPAC and PAC emerging from the comparisonFigs. 1 and 2 and Equations~9! to ~12!, respectively, can besummarized as follows:

1. Rather than alternating estimation of local intensparameters and pixel labels, MPAC alternates btween estimating local intensity parameters, pixelbels and global intensity parameters.

2. According to Equation~12!, when a local intensityaveragemWi

(xi), estimated in neighborhoodWi,W

Fig. 2 MPAC algorithm for image segmentation at a given resolu-tion level.



dityn

-

h

e

on

h

rd-allthansanoft

a-

xi-tathe

the

teds-iso

ic

l

.

be-ix-

r

two

u-erm

1.


Downl

centered on pixeli , does not exist or is considereunreliable, then the estimate of the global intensaveragemW(xi) is employed instead for comparisowith the pixel data. In line with Equation~9!, localestimatemWi

(xi) is not considered reliable by Equa

tion ~10! when the number of pixels of typexi withinwindow Wi is less than the adaptive window widtWi ,w . Exploitation of Equation~12! is sufficient toprevent MPAC from removing isolated but genuinregions whose area is smaller thanWi ,w .

3. In MPAC, when local intensity estimatemWi(xi) ex-

ists and is considered reliable by Equation~10!, bothlocal and global intensity estimates, mWi

(xi) and

mW(xi), respectively, are employed for compariswith the pixel data according to Equation~11!. Notethat while testing MPAC we found images to whicthe proposed version of Equation~11! applies suc-cessfully, while a simpler version of Equation~11!exploiting local estimatesmWi

(xi) exclusively, doesnot.

6 SEM Algorithm

From an information processing perspective, hacompetitive PAC and MPAC, enforcing a winner-take-assignment strategy, are expected to be less effective~fuzzy! segmentation algorithms employing soft decisioin estimating pixel labels. An interesting example ofiterative suboptimal segmentation method exploiting spixel labeling is the SEM algorithm.12 The traditional EMalgorithm performs maximum likelihood estimation of prameters for some observed incomplete data set~for moredetails on EM, refer to Refs. 13, 21, and 22!. SEM is acontextual adaptation of the EM algorithm applied to mamize the joint probability of an observed incomplete dadistribution modeled as a Gaussian mixture. Based onobservation that adjacent pixels are likely to belong tosame region~label!, priors in the original EM algorithm aremodeled as a second-order MRF in SEM. The presenversion of SEM differs from its published version in asuming that the Gaussian mixture of observable data istropic and in some changes underlined below~for directcomparison of SEM with the EM equations for an isotropGaussian mixture, see Ref. 13, p. 67!.

Step 0. Initialize Gaussian parameters@mc(t50) ,sc

(t50)#,c51, . . . ,C, and local priors a i ,c

(t50) , i 51, . . . ,N, c51, . . . ,C, whereN is the total number of pixels andC isthe total number of class labels.

Step 1 (E step). Posterior probability estimate.

hi ,c(t11)5

a i ,c(t)IG@xi ,mc

(t) ,sc(t)#

( l 51C a i ,l

(t)IG@xi ,m l(t) ,s l

(t)#,

i 51, . . . ,N, c51, . . . ,C, ~13!

such that isotropic Gaussian IG~•! is a class-conditionalikelihood ~goodness of fit! equal to



n

-

IG~xi ,mc ,sc!51

~2psc2!d/2 expS 2

1

2sc2 ixi2mci2D , ~14!

whered is the dimensionality of the pixel intensity space

Step 2 (M step). Label mean estimate.

mc(t11)5

( i 51N hi ,c

(t11)xi

( i 51N hi ,c

(t11) , c51, . . . ,C. ~15!

Step 3 (M step). Label variance estimate.

~sc2!(t11)5

1

d

( i 51N hi ,c

(t11)ixi2mc(t11)i2

( i 51N hi ,c

(t11) , c51, . . . ,C.

~16!

Step 4 (M step). Estimate of local prior probability~thislocal versus global computation is the main differencetween SEM and traditional EM for isotropic Gaussian mtures!.

a i ,c(t11)5

exp@( j ( i )518 b i ,sj

hsj ,c(t11)1hi ,c

(t11)#

( l 51C exp@( j ( i )51

8 b i ,sjhsj ,l

(t11)1hi ,l(t11)#

,

i 51, . . . ,N, c51, . . . ,C, ~17!

where sj5sj ( i ) is a pixel belonging to a second-ordeneighborhood systemNi centered on pixeli , while the two-point clique potentialb i ,sj

is equal to

b i ,sj

5511, if pixel pair „i ,sj ( i )… is aligned either

horizontally or vertically

11/&, if pixel pair „i ,sj ( i )… is aligned neither

horizontally nor vertically.

It is important to stress that Equation~17! differs fromEquation ~7! in Ref. 12 in two terms:~1! Equation ~17!presents no negative sign before the arguments of theexponents; and~2! in Equation ~17!, to account for theprobabilistic membership of the central pixel, the argments of the two exponents present a second additive tthat is absent from Equation~7! in Ref. 12.

Step 5. If termination is not reached, return to Step


b-

e-Me.

ita-ongrebecalosslesallym-ets

cepatioff

ron

t be

al-andhishatragchr o

ble.eryul

he

m-atina-

se-

cdif-resasfacus-

eex-

eg-ndg

,o-n

p-of

he,

ft-

rd-al,

ner-all

ef.ri-t tos

ToA,

d


Downl

To stress the relationship with the EM algorithm, oserve that by replacing Equation~17! with

ac(t11)5

1

N (i 51

N

hi ,c(t11) , c51, . . . ,C, ~18!

and symbola i ,c(t11) in Equation~13! with ac

(t11) , the clas-sical EM algorithm for isotropic Gaussian mixtures is rcovered. Finally, note that, unlike PAC and MPAC, SErequires no smoothing parameter, i.e., it is easier to us

7 Quantitative Evaluation of Segmentation

Most picture partitions are evaluated visually and qualtively on a subjective, perceptual basis. This is due, amother things, to the fact that traditional supervised measuemployed in image processing tasks, such as the mislaing rate, are global statistics unable to account for lovisual properties. For example, the segmentation of a crboard picture may improve when the number of small hodecreases even when its mislabeling rate actuincreases.23 Therefore, these global statistics cannot be eployed for quantitative evaluation of pictursegmentation.30 Since no single segmentation goal exisbecause of the subjective appraisal of continuous pertual features, a system developed to compare segmentresults must employ1,2: ~1! an entire set of measures osuccess~termedbattery test! to account for the fuzziness operceptual segmentation;~2! a test set of imagesto enableexploitation of supervised data, i.e., ofa priori knowledgeabout the objects in a scene, so that the external enviment ~supervisor! provides the generic~vague! segmenta-tion task with an explicit goal; and~3! a set of existingtechniques against which the proposed algorithm muscompared.

Let us start by defining the battery test. As we are deing with image segmentation algorithms that minimizegiven cost function, i.e., algorithms that are well-posed afeature one explicitly defined segmentation objective, tfunction is included in the battery test. Other statistics tare considered in the segmentation comparison are avevalue of the error term, number of cycles required to reaconvergence, number of label replacements and numbemisclassified pixels when ground truth data are availaNote that computation time is not considered in the batttest since MPAC and PAC share the same hierarchical mtiresolution implementation, which significantly reduces tamount of computation.

The test set of images must consist of a sufficient nuber of real and standard data sets capable of demonstrthe potential utility of MPAC, i.e., these images must feture piecewise constant or slowly varying intensity. Welected a standard achromatic human face~‘‘Lena’’ !, athree-band Satellite Pour l’Observation de la Terre~SPOT!high resolution visible~HRV! image, and a tomographiimage provided with supervised data fields. Because officulties in comparing alternative classification proceduin a meaningful way, the first application uses a faceopposed to natural scenes because we know what alooks like and can therefore judge the results of the cltering algorithm intuitively.


sl-

-

-n

-

e

f

-

g

e

To demonstrate its potential utility in different imagsegmentation applications, MPAC is compared againstisting techniques. These are general purpose~noncontex-tual! clustering algorithms as well as contextual image smentation algorithms based on probabilistic theory afeaturing a wide variety of functional properties. Existintechniques employed for comparison are

1. ~noncontextual! clustering algorithms such as HCMwhich is a Bayesian noncontextual ML labeling prcedure~see Section 1!, and SOM, whose descriptiois beyond the scope of this paper.13,27

2. SA, which is a general purpose minimization aproach, employed to detect the absolute minimumEquation~8! ~Ref. 28!. This segmentation approacis optimal, nonadaptive, spectral, hard-competitivBayesian, contextual, MAP~see Sections 1 and 3!.

3. SEM, which is suboptimal, adaptive, spectral, socompetitive, Bayesian, contextual~see Section 6!.

4. PAC, which is suboptimal, adaptive, spectral, hacompetitive, hierarchical, Bayesian, contextuMAP, like MPAC ~see Sections 4 and 5!.

7.1 Standard Image Application

The standard achromatic input image of ‘‘Lena’’ is showin Fig. 3. Six category templates are fixed by a photointpreter: m(1)568 ~corresponding to surface classes: wshadow, hair!, m(2)5100 ~hair!, m(3)5143 ~wall, hat!,m(4)5160 ~skin!, m(5)5190 ~hat!, andm(6)5213 ~wall,skin!. This set of templates is larger than suggested in R9, where two to four clusters were employed to obtain cacatures of the original images. These templates are inpunoncontextual HCM to provide segmentation algorithmunder testing with an initial segmentation to start from.highlight functional differences between procedures SPAC and MPAC, relating to Equations~8!, ~9! and ~10!respectively, free parameterb is kept low (b50.5).

Case 1: SA. According to the principles of simulate

Fig. 3 Standard achromatic input image of ‘‘Lena.’’



de

-a-atonn.uape

el,

b-

in

,EMownies

ea-. 7hile

of


Downl

annealing, the chance of random label assignmentscreases as temperatureT decreases.28 At each iteration,T islowered by a constant cooling rated. Pixels are visitedaccording to a raster scan and updated in batch~synchro-nous! mode at the end of each cycle.10 Figure 4 shows theoutput of the SA algorithm exploiting parametersT5800,d50.95, b50.5, tmax5150, wheretmax is the epoch atwhich the algorithm is stopped.28 Since weightb of theterm enforcing spatial continuity in pixel labeling in Equation ~8! is small, then, as expected, minimization of Eqution ~8! provides a segmentation result quite similar to thgenerated by HCM at the initialization step. For this reasthe initial segmentation provided by HCM is not showFigures 5 and 6 show the plots of the mean value of Eqtion ~8! and the percentage number of replacements

Fig. 4 Output of the SA algorithm applied to Fig. 3. SA parametersare: T5800, d50.95, b50.5, tmax5150. This final segmentation isindistinguishable from the initial segmentation provided by a (non-contextual) HCM clustering algorithm.

Fig. 5 Plot of the SA algorithm applied to Fig. 3: mean value ofEquation (8).



-

-r

pixel ~multiplied by factor 100!, respectively. All plots arein terms of the epoch number. Sinceb is small, i.e., thealgorithm follows the data rather than the region modthen minimization of Equation~8! is mostly focused onminimization of the error term. In agreement with this oservation, the mean value of the error term in Equation~8!was found to be almost identical to the mean cost shownFig. 5.

Case 2: SEM. Figure 7 shows the output image of SEMwhich employs no user-defined smoothing parameter. Sreaches convergence after about 10 iterations: this is shin Fig. 8 where the average value of posterior probabilitbased on hard label decisions is adopted as a possible msure of convergence. Overall, clustering shown in Figseems satisfactory by featuring smooth boundaries wimage details are preserved. However, a comparison

Fig. 6 Plot of the SA algorithm applied to Fig. 3: percentage numberof replacements per pixel.

Fig. 7 Output of the SEM algorithm applied to Fig. 3. SEM param-eter is: tmax5150.


re

re

o-

CCth

ee-f

r

thisn-

heet-

ig.theeredhe


Downl

Figs. 4 and 7 reveals that SEM preserves small imagegions that are blob-like~see the woman’s eyes!, but tendsto remove thin but genuine string-like regions that wedetected by the initialc-means clustering step~see thewoman’s hat!. Final templates estimated by the SEM algrithm are reported hereafter:m(1)568.9, m(2)5109.0,m(3)5139.4, m(4)5159.7, m(5)5182.3, and m(6)5205.4.

Case 3: PAC. Figure 9 shows the output of the PACalgorithm exploiting parametersb50.5 and tmax5150.Figures 10 and 11 provide meaningful plots of this PAapplication at full resolution. These figures show that PAreaches convergence after about 10 iterations, althoughcost function tends to oscillate once its asymptote has breached. Since weightb of the term enforcing spatial continuity in pixel labeling is small, then minimization o

Fig. 8 Plot of the SEM algorithm applied to Fig. 3: average value ofthe posterior probability for hard label decisions.

Fig. 9 Output of the PAC algorithm applied to Fig. 3. PAC param-eters are: b50.5, and tmax5150.


-

en

Equation~9! is mostly focused on minimization of the erroterm. This was proved by measuring, in Equation~9!, themean error term as a function of the processing epoch;function, which is not shown, was found to be almost idetical to the mean total cost shown in Fig. 10. Although terror term in PAC is designed to preserve local details bter than the error term in Equation~8!, small details arewashed out in Fig. 9 while they are well preserved in F4. This phenomenon must be caused exclusively byPAC policy of removing all regions whose size is smallthan the width of the estimation window. To be comparwith initial centroid values, final templates estimated by tPAC algorithm are reported hereafter:m(1)570.4, m(2)5107.9, m(3)5138.3, m(4)5160.7, m(5)5185.8, andm(6)5205.6.

Fig. 10 Plot of the PAC algorithm applied to Fig. 3: mean value ofEquation (9).

Fig. 11 Plot of the PAC algorithm applied to Fig. 3: percentagenumber of replacements per pixel.



e

in

a-teridm

ughiteCrs

ith

-

em

rea-

areith

dis-


Downl

Case 4: MPAC. Figure 12 shows the output of thMPAC algorithm exploiting parameterb50.5 and tmax

5150. Small details are preserved better in Fig. 12 thanFigs. 4, 7 and 9~see the woman’s hat and left eye!. Figures13 and 14 provide meaningful plots of this MPAC appliction at full resolution. MPAC reaches convergence afabout 10 iterations. To be compared with initial centrovalues, final templates estimated by the MPAC algorithare m(1)569.0, m(2)5107.2, m(3)5138.3, m(4)5160.9, m(5)5186.0, andm(6)5206.9. Note that thesemean values are similar to those detected by PAC, althothe two algorithms account for local visual properties qudifferently. To compare the smoothing properties of MPAwith those of SEM, MPAC was run with input parameteb59 ~s53 gray levels! and tmax510. This run generatesthe output image shown in Fig. 15, to be compared w

Fig. 12 Output of the MPAC algorithm applied to Fig. 3. MPACparameters are b50.5 and tmax5150.

Fig. 13 Plot of the MPAC algorithm applied to Fig. 3: mean value ofEquation (10).



Fig.7. Smoothness of borders~i.e., the length between common boundaries of different clusters! in Fig. 15 is compa-rable to that shown in Fig. 7~see the background wall!,while MPAC maintains its ability to detect small blob-likand thin string-like regions that have been removed froFig. 7 ~see the woman’s hat and left eye!.

Note. As expected, SA reaches termination much moslowly than SEM, PAC and MPAC. Moreover, segmenttion effectiveness of Equation~8! being minimized by SAis kept low by the fact that spectral model parametersnonadaptive. These theoretical obervations, combined wexperimental results in the standard image application,courage the use of SA in further experiments.

Fig. 14 Plot of the MPAC algorithm applied to Fig. 3: percentagenumber of replacements per pixel.

Fig. 15 Output of the MPAC algorithm applied to Fig. 3. MPACparameters are b59 and tmax510.


o

neaof

perthe7,re-i-ithmon-seht

ta-toMtis

18is

d laualen

-to

Micts

ingnd

righbutby

,

C

0ed17.ted,

notarethe

e

urecor-ls,ve

ralsedig.t

es


Downl

7.2 Unsupervised Satellite Image Application

A multispectral SPOT HRV image of the city of PortAlegre ~Rio Grande do Sul, Brazil!, acquired on Nov. 7,1987, is employed. Spectral bands are green, red andIR ~see Fig. 16!, respectively. We underline the presencethree bay bridges linked to the large island in the upleft-hand corner of this image. A zoomed area aroundcity airport extracted from Fig. 16 is shown in Fig. 1where a large number of genuine, structured and thingions are visible. Without explaining in detail what physcal correspondences a desirable segmentation algorshould produce, we focus our qualitative comparisonhow well SEM, PAC and MPAC~SA is ignored, see Section 7.1! account for local visual properties such as thesmall structured regions, either blob- or string-like. Eigcategory templates are fixed by a photointerpreter:m(1)5(49, 43, 18)~water!, m(2)5(39, 30, 66)~vegetation!,m(3)5(46, 30, 112) ~vegetation!, m(4)5(49, 42, 36)~airport, asphalt!, m(5)5(66, 66, 85) ~street!, m(6)5(207, 193, 152) ~metal roofs!, m(7)5(50, 46, 62)~houses!, m(8)5(57, 51, 62) ~houses!. These templatesare input to noncontextual HCM to provide the segmention algorithms under testing with an initial segmentationstart from. To relate the segmentation performance of SEPAC and MPAC to their initial condition, the HCM outpuimage is shown in Fig. 18, where one bridge out of threecompletely missing. The zoomed area taken from Fig.and corresponding to Fig. 17 is shown in Fig. 19. Thclustered image shows the presence of several isolatebels which are typical of salt-and-pepper noncontextclassification. To highlight functional differences betwePAC and MPAC, free parameterb in Equations~9! and~10! is set to zero~therefore, Equation~8! becomes equivalent to the cost function minimized by HCM, leadingFig. 18!. For all algorithms, parametertmax is fixed at 20.

Case 1: SEM. Figure 20 shows the output of the SEalgorithm after 20 processing epochs, while Fig. 21 dep

Fig. 16 SPOT HRV image of the city of Porto Alegre: band 3 (nearIR).


r

,

-

the zoomed area extracted from Fig. 20 and correspondto Fig. 17. Both images show that small blob-like areas athin string-like structures have been removed~e.g., thebridges!, while only a few thick linear regions are left. Ouconclusion is that SEM seems incapable of preserving hresolution image details, especially when these are thingenuine string-like structures. Final templates estimatedthe PAC algorithm arem(1)5(49.2, 43.3, 20.195730)m(2)5(39.9, 29.5, 71.5), m(3)5(45.8, 35.3, 92.7),m(4)5(49.9, 44.9, 47.9), m(5)5(66.6, 65.4, 77.7),m(6)5(98.3, 94.0, 85.6),m(7)5(50.3, 44.9, 68.7), andm(8)5(59.0, 55.4, 61.6).

Case 2: PAC. Figure 22 shows the output of the PAalgorithm exploiting parametersb50 and tmax520. Theasymptote of the cost function~9! is reached after about 2iterations at full resolution. Figure 23 shows the zoomarea extracted from Fig. 22 and corresponding to Fig.Although the smoothing parameter has been disactivasmall but genuine details~blob- or string-like! are lost. Thisundesired effect is totally due to the fact that PAC doesallow pixel assignments to a class whose local statisticsconsidered unreliable. Final templates estimated byPAC algorithm are m(1)5(49.1, 43.2, 21.0), m(2)5(41.6, 31.6, 73.2), m(3)5(45.4, 33.7, 97.1), m(4)5(50.2, 45.1, 44.7), m(5)5(67.3, 66.0, 79.9), m(6)5(162.9, 149.5, 114.3),m(7)5(50.5, 45.3, 65.3), andm(8)5(58.9, 55.0, 62.6).

Case 3: MPAC. Figure 24 shows the output of thMPAC algorithm exploiting parametersb50 and tmax

520. The asymptote of the cost function of Equation~10!is reached after about 15 iterations at full resolution. Fig25 shows the zoomed area extracted from Fig. 24 andresponding to Fig. 17. In Fig. 25, many isolated pixerather than being filtered out, as occurred in Fig. 23, habeen linked to neighboring pixels featuring similar spectsignatures. Figure 24 features not only overall increasharpness with respect to initial conditions depicted in F18, but also includes~recovers! several image details thawere completely absent in Fig. 18~e.g., the third bridge!.To be compared with initial centroid values, final templatestimated by the MPAC algorithm are:m(1)5(49.2, 43.3, 20), m(2)5(41.4, 31.6, 73.1), m(3)5(45.1, 33.6, 96.4), m(4)5(49.2, 44.1, 45.3), m(5)

Fig. 17 Zoomed area around the city airport extracted from Fig. 16.




Downl

Fig. 18 Initial segmentation of the SPOT image, obtained by a non-contextual c-means clustering algorithm.

Fig. 19 Zoomed area taken from Fig. 18 and corresponding to Fig.17.



Fig. 20 Output of the SEM algorithm applied to the SPOT image.Termination epoch: tmax520.

Fig. 21 Zoomed area taken from Fig. 20 and corresponding to Fig.17.



Downl

Fig. 22 Output of the PAC algorithm applied to the SPOT image.PAC parameters are: b50 and tmax520.

Fig. 23 Zoomed area extracted from Fig. 22 and corresponding toFig. 17.


Fig. 24 Output of the MPAC algorithm applied to the SPOT image.MPAC parameters are b50 and tmax520.

Fig. 25 Zoomed area extracted from Fig. 24 and corresponding toFig. 17.



cale-id

sedyhavthe-

ton

ers

,

-byrmare

,d asrn.edat-

d 2m-

v-fier.sses

susth

op-al,fors.

aler-ACthe

rtsPACr of

l

nd


Downl

5(70, 68.5, 78), m(6)5(148.6, 137.2, 105.1), m(7)5(50.8, 45.8, 65.4),m(8)5(59.6, 55.9, 63.8).

7.3 Supervised Medical Image Application

A different experiment, consisting of a supervised mediimage application, is set up as follows. A multiplprototype classifier31 is implemented as a two-stage hybrclassification system.32 It consists of a ‘‘hybrid’’ sequenceof an unsupervised~data-driven learning! first stage, imple-mented as a clustering algorithm, followed by a supervi~error-driven learning! second stage exploiting a majoritvote mechanism, i.e., relating each cluster to the classing the largest number of representatives insidecluster.13 A SOM is employed as a well-known and efficient first stage per-pixel~noncontextual! clusteringscheme.27

A three-band magnetic resonance image~MRI! of ahorizontal section of a brain is shown in Figs. 26–28~band1: T1 MP-RAGE; band 2: T2 spin-echo, SP; band 3: prodensity, PD!. Regions belonging to six classes~tissues! ofinterest are manually selected by expert photointerpret

Fig. 26 MRI, band 1: T1 MP-RAGE.

Fig. 27 MRI, band 2: T2 SP.



-

.

These classes are~see Fig. 29! white matter, gray mattercerebral spinal fluid, lesions, background, other~bone, fat,thalamus: white!.

SOM is employed to extract 22 clusters~i.e., statisticalregularities! from the MRI 3-D histogram. Then, each region ~label! type is assigned to one supervised classmajority vote. All categories relating to the same class foa so-called metacategory, i.e., six metacategoriesformed ~one for each class!, as shown in Fig. 30. Nextcluster centers detected by noncontextual SOM are useinput by contextual MPAC whose smoothing parametebin Equation ~10! is set to zero to avoid user interactioFinally, each region type recomputed by MPAC is relatto one supervised class by majority vote. Six new metacegories are formed, as shown in Fig. 31. Tables 1 anshow that, without requiring any additional system paraeter, the sequence of noncontextual~pixel-based! SOMwith contextual MPAC enables an improvement in the aerage classification performance of the two-stage classiAnalogous results have been obtained when eleven claof interest are selected in the MRI image~SOM1MPACaverage classification performance scoring 64.3 ver61.5% of SOM!, and when satellite images provided wiground truth regions are classified.

8 Conclusions

Unsupervised PAC and MPAC procedures are both subtimal, adaptive, spectral, hard-competitive, hierarchicBayesian, contextual, MAP segmentation algorithmspiecewise constant or slowly varying intensity imageMPAC differs from PAC in its spectral class-conditionmodel where global and local estimates of intensity avages are employed simultaneously. Advantages of MPwith respect to other segmentation algorithms found inliterature are that~1! by enabling the same region~label!type to feature different intensity averages in different paof the image, as long as they are separated in space, Mis less sensitive to changes in the user-defined numbeinput clusters than~noncontextual! clustering algorithms~see Section 2!; and~2! although it employs no MRF modesupporting special image features~e.g., thin lines!, MPACpreserves image details better than HCM, SEM, PAC a

Fig. 28 MRI, band 3: PD.



Downloaded From: http://o

Table 1 Medical image classification with SOM; confusion matrix.

Input Class

Total Pixels Purity (%)1 2 3 4 5 6

Metacategory 1 1232 164 4 119 0 505 2024 60.9

Metacategory 2 26 723 28 90 0 250 1117 64.7

Metacategory 3 0 10 1168 89 0 74 1341 87.1

Metacategory 4 5 2 32 157 0 1 197 79.7

Metacategory 5 0 0 0 0 727 322 1049 69.3

Metacategory 6 412 395 19 24 121 1928 2899 66.5

Total Pixels 1675 1294 1251 479 848 3080 8627

Efficiency (%) 73.6 55.9 93.4 32.8 85.7 62.6 68.8

Table 2 Medical image classification with SOM followed by MPAC; confusion matrix.

Input Class

Total Pixels Purity (%)1 2 3 4 5 6

Metacategory 1 1213 60 2 40 0 374 1689 71.8

Metacategory 2 23 737 35 91 0 243 1129 65.3

Metacategory 3 0 16 1168 85 0 77 1346 86.8

Metacategory 4 5 1 24 160 0 1 191 83.8

Metacategory 5 0 0 0 0 770 275 1045 73.7

Metacategory 6 434 480 22 103 78 2110 3227 65.4

Total Pixels 1675 1294 1251 479 848 3080 8627

Efficiency (%) 72.4 57.0 93.4 33.4 90.8 68.5 71.4

ible

.g.

ima

C

-the

ctis

erfinsesting

ra-

n-

e-be

C,is.

lu-

e

G.r,go-

sed

dage

m

ed

the SA algorithm employed to minimize Equation~8! ~seeSection 7!. In unsupervised image processing, a possuse of the proposed class-conditional spectral model is~1!in cascade to any noncontextual clustering algorithm, eHCM, SOM or LBG-U ~Ref. 33! ~see Section 7.3!; and~2!in Bayesian segmentation algorithms based on suboptapproaches.

Theoretical failure modes and limitations of the MPAalgorithm are

1. MPAC applies only to images with little useful texture and additive Gaussian noise independent ofscene~see Sections 2 to 5!.

2. It is incapable of detecting outliers, which may affethe estimate of spectral parameters. This problemrelated to the fact that MPAC employs hard raththan soft decision rules.4,12 Future developments oMPAC should employ soft decision strategiespixel labeling to identify pure pixels, mixed pixeland misclassified cases, this information being necsary in map accuracy assessment and/or for direcground surveys.12,34

3. Although it is less sensitive~more robust! to changesin the user-defined number of input clusters than tditional ~noncontextual! clustering algorithms,MPAC is still a suboptimal clustering procedure sesitive to initial conditions~see Section 2!. Therefore,

pticalengineering.spiedigitallibrary.org/ on 09/24/2013 T

,

l

-

one main issue in user interaction with MPAC rmains the choice of the number of clusters todetected.5,29

4. It exploits a higher degree of heuristics than PAi.e., MPAC features a statistical framework, whichsomehow less rigorous than that featured by PAC

Acknowledgments

We are grateful to the anonymous referees for their vaable comments in improving the quality of this paper.

References1. L. Delves, R. Wilkinson, C. Oliver, and R. White, ‘‘Comparing th

performance of SAR image segmentation algorithms,’’Int. J. RemoteSens.13~11!, 2121–2149~1992!.

2. A. Hoover, G. Jean-Baptiste, X. Jiang, P. J. Flynn, H. Bunke, D.Goldgof, K. Bowyer, D. W. Eggert, A. Fitzgibbon, and R. B. Fishe‘‘An experimental comparison of range image segmentation alrithms,’’ IEEE Trans. Pattern. Anal. Mach. Intell.18~7!, 673–688~1996!.

3. Y. Jhung and P. H. Swain, ‘‘Bayesian contextual classification baon modifiedM -estimates and Markov Random Fields,’’IEEE Trans.Geosci. Remote Sens.34~1!, 67–75~1996!.

4. J. Zhang, J. W. Modestino, and D. A. Langan, ‘‘Maximum likelihooparameter estimation for unsupervised stochastic model-based imsegmentation,’’IEEE Trans. Image Process.3~4!, 404–419~1994!.

5. A. H. Schistad Solberg, T. Taxt, and A. K. Jain, ‘‘A Markov randofield model for classification of multisource satellite imagery,’’IEEETrans. Geosci. Remote Sens.34~1!, 100–113~1996!.

6. C. Bouman and B. Liu, ‘‘Multiple resolution segmentation of texturimages,’’ IEEE Trans. Pattern. Anal. Mach. Intell.13~2!, 99–113~1991!.




Downl

Fig. 29 MRI, areas of interest.

Fig. 30 Multiple-category classification of Figs. 26–28, based onSOM.



Fig. 31 Multiple-category classification of Figs. 26–28, based onSOM followed by MPAC.


or

ons,

en-

nd

to-es,’put

d

ural

and

ian

ing.

ageap-

d

t

eca-

,’’

ch-

od-

d

r of’’ in

el,-

uesortCA

-

i-ship


Downl

7. C. Bouman and M. Shapiro, ‘‘A multiscale random field model fBayesian image segmentation,’’IEEE Trans. Image Process.3~2!,162–177~1994!.

8. S. Geman and D. Geman, ‘‘Stochastic relaxation, Gibbs distributiand the Bayesian restoration of images,’’IEEE Trans. Pattern. Anal.Mach. Intell.6~6!, 721–741~1984!.

9. T. N. Pappas, ‘‘An adaptive clustering algorithm for image segmtation,’’ IEEE Trans. Signal Process.40~4!, 901–914~1992!.

10. J. Besag, ‘‘On the statistical analysis of dirty pictures,’’J. R. Stat.Soc. B48~3!, 259–302~1986!.

11. H. Derin and H. Elliott, ‘‘Modeling and segmentation of noisy atextured images using Gibbs random fields,’’IEEE Trans. Pattern.Anal. Mach. Intell.9~1!, 39–55~1987!.

12. J. Dehemedhki, M. F. Daemi, and P. M. Mather, ‘‘An adaptive schastic approach for soft segmentation of remotely sensed imagin Series in Remote Sensing: 1, Proc. Int. Workshop on Soft Coming in Remote Sensing Data Analysis, E. Binaghi, P. A. Brivio, and A.Rampini, Eds., pp. 211–221, World Scientific, Singapore~1996!.

13. C. M. Bishop,Neural Networks for Pattern Recognition, ClarendonPress, Oxford~1995!.

14. A. K. Jain and R. C. Dubes,Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs, NJ~1988!.

15. R. Chellappa and A. Jain, Eds.,Markov Random Fields: Theory anApplication, Academic, New York~1993!.

16. F. Girosi, M. Jones, and T. Poggio, ‘‘Regularization theory and nenetworks architectures,’’Neural Comput.7, 219–269~1995!.

17. I. Elfadel and R. Picard, ‘‘Gibbs random fields, cooccurrences,texture modeling,’’IEEE Trans. Pattern. Anal. Mach. Intell.16~1!,24–37~1994!.

18. P. B. Chou and C. M. Brown, ‘‘The theory and practice of Bayesimage labeling,’’Int. J. Comput. Vis.4, 185–210~1990!.

19. D. Geman and B. Jedynak, ‘‘An active testing model for trackroads in satellite images,’’IEEE Trans. Pattern. Anal. Mach. Intell18~1!, 1–13~1996!.

20. P. C. Smits and S. G. Dellepiane, ‘‘Synthetic aperture radar imsegmentation by a detail preserving Markov Random Fieldproach,’’ IEEE Trans. Geosci. Remote Sens.35~4!, 844–857~1997!.

21. A. P. Dempster, N. M. Laird, and D. B. Rubin, ‘‘Maximum likelihoofrom incomplete data via the EM algorithm,’’J. R. Stat. Soc. Ser. B39, 1–38~1977!.

22. R. M. Neal and G. E. Hinton, ‘‘A view of the EM algorithm thajustifies incremental, sparse, and other variants,’’ inLearning inGraphical Models, M. I. Jordan, Ed., pp. 355–368, NATO SciencSeries D: Behavioural and Social Sciences, Vol. 89, Kluwer Ademic, Dordrecht, The Netherlands~1998!.

23. J. Liu and Y. Yang, ‘‘Multiresolution color image segmentationIEEE Trans. Pattern. Anal. Mach. Intell.16~7!, 689–700~1994!.

24. R. M. Haralick and L. G. Shapiro, ‘‘Survey: image segmentation teniques,’’ Comput. Vis. Graph. Image Process.29, 100–132~1985!.

25. K. S. Fu and J. K. Mui, ‘‘A survey on image segmentation,’’PatternRecogn.13, 3–16~1981!.

26. A. Rosenfeld and L. S. Davis, ‘‘Image segmentation and image mels,’’ Proc. IEEE67~5!, 764–772~1979!.

27. T. Kohonen,Self-Organizing Maps, Springer Verlag, Berlin~1995!.28. M. Sonka, V. Hlavac, and R. Boyle,Image Processing, Analysis an

Machine Vision, Chapman & Hall, London~1993!.29. S. Medasani and R. Krishnapuram, ‘‘Determination of the numbe

components in Gaussian mixtures using agglomerative clustering,Proc. IEEE Int. Conf. on Neural Networks ’97, pp. 1412–1417, Hous-ton, TX ~1997!.

30. B. D. Ripley, ‘‘Statistics, images, and pattern recognition,’’Can. J.Stat.14~2!, 83–111~1986!.

31. J. C. Bezdek, T. Reichherzer, Gek Sok Lim, and Y. Attikiouz‘‘Multiple-prototype classifier design,’’IEEE Trans. Syst. Man Cybern.28~1!, 67–79~1998!.

32. A. Baraldi and N. A. Borghese, ‘‘Learning from data: general issand special applications of radial basis function networks,’’ RepTR-98-028, International Computer Science Institute, Berkeley,~1998!.

33. B. Fritzke, ‘‘The LBG-U method for vector quantization—an improvement over LBG inspired from neural networks,’’Neural Pro-cess. Lett.5~1!, 83–111~1997!.


’-

34. G. M. Foody, N. A. Campbell, N. M. Trodd, and T. F. Wood, ‘‘Dervation and applications of probabilistic measures of class memberfrom the maximum-likelihood classification,’’PE&RS58~9!, 1335–1341 ~1992!.

Andrea Baraldi graduated in electronic engineering from the Uni-versity of Bologna, Italy, in 1989. His master thesis focused on thedevelopment of unsupervised clustering algorithms for optical satel-lite imagery. From 1989 to 1990 he was a research associate atCIOC-CNR, an Institute of the National Research Council (CNR) inBologna, and served in the army at the Istituto Geografico Militare inFlorence, working on satellite image classifiers and GIS. As a con-sultant at ESA-ESRIN in Frascati, Italy, he worked on object-oriented applications for GIS from 1991 to 1993. From December1997 to June 1999 he was with the International Computer ScienceInstitute, Berkeley, California, with a postdoctoral fellowship in arti-ficial intelligence. Since his master’s thesis he has continued hiscollaboration with ISAO-CNR in Bologna, where he currently worksas a research associate. His main interest is low-level vision pro-cessing, with a special emphasis on texture analysis and neuralnetwork applications.

Palma Blonda graduated in physics from the University of Bari in1980, and her research activity has since been in the area of imageprocessing, with an application to remotely sensed data. In 1984she joined the Institute for Signal and Image Processing (IESI) atthe Italian National Research Council (CNR), in Bari, Italy. She isinvolved in research studies on both synthetic aperture radar andTM data analysis for land cover mapping. With neuroradiologistsfrom the University of Bari she is also involved in a research projecton the segmentation of magnetic resonance images. Her researchinterests include supervised and unsupervised image processing,fuzzy logic and neural networks.

Flavio Parmiggiani graduated in physics from the University of Mi-lan in 1970. From 1970 to 1982, he was with the Italian NationalResearch Council, Milan, working in the field of biological cybernet-ics. From 1978 to 1980, he was with the Laboratory of Neurophysi-ology, University of Alberta, Edmonton, Canada. In 1978, he joineda new National Research Council (CNR) Institute, IMGA-CNR,working in the field of remote sensing and satellite image process-ing. He is responsible for the AVHRR receiving station installed forreal-time operations at the Italian Base in Antartica. In 1992 and1993, he was with the Scott Polar Research Institute, University ofCambridge, United Kingdom, on the problem of sea-ice dynamicsusing both AVHRR and ERS-1 synthetic aperture radar images.

Giuseppe Satalino received the laurea degree in computer sciencefrom the University of Bari, Italy, in 1991, with a thesis on neuralnetworks. In 1991 he was a ‘‘summer student’’ at the EuropeanOrganization for Nuclear Research (CERN) in Geneva, for applica-tions of neural networks to high energy physics. Since 1993 he hasbeen with the Institute for Signal and Image Processing (IESI) of theNational Research Council (CNR) in Bari, where he has been work-ing on research projects dealing with neural networks applied toradar and medical images, remotely sensed data and physics ex-perimental data. His research focuses on neural network applica-tions for image processing and data classification. He is currentlyinvolved in the application of supervised neural networks architec-tures to the inversion of backscattering models in the framework ofASI contracts.



Date post:	14-Dec-2016
Category:	Documents
Upload:	palma
View:	214 times
Download:	1 times

Contextual clustering for image segmentation

Documents