+ All Categories
Home > Documents > [IEEE 2013 11th International Workshop on Content-Based Multimedia Indexing (CBMI) - Veszpr m ,...

[IEEE 2013 11th International Workshop on Content-Based Multimedia Indexing (CBMI) - Veszpr m ,...

Date post: 08-Dec-2016
Category:
Upload: maha
View: 212 times
Download: 0 times
Share this document with a friend
6
Improved segmentation of a series of remote sensing images by using a fusion MRF model Tamas Sziranyi Distributed Events Analysis Research Laboratory MTA SzTAKI, Hungary 1111 Budapest, Kende u. 13/17. Email: [email protected] Maha Shadaydeh Distributed Events Analysis Research Laboratory MTA SzTAKI, Hungary 1111 Budapest, Kende u. 13/17. Email: [email protected] Abstract—Classifying segments and detection of changes in terrestrial areas are important and time-consuming efforts for remote-sensing image repositories. Some country areas are scanned frequently (e.g. year-by-year) to spot relevant changes, and several repositories contain multi-temporal image samples for the same area in very different quality and details. We propose a Multi-Layer Markovian adaptive fusion on Luv color images and similarity measure for the segmentation and detection of changes in a series of remote sensing images. We aim the problem of detecting details in rarely scanned remote sensing areas, where trajectory analysis or direct comparison is not applicable. Our method applies unsupervised or partly supervised clustering based on a cross-image featuring, followed by multilayer MRF segmentation in the mixed dimensionality. On the base of the fused segmentation, the clusters of the single layers are trained by clusters of the mixed results. The improvement of this (partly) unsupervised method has been validated on remotely sensed image series. I. I NTRODUCTION Earth observation based on aerial and satellite image series, including high-resolution satellite image time-series (SITS) [1], results in a large data volume containing several known and maybe unknown details. The high resolution in space, and the different sampling rate in time leads to having SITS of regular (e.g. monthly) or irregular [1] timing samples. The sample series can be very rare and irregular [2], where usual time-series’ evaluation methods cannot be applied. The increasing volume of remote sensing repositories needs unsupervised comparison and change detection methods. For high-resolution satellite image time-series (SITS), [1] presents an information mining concept which enables a user to learn and retrieve spatio-temporal structures in SITS. The con- cept is based on a hierarchical Bayesian modeling of SITS information of spatio-temporal structures. The hierarchy is composed of two inference steps: an unsupervised modeling of dynamic clusters resulting in a graph of trajectories, and an interactive learning procedure based on graphs which leads to the semantic labeling of spatio-temporal structures, performed on a SPOT image time-series. The definition of changes is usually related to some su- pervised based segmentation method; a semantic [3] or a statistical [2] definition may help in defining terrestrial details of semantic meaning into land cover classes. These semantic rules, linking the low-level features of clusters and segments to the high-level map labels manually developed by a human operator, however, have the lack of generality for occurring new details not defined before. Exploring remote sensing (aerial or satellite) time-series, the images in the series are usually different in light and weather conditions, season, traffic, flooding or blooming conditions. For this reason some type of segmentation, based on object [4], structure [5] or pixel [6] level decision, should precede the comparison of the different time-layers. It means that the one-layer segmentation (often Markov Random Fields, [7][2] should be parametrized by some preliminary cluster-consensus method, where the map of clusters related to the different layers are assigned to a common basis. In case of human interaction defining semantic labels (e.g. in [3] ) it is really helpful if the clustering is steady over the time. If new details related to new clusters occur then it should be detected and the cluster schema should be updated. For this reason a multi-layer classification or change detection method is better to apply an unsupervised or semi-supervised label clustering procedure. For Markov Random Field [7], unsupervised labeling is often used [6] [8]. If the number of clusters is undefined, reversible jump MCMC sampling [9] can be applied for getting an optimum of labeling. Hierarchical models or tree structures are also applicable for Markov chain models for unsupervised texture segmentation [10],[11]. For remote sens- ing tasks a color segmentation method, performed by using the unsupervised TS-MRF algorithm described in [11], can be successfully used, but this method is divisive, meaning largely unbalanced clusters. This problem is partly solved in [5] by a graph based representation that allows for the set of color-uniform fragments the definition of neighborhood relationships, by measuring the context similarity between the fragments associated with the link vertices. Graph cuts with automatically determined threshold carry out the segmentation of texture patterns. The last state-of-the-art results suggest that a higher level complexity of data hierarchy may lead to automatic unsuper- vised methods to avoid supervised human interaction. It is also a need when comparing two or more arbitrary time-samples (images) in a time-series that we should get the changes as labeling differences, and the trajectory analysis [1] of such a time-series is not applicable because of the rare (low sampling 2013 11TH INTERNATIONAL WORKSHOP ON CONTENT-BASED MULTIMEDIA INDEXING (CBMI) 17 – 19. JUNE 2013 VESZPR ´ EM, HUNGARY 978-1-4799-0956-8/13/$31.00 ©2013 IEEE 137
Transcript

Improved segmentation of a series of remotesensing images by using a fusion MRF model

Tamas SziranyiDistributed Events Analysis Research Laboratory

MTA SzTAKI, Hungary1111 Budapest, Kende u. 13/17.

Email: [email protected]

Maha ShadaydehDistributed Events Analysis Research Laboratory

MTA SzTAKI, Hungary1111 Budapest, Kende u. 13/17.

Email: [email protected]

Abstract—Classifying segments and detection of changes interrestrial areas are important and time-consuming efforts forremote-sensing image repositories. Some country areas arescanned frequently (e.g. year-by-year) to spot relevant changes,and several repositories contain multi-temporal image samplesfor the same area in very different quality and details. We proposea Multi-Layer Markovian adaptive fusion on Luv color imagesand similarity measure for the segmentation and detection ofchanges in a series of remote sensing images. We aim the problemof detecting details in rarely scanned remote sensing areas,where trajectory analysis or direct comparison is not applicable.Our method applies unsupervised or partly supervised clusteringbased on a cross-image featuring, followed by multilayer MRFsegmentation in the mixed dimensionality. On the base of thefused segmentation, the clusters of the single layers are trainedby clusters of the mixed results. The improvement of this (partly)unsupervised method has been validated on remotely sensedimage series.

I. INTRODUCTION

Earth observation based on aerial and satellite image series,including high-resolution satellite image time-series (SITS)[1], results in a large data volume containing several knownand maybe unknown details. The high resolution in space,and the different sampling rate in time leads to having SITSof regular (e.g. monthly) or irregular [1] timing samples. Thesample series can be very rare and irregular [2], where usualtime-series’ evaluation methods cannot be applied.

The increasing volume of remote sensing repositories needsunsupervised comparison and change detection methods. Forhigh-resolution satellite image time-series (SITS), [1] presentsan information mining concept which enables a user to learnand retrieve spatio-temporal structures in SITS. The con-cept is based on a hierarchical Bayesian modeling of SITSinformation of spatio-temporal structures. The hierarchy iscomposed of two inference steps: an unsupervised modelingof dynamic clusters resulting in a graph of trajectories, and aninteractive learning procedure based on graphs which leads tothe semantic labeling of spatio-temporal structures, performedon a SPOT image time-series.

The definition of changes is usually related to some su-pervised based segmentation method; a semantic [3] or astatistical [2] definition may help in defining terrestrial detailsof semantic meaning into land cover classes. These semanticrules, linking the low-level features of clusters and segments

to the high-level map labels manually developed by a humanoperator, however, have the lack of generality for occurringnew details not defined before.

Exploring remote sensing (aerial or satellite) time-series, theimages in the series are usually different in light and weatherconditions, season, traffic, flooding or blooming conditions.For this reason some type of segmentation, based on object[4], structure [5] or pixel [6] level decision, should precedethe comparison of the different time-layers. It means that theone-layer segmentation (often Markov Random Fields, [7][2]should be parametrized by some preliminary cluster-consensusmethod, where the map of clusters related to the differentlayers are assigned to a common basis. In case of humaninteraction defining semantic labels (e.g. in [3] ) it is reallyhelpful if the clustering is steady over the time. If new detailsrelated to new clusters occur then it should be detected and thecluster schema should be updated. For this reason a multi-layerclassification or change detection method is better to apply anunsupervised or semi-supervised label clustering procedure.

For Markov Random Field [7], unsupervised labeling isoften used [6] [8]. If the number of clusters is undefined,reversible jump MCMC sampling [9] can be applied forgetting an optimum of labeling. Hierarchical models or treestructures are also applicable for Markov chain models forunsupervised texture segmentation [10],[11]. For remote sens-ing tasks a color segmentation method, performed by usingthe unsupervised TS-MRF algorithm described in [11], canbe successfully used, but this method is divisive, meaninglargely unbalanced clusters. This problem is partly solvedin [5] by a graph based representation that allows for theset of color-uniform fragments the definition of neighborhoodrelationships, by measuring the context similarity between thefragments associated with the link vertices. Graph cuts withautomatically determined threshold carry out the segmentationof texture patterns.

The last state-of-the-art results suggest that a higher levelcomplexity of data hierarchy may lead to automatic unsuper-vised methods to avoid supervised human interaction. It is alsoa need when comparing two or more arbitrary time-samples(images) in a time-series that we should get the changes aslabeling differences, and the trajectory analysis [1] of such atime-series is not applicable because of the rare (low sampling

2013 11TH INTERNATIONAL WORKSHOP ON CONTENT-BASED MULTIMEDIA INDEXING (CBMI) • 17 – 19. JUNE 2013 • VESZPREM, HUNGARY

978-1-4799-0956-8/13/$31.00 ©2013 IEEE 137

rate) data sequence.In this paper, remote sensing areas of mixed feature sets

are examined; the goal is to automatically detect the yearlytransmuting indefinite subclass areas by using more samplelayers, where the overlapping combination of subclasses canbe collected in a multilayer superclass [12] for overall seg-mentation. This multilayer superclass segmentation may helpin layer-by-layer segmentation and change detection later. Inaddition to the stacking of the layers’ color and texture infor-mation, we propose to apply fusion to block-wise similaritymeasures calculated between each pair in a series of remotesensing images. Similarity measures provide useful tool forchange detection and tracking analysis when dealing withimage time series data that comes from different sensors.So far, different similarity measures have been proposed,such as distance to independence, mutual information, clusterreward algorithm, Kullback Leibler divergence, (see [13] andreferences therein). In this work, we propose to use the ClusterReward Algorithm (CRA) [14] as it gives better segmentationand change detection results than other similarity measuressuch as correlation and mutual information.

II. MARKOV MODEL FOR MULTI-FRAME SEGMENTATION

First, we briefly give an introduction to the theory of MRF[15][16], then we describe a multilayer image model used inthe following sections.

An image S = s1, s2, ...sN is considered to be a two-dimensional grid of pixels (sites), with a neighborhood systemon the lattice: G = Gs | s ∈ S is a neighborhood system forS if

1) s /∈ Gs2) s ∈ Gr ⇔ r ∈ GsLet X = Xs : s ∈ S denote any family of random

variables so that ∀s ∈ S : Xs ∈ Λ, where Λ = 1, . . . ,Mis a common state space of label-set assigning image classes.Furthermore, let Ω = ω = (ωs1 , . . . , ωsN ) : ωsi ∈ Λ, 1 ≤i ≤ N be the set of all possible configurations. X is a MRFwith respect to G if

1) for all ω ∈ Ω: P (X = ω) > 0,2) for every s ∈ S and ω ∈ Ω:

P (Xs = ωs | Xr = ωr, r 6= s) = P (Xs = ωs | Xr =ωr, r ∈ Gs).

The function in (2) is called the local characteristics ofthe MRF, and the probability distribution P (X = ω) ofany process satisfying (1) is uniquely determined by theseconditional probabilities. However, it is extremely difficult todetermine these characteristics in practice. Gibbs distributionand the Hammersley-Clifford theorem provides us a simpleway to overcome this problem.

A Gibbs distribution relative to the neighborhood system Gis a probability measure π on Ω with the following represen-tation:

π(ω) =1

Zexp

(−E(ω)

T

), (1)

where Z is the normalizing constant or partition function:

Z =∑ω

exp

(−E(ω)

T

),

T is a constant called the temperature, and the energy functionE is of the form

E(ω) =∑C∈C

EC(ω).

Each EC is a function defined on Ω depending only on thoseelements ωs of ω for which s ∈ C. The restriction of ω to thesites of a given clique C is denoted by ωC . Such a functionis called a potential. One of the most important theorem isprobably the Hammersley-Clifford theorem which points outthe relation between MRF and Gibbs distribution:X is a MRF with respect to the neighborhood system G if

and only if π(ω) = P (X = ω) is a Gibbs distribution withrespect to G.

Using the above theorem, the definition of the MRF iscompleted by the knowledge of the clique potentials EC(ωC)for every C in C and every ω in Ω.

The image segmentation is equivalent to a global labelingΩ = ωs | s ∈ S. As it is typical, the label field Ω is modeledas a Markov Random Field based on [7].

A. Multilayer Markov model

The image data at pixel s is characterized by a J dimen-sional feature vector xs. Examples J = 3 in case of describingthe pixels by their Luv color values, or by additional texturefilters with e.g. J = 12 :

xs = [x3C(s), x

9T (s)]

T (2)

where x3C(s) contains the first three elements with the Luv

color components of the pixel, and x9T (s) is a microstructural

response of the 9 Laws filters (see more in Sect. III-A). SetX = xs|s ∈ S marks the global image data.We use a Maximum A Posteriori (MAP) estimator for thelabel field, where the optimal labeling Ω, corresponding tothe optimal segmentation, maximizes the probability:

P (Ω|X) ∝ P (X|Ω) · P (Ω) (3)

The key point in the model is to define the conditionaldensity functions pk(s) = P (xs|ωs = k), for all k ∈ Λ ands ∈ S.

The εk(s) = − log pk(s) terms can be directly derived fromthe one dimensional marginal probabilities:

εk(s) = C+1

2log |det Σk|+

1

2(x(s)−µk(s))TΣ−1

k (x(s)−µk(s))

(4)with C = 6 · log 2π.

IMPROVED SEGMENTATION OF A SERIES OF REMOTE SENSING IMAGES BY USING A FUSION MRF MODEL

138

B. MRF optimizationThe MAP estimator in eq. (3) is realized by combining

a conditional independent random field of signals and anunconditional Potts model [17]. The optimal segmentationcorresponds to the global labeling, Ω, defined by

Ω = argminΩ

∑s∈S− logP (xs|ωs)︸ ︷︷ ︸

εωs (s)

+∑r,s∈S

Θ(ωr, ωs) (5)

where the minimum is searched over all the possible segmen-tations (Ω) of a given input frame, and Θ(ωr, ωs) = 0 if sand r are not neighboring pixels, otherwise:

Θ(ωr, ωs) =

0 if ωr = ωs+β if ωr 6= ωs

In our application we used a graph cut based α-expansionalgorithm for energy minimization, with the implementationaccompanying [18].

III. THE PROPOSED MULTILAYER FUSION MODELS

Based on the multilayer fusion MRF procedure, two mainmethods are presented in the following. The first one is appliedbased on the fusion of stack of features, resulting in properunsupervised single layers segmentation and change detection.The second one additionally uses similarity measure amongthe different layers when building up the previous multilayerstructure. It results in a better segmentation result in case ofmultiple but unsupervised subclasses.

A. Fusion MRF: multilayer segmentation and change detec-tion

A pixel in any layer is represented by the 3 color values(x3C(s)) and/or the kernel responses of B (max. 9) texture

filters (xBT (s)). The pixels of the multiple layers of imagesare characterized by the stack of these vectors of lengthJ(= 3 +B):

xMLns = xL1

s , xL2s , ...xLn

s (6)

where xLis is the xs feature vector of layer Li. The

microstructure detection (xBT (s)) in eq. (2) is done byusual zero-mean kernels (∀s :

∑r∈Ns

as(r) = 0) as ageneralization of simple first-order edge features by [19]. Inthe following we use the kernels of Laws-filters.

The segmentation and change detection procedure containsthe following main steps:

1) Selecting and registering the image layers. In case ofprofessional data suppliers orthonormed and geographi-cally registered images are given; no further registrationis needed. In our method no color-constancy or anyshape/color semantic information is needed; the colorof the corresponding areas and the texture can differstrongly layer-by-layer.

2) Finding clusters in the set of (xMLns ) vectors by K-

means algorithm. The statistical data (mean and covari-ance) for the fusion based ”super-clusters” (k ∈ Λ) aregiven;

3) Running MRF segmentation (as eq.5) on the fused layerdata (xMLn

s ) with the super-cluster parameters, resultingin a multilayer labeling ΩML;

4) Single-layer training: the multilayer labeling ΩML isused as a preliminary training map for each image layer.For each label k ∈ Λ the corresponding subspace of the(xMLns ): xLi

s , s ∈ S, i ∈ [1...n] are collected;5) For each single layer a MRF segmentation is processed,

resulting a labeling: ΩLi ;6) The consecutive image layers (..., (i − 1), i, ...) are

compared to find the changes: ∆(i− 1, i).The result of this process is ∆(i − 1, i): Change detection

between layers of high in-class and inter-class variability.

B. Cross-layer blockwise similarity measures

Change detection methods based on radiometry measure-ment alone are not useful when dealing with image timeseries data that comes from different sensors such as opticaland synthetic aperture radar. In such case, similarity measuresprovide useful tool for change detection and image times seriesanalysis. In this work, we propose to use the Cluster RewardAlgorithm (CRA) [14] as similarity measure. The CRA(I, J)of two images I, and J is calculated using the joint histogrampI,J and the marginal histograms pI , pJ as follows [13]:

CRA(I, J) =

∑i,j p

2IJ(i, j)−

∑i p

2I(i) ·

∑j p

2J(j)√∑

i p2I(i) ·

∑j p

2J(j)−

∑i p

2I(i) ·

∑j p

2J(j)

(7)The value of CRA(I, J) is large when there is high correlationbetween the two images or the joint histogram has little disper-sion. The CRA similarity measure is chosen as it gives bettersegmentation and change detection results than other similaritymeasures such as correlation and mutual information. This isdue to the fact that joint histogram estimation noise has weakinfluence on the CRA values and thus smaller window sizecan be used [13], which in turn enables detection of changesin small areas.

The new idea in the proposed segmentation algorithm is toapply the fusion model MRF on CRA similarity measurescalculated between each pair in a series of remote sensingimages. In our presentation here we used three consecutiveimages only, however the algorithm can be easily extendedto more layers. The segmentation procedure is carried out asfollows:

1) Selecting and registering the image layers.2) For each pair of three consecutive images I(t −

1), I(t), I(t + 1), the CRA image is calculated. Inthe calculation of each pixel of the CRA image, weuse D × D-pixel estimation window to calculate thehistograms; The window size can be varied according tothe required scale of change detection. Each CRA imageis then normalized to have values in the range [0, 1].Let the obtained CRA images be CRAt−1,t, CRAt,t+1,CRAt−1,t+1.

3) In the color space Luv, let xt(s) denotes the L colorvalue of pixel s in image I(t). Construct a combined

2013 11TH INTERNATIONAL WORKSHOP ON CONTENT-BASED MULTIMEDIA INDEXING (CBMI) • 17 – 19. JUNE 2013 • VESZPREM, HUNGARY

139

feature vector for pixels s in the three images I(t −1), I(t), and I(t+ 1) as follows:

xt(s) = [xt−1(s) + αCRAt−1,t(s), xt(s)

+αCRAt,t+1(s), xt+1(s) + αCRAt−1,t+1(s)]

(8)

where α is a positive scalar, usually set to have valueclose to the maximum value in the processed images.

4) Defining training areas. Evaluating the (xt(s)) vectorson the training areas, the statistical data (mean andcovariance) for the fusion based clusters are given;

5) Running MRF segmentation on the fused layer data(xt(s)) resulting in a multilayer labeling Ωt−1,t,t+1;

6) Single-layer training: the multilayer labeling Ωt−1,t,t+1

is used as a preliminary training map for each image.7) For each single layer a MRF segmentation is processed,

resulting in a labeling: Ωt; In this step, The featurevector of each pixel consists of its three Luv color valuesonly.

IV. EXPERIMENTAL RESULTS

The method is validated on aerial images from very differentscanning time instants and seasonal conditions, having com-plex (multiple pattern) classes. First, we show segmentationresults of a complex meadow/forest image-series, based on theprocedures in sections III-A and III-B. Then a multiclass (hav-ing 4 labels) segmentation is demonstrated by using similaritymeasure to find unsupervised classes in a partly supervisedmultilayer segmentation procedure. Classes are sand, meadow,forest, and water textures.

A. Two class change detection

A practically difficult example is shown (first in [12]) forthe image series in Fig.1. We can see the new forest partin 2005 and a cut of forest in the year 2007 on a verycomplex area with high in-class variability. The unsupervisedMRF segmentation (Sect. III-A) on the fused images isapplied following a K-means clustering of the fused data.The segmented fusion labeling is then fed into the singlelayers for training the Gaussian mixture models of the inlayerclusters, resulting in appropriate MRF segmentation of forestand meadow areas. The differences of the forest/meadow masklocate the new forest or forest-cut areas in Fig.2/up. The CRAbased blockwise segmentation and change detection results ina better segmented greater scaled comparison for the samearea. While local features (texture and color) help us to findsalient changes, the cross-layer blockwise modality resultsin semantically more reasonable detection of changes. Thismultilayer based change detection in Fig.2/bottom gives usthe precise locations of the new forest and the forest-cut in theappropriate scale. We have also tested the single-layer basedMRF segmentation and change detection in similar conditions,with similar results in case of supervised cluster definition;however, with the proposed fusion method we can achieveappropriate unsupervised segmentation.

(2000) (2005) (2007)Fig. 1. Aerial photos about a new-forest/forest-cut area (Szada, Hungary,photos by FOMI) from the years 2000, 2005 and 2007. On the mid-rangethe new forest of 2005 is found, while on right-up corner another part of theforest is cut in 2007.

Unsupervised segmentation and change detection by using texture and color info

Changes through 2000-2005 Changes through 2005-2007Unsupervised segmentation and change detection by using CRA cross-layer measure and color info

Fig. 2. Detection of changes (colored) as new-forest between imagesFigs.1/(a)-(b) in years 2000-2005 (left), and forest-cut between imagesFigs.1/(b)-(c) in years 2005-2007 (right).The unsupervised segmentation is done by multi-layer fusion MRF modelfollowed by K-means clustering of the fused data. The segmented fusionlabeling is then fed into the single layers for segmentation and finallychange detection among the single layers. Upper row: using texture & colorinformation, Bottom row: CRA cross-layer info & color information

B. Multi-class segmentation of image time series using cross-layer similarity measures

The proposed cross-layer feature of Sect.III-B was appliedto the three aerial images shown in Fig. 4 (Row 1). The imagesare from different scanning time conditions. They consist offour main classes, meadow, forest, river, and sand areas. Theimages from left to right are taken in years 2000, 2005, and2007, respectively. The small island that appears in year 2000image, does not exist in the other two images. The riverscontain dense vegetation that is very different from one yearto the other; this makes it difficult to identify the water class

IMPROVED SEGMENTATION OF A SERIES OF REMOTE SENSING IMAGES BY USING A FUSION MRF MODEL

140

Fig. 3. Training areas used in the segmentation process, Meadow (M), Forest(F), Sand (S), and River (R) in the Tiszadob area (by FOMI)

based on color or texture alone. Fig. 3 shows the selectedtraining areas in our experiments; since only one subclassper class is applied, subclasses (e.g. oxbow sections in 2007)are to be trained from the similarity measure among layers.We compare the performance of the multilayer MRF fusedsegmentation with that of MRF based segmentation performedon each single layer independently without fusion. The MRFis used as explained in Section II-A. The comparison is carriedout with and without including the CRA values in the featurevector to highlight the effect of CRA values. That is, we testedfour methods for comparison,

• Single Layer MRF optimization on Luv color values onthe separate layers without the use of CRA similaritymeasures (SL-MRF).

• MRF without fusion but using Luv color values as wellas CRA similarity measures (SL-MRF-CRA).

• The proposed multilayer fusion MRF on color valuesonly, without the use of CRA similarity measures (ML-MRF).

• The proposed Fusion model MRF using Luv color valuesas well as CRA similarity measures (ML-MRF-CRA) asgiven in eq. (8).

Segmentation results are shown in Fig. 4 (Rows 2-5) insimilar order. In these experiments, we used 7 × 7-pixelestimation window for the calculation of the CRA histograms.These results show clearly that the use of fusion segmentationwith CRA values has improved the segmentation resultssignificantly. As it can be seen in these figures, the water class(the two oxbow river parts in the images) was only detectedwhen using fusion segmentation with CRA cross-layer values.

For Ground-Truth illustration purpose, we further run singlelayer MRF optimization on Luv color values on separatelayers; however, only in the MRF optimization of the 2007image, we used in addition to the Luv color values, thevalues of the 2007 infrared image. The infrared image andsegmentation results of this experiment are shown in Fig.5. These results show that the use of infrared image helpedin identifying correctly the water class, which could not beidentified in the method SL-MRF. Note however that we could,as shown in Fig. 4, obtain similar results using the proposedML-MRF-CRA without the use of the infrared image. Thatis to say, by using the fusion model MRF on CRA similaritymeasures we can avoid the use of the infrared image in theseexperiments.

Fig. 5. Ground-Truth result by a one layer fusion with infrared image tofind a subclass (different water covers):Infrared image from the year 2007 (Left) and the segmentation results for 2007(Right) using single layer MRF on Luv color values (Fig.4 topright inputlayer) fused by the infrared image. Compare the results to the 3rd column ofFig.4.

V. CONCLUSION

We have proved that using cross-image featuring may resultin better segmentation or change detection result for sparelysampled remote sensing images. Having more layers andundefined subclasses, similarity measure may help in theexploitation of the multiple information from the multiplelayers. The unsupervised or partly supervised (no predefinedsubclasses) method results in getting the common classeson the different layers, and its projection to the single-layersegmentation works as an adaptive training.

ACKNOWLEDGMENT

The authors would like to thank Miklos Homolya for takingpart in the preliminary experiments. This work has beensupported by the National Research Fund, OTKA 106.374.

REFERENCES

[1] P. Heas and M. Datcu, “Modeling trajectory of dynamic clusters inimage time-series for spatio-temporal reasoning,” IEEE Tr. Geoscienceand Remote Sensing, vol. 43, pp. 1635–1647, 2005.

[2] C. Benedek and T. Sziranyi, “Change detection in optical aerial imagesby a multilayer conditional mixed markov model,” IEEE Tr. Geoscienceand Remote Sensing, vol. 47, no. 10, pp. 3416–3430, October 2009.

[3] D. Bratasanu, I. Nedelcu, and M. Datcu, “Bridging the semantic gap forsatellite image annotation and automatic mapping applications,” IEEEJ. Selected Topics in Applied Earth Observation and Remote Sensing,vol. 4, pp. 193–204, 2011.

[4] C. Benedek, X. Descombes, and J. Zerubia, “Building developmentmonitoring in multitemporal remotely sensed image pairs with stochasticbirth-death dynamics,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34,no. 1, pp. 33–50, 2012.

[5] R. Gaetano, G. Scarpa, and T. Sziranyi, “Graph-based analysis oftextured images for hierarchical segmentation,” in British MachineVision Conference (BMVC). BMVA, 2010, p. #318.

[6] Z. Kato, J. Zerubia, and M. Berthod, “Unsupervised parallel imageclassification using markovian models,” Pattern Recognition, vol. 32,pp. 591–604, 1999.

[7] S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions andthe Bayesian restoration of images,” IEEE Trans. Pattern Analysis andMachine Intelligence, pp. 721–741, 1984.

[8] T. Sziranyi and J. Zerubia, “Markov random field image segmentationusing cellular neural network,” IEEE Tr. Circuits and Systems I., vol. 44,no. 1, pp. 86–89, 1997.

[9] Z. Kato, “Segmentation of color images via reversible jump mcmcsampling,” Image and Vision Computing, vol. 26, no. 3, pp. 361–371,2008.

[10] G. Scarpa, R. Gaetano, M. Haindl, and J. Zerubia, “Hierarchical multiplemarkov chain model for unsupervised texture segmentation,” IEEETrans. on Image Processing, vol. 18, no. 8, pp. 1830–1843, August2009.

[11] C. D’Elia, G. Poggi, and G. Scarpa, “A tree-structured markov randomfield model for bayesian image segmentation,” IEEE Trans. on ImageProcessing, vol. 12, no. 10, p. 12591273, October 2003.

2013 11TH INTERNATIONAL WORKSHOP ON CONTENT-BASED MULTIMEDIA INDEXING (CBMI) • 17 – 19. JUNE 2013 • VESZPREM, HUNGARY

141

(2000) (2005) (2007)

Fig. 4. Aerial photos around the Tiszadob oxbow area (Hungary, photos by FOMI) from the years 2000, 2005 and 2007. See the temporary island in the topin 2000, the transitions among sandy, meadow and forest areas, and the different water-colors in 2007 caused by the different vegetation covering the watersurface. The rows show from the top to down :(1) original images(2) segmentation results using single layer MRF optimization on Luv color values on the separate layers (SL-MRF)(3) segmentation results using single layer MRF optimization on Luv color values on the separate layers and CRA similarity measure values among thelayers (SL-MRF-CRA)(4) segmentation results using the proposed multilayer fusion MRF on Luv color values only (ML-MRF)(5) segmentation results using the proposed multilayer fusion MRF on color values and the CRA similarity measure values among layers (ML-MRF-CRA).

[12] T. Sziranyi and M. Homolya, Unsupervised detection of changes in aseries of remote sensing images by using a fusion MRF model - Researchreport. PPKE ITK and MTA SZTAKI, September 2012.

[13] V. Alberga, “Similarity measures of remotely sensed multi-sensor imagesfor change detection applications,” IEEE Trans. Geosci. Remote Sens.,vol. 1, pp. 122–143, 2009.

[14] J. Inglada and A. Giros, “On the possibility of automatic multisensorimage registration,” IEEE Trans. Geosci. Remote Sens., vol. 42, pp.2104–2120, 2004.

[15] Y. Rosanov, Markov Random Fields. Springer Verlag, 1982.[16] Z. Kato and J. Zerubia, Markov Random Fields in Image Segmentation.

Collection Foundation and Trends in Signal Processing. Now Editor,

World Scientific, September 2012.[17] R. Potts, “Some generalized order-disorder transformation,” in Proceed-

ings of the Cambridge Philosophical Society, no. 48, 1952, p. 106.[18] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov,

A. Agarwala, M. Tappen, and C. Rother, “A comparative study of energyminimization methods for markov random fields,” in Ninth EuropeanConference on Computer Vision (ECCV 2006), vol. 2, 2006, pp. 16–29.

[19] Y. Wang, K.-F. Loe, and J.-K. Wu, “A dynamic conditional random fieldmodel for foreground and shadow segmentation,” IEEE Trans. PatternAnalysis and Machine Intelligence, vol. 28, no. 2, pp. 279–289, 2006.

IMPROVED SEGMENTATION OF A SERIES OF REMOTE SENSING IMAGES BY USING A FUSION MRF MODEL

142


Recommended