Tumor segmentation of multi-echo MR T2-weighted images with morphological operators

W. Torresa,c, M. Martín-Landrove†b,d, M. Paluszny‡

aFundación Instituto de Ingeniería, FII, Caracas, Venezuela; bCentro de Física Molecular y Médica, Universidad Central de Venezuela, Caracas, Venezuela; cLaboratorio de Computación Gráfica y Geometría Aplicada, Centro de Geometría, Universidad

Central de Venezuela, Caracas, Venezuela; dCentro de Diagnóstico Docente Las Mercedes, Caracas, Venezuela;

eDepartamento de Matemática, Universidad Nacional de Colombia, Medellín, Colombia

e, G. Figueroac, G. Padillac

ABSTRACT

In the present work an automatic brain tumor segmentation procedure based on mathematical morphology is proposed. The approach considers sequences of eight multi-echo MR T2-weighted images. The relaxation time T2 characterizes the relaxation of water protons in the brain tissue: white matter, gray matter, cerebrospinal fluid (CSF) or pathological tissue. Image data is initially regularized by the application of a log-convex filter in order to adjust its geometrical properties to those of noiseless data, which exhibits monotonously decreasing convex behavior. Finally the regularized data is analyzed by means of an 8-dimensional morphological eccentricity filter. In a first stage, the filter was used for the spatial homogenization of the tissues in the image, replacing each pixel by the most representative pixel within its structuring element, i.e. the one which exhibits the minimum total distance to all members in the structuring element. On the filtered images, the relaxation time T2 is estimated by means of least square regression algorithm and the histogram of T2 is determined. The T2 histogram was partitioned using the watershed morphological operator; relaxation time classes were established and used for tissue classification and segmentation of the image. The method was validated on 15 sets of MRI data with excellent results.

Keywords: Mathematical morphology, segmentation, classification, pattern recognition.

1. INTRODUCTION T2-weighted Carr-Purcell-Meiboom-Gill (CPMG) MRI has been previously used1,2,3 for tissue classification and tumor segmentation, particularly for obtaining nosologic maps of tumor lesions in brain4. In the present work, sequences of eight images, each one corresponding to a different echo time TE will be considered. In general, soft tissues are characterized by long T2’s. In brain, normal tissues like white matter (WM) and gray matter (GM) exhibit specific relaxation times T2, usually shorter than pathological tissues such as tumor tissue, necrosis, edema or cerebrospinal fluid (CSF). The set of eight multi-echo MR T2-weighted images define an MxNx8-dimensional space, i.e., each pixel in the image MxN image matrix is described by an eight component vector, comprising the transversal magnetization decay. To start with, a sequential eccentricity 8-dimensional filter is applied5, producing a new homogenized 8-dimensional image, for which each pixel is replaced by the closest one to the centroid in the structuring element. The pixels of the regularized images tend to be less diverse, assigning more frequent 8-dimensional labels to the pixels that deviate from the 8-dimensional labels in their neighborhood. In this new image the pixels represent a somehow less mixed tissue. The T2 for each pixel is estimated by least square regression and the histogram is computed. Then using the watershed morphological operator6, the histogram is partitioned and the soft tissues are identified by their largest T2.

† mmartin@fisica.ciens.ucv.ve; phone 58 212 6051516; fax 58 212 6051675 ‡ Profesor Jubilado, Universidad Central de Venezuela

2. METHODS 2.1 Image measurement

Multi-echo T2-weighted images were acquired in a Siemens Magnetom MRI system, working at 1.5 Tesla, using a Carr-Purcell-Meiboom-Gill (CPMG) sequence with a total of 16 equally separated echoes, starting at TE = 22 ms. To improve the accuracy in image data points only those images corresponding to even numbered echoes were used. Tumoral lesions were covered in full by sampling 8 axial slices, each one 5 mm thick and 5 mm in separation. Pixel intensity is generally given by an exponential decay

( )2exp nTERpp On −= (1)

where R2 = 1/T2, n being the echo index and T2 the transversal relaxation time. As a consequence of MRI slice thickness and voxel size, equation (1) is commonly an approximation, because more than a single tissue could be present within the voxel, i.e., there is a partial volume problem. In the present paper, relaxation rates will be calculated according to equation (1). Pixel attenuation is clearly exemplified in Figure 1. White pixels correspond to tissue with the longest relaxation time T2, associated to cerebrospinal fluid (CSF) or necrotic tissue.

Fig. 1. Images for echoes 1 through 8, echo time (TE) ranging from 44 ms to 352 ms with regular intervals of 44 ms.

2.2 Image data regularization and the morphological eccentricity filter

Images were segmented to extract relevant pixels related to brain tissue only. This procedure was performed by the application of morphological gradient filters and watershed7 operations. In this way interior and exterior limits of the skull were determined. On a first stage, a morphological gradient filter was applied to the image, followed by the watershed operator to define a head mask. Similarly, a brain tissue mask was obtained, leading to a removal of pixels not related to brain tissue. The sequence of events is depicted in Figure 2. Image data were regularized by the application of a log-convex filter which essentially adjusts the original decay data to the geometrical properties of the expected (noiseless) decay, i.e., monotonicity and convexity. This regularization is made in the following way: starting with the original polygonal curve, g, a band is defined with limits ginf and gsup, both monotonous and convex polygonal curves, such that inf supg g g≤ ≤ ; next g is compared with the convex average, ( )sup inf / 2midg g g= + . If g is originally

monotonous and convex, i.e., noise distortions do not modify the geometrical properties of the data, then midg g= , otherwise, g is replaced by gmid.

(a) (b) (c)

Fig. 2. Application of the segmentation algorithm to isolate brain tissue; (a) original image, (b) watershed limits: inside (green) and outside skull (red) and (c) extraction of relevant image pixels (brain tissue alone).

Mathematical morphology (MM) is a non-linear technique for the analysis of geometric spatial structures within an image7. It was initially developed for the case of binary images and then extended to the grayscale case. For extracting relevant structures a small shape denominated structuring element (SE), is used in order to probe the image under study. The two basic morphological operators are erosion and dilation, the application of the erosion (dilation) operator to an image f gives an output image, which shows the minimum (maximum) of the pixels belonging to the structuring element B. For 8-dimensional images, a notion of minimum (maximum) has to be given, since there are no natural means of defining order in 8-dimensional spaces. Our approach to the problem is to substitute the 8-dimensional vector ( ) ( ) ( )( )1 8, , , , ,f x y f x y f x y=

for pixel ( ),x y by the 8-dimensional vector for pixel ( ),x y within the structuring

element B centered at ( ),x y for which ( ),BD x y attains a minimum. ( ),BD x y is defined as

( ) ( ) ( )( ), , ,Bs

D x y d ist f x y f s=∑

(2)

where

( ) ( )( ) ( )8

1, , ( , )i i

id ist f x y f s f x y f s

=

= −∑

(3)

Based on the cumulative distance above, the 8-dimensional erosion of f by B consists in the selection of the B-neighborhood pixel vector that produces the minimum value for DB, this is the pixel vector most highly similar to all the other pixels in B. This process is illustrated in Figure 3 where DB is determined for a and b among four pixel-vectors denoted by a, b, c and d. The eccentricity filter is a 8-dimensional erosion operator where the structuring element is a circle of fixed radius. The vector associated to each pixel in the resulting image corresponds to the centroid of the neighborhood with respect to the cumulative distance, see equation (3). Several radii were tried for the circular structuring element and we found that the choice of any radius greater than 3 led to blurring of the image. We also tried sequential filtering techniques varying the radius of the structuring element. Figure 4 shows the effect of the application of the eccentricity filter to the original set of images (see Figure 1).

Fig. 3. Schematic representation of the DB values for pixels a and b in the structuring element B

1 2 3 4

5 6 7 8

Fig. 4. Images corresponding to Figure 1 filtered using a circular structuring element with a 3 pixel radius.

2.3 Relaxation time T2 map and histogram partitioning

For each pixel of the filtered image, the relaxation time T2 was estimated using linear least squares regression assuming a single exponential decay, see equation (1). Figure 5 shows the resulting T2 map. The light-gray tones correspond to long relaxation times which are associated to soft tissue. In Figure 5b, the regions T, Nc and N, which correspond to tumor, necrosis and normal or unaffected tissue, respectively, are clearly differentiated.

Fig. 5. (a) Relaxation time T2. (b) Detail in regions T (tumor), N (normal or unaffected tissue) and Nc (necrosis).

A histogram of relaxation times T2 is obtained from the T2 map, determining the most representative T2 values since the images were already filtered. In order to apply the watershed morphological operator6 to define an appropriate partition and definition of T2 classes, the histogram has to be inverted, i.e., it is reflected against the horizontal axis such that maxima are transformed to minima and vice versa. Histogram variability is reduced by means of a sequential 1-dimensional morphological filter: opening-closing-opening with a structuring element of size 3. Figure 6 illustrates the effect of the filter upon a sector of the histogram.

Fig. 6. Detail of the smoothing effect of the sequential 1-dimensional morphological opening-closing-opening filter over a

sector of the T2 histogram. Blue lines represent the original histogram, green lines the filtered one.

The histogram of Figure 6 is partitioned in order to identify the various T2 ranges corresponding to main peaks, i.e., those related to more pronounced maxima, these are identified and ordered decreasingly according to their dynamics6,7, i.e., the minimum height that have to be descended from a peak to reach a neighboring peak with a higher altitude. A choice has to be made for the number of classes, each one corresponding to a main peak; the boundaries of the influence zones associated to each class are determined through a watershed filter. Figure 7 shows the localization of the main peaks in red and the boundaries in green. Those pixels with T2 values within the influence zone of a main peak belong to the same class and allow for the segmentation of the image for that class.

Fig. 7. Classification of peaks based on their dynamics. Lines indicated in red denote the position of peaks (p) with the

highest values for the dynamics. Boundaries for class partition are represented in green (b).

Once the class partition is obtained, a structuring element is defined for each class, including all those pixels that belong to that class, i.e., their T2 values are within the boundaries of a class, and the eccentricity filter is again applied, leading to a identification for each class of the most representative T2 value and its corresponding 8-dimensional class centroid. This depends on the choice for the number of classes. Figure 8 shows the histogram partition and also the magnetization decays corresponding to class centroids.

Fig. 8. On top of the figure, histogram of T2 and segmentation thresholds obtained by the watershed operator indicated by

vertical bars. Bottom, magnetization decays classified according to class; each decay corresponds to a different class given by the thresholds established in the histogram.

Since the number of classes is arbitrary, we worked with several choices. A small number of classes lead to brain tissue segmentation which did not allow for the identification of pathological tissue. On the other hand too high a choice for the number of classes introduced artifacts in the segmentation, i.e., similar tissues are classified differently. This is illustrated in Figure 8, where the numbers of classes is 15 and many of the representative magnetization decays, i.e., 8-dimensional class centroids, are very similar.

(a) (b) Fig. 9. Comparison of the image segmentation obtained by assuming 8 classes (a) and 15 classes (b).

Figure 9 shows a comparison between segmentations performed with 8 and 15 classes. All the remaining images in this work were segmented assuming 8 tissue classes. Image segmentation procedure is carried out very simply by comparing the 8-dimensional centroid for a particular pixel with all the 8-dimensional class centroids and deciding which one is closer. A color scale is then assigned arbitrarily to each class. It is important to recall that the class structure relies on the

T2 histogram determination, and as it was mentioned before due to the voxel size in MRI, the single exponential behavior given by equation 1 might no longer be valid. For there are further methods to evaluate relaxation time distributions or histograms that could be conveniently combined with this work8.

3. RESULTS Some of the results of the segmentation procedure are shown in Figures 10.1 through 10.3. For all the segmentations a set of eight T2 classes was assumed. Color scale is relative to the actual T2 values for each tumor and is represented in increasing order so white is assigned to the class with the highest value of T2 for that particular tumor. General inspection of Figures 10 reveals a clear separation between tumoral lesions and other types of tissue present in the brain, such as grey and white matter, both labeled with the same color (brown); liquid like tissues such as cerebrospinal fluid (CSF) or necrotic tissue exhibit colors corresponding to long relaxation times (purple, white) and in some cases are differentiated.

GM

O GII

(a) (b) Fig 10.1. Segmentation of MR images for glioblastoma multiforme (GM) and oligodendroglioma Grade II (O GII): (a)

Original axial image, (b) segmented image. Colors are assigned arbitrarily.

FA GII

FA GIII

(a) (b) Fig 10.2. Segmentation of MR images for fibrillary astrocytoma Grade II (FA GII) and fibrillary astrocytoma Grade III (FA

GIII): (a) Original axial image, (b) segmented image. Colors are assigned arbitrarily.

FA GII

C

(a) (b) Fig 10.3. Segmentation of MR images for fibrillary astrocytoma Grade II (FA GII) and a cyst (C): (a) Original axial image,

(b) segmented image. Colors are assigned arbitrarily.

4. CONCLUSIONS In this paper a technique for the segmentation of tumor lesions in brain MR images is presented. The method uses geometrical regularization and various morphological operators, such as erosion, opening, closing and watershed applied to a set of multidimensional images, multiecho T2-weighted MRI. Although the method requires the evaluation of relaxation times, its strength relies on the comparison of decay behaviors or patterns in the multidimensional image set. A more precise evaluation of relaxation times could only improve the effectiveness of the method, a task that will be tackled in future work. Also, it can be easily extended to other sets of multidimensional MR images, such as diffusion-weighted MRI, perfusion CT or MRI and in vivo MRS.

ACKNOWLEDGEMENTS

The authors would like to thank the MRI radiologists and technologists from the Instituto de Resonancia Magnética La Florida-San Román, Caracas, Venezuela who helped in the acquisition and analysis of the images used in this work. Research was partially supported by the Universidad Central de Venezuela, under grant CDCH PI 03-00-6267-2006/1.

REFERENCES

[1] Martín, R. and Martín-Landrove, M., “A novel algorithm for tumor characterization by analysis of transversal relaxation rate distributions in MRI,” [Spatially Resolved Magnetic Resonance], Blümler, P., Blümich, B., Botto, R., Fukushima, E., eds., Wiley-VCH, chap. 11, 133-138 (1998).

[2] Martín-Landrove, M., Bautista, I., Mayobre, F., Villalta, R. and Contreras, A., “Tumor assessment by in vivo proton spectroscopy and relaxometry,” MAGMA 15, 225-226 (2002).

[3] Martín-Landrove, M., Mayobre, F., Bautista, I. and Villalta, R., “Brain tumor evaluation and segmentation by in vivo proton spectroscopy and relaxometry,” MAGMA 18, 316-331 (2005).

[4] Martín-Landrove, M., “Nosologic maps of brain tumor images obtained from combination of different MRI modalities,” Proc. 28th IEEE EMBS Annu. Int. Conf. 759-762 (2006).

[5] Plaza, A., [Hyperspectral Data Exploitation: Theory and Applications], Chen-I Chang, ed., John Wiley and Sons, Inc., chap. 13 (2007).

[6] Soille, P., [Morphological Image Analysis, Principles and Applications], Springer, Berlin, (1998). [7] Climent, J. and Sanfeliu, A., “Visually Significant Dynamics for Watershed Segmentation,” Proc. 18th IEEE

International Conference on Pattern Recognition (ICPR'06). [8] Martín-Landrove, M., Figueroa, G., Paluszny, M. and Torres, W., “A Quasi-Analytical Method for Relaxation Rate

Distribution Determination of T2-Weighted MRI in Brain,” Proc. 29th IEEE EMBS Annu. Int. Conf. 1318-1321 (2007).