+ All Categories
Home > Documents > Atlas based liver segmentation using nonrigid registration ... · segmentation methods for clinical...

Atlas based liver segmentation using nonrigid registration ... · segmentation methods for clinical...

Date post: 25-Jan-2021
Category:
Upload: others
View: 9 times
Download: 0 times
Share this document with a friend
10
Atlas based liver segmentation using nonrigid registration with a B-spline transformation model Pieter Slagmolen 1,2 , An Elen 1 , Dieter Seghers 1 , Dirk Loeckx 1 , Frederik Maes 1 , and Karin Haustermans 2 1 Medical Image Computing (ESAT/PSI), Faculties of Medicine and Engineering, UH Gasthuisberg, Herestraat 49, B-3000 Leuven, Belgium. 2 Department of Radiation Oncology, UH Gasthuisberg, Leuven, Belgium [email protected] Abstract. Liver segmentation is an important step for the therapeutic decision making in liver surgery. However, manual segmentation is time- consuming and tedious and so the need for accurate and robust automatic segmentation methods for clinical data arises. In this work an atlas in combination with nonrigid registration is used to segment the liver in actual clinical CT images. First, the atlas is built on twenty training images using nonrigid registration with a novel surface distance penalty. Next, this atlas is nonrigidly registered to ten test images. Currently, the user interaction is limited to the initialization of a rigid registration and to the definition of a region of interest for the nonrigid registration. Future work will focus on replacing the remaining user interaction with fully automatic procedures. Results are promising with an average overlap error of 10.4% and an average RMS distance of 5.0mm for the ten test images. Errors occur mainly at sites where the atlas is ill-defined such as the border between the heart and the liver. 1 Introduction Computer aided planning of liver surgery can greatly improve the choice of a suitable treatment strategy [1]. Liver segmentation is a crucial step for this com- puter aided planning. In a more general context, segmentation still is a bottleneck for the breakthrough of many computer assisted procedures because it remains tedious, time-consuming and subjective. New algorithms are constantly being developed and published. However, the step from research to clinical practice has proven to be very difficult because algorithms need to perform robustly and accurately on actual clinical data [2]. In this work a clinical dataset of ten liver patients is automatically segmented. We use a method based on an atlas and nonrigid registration similar to the one used in [3]. The atlas is built from a training database of 20 images with manual segmentations.
Transcript
  • Atlas based liver segmentation using nonrigidregistration with a B-spline transformation

    model

    Pieter Slagmolen1,2, An Elen1, Dieter Seghers1, Dirk Loeckx1,Frederik Maes1, and Karin Haustermans2

    1 Medical Image Computing (ESAT/PSI), Faculties of Medicine and Engineering,UH Gasthuisberg, Herestraat 49, B-3000 Leuven, Belgium.

    2 Department of Radiation Oncology, UH Gasthuisberg, Leuven, [email protected]

    Abstract. Liver segmentation is an important step for the therapeuticdecision making in liver surgery. However, manual segmentation is time-consuming and tedious and so the need for accurate and robust automaticsegmentation methods for clinical data arises.

    In this work an atlas in combination with nonrigid registration is usedto segment the liver in actual clinical CT images. First, the atlas isbuilt on twenty training images using nonrigid registration with a novelsurface distance penalty. Next, this atlas is nonrigidly registered to tentest images. Currently, the user interaction is limited to the initializationof a rigid registration and to the definition of a region of interest for thenonrigid registration. Future work will focus on replacing the remaininguser interaction with fully automatic procedures.

    Results are promising with an average overlap error of 10.4% and anaverage RMS distance of 5.0mm for the ten test images. Errors occurmainly at sites where the atlas is ill-defined such as the border betweenthe heart and the liver.

    1 Introduction

    Computer aided planning of liver surgery can greatly improve the choice of asuitable treatment strategy [1]. Liver segmentation is a crucial step for this com-puter aided planning. In a more general context, segmentation still is a bottleneckfor the breakthrough of many computer assisted procedures because it remainstedious, time-consuming and subjective. New algorithms are constantly beingdeveloped and published. However, the step from research to clinical practicehas proven to be very difficult because algorithms need to perform robustly andaccurately on actual clinical data [2].

    In this work a clinical dataset of ten liver patients is automatically segmented.We use a method based on an atlas and nonrigid registration similar to the oneused in [3]. The atlas is built from a training database of 20 images with manualsegmentations.

  • First, an overview will be given of how the atlas was built. Next, the regi-stration of the atlas to the individual images is described. Finally, segmentationresults are given for the training images and for the test images.

    2 Materials and Methods

    2.1 Images and Segmentations

    A database of thirty CT images is available. Twenty randomly selected imagesare used for training while ten other test images are used for validation.

    All CT images are enhanced with contrast agent and scanned in the centralvenous phase on a variety of scanners (different manufacturers, 4, 16 and 64detector rows). As it is CT, all datasets have been acquired in transversal direc-tion. The pixel spacing varies between 0.55 and 0.8mm, the inter-slice distancevaries from 1 to 3mm. There is no overlap between neighbouring slices.

    All segmentations were created manually by radiological experts, workingslice-by-slice in transversal view. The first tool they employed was an intensity-based region grower. In case of leakage, these leaks were removed by drawingmanual cut-lines. The segmentation is defined as the entire liver tissue includingall internal structures like vessel systems, tumours etc. In general, a vessel countsas internal if it is completely surrounded by liver tissue (in the transversal view).The large vessels that enter the liver (V.Cava and portal vein) are segmentedin the part which is enclosed by liver tissue, i.e. as the convex hull of the livershape in that area. The segmentations are available as binary maps.

    2.2 Atlas Building

    Affine Registration and Resampling First, one image with an average liver waschosen from the training set (image 1 in training set). All other images andtheir corresponding segmentations were affinely registered and resampled to thisimage. Affine registration was performed using the MIRIT software and is basedon the maximization of mutual information [4]. However, due to the diversity ofthe images, initialization of the affine registration is needed and therefore imageswere manually shifted to provide an initial, rough alignment. Our future workwill contain the automatization of this initialization. The resulting resampledimages Ii and segmentations, Si all have a size of 512 x 512 x 183 voxels with avoxelsize of 0.74 x 0.74 x 1.5 mm.

    Nonrigid registration Nonrigid registration is performed using a B-spline trans-formation model [5, 6]. A grid of mesh control points is positioned over thereference image and the displacements of these control points act as parametersfor the deformation field. A gradual refinement of the grid allows more local de-formations to be modelled. Again, mutual information is used as the similaritymeasure.

  • For the atlas building two penalty terms are optimized along with the mu-tual information. First, the smoothness penalty will disfavour unlikely transfor-mations by promoting a smooth transformation field. Next, a surface distancepenalty will minimize the distance between the segmentations on reference andfloating image. This penalty is described in detail in the next paragraph.

    The cost function Ec to optimize is a linear combination of the mutual in-formation Emi and the two penalty factors Esm and Esd, each with their ownweighting factor.

    Ec = ωmiEmi + ωsmEsm + ωsdEsd

    Optimization is carried out using a multiresolution approach. Starting fromdownscaled images and a coarse mesh, the image and/or mesh resolution areincreased at each stage. Within each stage, the optimal set of parameters issought using a limited memory quasi Newton optimizer.

    Surface Distance Penalty We introduce a new registration penalty, which pe-nalizes the remaining distance between surfaces of corresponding structures inreference and floating image. First, the known surfaces in the reference and float-ing images are approximated with a triangular mesh using the marching cubesalgorithm [7]. The thus obtained mesh points in the reference image are consid-ered as a dense sampling of the reference image surface. At each optimizationstep of the registration, the inverse transformation is applied to the coordinatesof these samples.

    At the same time, a polyharmonic Radial Basis Function (RBF) is fittedthrough the mesh points of the floating image surface [8]. The used fast RBFmethod constructs a signed distance function, which evaluates to zero in the meshpoints, to one in a set of points outside the surface at unit distance along thenormal of each mesh triangle and to minus one in a corresponding set of pointsinside the surface. This signed distance function of the floating image surfaceis evaluated in the transformed samples of the reference image surface. Themean squared value of these results is considered as a measure for the remainingdistance between the two surfaces and is minimized during optimization.

    Mean Morphology For each of the training images Ii a mean deformation fieldTi is determined. Each image from the training set (floating image) is registeredto all other training images (reference image).

    Ii→j = Tij(Ii)

    For all 20 training images, 19 registrations were calculated resulting in atotal of 380 registrations. Consequently, for each image Ii, 19 deformation fieldsTij are available that define its transformation to any other training image.Averaging these 19 deformation fields for each image gives a mean deformationmap Ti for all training images.

  • Ti =1

    n− 1∑j 6=i

    Tij

    All images and segmentations are then deformed with their correspondingmean deformation map to produce 20 deformed images Ii and 20 deformedsegmentations Si.

    Ii = Ti(Ii) Si = Ti(Si)

    Mean Intensities The 20 deformed images each are biased towards their originalimage. To overcome this bias, all images and segmentations are averaged thusproducing a single atlas image IAtlas with the corresponding atlas segmentationSAtlas.

    IAtlas =1n

    n∑i=1

    Ii SAtlas =

    1n

    n∑i=1

    Si

    To overcome possible cut-off problems at the resampling with some images,empty slices are added to the cranial side of the atlas to produce an atlas imagethat contains 512 x 512 x 210 voxels with a 0.74 x 0.74 x 1.5 voxelsize. Theresulting atlas is shown in figure 1.

    Fig. 1. Atlas image IAtlas and corresponding Atlas segmentation SAtlas

    2.3 Atlas Registrations

    The atlas built in the previous section is used to segment the training andtest images by nonrigidly registering the atlas image to all these images. In

  • the following, the image we want to segment will be referred to as the targetimage.

    Affine Registration and Resampling First, the target image is affinely registeredand resampled towards the atlas image after manual initialization to overcomelarge differences in imaging position. The resampling ensures that the parametersfor the nonrigid registration such as the multiresolution settings are reproducibleand can be kept constant for all possible target images.

    Region of interest To decrease registration time and to focus the registration onthe region of the liver, a region of interest (ROI) is defined for the target image.This region of interest is chosen around the liver and is currently manuallydefined. The segmentation also works without the ROI but this makes the nextstep significantly slower. An automatic detection of the ROI is possible since itdoesn’t need to be very precise. However, this has not yet been implemented.

    Atlas Segmentation The nonrigid registration used to register the atlas imageis the same as the one used to construct the atlas but without the surface dis-tance penalty which obviously can’t be used for segmentation. The resulting costfunction is:

    Ec = ωmiEmi + ωsmEsm

    Registration results were best when the atlas was deformed, and thus thetarget image remained fixed. The reason behind this is that the atlas image ismuch smoother than the target image. If we would perform the registration theother way round, the nonrigid transformation field would be tempted to wipeout smaller image details in the target image, trying to make it as smooth asthe atlas image.

    After nonrigid registration, the atlas segmentation is deformed with the founddeformation field and it is thresholded at 50% of the maximum value. To finishthe segmentation a morphological opening operation is performed and possibleunconnected segments are removed. Finally, the segmentation is resampled backto the original test image.

  • 3 Results

    3.1 Atlas Building

    The results of the nonrigid registrations used for the atlas building are shownin Table 1. The correspondence between the segmentations is very high in mostcases. Some registrations failed due to folding induced by the surface distancepenalty. However, this occurs in very few registrations and thus their influenceon the mean deformation field is minimal.

    The evaluation metrics used here are the volume overlap error, the volumedifference, the average surface distance, the RMS surface distance and the maxi-mum surface distance [9]. The same registrations but without surface registrationpenalty yielded much poorer results with an average overlap error of 14.5% andvolume difference of 8.95%. Thus, the resulting atlas will be more accurate whenusing the surface distance penalty.

    Table 1. Results of the comparison metrics for the atlas building [9]. These parametershave been calculated on a total of 380 registrations.

    Overlap Error Volume Diff. Avg Dist. RMS Dist. Max. Dist.[%] [%] [mm] [mm] [mm]

    Average 7.30 1.40 1.52 3.70 35.28Standard Dev 4.07 5.11 1.55 4.07 21.64

    Median 6.00 0.43 1.02 2.22 28.36

    3.2 Training Set

    The results shown in Table 2 are calculated by segmenting the trainig imageswith the proposed method and comparing these automatic segmentations withthe manual segmentations made by an experienced radiologist.

    3.3 Testing Set

    The results shown in Table 3 are calculated by segmenting the test images withthe proposed method in comparison with the manual segmentations. The resultson the training images are slightly better than the results on the test images.This is probably because the atlas contains information about each trainingimage and not about the test images. Figure 2 gives some visual examples of oursegmentations in an easy, intermediate and difficult case. In the difficult case (testimage 3), our method was unable to include the tumour in the segmentation.This is reflected by the score for this segmentation which is lower than theaverage (see Table 3).

  • Table 2. Results of the comparison metrics [9] for the training database

    Training Overlap Error Volume Diff. Avg Dist. RMS Dist. Max. Dist.Image [%] [%] [mm] [mm] [mm]

    1 9.71 -1.53 1.84 3.69 35.502 7.15 -3.72 1.18 2.12 18.003 5.68 -1.04 1.00 2.00 24.234 8.41 6.24 1.82 5.18 70.085 7.56 5.44 1.16 3.22 34.816 10.44 8.05 2.26 5.01 44.797 6.08 2.02 0.94 1.74 22.718 8.66 -3.15 1.68 3.16 33.269 6.71 3.59 1.08 2.45 23.0210 15.72 17.60 2.84 5.08 28.4411 11.01 4.66 2.68 6.50 53.1212 5.91 4.47 1.12 2.71 36.1013 12.13 10.30 2.94 7.26 51.3614 15.36 -6.10 3.37 6.90 53.8515 7.63 6.97 1.58 3.37 35.1216 5.43 1.97 1.13 2.31 28.1217 6.11 4.37 0.98 2.00 21.0518 6.19 3.68 1.35 3.47 36.7719 8.28 0.51 2.48 7.84 66.0220 4.93 -0.69 0.78 1.44 14.18

    Average 8.46 3.18 1.71 3.87 36.53

    Table 3. Results of the comparison metrics [9] and corresponding scores for all tentest cases.

    Dataset Overlap Error Volume Diff. Avg. Dist. RMS Dist. Max. Dist. Total[%] Score [%] Score [mm] Score [mm] Score [mm] Score Score

    1 9.2 64 5.2 72 2.1 48 6.0 17 49.8 34 472 13.1 49 10.4 45 2.4 40 5.9 18 48.3 36 383 14.5 43 -5.5 71 2.8 30 6.4 11 49.4 35 384 5.9 77 2.1 89 0.9 77 1.8 76 12.7 83 805 6.8 73 -0.5 98 1.2 70 2.4 67 21.3 72 766 8.9 65 -4.0 79 2.1 47 6.5 10 59.6 22 457 15.1 41 14.4 23 3.4 15 9.6 0 70.5 7 178 8.9 65 4.7 75 1.8 55 4.6 36 37.4 51 569 12.0 53 8.2 56 1.6 60 3.5 51 24.5 68 58

    10 9.9 61 1.9 90 1.6 60 3.3 54 31.0 59 65Average 10.4 59 3.7 70 2.0 50 5.0 34 40.5 47 52

    3.4 Registration Time

    Table 4 gives the average time needed to perform a single segmentation (meanof the segmentation time for the ten test images). The largest portion of timeis spent on calculating the nonrigid registration. About two minutes of user

  • Fig. 2. From left to right, a sagittal, coronal and transversal slice from a relatively easycase (1, top), an average case (4, middle), and a relatively difficult case (3, bottom). Theoutline of the reference standard segmentation is in red, the outline of the segmentationof the method described in this paper is in blue. Slices are displayed with a window of400 and a level of 70.

    interaction are required for each image. Removing the region of interest fromthe registration would decrease this manual interaction but consequently wouldincrease registration time significantly. The manual initialization of the affineregistration could be made obsolete by making the affine registration more robustto account for large variations in imaging position.

    4 Discussion and Future Work

    A segmentation framework was presented based on nonrigid registration with anatlas image. Results show that our segmentation is able to segment all imagesand that it doesn’t fail on any of the proposed images. For the 20 training imagesan average volume overlap error of 8.5% and an RMS surface distance of 3.9mmare obtained. For the 10 test images an average volume overlap error of 10.4%and an RMS surface distance of 5.0mm are obtained.

  • Table 4. Time needed to perform a full segmentation of a single image.

    Step Average time

    Manual Initialization 1 minAffine Registration 30 sec

    Resampling 30 secDefinition Region of Interest 1 min

    Nonrigid Registration 59 minBinary Operations 1.5 min

    Resampling to Original 30 sec

    Total Time 64 min

    A few shortcomings of this method still need to be solved. First, some reg-istrations in the atlas building process still fail due to folding induced by thesurface distance penalty. This could be solved by decreasing the weigth of thesurface distance penalty in the first multiresolution stages. Even though the in-fluence of a single failed registration is limited in the atlas due to the averaging ofthe deformation field, a slight increase in accuracy for the atlas can be expected.

    Next, manual initialization of the affine registration and manual definitionof a region of interest for the segmentation make this a semi-automatic methodrather than a fully automatic one. Both initializations can probably be doneautomatically in the future but are not yet implemented.

    Fig. 3. Similar intensities in the target image can cause the registration to fail locally.This is mainly the case at the border between the heart and the liver (left). Also, inone case (test image 7) a part of the bowel was mistaken for a part of the liver. Theoutline of the manual segmentation is in red, the outline of our segmentation is in blue.Slices are displayed with a window of 400 and a level of 70.

    Finally, when looking at the images with the corresponding segmentations inFigure 2 and Figure 3, an additional shortcoming of this method becomes clear.Small differences in intensities can cause it to fail in certain regions. In Figure 2,a tumor is not included in the segmentation where it should have been. In Figure3, either a part of the heart or a part of the bowel are incorrectly included in thesegmentation. This occurs mainly where the liver and a surrounding organ withsimilar intensity lie closely together as is the case in the examples shown. Also,

  • the atlas is ill-defined at the border between the heart and the liver (see Figure1) and this obviously limits the information available for registration in thisregion. To overcome these problems statistical information about the shape andintensities of the liver could be implemented as an additional penalty. This woulddiscourage unlikely shape changes or intensity profiles. Our future work will tryto extend our atlas approach with this statistical information as we expect thiscould diminish the problems we encounter because they usually involve unlikelyshape (liver-heart boundary) or intensity (tumour) information.

    References

    1. Meinzer, H.P., Thorn, M., Cardenas, C.E.: Computerized planning of liver surgery–an overview. Computers & Graphics 26(4) (2002) 569–576

    2. Heimann, T., Wolf, I., Meinzer, H.P.: Active shape models for a fully automated3d segmentation of the liver - an evaluation on clinical data. In Larsen, R., Nielsen,M., Sporring, J., eds.: MICCAI (2). Volume 4191 of Lecture Notes in ComputerScience., Springer (2006) 41–48

    3. Seghers, D., D’Agostino, E., Maes, F., Vandermeulen, D., Suetens, P.: Constructionof a brain template from MR images using state-of-the-art registration and segmen-tation techniques. In Barillot, C., Haynor, D.R., Hellier, P., eds.: Lecture notes incomputer science. Volume 3216 of Lecture Notes in Computer Science., Springer(2004) 696–703

    4. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodalityimage registration by maximization of mutual information. IEEE Transactions omMedical Imaging 16(2) (1997) 187–198

    5. Loeckx, D., Maes, F., Vandermeulen, D., Suetens, P.: Nonrigid Image Registra-tion Using Free-Form Deformations with a Local Rigidity Constraint. In Barillot,C., Haynor, D.R., Hellier, P., eds.: MICCAI (1). Volume 3216 of Lecture Notes inComputer Science., Springer (2004) 639–646

    6. Loeckx, D.: Automated nonrigid intra-patient image registration using B-splines.PhD thesis, K.U.Leuven, http://hdl.handle.net/1979/298 (2006)

    7. Lorensen, W.E., Cline, H.E.: Marching cubes: A high resolution 3D surface con-struction algorithm. In: SIGGRAPH ’87: Proceedings of the 14th annual conferenceon Computer graphics and interactive techniques. Volume 21., New York, NY, USA,ACM Press (1987) 163–169

    8. Carr, J.C., Beatson, R.K., Cherrie, J.B., Mitchell, T.J., Fright, W.R., McCallum,B.C., Evans, T.R.: Reconstruction and Representation of 3D Objects with RadialBasis Functions. In: SIGGRAPH ’01: Proceedings of the 28th annual conference onComputer graphics and interactive techniques, New York, NY, USA, ACM Press(2001) 67–76

    9. Gerig, G., Jomier, M., Chakos, M.: Valmet: A New Validation Tool for Assessingand Improving 3D Object Segmentation. In Niessen, W.J., Viergever, M.A., eds.:MICCAI. Volume 2208 of Lecture Notes in Computer Science., Springer (2001)516–523


Recommended