VESSEL SEGMENTATION USING 3D ELASTICA REGULARIZATION

Noha Youssry El-Zehiry and Leo Grady

Siemens Corporate Research, Princeton, NJ, USA.

ABSTRACT

Vascular diseases are among the most important health prob-lems. Vessel segmentation is a very critical task for steno-sis measurement and simulation, diagnosis and treatment plan-ning. However, vessel segmentation is much more challengingthan blob-like object segmentation due to the thin elongatedanatomy of the blood vessels, which can easily appear discon-nected in the acquired images due to noise and occlusion. Inthis paper, we present a generic vessel segmentation approachthat extracts the vessels by globally minimizing the surface cur-vature. The low curvature model enforces surface continuityand prevents the formation of false positives (leakages) andfalse negatives (holes). We present two contributions: First,we introduce a generic 3D vessel segmentation model by pe-nalizing the boundary surface curvature. Second, we introducean attraction force as a generalization of the boundary lengthin the elastica model, which guarantees a complete global so-lution and avoids shrinkage bias of length regularization. Ourresults will illustrate that the approach works efficiently acrossdifferent acquisition modalities and for different applications.

Index Terms— Segmentation, Vessel Segmentation, Com-binatorial Optimization, Curvature, Graph Methods

1. INTRODUCTION

Vessel segmentation approaches vary widely according to theprior information used in the segmentation, e.g.; appearancemodels, geometric models, and hybrid models (see [1] for ex-tended reviews). Despite this wide variety of vessel extractionschemes, there is not a single model that can work efficientlyacross modalities. Each vessel segmentation approach seemsto be customized to handle a specific application in a particularmodality. Here, we highlight the drawbacks associated withthe state-of-the-art vessel extraction schemes as reviewed byLesageet al. [1] and propose a generic model that overcomethese drawbacks.Centerline based methodsaim at extractingthe vessel centerline without providing any surface informa-tion. Region growingextraction techniques extend the vesselsby testing the neighbors of a given voxel against a predefinedinclusion criterion. They suffer from topological problems thatresult in holes (false negatives) and leakages (false positives)in the final segmentation. Active contoursrepresent anothermajor category of vessel extraction schemes; Parametric activecontours are proven to be efficient in 2D segmentation but theparameterizations get very complicated in 3D. On the otherhand, the formulation of geometric active contours (level sets

formulation) can simply be extended to 3D but at a very highcomputational cost. Thin structures can also be segmentedby imposing directional propagation information such as fluxbased approaches [2]. Although these approaches help elimi-nate leakage and premature stopping, they can not bridge gapsin the vessels. Additionally, the Euler-Lagrange optimizationschemes, used to minimize active contour models, convergeeasily to local minima. Hence, we conclude that agenericvessel segmentation model should satisfy the following crite-ria: 1) Topology preservation: The algorithm is respectiveof the vessel topology,i.e., preserve continuity of the surfaceand prevent the formation of false negatives (holes) and falsepositives (leakages). 2)Global optimization: The algorithmcan be optimized globally to capture the global solution andeliminate the need of tailored initialization. 3)Reusability:The algorithm is reusable across different modalities.

In order to provide such agenericmodel, we build on therecent work in [3] to propose a segmentation method whichgenerates the vessel segmentation as theglobal optimizationof a model that penalizes boundary curvature. The advantageof curvature optimization (shown theoretically by Mumford[4]) is that, in the absence of reliable image data, it encouragesthe segmentation of straight lines, which can be used to bridgegaps. Additionally, leaking is suppressed because the leakpoints often create unnecessary curvature, which is penalized.Curvature minimization has been proven in practice to providegood vessel segmentation (e.g., [5, 6]), but previous work hasnot been able to provide a global optimization.

The algorithm developed in [3] was demonstrated to havethe ability to bridge regions of a vessel for which the imageinformation was incomplete as a result of stenosis, occlusion,etc. In this work, we extend this method in two importantways. First, we extend the formulation of the method to 3D,which is necessary in order to apply the method in severalimaging modalities. However, this 3D formulation has theproblem that the optimization had more difficulty producing aglobal optimum (see Section 2). Additionally, when leakingis a concern, we want the flexibility to deemphasize the inten-sity model and also to fix background seeds in confoundingstructures. In the original formulation of [3], the reductionof influence for the intensity model in favor of seeds wouldalso cause the optimization method to fail. Therefore, thesecond contribution of this work is to decompose the bound-ary length term in the elastica model [4] into foreground andbackgroundattraction forcesyielding a generalized elastica

model. This decomposition allows independent control overforeground and background voxels that yields a solution withthe foreground voxels intact and without causing the shrinkagebias associated with length regularization. More importantly, itenables the Quadratic Psuedo Boolean Optimization (QPBO)and Quadratic Psuedo Boolean Optimization with Probing(QPBOP) to provide a complete global solution. Eliminationof unary terms enables the algorithm to segment, with aid ofseeds, objects the share the same intensity profile.

2. METHODS

We begin this section with a review of the 2D curvature opti-mization framework presented in [3] before proceeding to our3D extension of this approach, addition of an attraction forceand details on the optimization.

The continuous formulation of Mumford’s Elastica model[4] is defined for curveC as

E(C) =∫C(a + bκ2)ds a, b > 0. (1)

whereκ denotes the scalar curvature andds represents the arclength element. Whena = 0, the model reduces to the integralof the boundary squared curvatureE(C) =

∫C κ2ds.

The use of combinatorial optimization by [3] to minimizethe elastica model prompted the discrete formulation of the cur-vature on a graph. A graphG = {V, E} consists of a set ofverticesv ∈ V and a set of edgese ∈ E ⊆ V × V. An edgeincident to verticesvi andvj is denotedeij . In our formulation,each voxel is identified with a node,vi. A weighted graph isa graph in which every edgeeij is assigned a weightwij . Anedge cut is a set of edges that separates the graph into two sets,S ⊆ V andS, which may be represented by a binary indicatorvectorx, such that,xi = 1 if vi ∈ S and 0, otherwise. Thecost of the cut represented by anyx is given by

Cut(x) =∑eij

wij |xi − xj |. (2)

In [3], we have introduced the discrete representation ofcurvature on the primal graph. In this formulation, if two edgesincident on a nodevi, eij andeik, are cut then the cut is penal-ized with valuewijk = αp

min(||−→eij ||,||−→eik||)

, whereα is the angle

between the edges. This cut penalty is then exactly decom-posed into three edge weights

E(xi,xj ,xk) = wij |xi−xj |+wik|xi−xk|−wjk|xj−xk|, (3)

wherewij = wik = wjk = 12wijk. Despite the negative

weights, it was shown in [3] that QBPOP was able to find aminimum cut in most circumstances. Notice that although thecurvature clique was designed to penalize the cut of both edgeseij andeik, the decomposition to pairwise interactions add anedgeejk with negative weight. We denote the set of effectiveedges with nonzero weights asE∗ ⊇ E .

Extension to 3D: The graph-based formulation presentedin [3] and reviewed above associates cut costs with the curva-ture for the boundary on a dual graph. Unfortunately, manygraphs of interest in 2D (e.g., an 8-connected lattice) are non-planar and therefore have no dual. However, in [3] it was shown

Fig. 1. Six point neighborhood used in evaluating the 3D curvature.

that the formulation may also be applied to nonplanar graphsby simply computing angles between the edges incident on anode, applying the decomposition in (3).

Extending this method to 3D follows the same approach.Since a 6-connected lattice, depicted in Figure 1, has a dualcomplex [7], it is straightforward to interpret this curvature for-mulation in terms of penalizing the corners formed on the 3Dsurface of the dual complex. However, boundary optimizationon a 6-connected lattice is well-known to produce undesirablegridding artifacts. Consequently, it would be desirable to ex-tend this method to apply to 3D lattices of higher-order con-nectivity, such as a 26-connected lattice. Unfortunately, sincethe dual representation is not as clear in this case, it is quitedifficult to determine which pairs of edges should be penal-ized. Here, we performed this extension by penalizing 8 planarcliques (same cliques of the 8-connected lattice in 2D [3]) and16 cliques; eight to each of the upper and lower planes. Theadded cliques represent a higher resolution partitioning of theunit sphere which should yield a smoother surface.

Optimization: In the previous sections, curvature regular-ization was formulated into the problem of finding a minimumcut on a graph in which some of the edge weights were neg-ative. Unfortunately, the negative edge weights introduced bythe third term of (3) cause the min-cut problem to be nonsub-modular [8],i.e., straightforward max-flow/min-cut algorithmswill not yield a minimum cut. However, it was shown in[3] that the Quadratic Pseudo Boolean Optimization (QPBO)and Quadratic Pseudo Boolean Optimization with Probing(QPBOP)[8] frequently offered a solution to the optimizationproblem that is complete and optimal.

In our extension of this work to vessel segmentation, weencounter two difficulties with this optimization approach. 1)In our experience the structure of the negative weights encoun-tered in the 3D construction more often leads to incompletesolutions from QBPO. 2) Intensity models are often unable todistinguish vessels from other proximal structures (e.g., twotouching vessels). Therefore, in these circumstances we wantthe ability to modify the construction to remove the intensitymodel and instead supply a seed in the confounding structure.Unfortunately, without an intensity model (unary term), thenQBPO will be completely unable to label the voxels. To re-solve this problem, it is possible to add the length term of theElastica to decrease the non submodularity of the energy func-tion. However, it is well-known that the length regularizationintroduces a shrinkage bias so we decompose the length terminto foreground and backgroundattraction energiesin order to

produce a complete labeling of the vessel and control the fore-ground and background independently to avoid shrinkage.

Attraction Energy: The second term in the elastica energyof (1) is the boundary length term. The boundary length termcorresponds to the minimum cut term in graph-based meth-ods, which identifies the boundary length with a cut using edgeweights which may be weighted to reflect Euclidean boundarylength [9]. Therefore, combining our formulation of discretecurvature with boundary length, we obtain a discrete formula-tion of the elastica model as:

E(x) = λ∑

eij∈E∗wij |xi − xj |+ µ

∑eij∈E∗

w∗ij |xi − xj |, (4)

wherewij could be set towij = 1 or to reflect the Euclideanboundary length (as in [9]) and the weightsw∗

ij are the cur-vature penalties in (3). We may generalize the elastica modelby decomposing the length term into aforeground attractionforce and abackground attraction force. The minimizationof the length term expressed by

∑eij∈E∗ wij |xi−xj | is equiv-

alent to the maximization of the function∑eij∈E∗

wij (xixj + (1− xi)(1− xj)) . (5)

Foreground and background attraction forces are analogous toinflation and deflationballoon forcespresented in [10], withthe difference that the attraction forces operate on pairs of vox-els. While an inflation force encourages every individual voxelto be labeled foreground, a foreground attraction force encour-ages attraction between pairs of voxels by enforcing neighbor-ing voxels to have the same foreground label. Consequently,we may consider a generalized elastica model that consists ofthe curvature term with two attraction forces. This generalizeddiscrete elastica model is written as

Eelastica(x) = −λ1

∑eij∈E∗

wijxixj−

λ2

∑eij∈E∗

wij(1− xi)(1− xj) + µ∑

eij∈E∗w∗

ij |xi − xj |(6)

whereλ1 andλ2 may be independently controlled to weightthe foreground or background attraction forces.

A key value of the attraction force is that it allows for anoptimization of the curvature energy even if the data (unary)terms are removed. Specifically, the graph construction rep-resents the negative attraction energy by adding an edgee12

and an edgee2T with positive weights. The addition of posi-tive weights changes the sign of some of the negative weightsintroduced by the curvature term. These sign changes affectthe optimization problem by strongly decreasing the number ofnon submodular terms in the energy.

Summary: The segmentation problem is modeled as thesolution,x, which minimizesE(x) = Edata(x) + Eelastica.

The data term is typically instantiated by an intensity modelfor the object (vessel). In our experiments, we employ a verysimple Chan-Vese data term [11] which models the foregroundand background each with a single intensity.

Edata(x) =∑vi∈V

xi(gi−µF )2 +∑vi∈V

(1−xi)(gi−µB)2, (7)

where µF represents the expected foreground intensity andµB represents the background intensity and the elastica energyEelastica is defined in (6). We intentionally chose a simpledata model in order to highlight the contribution of the otherenergy terms. In practice, the algorithm could be customizedfor a specific application by replacing this data term with a dataterm that models the appearance of the target vessel.

3. RESULTS

This section demonstrates that our method is capable of seg-menting a vessel in 3D under challenging conditions. We willpresent results for three challenging cases in three differentmodalities (CTA, MRI and US) to demonstrate that:

1. The algorithm works across modalities with no changes.

2. The algorithm can use the curvature regularization andglobal optimization to connect a vessel in which a signaldropout appears to disconnect the vessel.

3. The algorithm is capable of separating two structureshaving similar intensity.

We begin by addressing situations in which a signal dropoutin the image makes the vessel segmentation challenging. Thefirst case shows an example of a Computed Tomography An-giogram (CTA) acquisition where the Right Coronary Arterysuffers a signal drop during descent. Using a simple data modelin which µF and µB in (7) are fixed based on the maximaland minimal intensities in the input volumes, we see in Fig-ure 2 that the curvature regularization is sufficient to connectthe vessel under these challenging conditions. A more extremeexample is given by Figure 3, which depicts a vessel in a 3Dultrasound . In this case, acquisition resulted in a series of high-intensity blobs which are separated by dark regions. However,by using exactly the same weak intensity model as before (i.e.,settingµF andµB the same as in the CT case) Figure 3 demon-strates that our curvature regularization method is able retrievethe vessel from this series of blobs.The second major benefit of the algorithm is demonstrated bythe ability of the regularization method to separate two struc-tures with a similar intensity, we placed a single seed on oneslice to mark a target vessel and a second seed to mark a back-ground vessel. When seeds are incorporated into the segmenta-tion, a foreground seedvi is set toxi = 1 while a backgroundseed would be set toxi = 0. When seeds are incorporatedinto the segmentation, a data term is not necessary to avoidthe trivial minimum (i.e.,xi = 1,∀vi ∈ V). However, if thedata term is removed, then at least one of the attraction termsmust be included (λ1 > 0 or λ2 > 0) to allow the QBPO op-timization procedure to find a solution forx. In this scenario,no data term was used. Figure 4 shows an example of thismethodology applied to the separation of two blood vessels inan MR acquisition. Needless to say that any data model (alone)would have failed to perform this segmentation because bothvessels share the same intensity profile. Moreover, the mostcommon regularization in the literature, length regularization,

Fig. 2. Segmentation of the Right Coronary Artery in CTA. The first im-age is a coronal slice of a CTA , the second image is zooming on the RightCoronary Artery (RCA), the third image shows our result of the 3D segmenta-tion for the given slice and the fourth image is an orthogonal view that depictsthe dropout in the signal. The second row consists of six consecutive slices ofthe 3D volume cropped around the RCA, the third row exhibits the segmen-tation result using the data fidelity only and the fourth row depicts the resultsobtained by data and curvature regularization.

Fig. 3. Blood vessel segmentation in ultrasound.Color code: yellow -vowels excluded form the segmentation domain by simple thresholds, Red-Object of interest, Blue- Background. The top row shows a slice of the inputvolume. The second image is a magnified portion of the first image, the thirdand fourth images are the segmentation of the second image with data termonly and with data term and curvature, respectively. Second row: Six slices ofthe input volume. Third row: Segmentation using data term only. Fourth row:Segmentation using data term and curvature [µ=15,λ1 = 10, λ2 = 20].

would have yielded a cut around one the seeds. This shrinkingbias is eliminated by the curvature regularization. Another as-pect that worth highlighting is the computational complexity ofour algorithm relative to the state-of-the art curvature optimiza-tion schemes (that are mostly local). The most recent curvaturebased segmentation approach [12] reported a time varying from10 minutes to 3.5 hours for the segmentation of a 2D image,without even guaranteeing optimality, while our approach per-forms volumetric segmentation of a small volume in less thana minute on a similar machine. For example, The segmentationof an ultrasound data set of size 128× 128× 15 was performedin 25 seconds.

4. CONCLUSIONThe paper presented a generic curvature based vessel seg-mentation model that extracts vessels across data acquisitionmodalities without any modifications. Moreover, our model is

Fig. 4. Separation of vessels in MR. Top row shows a sample of the inputslices in the first image. Second and third images are a magnified portion of theimage and its segmentation, respectively. Second row: six consecutive inputslices with seeds. Third row: segmentation result.

globally optimized and does not require any initialization ma-nipulation to provide the desired results. Curvature has beenshown to provide a good mechanism for segmenting vessels inboth theory [4] and practice [5, 6]. Here we showed how the2D method for global optimization of curvature in [3] could beextended to 3D with the addition of an attraction force (whichalso allows us to employ seeds rather than a data term). Thisglobal optimization approach avoids the difficulties of tradi-tional vessel-following methods by considering the data as awhole, which allows it to connect regions of a vessel which aredisjoint as a result of noise or pathology. Future work willaddress more sophisticated data models and customization ofour method to specific problems in vessel segmentation.

5. REFERENCES

[1] D. Lesage, E. Angelini, I. Bloch, and G. Funka-Lea, “A review of 3Dvessel lumen segmentation techniques: models, features and extractionschemes,”Med. Imag. Anal., vol. 13, no. 6, 2009.

[2] M. Descoteaux, L. Collins, and K. Siddiqi, “A multi-scale geometric flowfor segmenting vasculature in MRI,” inMICCAI, 2004.

[3] N. El-Zehiry and L. Grady, “Fast global optimization of curvature,” inCVPR, June 2010, pp. 3257 –3264.

[4] David Mumford, “Elastica and computer vision,”Algebraic Geometryand Its Applications, pp. 491–506, 1994.

[5] K. Ram, Y. Babu, and J. Sivaswamy, “Curvature orientation histogramsfor detection and matching of vascular landmarks in retinal images,” inSPIE, 2009, vol. 7259, p. 52.

[6] S. Garg, J. Sivaswamy, and S. Chandra, “Unsupervised curvature-basedretinal vessel segmentation,” inISBI, 2007.

[7] L. Grady, “Minimal surfaces extend shortest path segmentation methodsto 3D,” PAMI, vol. 32, no. 2, pp. 321–334, Feb. 2010.

[8] V. Kolmogorov and C. Rother, “Minimizing nonsubmodular functionswith graph cuts: A review,”PAMI, vol. 29, no. 7, pp. 1274–1279, 2007.

[9] Y. Boykov and V. Kolmogorov, “Computing geodesics and minimal sur-faces via graph cuts,” inICCV, Oct. 2003, pp. 26–33.

[10] L. D. Cohen, “On active contour models and balloons,”CVGIP: Imageunderstanding, vol. 53, no. 2, pp. 211–218, 1991.

[11] Tony F. Chan and Luminita A. Vese, “Active contours without edges,”TIP, vol. 10, no. 2, pp. 266–277, 2001.

[12] T. Schoenemann, F. Kahl, and D. Cremers, “Curvature regularity forregion-based image segmentation and inpainting: A linear programmingrelaxation,” inICCV, Kyoto, Japan, 2009.