62
List of Papers
This thesis is based on the following papers, which are referred to in the textby their Roman numerals.
I Malmberg, F., Vidholm, E., Nyström, I. (2006) A 3D Live-Wire Segmen-tation Method for Volume Images Using Haptic Interaction. In Proceed-ings of Discrete Geometry for Computer Imagery (DGCI), pp. 663-673.
II Strand, R., Malmberg, F., Svensson, S. (2007) Minimal Cost-Path forPath-Based Distances. In Proceedings of 5th International Symposiumon Image and Signal Processing and Analysis (ISPA), pp. 379-384.
III Malmberg, F., Lindblad, J., Nyström, I. (2009) Sub-pixel Segmentationwith the Image Foresting Transform. In Proceedings of the 13th Interna-tional Workshop on Combinatorial Image Analysis (IWCIA), pp. 201-211.
IV Malmberg, F., Nyström, I., Mehnert, A., Engstrom, C., Bengtsson, E.(2010) Relaxed Image Foresting Transforms for Interactive Volume Im-age Segmentation. In Proceedings of SPIE Medical Imaging 2010, Vol-ume 7632, Issue 762340.
V Malmberg, F., Lindblad, J., Sladoje, N., Nyström, I. (2011) A Graph-based Framework for Sub-pixel Image Segmentation. Theoretical Com-puter Science, Volume 412, Issue 15, pp. 1338-1349.
VI Malmberg, F. (2011) Image Foresting Transform: On-the-fly Computa-tion of Segmentation Boundaries. In Proceedings of the 17th Scandina-vian Conference on Image Analysis (SCIA).
VII Malmberg, F., Strand, R., Nyström, I. (2011) Generalized Hard Con-straints for Graph Segmentation. In Proceedings of the 17th Scandina-vian Conference on Image Analysis (SCIA).
For each paper, the authors are ordered according to their individual contributions.Reprints were made with permission from the publishers.
Related Work
In the process of performing the research leading to this Thesis, the author hascontributed also to the following publications.
Licentiate ThesisSegmentation and Analysis of Volume Images, with Applications. (2008)Swedish University of Agricultural Sciences. The work leading to thislicentiate thesis was performed under the supervision of Professor GunillaBorgefors.
Journal publications1. Malmberg, F., Lindblad, J., Östlund, C., Almgren, K.M., Gamstedt, E.K.
(2011) An Automated Image Analysis Method for Measuring Fibre Contactin Fibrous and Composite Materials. Nuclear Instruments and Methods inPhysics Research Section B: Beam Interactions with Materials and Atoms.In press.
2. Almgren, K.M., Gamstedt, E.K., Nygård, P., Malmberg, F., Lindblad, J.,Lindström, M. (2009) Role of fibre-fibre and fibre-matrix adhesion in stresstransfer in composites made from resin-impregnated paper sheets. Interna-tional Journal of Adhesion and Adhesives, volume 29, number 5, pp 551-557.
Refereed conference publications1. Malmberg, F., Östlund, C., Borgefors, G. (2009) Binarization of Phase
Contrast Volume Images of Fibrous Materials: A Case Study. In Proceed-ings of International Conference on Computer Vision Theory and Applica-tions (VISAPP 2009).
Other publications1. Malmberg, F., (2010) Image Foresting Transform: On-the-fly Computation
of Region Boundaries. In Proceedings of Swedish Symposium on ImageAnalysis (SSBA), pp. 51-54.
2. Nyström, I., Malmberg, F., Vidholm, E., Bengtsson, E. (2009) Segmenta-tion and Visualization of 3D Medical Images through Haptic Rendering.Proceedings of the 10th International Conference on Pattern Recognitionand Information Processing (PRIP 2009), pages 43-48. Publishing Centerof BSU, Minsk, Belarus, 2009.
3. Malmberg, F., Nyström, I. (2009) Interactive Segmentation with RelaxedImage Foresting Transforms. In Proceedings of Swedish Symposium onImage Analysis (SSBA), pp. 17-20.
4. Malmberg, F., Östlund, C., Borgefors, G. (2008) Graph Cut Based Segmen-tation of Phase Contrast Volume Images of Fibrous Materials. In Proceed-ings of Swedish Symposium on Image Analysis (SSBA), pp. 131-134.
5. Malmberg, F., Vidholm, E., Nyström, I. (2006) Live-wire based interactivesegmentation of volume images using haptics. In Proceedings of SwedishSymposium on Image Analysis (SSBA), pp. 57-60.
Contents
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 Digital images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Interactive image segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3.1 Desired properties of delineation methods . . . . . . . . . . . . . . . . 173.2 Paradigms for user input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.3 Interaction with volume images . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3.1 Volume visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.3.2 Haptics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.4 Evaluation of interactive segmentation methods . . . . . . . . . . . . 234 A graph theoretic approach to image processing . . . . . . . . . . . . . . . 25
4.1 Basic graph theory and notation . . . . . . . . . . . . . . . . . . . . . . . . 254.2 Images as graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.3 Graph partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.3.1 Vertex labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.3.2 Graph cuts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
4.4 Graph-based segmentation methods: A brief overview . . . . . . . 295 Minimum cost path forests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.1 Notation and definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315.2 Computing minimum cost path forests . . . . . . . . . . . . . . . . . . . 325.3 Applications in image processing . . . . . . . . . . . . . . . . . . . . . . . 34
5.3.1 Distance transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345.3.2 Live-wire segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . 355.3.3 Seeded segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
6 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396.1 A 3D extension of live-wire . . . . . . . . . . . . . . . . . . . . . . . . . . . 396.2 Minimal cost paths with neighborhood sequences . . . . . . . . . . 406.3 Partial coverage segmentation on graphs . . . . . . . . . . . . . . . . . 426.4 The relaxed image foresting transform . . . . . . . . . . . . . . . . . . . 456.5 Fast computation of boundary vertices . . . . . . . . . . . . . . . . . . . 466.6 Generalized hard constraints for graph partitioning . . . . . . . . . 48
7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517.1 Summary of contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517.2 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Summary in Swedish . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
Acknowledgements
This thesis would not have been completed without the help and support of anumber of people. In particular, I would like to thank the following:
• My supervisor Ingela Nyström – I could not have wished for better super-vision! During my years as a PhD student, it has been reassuring to knowthat I could always count on your support, in any matter. Thank you forshowing such confidence in my work and for encouraging me to pursuemy research ideas, even when they sometimes brought me away from theoriginal project plan.• My assistant supervisor Ewert Bengtsson for scientific support, for wise
guidance in various matters, and for giving me the opportunity to do re-search in this exciting field.• My other supervisors during the years: Gunilla Borgefors, Joakim Lind-
blad, and Catherine Östlund, for help and support.• Stina Svensson for valuable support during the first years of my PhD stud-
ies, and for good collaboration on Paper II.• Robin Strand for being an inspiring and patient teacher in the art of writing
mathematical papers, and for good collaboration on Papers II and VII.• Joakim Lindblad and Nataša Sladoje for many fun, interesting, and lively
discussions on various topics, some of which led to the ideas presented inPapers III and V.• All other co-authors and collaborators: Karin Almgren, Craig Engstrom,
Kristoffer Gamstedt, Andrew Mehnert, and Erik Vidholm – it has been apleasure to work with you!• Olof Dahlqvist-Leinhard, Milan Golubovic, Jan Hirsch, Joel Kullberg, and
Sven Nilsson, for interesting and fruitful discussions on applying the resultspresented in this thesis to problems in medical research.• Anders Brun for contributing greatly to the inspiring and creative atmo-
sphere at CBA.• Olle Eriksson for keeping my computer running (often fixing it before I
even knew it was broken), and Lena Wadelius for help with all administra-tive matters.• All my friends and colleagues, past and present, at CBA, for making it a
great place to work.
11
• Ewert Bengtsson, Gunilla Borgefors, Cris Luengo, Anders Malmberg, BoNordin, Ingela Nyström, and Robin Strand for proof-reading and comment-ing on drafts of this thesis.
♥ My family and my friends.♥ My wife Annika, for all the love and happiness you give me.
Uppsala, March 2011
Filip Malmberg
12
1. Introduction
The subject of digital image analysis deals with extracting relevant informa-tion from image data, stored in digital form in a computer [31]. Research inthis field started in the 1960’s, when some fundamental properties of digi-tal images were investigated [25]. The idea of using graph theoretic conceptsfor image processing and analysis can be traced back to, e.g., the work ofZahn [36] in the early 1970’s. Since then, many powerful image process-ing methods have been formulated on pixel adjacency graphs, i.e., a graphwhose vertex set is the set of image elements (pixels), and whose edge set isdetermined by an adjacency relation among the image elements. Due to itsdiscrete nature and mathematical simplicity, this graph based image represen-tation lends itself well to the development of efficient, and provably correct,methods. This thesis concerns the development of graph-based methods forinteractive image segmentation.
Image segmentation is the process of identifying and separating relevantobjects and structures in an image. This is a fundamental problem in imageanalysis – accurate segmentation of objects of interest is often required beforefurther processing and analysis can be performed.
Despite years of active research, fully automatic segmentation of arbitraryimages remains an unsolved problem. At first, this may seem somewhat sur-prising. Why is segmentation such a hard problem? Part of the answer to thisquestion lies in the definition of the segmentation problem as the the task ofidentifying relevant objects in an image. The notion of a relevant object ishighly context dependent, and is in general not possible to define based onthe image data alone. The identification of relevant objects may require, e.g.,experience, knowledge of the task at hand, and knowledge of the imagingprocess. These are qualities that humans possess, but that computers are no-toriously lacking. Semi-automatic, or interactive, segmentation methods usehuman expert knowledge as additional input, thereby making the segmenta-tion problem more tractable. The goal of interactive segmentation methods isto minimize the required user interaction time, while maintaining tight usercontrol to guarantee the correctness of the results.
Research in image segmentation can be divided into two types of activities:(1) development of general purpose tools and methods, and (2) constructionof domain-specific solutions. The work presented in this thesis is primarilyfocused on the former activity. To illustrate the benefits of the proposed meth-ods, we use examples from the medical field.
13
2. Digital images
An image in the usual intuitive meaning, e.g., the images captured by a cam-era, can be modeled as a continuous function I(x,y) of two variables, where xand y are coordinates in the plane. With a conventional camera, the values ofthe image function corresponds to some property, such as brightness or color,of the incident light at points in the image.
To store an image in a computer, it must first be digitized. Digitization re-quires sampling, i.e., recording the value of the image function at a finite set ofsampling points, and quantization, i.e., discretization of the continuous func-tion values. The obtained data is called a digital image. Predominantly, thesampling points are located on a Cartesian grid, with grid points having integercoordinates. The basic definition given above may be generalized in severalways. We may divide such generalizations into three categories:
Generalized image modalities The values of the image function may beused to represent other physical properties than incident light. Today,many specialized imaging devices are available that are capable ofcapturing, e.g., temperature, material density, water content, or distanceto the observer, at points in the image.
Generalized image domains This category of generalizations extend the do-main of the image function in various ways. The most basic example ofsuch generalizations is temporal images, i.e., video, where a sequenceof two-dimensional (2D) images captured at different times may be con-sidered a function of two spatial variables, and one time variable t.
Some imaging techniques are capable of generating three-dimensional(3D) volume images. In this case, the image function is defined overa portion of R3. Volume imaging is particularly common in medicine,where techniques such as computed tomography (CT), and magneticresonance imaging (MRI), are routinely used to generate high resolu-tion volume images of the human body.
In 2D images, the sampling points1 are often called pixels (picture ele-ments). In 3D images, the term voxel (volume picture element) is oftenused. In this thesis, the term image element will be used to denote ei-ther a pixel or a voxel, depending on the dimensionality of the image athand.
1Or, rather, the Voronoi regions associated with the sampling points.
15
Generalized sampling point distributions Although most imaging devicesnaturally produce images sampled on the Cartesian grid, it has beenshown that there are several reasons to consider alternative samplingpoint distributions. Strand [33] investigated non-Cartesian grids, e.g.,the hexagonal grid and its generalizations to 3D, and showed that thesegrids have many favorable properties.
Some authors have also considered images with arbitrarily distributedsampling points. This allows, e.g., images with high sampling densityin an area of interest, and lower sampling density in other regions. Thisreduces the total number of sampling points, thereby allowing the imageto be processed faster, while maintaining a high peak resolution. See,e.g., [14].
Ideally, methods for image processing and analysis should be applicable toimages defined in this broader sense. This is, however, not always the case. Inparticular, many methods implicitly assume a Cartesian sampling point dis-tribution. Extending such methods to images with alternative sampling pointdistributions is often non-trivial. The graph-based image representation con-sidered in this thesis is particularly flexible in this respect. In general, methodsformulated on arbitrary graphs are directly applicable to images of any struc-ture and dimensionality.
16
3. Interactive image segmentation
As stated in Chapter 1, image segmentation is the process of identifying andseparating relevant objects and structures in an image. The segmentation pro-cess can be divided into two tasks: recognition and delineation [12]. Recogni-tion is the task of roughly determining where in the image an object is located,while delineation consists of determining the exact extent of the object. Hu-man users outperform computers in most recognition tasks, while computersare often better at delineation. Interactive or semi-automatic methods attemptto combine human and computer abilities by letting a human user perform therecognition, while the computer performs the delineation. A successful semi-automatic method combines these abilities to minimize user interaction time,while maintaining tight user control to guarantee the correctness of the result.
The interactive segmentation process is illustrated in Figure 3.1. In thischapter, we discuss the various components involved in this process, and con-clude with some observations regarding the evaluation of interactive segmen-tation methods.
3.1 Desired properties of delineation methodsA delineation takes an image, together with user input given in some form,and produces a segmentation of the image. Grady [15] proposed the followingproperties, that a successful delineation method should satisfy:
1. Fast computation.2. Fast editing.3. An ability to produce, with sufficient interaction, an arbitrary segmentation.4. Intuitive segmentations.
The first two requirements are related to the speed of the computationalpart of the segmentation process. Ideally, the segmentation result should beupdated instantly when the user changes the input to the algorithm. As illus-trated in Figure 3.1, interactive segmentation is an iterative process. Typically,the changes in user input from one iteration to the next are relatively small.Often, it is possible to accelerate the computation of the solution for the cur-rent input by re-using information from the previous solution. In this way, fastediting can be achieved.
The third requirement is related to user control. A good delineation methodtypically requires only modest user interaction to produce a desired result.
17
Figure 3.1: The interactive segmentation process and its components. The process isrepeated iteratively, until a desired result has been obtained.
There will, however, always be cases when the delineation method fails toproduce a desired segmentation. In these cases, it is important that the usercan override the results of the delineation method, and in the worst case resortto manual delineation.
The goal of automatic segmentation methods is to produce correct seg-mentations. In interactive segmentation, the correctness of the result is ul-timately judged by the user. Thus, the goal of a delineation method is notprimarily to produce segmentations that are correct, in an absolute sense, butrather to produce segmentations that capture the intent of the user. This dis-tinction is emphasized by the fourth requirement. Obviously, this requirementis rather vague, and therefore hard to quantify. A common assumption is thatthe boundary of the desired segmentation should coincide with regions of highcontrast, e.g., strong edges, in the image. The delineation method should alsoperform consistently and predictably on degraded images, e.g., images withnoisy or missing data.
All interactive segmentation methods are subject to variations in user in-put. For the segmentation results to be repeatable, it is therefore desirable fora delineation to be robust with respect to “small” changes in user input. An-other feature that distinguishes different delineation methods is the ability tosegment multiple objects simultaneously.
3.2 Paradigms for user inputWe now turn our attention to the mechanisms by which the user providesrecognition information, i.e., the type of input that the user provides to the de-lineation algorithm during the segmentation process. At the most basic level,user interaction may involve the specification of some set of parameters thatcontrol the segmentation algorithm. This type of interaction, however, doestypically not allow the high degree of user control that we seek. Instead, we are
18
primarily concerned with methods that use pictorial input [23], i.e., methodswhere the user guides the segmentation by making annotations in the imagedomain. This type of input is typically provided in one of three forms:
Initialization The user is asked to provide the boundary of an initial segmen-tation that is “close” to the desired one.
Boundary constraints The user is asked to provide pieces of the desired seg-mentation boundary.
Regional constraints The user is asked to provide a partial labeling of theimage elements (e.g., marking a small number of image elements as“object” or “foreground”).
The first type of user input, initialization, is commonly used with activecontour [18] and level-set methods [26]. In these methods, the initial boundaryis evolved to a local optimum of some energy function. This energy functionshould be defined so that the desired segmentation corresponds to an optimumof the energy function. With this approach, the user input is treated as a softconstraint – it guides the delineation method towards a particular result, butdoes not reduce the set of feasible segmentations in any way. No guaranteesare given regarding the relation between the initial boundary and the final seg-mentation, and so the user only has limited control of the result. In particular,if the desired result does not correspond to an optimum of the energy function,there are no mechanisms for manually “overriding” the delineation method.
In contrast, boundary and regional constraints are typically treated as hardconstraints, i.e., any feasible segmentation must satisfy the constraints exactly.For boundary constraints, this means that all boundary elements specified bythe user must be included in the final segmentation boundary. For regionalconstraints, this means that the labels provided by the user must be preservedin the final labeling. In Section 4.4, an overview of segmentation methods thatutilize boundary or regional constraints is given.
In general, hard constraints provide a higher degree of control than softconstraints. For that reason, this work has primarily focused on methods em-ploying hard (regional or boundary) constraints. In Paper IV, we treat initialcontours as hard constraints by requiring the boundary of the final segmenta-tion to be located within some specified distance from the initial contour. Thisis achieved by converting the initial contour into a set of regional constraints.
In Paper VII, we show that both regional and boundary constraints can beseen as special cases of what we refer to as generalized hard constraints. Animportant consequence of this result is that it facilitates the development ofgeneral-purpose methods for interactive segmentation, that are not restrictedto a particular paradigm for user input.
19
Figure 3.2: Tasks involved in interactive segmentation with pictorial input. Compo-nents in gray correspond to tasks that are performed by the user.
3.3 Interaction with volume imagesAs illustrated in Figure 3.2, the process of interactive segmentation with pic-torial input requires the user to perform several tasks:
• Identify regions where pictorial input is needed.• Give pictorial input with sufficient precision.• Inspect the segmentation result and determine if it is satisfactory.
For interactive segmentation to be effective, the interface presented to theuser during segmentation must support all these tasks in a good way. For 2Dimages, it is relatively straightforward to design efficient interfaces that sup-port these tasks. Interaction with volume images, however, presents a range ofadditional difficulties, that make the problem more challenging.
3.3.1 Volume visualizationWhile 2D images are straightforward to display on a computer screen, vol-ume images require more sophisticated visualization techniques. In this sec-tion, the volume visualization techniques that have been used in this thesis aredescribed briefly.
A trivial way to visualize a volume image is to extract slices from the dataalong one of the principal axes (x, y, or z) and display the slices as 2D im-ages on the screen. While this gives a direct view of the data, it may behard to perceive how different structures relate to each other in the volume.A slightly more sophisticated version of this technique is multi-planar refor-matting (MPR), where arbitrarily positioned and oriented planes are used tovisualize multiple cross-sections of the 3D data-set. A common application ofMPR is to display three planes, each one orthogonal to one of the principal
20
Figure 3.3: A CT volume image of a human abdomen, visualized using multi-planarreformatting (MPR).
axes, next to each other along with a user interface that allows for translationof the planes, see Figure 3.3.
In surface rendering, polygonal surfaces are extracted from the volume anddisplayed using standard computer graphics techniques. A well-known tech-nique for surface extraction is the marching cubes (MC) method [21]. Thismethod extracts a polygonal approximation of an iso-surface, i.e., a surfacealong which the volume data attains some constant value, from the volume.This is useful for, e.g., displaying segmentation results, see Figure 3.4.
The above techniques all visualize volume data by converting it to an in-termediate representation, that can be displayed using standard visualizationtechniques. In contrast, direct volume rendering methods operate directly onthe full 3D data-set. The most common approach to direct volume render-ing is ray casting. Through each pixel in the image plane, a ray is cast fromthe view position into the volume. The color of the pixel is determined byintegration along the intersection of the ray and the bounding box of the vol-ume, using some selected compositing technique. Common compositing tech-niques include maximum intensity projection (MIP) and alpha-blending, seeFigure 3.5.
21
Figure 3.4: Surface rendering of the skeleton and a number of internal organs, seg-mented from a CT volume image. The segmentations were obtained using the relaxedIFT, proposed in Paper IV. Polygonal representations of the segmented organs wereextracted using the marching cubes algorithm.
3.3.2 HapticsTo interact with the surrounding world, humans rely not only on vision, butalso on our sense of touch. The subject of computer haptics deals with gen-erating tactile feedback, often with the aim of simulating the touch and feelof virtual objects. It is analogous to computer graphics, where the aim is togenerate visual impressions of a virtual scene.
A haptic device is a piece of equipment that is capable of generating tactilefeedback. In recent years, several devices that combine tactile feedback with3D input capabilities have become commercially available, e.g., the PHAN-ToM series from Sensable Technologies1. Commonly, these devices are de-signed as a stylus that the user can move and rotate in three dimensions. Asingle point, the haptic probe, is located at the tip of the stylus, and is used tointeract with objects in a virtual scene. Haptic interaction with objects in a 3Dcomputer graphics environment involves generating appropriate tactile feed-back when the haptic probe comes in contact with virtual objects. The processof calculating and generating tactile feedback is called haptic rendering.
The use of haptics for interactive image segmentation has been studiedby, e.g., Vidholm [35]. In Paper I, we use haptic feedback to facilitate theplacement of seed-points on the boundary of objects in a volume image. In
1URL: http://www.sensable.com
22
(a) (b)
Figure 3.5: A CT volume image of a human abdomen, visualized using direct vol-ume rendering with ray casting. (a) Maximum intensity projection (MIP). (b) Alphablending.
this work, we have used special haptic displays from the Swedish compa-nies Reachin 2 and SenseGraphics3. These display solutions combine a hapticdevice with a setup that allows co-localization of haptics and graphics, seeFigure 3.6.
3.4 Evaluation of interactive segmentation methodsAs pointed out by Olabarriaga and Smeulders [23], evaluation of interactivesegmentation methods differs slightly from evaluation of automatic segmen-tation methods.
A common criterion for evaluating automatic methods is accuracy, i.e., thedegree to which a delineation produced by the segmentation method corre-sponds to the truth. Accuracy may be measured subjectively, by letting a hu-man expert rank the correctness of the result, or objectively, by comparing itto a known ground-truth. In the context of interactive segmentation, the ac-curacy of the resulting segmentation is determined by the user. In this sense,the output of interactive segmentation is “always” a correct segmentation, pro-vided that the user control is not limited by the user interface or the delineationmethod. Thus, other criteria, such as efficiency and repeatability, may be moreappropriate for evaluating interactive segmentation methods.
2URL: http://www.reachin.se3URL: http://www.sensegraphics.com
23
Figure 3.6: SenseGraphics 3D-IW haptic display with a PHANToM Omni haptic de-vice. The haptic device is positioned beneath a semi-transparent mirror. The graphicsare projected through the mirror, in order to obtain co-localization of haptics andgraphics.
Efficiency relates to the total time required to complete a given segmenta-tion task. This may be separated into the time required for the computationalpart, and the time required for user interaction.
When a user (or multiple users) segments a specified object in the sameimage multiple times, the results should ideally be identical. The repeatability,or precision, of a method indicates the degree to which this is true for theparticular method. Variations in the results may be due to differences in therecognition step or in the delineation step. While nothing can be done aboutvariations of the first type, a successful method should minimize variationsof the second type. Repeatability may be evaluated empirically, by repeatedlyperforming the same segmentation task and measuring the amount of variationin the results, or theoretically, as in, e.g., [1].
24
4. A graph theoretic approach to imageprocessing
In this chapter, we give a formal definition of edge weighted graphs, and dis-cuss how these may be used to represent and segment digital images. Addi-tionally, we give a brief overview of previous work in the field of graph-basedimage segmentation.
4.1 Basic graph theory and notationA graph is a pair G = (V,E) consisting of vertices V and edges E, where Vis a set and E is a set of pairs of elements in V . The pairs of vertices in Emay be ordered or unordered. In the former case, we say that G is directed,and in the latter case, we say that G is undirected. In this thesis, we onlyconsider undirected graphs1. Commonly, graphs are visualized by drawing adot or circle for each vertex, and drawing arcs or lines between two vertices ifthey are connected by an edge, see Figure 4.1.
An edge spanning two vertices v and w is denoted ev,w. If ev,w ∈ E, thenthe vertices v and w are adjacent. The set of vertices adjacent to a vertex v isdenoted by N (v). In an edge weighted graph, each edge e ∈ E is associatedwith a real-valued weight, W (e). Depending on the context, we will interpretthe weight as either the affinity or the distance between two adjacent nodes. Inthe former case, two adjacent nodes are considered to be closely related if theweight of the edge connecting them is high. In the latter case, two adjacentnodes are considered to be closely related if the weight of the edge connectingthem is low.
A path in G is an ordered sequence of vertices π = 〈v1,v2, . . . ,vk〉 such thatevi,vi+1 ∈ E for all i ∈ [1,k− 1]. Two vertices v and w are linked in G if thereexists a path in G that starts at v and ends at w. The notation v ∼
Gw will here
be used to indicate that v and w are linked on G. If all pairs of vertices in agraph are linked, then the graph is connected, otherwise it is disconnected.
1The methods proposed in Papers IV and V are formally defined for directed graphs. In bothcases, however, we require the adjacency function to be symmetric. Thus, the graphs are ineffect undirected, but the weight of the edges in the graph may depend on the direction inwhich the edge is traversed.
25
A
B
C D
Figure 4.1: A drawing of an undirected graph with four vertices {A,B,C,D} and fouredges {eA,B, eA,C, eB,C, eC,D}.
(a) (b) (c)
Figure 4.2: (a) A 2D image with 4× 4 pixels. (b) A 4-connected pixel adjacencygraph. (c) An 8-connected pixel adjacency graph.
If G and H are graphs such that V (H)⊆V (G) and E(H)⊆ E(G), then H isa sub-graph of G. If H is a connected sub-graph of G and v 6∼
Gw for all vertices
v ∈ H and w /∈ H, then H is a connected component of G.
4.2 Images as graphsAs previously mentioned, graph-based image processing methods typicallyoperate on pixel adjacency graphs, i.e., graphs whose vertex set V is the setof image elements, and whose edge set E is given by an adjacency relation onthe image elements. Commonly, E is defined as all pairs of vertices v and wsuch that
d(v,w)≤ ρ , (4.1)
where d(v,w) is the Euclidean distance between the points associated with thevertices v and w and ρ is a specified constant. This is called the Euclideanadjacency relation. In 2D images, with pixels sampled in a regular Cartesiangrid, ρ = 1 gives a 4-connected graph and ρ =
√2 gives an 8-connected graph,
see Figure 4.2. In 3D images, ρ = 1 gives a 6-connected graph and ρ =√
3gives a 26-connected graph, see Figure 4.3.
26
(a) (b) (c)
Figure 4.3: (a) A volume image with 3×3×3 voxels. (b) A 6-connected voxel adja-cency graph. (c) A 26-connected voxel adjacency graph.
The edge weights in a pixel adjacency graph are typically chosen to reflectthe image content in some way. The weights may be based on, e.g., localdifferences in intensity, or other features, between adjacent image elements.A thorough discussion on how the graph definition affects the results of graphbased segmentation results can be found in [17].
In some cases, it may be of interest to consider graph structures other thanpixel adjacency graphs. For example, one may associate graph vertices withpre-segmented clusters (super-pixels) of image elements, rather than singleelements. The resulting graph has a smaller number of nodes, thus allowingcomputations on the graph to be performed faster. If the super-pixels representa meaningful partition of the image elements, then a good segmentation of theregion adjacency graph is likely to correspond to a good segmentation of theunderlying image. See, e.g., [20] for an example of this approach. Grady [14]proposed a pyramid graph as a multi-scale image representation, and demon-strated improved results for segmenting objects with blurred boundaries.
The above examples highlight the flexibility of the graph-based approachto image processing. Methods formulated on arbitrary graphs can readily beapplied in a wide range of contexts.
4.3 Graph partitioningTo segment an image represented as a graph, we are interested in partition-ing the graph into a number of separate connected components. A partitioningof a graph is commonly represented either as a vertex labeling or as a graphcut. These two representations are closely related, and the choice of one rep-resentation over the other is largely a matter of preference. In this section,we provide formal definitions of both representations, and clarify the relationbetween them.
27
4.3.1 Vertex labelingInformally, a vertex labeling associates each node of the graph with an elementin some set of labels. Each element in this set represents an object category,e.g., object or background.
Definition 1. A (vertex) labeling L of G is a map L : V → L, where L is anarbitrary set of labels.
A vertex labeling according to the above definition is crisp, in the sensethat each vertex is mapped to exactly one element in the set of object cat-egories. In contrast, a fuzzy image segmentation allows each image elementto belong partially to more than one object category. It has been shown thatthe extra information contained in a fuzzy segmentation may be utilized toachieve improved precision and accuracy when measuring geometric featuresof segmented objects [29, 30]. We now describe how fuzzy segmentations canbe formulated in terms of a vertex labeling. Consider a set of object categoriesL such that |L| = k. Rather than performing a vertex labeling L : V → L di-rectly, we consider a mapping L : V → Uk, where Uk is the set of vectorsx = (x1,x2, . . . ,xk) ∈ [0,1]k such that
xi ≥ 0 for all i ∈ {1,2, . . . ,k} (4.2)
and
‖x‖1 = 1 . (4.3)
In other words, we associate each vertex with a vector x∈Uk. Each componentxi in x represents the degree to which the vertex belongs to the correspondingclass in L. If all xi ∈ {0,1}, then x is crisp, otherwise it is fuzzy. Note that if xis crisp for all vertices in the graph, then this representation is equivalent to adirect mapping L : V → L.
4.3.2 Graph cutsInformally, a cut is a set of edges that, if they are removed from the graph,separates the graph into two or more connected components.
Definition 2. Let S ⊆ E, and G′ = (V,E \ S). If, for all ev,w ∈ S, it holds thatv 6∼
G′w, then S is a (graph) cut on G.
The boundary, ∂L, of a vertex labeling L is defined as the edge set ∂L ={ev,w ∈ E | L(v) 6= L(w)}. The relation between labelings and cuts is summa-rized in Theorem 1.
Theorem 1. For any graph G = (V,E) and set of edges S ⊆ E, the followingstatements are equivalent:
28
Figure 4.4: A vertex labeling of a graph. In this case, two labels (shown in the figureas black and white) are used. The boundary of the labeling is shown as dotted lines.By Theorem 1, the boundary of a vertex labeling is always a graph cut.
1. There exists a vertex labeling L of G such that S = ∂L.2. S is a cut on G.
A proof of Theorem 1 can be found in Paper V.
4.4 Graph-based segmentation methods: A briefoverviewThe literature on interactive segmentation is vast. In this section, we give abrief overview of a selection of graph-based method for interactive segmen-tation. A common theme for many of these methods is that they view graphpartitioning as an optimization problem. Thus, they seek to find a labeling orcut that optimizes some criterion on segmentation “goodness”, while satisfy-ing a set of constraints provided by the user.
The most prominent example of graph segmentation with respect to bound-ary constraints is the live-wire method [12]. Given a sequence of user-definedpoints on the boundary of an object, the live-wire method computes an op-timal path that encloses the object. In its original form, this method is re-stricted to 2D image segmentation. Many attempts have been made to extendthis paradigm to 3D, see, e.g., [10, 24].
Computing graph cuts with respect to regional constraints is a well studiedproblem, and many methods have been proposed for this purpose. The min-imal graph cuts [3] method calculates a cut separating the seed-points, suchthat the sum of the edge weights along the cut is minimal. A variant of thismethod is the normalized cuts algorithm [27, 7]. Another family of methodsis based on the calculation of a minimum cost path forest. These methods cal-culate a cut such that each vertex is connected to the closest seed-point, asdetermined by some path cost function. Examples of this approach includethe Image Foresting Transform (IFT) [9, 8], and the Relative Fuzzy Connect-edness method [34]. The Random Walker [15] method computes cuts suchthat each vertex is connected to the seed-point that a “random walker”, start-
29
ing at the vertex, is expected to reach first. The classical watershed approachhas also recently been reformulated on edge-weighted graphs [5].
Many of the above methods are closely related, and several efforts havebeen made to clarify the theoretical relation between the methods. A unifyingframework for seeded segmentation was presented by Sinop and Grady [28],and extended by Couprie et al. [4]. In [22], Miranda et al. established a linkbetween segmentation based on minimum cost paths and the minimal graphcuts approach.
In this thesis, we have primarily focused on methods based on the compu-tation of minimum cost paths. This concept is described in detail in Chapter 5.In the author’s opinion, these methods strike a good balance between speed ofcomputation, on the one hand, and segmentation quality, on the other hand.
30
5. Minimum cost path forests
Given two given vertices v and w, such that v ∼G
w, there exists one or morepaths in G that starts at v and ends at w. Assume that we are given a functionthat assigns a real value, a cost, to each path in the graph. Then there is, amongall possible paths between v and w, at least one path for which the cost isminimal. In this chapter, we consider the problem of finding such minimumcost paths between pairs of vertices in a graph. The cost of a minimum costpath may be interpreted as the distance or degree of connectedness betweenpairs of vertices. As such, it is a very useful concept, with applications in manyresearch fields. In Section 5.3, we discuss some applications of minimal costpaths in image processing and segmentation. While this chapter deals withminimal cost paths, we note that all concepts presented here may equivalentlybe formulated for maximal paths, as in, e.g., [22].
For graphs of practical interest in image processing, the number of possiblepaths between a given pair of vertices is typically huge, and searching thisspace for an optimal solution may appear to be a daunting task. Fortunately,efficient algorithms exist for this purpose. Given a set S ⊆ V of seed-points,it is in fact possible to simultaneously compute minimal cost paths from Sto all other vertices in V , using only O(|V |) operations. The output of thiscomputation, a minimum cost path forest, is formally defined in Section 5.1.In Section 5.2, we discuss the efficient computation of minimum cost pathforests.
5.1 Notation and definitionsWe now define a number of concepts, which are needed in the continued dis-cussion. As stated in Chapter 4, a path is a sequence of adjacent vertices.We denote the origin p1 and the destination pk of π by org(π) and dst(π),respectively. If π and τ are paths such that dst(π) = org(τ), we denote byπ · τ the concatenation of the two paths. A path cost function f (π) assigns areal-valued cost to any path in the graph. The choice of path cost function isapplication dependent. Commonly, the cost is a function of the edge weightsalong the path, e.g., the sum of all the edge weights along the path or themaximum edge weight along the path.
Definition 3. A path π is a minimum cost path if f (π) ≤ f (τ) for any otherpath τ with org(τ) = org(π) and dst(τ) = dst(π).
31
In general, the minimum cost path between two vertices is not unique. The setof minimum cost paths between two vertices v and w is denoted πmin(v,w).Since all paths in πmin(v,w) have the same (minimal) cost, f (πmin(v,w)) iswell defined even if |πmin(v,w)| > 1. The definition of a minimum cost pathbetween two sets of vertices is analogous. For two sets A⊆V and B⊆V , π isa path between A and B if org(π) ∈ A and dst(π) ∈ B. If f (π)≤ f (τ) for anyother path τ between A and B, then π is a minimum cost path between A andB. The set of minimum cost paths between A and B is denoted πmin(A,B).
Definition 4. A predecessor map is a mapping P that assigns to each vertexv ∈V either an element w ∈N (v), or /0.
For any v ∈V , a predecessor map P defines a path P∗(v) recursively as
P∗(v) =
{〈v〉 if P(v) = /0P∗(P(v)) · 〈P(v), v〉 otherwise
.
We denote by P0(v) the first element of P∗(v).
Definition 5. A spanning forest is a predecessor map that contains no cycles,i.e., |P∗(v)| is finite for all v ∈V . If P∗(v) = /0, then v is a root of P.
Definition 6. Let S⊆V . If P is a spanning forest such that P∗(v) ∈ πmin(v,S)for all vertices v ∈V , then we say that P is an minimum cost path forest withrespect to S.
5.2 Computing minimum cost path forestsThe problem of computing minimal cost paths has a long history in graphtheory. In 1959, Dijkstra [6] presented an efficient algorithm for computing aminimal cost path between two vertices v and w, under the assumption that fis the additive path cost function
fsum(π) =k−1
∑i=1
W ({vi,vi+1}) . (5.1)
Dijkstra’s algorithm is based on the observation that if P is a predecessormap such that
P(v) =
/0 if v ∈ Sw : w ∈ argmin
u∈N (v)f (P∗(u) · 〈u, v〉) otherwise (5.2)
for all v∈V , then P is a minimum cost path forest with respect to S. Accordingto this recursive definition of minimum cost path forests, it is trivial to com-
32
Algorithm 1: The Image Foresting TransformInput: A graph G = (V,E) and a set S⊆V of seed-points.Output: A predecessor map P, such that P is a minimum cost path
forest with respect to S.Auxiliary: Two sets of vertices F ,Q whose union is V .
1 Set F ← /0, Q←V . For all v ∈V , set P(v)← /0;2 while Q 6= /0 do3 Remove from Q a vertex v such that f (P∗(v)) is minimum, and add
it to F ;4 foreach w ∈N (v) do5 if f (P∗(w) · 〈w,v〉< f (P∗(v))) then6 Set P(w)← v;
pute a minimal cost path from S to v, provided that we have already computedall minimum cost paths whose cost is smaller than f (πmin(v)).
Falcão et al. [9] showed that Dijkstra’s algorithm may be generalized toallow multiple seed-points, and more general path-cost functions. This gen-eralized algorithm is called the image foresting transform (IFT). Pseudo-codefor the IFT is given in Algorithm 11. It was shown in [9] that Algorithm 1 pro-duces correct results for a fairly general class of path cost functions, including,e.g., all path cost functions that are monotonically increasing with respect topath length.
Asymptotically, the bottleneck of Algorithm 1 is the selection, on line 3, ofa vertex v ∈ Q for which f (P∗(v)) is minimal. Thus, the key to the efficientimplementation of Algorithm 1 is to store Q in a data structure that allowsrapid extraction of the element with minimum cost, e.g., some kind of priorityqueue. Typically, an efficient implementation of Algorithm 1 requires O(|V |)operations, for the type of graphs commonly occurring in image analysis ap-plications [9].
In [8], it was shown that seed-points may be added to, or removed from, aminimum cost path forest without recomputing the entire solution. This mod-ified algorithm, called the differential IFT (DIFT), dramatically improves theperformance of the IFT in interactive segmentation applications.
An alternative approach for computing minimum cost path forests is theBellman-Ford algorithm (BFA) [2, 13]. Pseudo-code for the BFA is given inAlgorithm 2. Just like the IFT, the BFA iteratively selects vertices for whichEquation 5.2 is not satisfied, and updates them. The difference is that whilethe IFT selects, at each step, a vertex v for which f (P∗(v)) is minimal, theBFA allows the vertices to be processed in any order.
1Note that in the formulation of Algorithms 1 and 2, we have adopted the convention thatf (P∗(v)) = ∞ whenever P0(v) /∈ S.
33
Algorithm 2: The Bellman-Ford algorithmInput: A graph G = (V,E) and a set S⊆V of seed-points.Output: A minimum cost path forest P with respect to S.
1 For all v ∈V set P(v)← /0. ;2 while there exists a v ∈V and w ∈N (v) such that
f (P∗(w) · 〈w,v〉) < f (P∗(v)) do3 Set P(v)← w ;
(a) (b) (c)
Figure 5.1: Distance transforms in different metrics, with level curves superimposedin red. The distance is computed from a single pixel, located at the centre of the image(+). (a) City-block distance. (b) Chessboard distance. (c) Euclidean distance.
When implemented on a computer with a standard sequential processor,the BFA is in general less efficient than the IFT. An advantage of the BFA,however, is that it is straightforward to implement on massively parallel pro-cessors, such as the programmable graphics processing units (GPUs) availablein commodity graphics cards [19].
5.3 Applications in image processingWe now present some applications of minimum cost path forests in imageprocessing.
5.3.1 Distance transformsFor many image analysis tasks, it is of interest to measure distances betweenimage elements. Given an image where a subset of the image elements havebeen labeled as foreground, and the remaining image elements have been la-beled as background, a distance transform (DT) assigns to each backgroundelement the distance from the element to the closest foreground element (ac-cording to some metric). See Figure 5.1. There are many variations on this
34
Figure 5.2: Segmentation of the liver in a slice from an MR volume image. The userinteractively positions seed-points (red) on the liver boundary. As the user moves thecursor, the minimum cost path (yellow) from the last seed-point to the current cursorposition is displayed in real-time.
basic concept. A signed DT assigns to each image element the distance to theclosest point on the border of the object. In this case the sign (+/-) of the dis-tance values depends on whether the image element belongs to the foregroundor the background. A constrained DT computes distance values in the pres-ence of a set of obstacles, that the shortest path between the image elementand the object must not pass.
Many different algorithms have been proposed for computing DTs, see,e.g., [33] for a good overview. Here, we note that the IFT may be used tocompute exact distance transforms for path-based metrics, e.g., the city-blockand chessboard metrics. In [9], it was shown that the IFT may also be usedto compute the Euclidean DT. That approach, however, is not applicable tocomputing constrained DTs.
5.3.2 Live-wire segmentationThe perhaps most straightforward way of utilizing minimal cost path calcu-lations in image segmentation is to consider the path itself as a boundary be-tween two regions. This idea forms the basis of the live-wire method [12, 11].To segment an object in a 2D image with live-wire, the user selects a seed-point on the object boundary. Dijkstra’s algorithm is then used to computeminimal cost paths from this point to all other points in the image. As theuser moves the pointer through the image, a minimal cost path from the cur-rent position to the seed-point – the live wire – is displayed in real-time. seeFigure 5.2. The idea is to design the path cost function so that low-cost pathscorrespond to desired boundaries in the image, thereby forcing the live-wireto snap onto the object boundary. When the user is satisfied with a live-wiresegment, he or she continues by placing a new seed-point. In this way, an en-
35
tire object boundary can be delineated with a rather small number of live-wiresegments.
While the computation of minimal cost paths is defined for arbitrary graphs,the nature of a path as a boundary between regions is not preserved for non-planar graphs. Thus live-wire, in its original form, is only applicable to 2Dimages.
5.3.3 Seeded segmentationTo use the IFT for seeded segmentation, we may associate each seed-pointwith a label, and assign to all other vertices the label of the “closest” seed-point as determined by the minimum cost path forest. See Figure 5.3. Forthis purpose, we can modify Algorithm 1 so that the labels of the seed-pointsare propagated along with the minimum cost paths [9]. Unlike the live-wiremethod, this approach is directly applicable to images of any dimensionality.
The quality of the segmentations obtained with this approach depends onthe choice of an appropriate path cost function. Recently, it was shown byMiranda et al. [22] that the fmax function, defined as
fmax = maxi∈[1,k−1]
(W ({vi,vi+1})) , (5.3)
has some properties that make it particularly well-suited for this purpose. Thisis the path cost function used in the fuzzy-connectedness framework [34].Specifically, the cuts obtained with this path cost function are shown to beglobally minimal with respect to a graph cut metric. The segmentation resultsare also provably robust with respect to “small” changes in the seed-pointplacement [1].
In this work, we have primarily used path cost functions of the form
f (π) =k−1
∑i=1
W ({vi,vi+1})p , (5.4)
where p ∈ R is a constant. When p is large, this function closely approxi-mates the fmax function. In contrast to fmax, however, the above function isstrictly increasing with respect to the path length2, as required by the methodproposed in Paper III.
2Provided that all edge weights are positive.
36
Figure 5.3: Seeded segmentation of the kidneys in an MR volume image, using theIFT. The user interactively selects seed-points labeled as foreground (green) and back-ground (red), respectively. When a new seed-point is added, the segmentation result(yellow) is updated in real-time, for the entire volume.
37
6. Contributions
In this chapter, the methods and results described in detail in the appendedpapers are presented briefly.
6.1 A 3D extension of live-wireAs described in Section 5.3.2, the live-wire method finds boundaries of objectsin 2D images by computing minimal cost paths through a sequence of user-defined points on the object boundary. While it is possible to compute minimalcost paths on general graphs, the character of a path as a boundary betweenobjects is not preserved in the 3D domain. In Paper I, we propose a new 3Dextension of the live-wire method. The method operates on a 26-connected3D lattice.
Our method allows the user to draw a number of live-wire curves on theboundary of the object of interest. These curves are then connected to form adiscrete surface, a process we call bridging. The aim is to segment entire ob-jects by drawing a relatively small number of live-wire curves on the boundaryof the object. The live-wire curves are not required to be planar. For drawingcurves in the 3D domain, we have implemented a user interface where theuser has two options: (1) place seed-points freely in the volume guided byvolume haptics and volume rendering, and (2) draw the curve onto an arbi-trarily oriented plane, see Figure 6.1. The haptic feedback in the first case isproxy-based volume haptics tuned to feel the surface of the object, and in thesecond case the slice plane is a haptic surface that the user can feel whiledrawing.
The bridging algorithm for connecting two curves uses the IFT to computea network of minimal cost paths between the two curves. This network is thenused to define a polygonal surface, that is subsequently rasterized to obtain atunnel-free discrete surface that closely matches the underlying object in theimage, see Figure 6.2.
Since the publication of Paper I, several interesting advances have beenmade in this area. Grady [16] proposed a method for computing globally min-imal discrete surfaces with prescribed boundary, thereby providing a moredirect extension of the live-wire paradigm to higher dimensions. In Paper VII,we present a method for computing graph cuts that satisfy a set of generalizedhard constraints, while globally minimizing a graph cut measure. In addition,
39
(a) (b)
Figure 6.1: Illustration of the 3D live-wire method proposed in Paper I. (a) Placingseed-points freely in the volume using volume rendering and volume haptics to locatethe boundary of the object. (b) Drawing a live-wire curve on an arbitrarily orientedslice.
(a) (b) (c) (d)
Figure 6.2: Illustration of the bridging procedure proposed in Paper I. (a) A syntheticobject. (b) Two live-wire curves drawn on the surface of the object. (c) Result of con-necting the two curves using the IFT. (d) Result of the proposed algorithm, includingrasterization.
we show that the proposed generalized constraints include both boundary andregional constraints as special cases. In this sense, the results in Paper VIIallow “live-wire”-style segmentation to be performed on arbitrary undirectedgraphs. The contents of Paper VII are further described in Section 6.6.
6.2 Minimal cost paths with neighborhood sequencesThe Euclidean distance function is used in many image analysis applica-tions, since it has minimal rotational dependence. However, in some appli-cations, path-based distance functions, such as the city-block and chessboarddistances, are preferable. One such example is the computation of constraineddistances, where a subset of the image elements are labeled as obstacles, that
40
Figure 6.3: Minimal cost paths for some constrained distance functions in Z2. Theset of minimal cost paths, shown in gray, is computed between two points (+). Whitepixels indicate obstacles, note the gaps in the obstacle lines. (a) Euclidean distance. (b)City-block distance. (c) Chessboard distance. (d) Weighted neighborhood sequencedistance.
the minimal cost paths are not allowed to pass through. For path-based dis-tance functions, this problem can be solved efficiently using Dijkstra’s algo-rithm.
In Euclidean geometry, the shortest path between two points is unique – it isa straight line between the points. In segmentation methods such as live-wire,where the minimal cost path represents the boundary of an object, this is alsothe result we expect in homogeneous regions of the image. Unfortunately, forpath based-distances the minimal cost path between two points is not neces-sarily unique. In Paper II, we consider the problem of finding one minimalcost-path π between two vertices v and w. If there are several minimal cost-paths between v and w, the minimal cost-path π might have a large deviationfrom a straight (Euclidean) line between v and w. The performance of a num-
41
(c) (d)
(a) (b)
(a) (b) (c)
Figure 6.4: Pixel coverage digitization. (a) A crisp continuous object (white) super-imposed on a pixel grid. (b) Crisp digitization of the object (Gauss digitization). (c)Pixel coverage digitization of the object.
ber of path-based distance functions is evaluated using a new error function,which links the number of possible minimal cost-paths with the asymptoticshape of the “balls” induced by the path-based distance functions.
So far, we have considered graphs with a fixed adjacency relation. How-ever, it has been shown [32] that by allowing the adjacency function to varyalong the length of a path, it is possible to obtain distance functions with alower rotational dependency. Such distance functions are called neighborhoodsequence distances. In Paper II, we show that for a fixed maximum neighbor-hood size, distance functions based on neighborhood sequences achieve betterscores with respect to the proposed error function than distance functions witha fixed adjacency relation. Figure 6.3 shows the set of (constrained) minimalcost-paths for some different distance functions considered in Paper II.
The results in Paper II are directly applicable to live-wire segmentation. Re-sults should hold for non-binary weights as well. By optimizing the distancefunction used to compute live-wire curves according to the criteria derivedin Paper II, more regular live-wire curves can be obtained in homogeneousregions of the image.
6.3 Partial coverage segmentation on graphsA common task in image analysis is to measure geometric features, such asarea/volume or perimeter/enclosed surface area, of objects. Such feature mea-surements usually rely on a correct segmentation of the object of interest.Even under the assumption that a correct segmentation is given, however, theaccuracy of such measurements is still limited by the fact that we are tryingto estimate features of continuous (real-world) objects based on a discrete,sampled, representation of the objects.
42
(a) (b) (c) (d)
Figure 6.5: Components of the framework, proposed in Paper V, for partial coveragesegmentation on graphs. (a) A crisp vertex segmentation of a graph. The boundary ofthe segmentation is shown as dashed lines. (b) A corresponding located cut. (c) Theedge segmentation induced by the located cut. (d) One component of the correspond-ing vertex coverage segmentation.
By utilizing fuzzy (rather than crisp) segmentations, the loss of informa-tion associated with the process of image segmentation can be significantlyreduced. To fully utilize the potential of a fuzzy representation, the fuzzy la-bels must be selected carefully. In particular, it has been shown that pixelcoverage representations outperform crisp representations for the purpose offeature measurements [29, 30]. Such representations are characterized by im-age values proportional to the relative area of an image element covered bythe imaged (presumably crisp continuous) object, see Figure 6.4. To utilizethis concept in practice, we need segmentation methods that produce fuzzysegmentations based on this principle – partial coverage segmentations.
Since the definition of pixel coverage digitization involves integration overthe shape of each image element, the concept does not translate directly tograph-based image representations where the “shape” of an image element isnot well defined.
In Paper V, we present a framework for extending the concept of partialcoverage segmentation to graphs. The components of this framework are illus-trated in Figure 6.5. Commonly, a segmentation is only defined at the verticesof a graph. In Paper V, we interpret the edges of the graph as paths betweenthe vertices. This allows us to define points along the edges of the graph, andto assign a (crisp or fuzzy) label to each such point. Thereby, we obtain anedge segmentation of the graph. We define the domain of a vertex as the set ofpoints on the “half-edges” adjacent to the vertex, see Figure 6.6. Conceptually,the domain of a vertex corresponds to the shape of an image element. Further-more, we define vertex coverage segmentation as a graph theoretic equivalentto pixel coverage segmentation. For each vertex, a vertex coverage segmen-tation is obtained by integrating the labels of an edge segmentation over thedomain of the vertex.
In order to make our framework usable in practical applications, we need away of obtaining edge segmentations. For this purpose, we introduce the con-cept of located cuts. As we have previously established, a graph cut separates
43
Figure 6.6: The domain (shown in gray) of a vertex with four neighbors.
(a) (b) (c)
Figure 6.7: Segmentation of the liver in a slice from an MR volume image. (a) Origi-nal image, with seed-points representing liver (green) and background (red). (b) Crispsegmentation. (c) Sub-pixel (vertex coverage) segmentation, obtained using the meth-ods proposed in Paper V.
the graph into two or more connected components. A located cut increasesthe precision of this separation by specifying a point (a parameter t ∈ [0,1])along each edge in the cut where the transition between different objects occur.In Paper III, we present a way of defining located cuts for segmentations ob-tained using the IFT (as a seeded segmentation method). The resulting methodis called the subpixel-IFT. In Paper V, we show that located cuts may be ob-tained as part of a defuzzification process, starting from an arbitrary fuzzysegmentation.
Via the concept of induced edge segmentation, located cuts provide a con-venient way of extending a segmentation defined on the vertices of the graphto all points along the edges of the graph. We show that for edge segmenta-tions induced by located cuts, the integrals involved in the calculation of avertex coverage segmentation may be reduced to simple closed formulas thatare easy to evaluate.
The practical utility of the proposed framework is demonstrated in two em-pirical studies. In Paper III, we perform a study on seeded segmentation ofmedical data, and conclude that the sub-pixel IFT is less sensitive to smallvariations in seed-point placement than the crisp IFT (for the additive path
44
cost function). In Paper V, we evaluate the proposed framework by measuringthe area of a large number of synthetic 2D shapes, comparing traditional crispobject representation with the proposed vertex coverage representation. Sig-nificant improvements in measurement precision are observed. An illustrationof vertex coverage segmentation in the context of medical image segmentationis shown in Figure 6.7.
6.4 The relaxed image foresting transformNumerous studies have shown that the IFT, as a seeded segmentation method,is capable of producing high quality segmentations in a wide range of con-texts. However, in images with weak or missing boundaries the IFT tends toproduce irregular segmentation boundaries. An explanation for this is that theIFT propagates information from the seed-points only along minimum costpaths. Since two adjacent image elements may receive their information fromdifferent seed-points, regularity of the segmentation boundary is not enforced.
In Paper IV, we address this weakness of the IFT by proposing the relaxedIFT (RIFT). This modified version of the IFT features an additional parameterthat controls the smoothness of the segmentation boundary, thereby makingthe results more predictable in the presence of noise and weak edges. Intu-itively, a vertex labeling is smooth if there is a high degree of correlation be-tween the labels of adjacent vertices. Based on this idea, our proposed methodworks by iteratively applying a relaxation procedure, where the label of eachvertex is replaced with a weighted average of the labels of all adjacent vertices.The number of iterations is used as a parameter to control the smoothnessof the segmentation. We show that these computations can be restricted to anarrow band around the segmentation boundary, yielding a fast segmentationalgorithm suitable for interactive applications. This results in a fuzzy vertexlabeling, which we subsequently defuzzify to obtain a final crisp labeling. Theefficacy of the relaxation procedure is demonstrated in Figure 6.8.
In addition, we present a study on the application of the RIFT method tothe problem of segmenting individual trunk muscles in MR volume imagesof human athletes (javelin throwers). In these images, contrast between adja-cent muscles is poor. The original IFT therefore produces segmentation resultswith noisy boundaries. Our tests indicate that the relaxed IFT produces morepredictable segmentation results for these images.
A vertex labeling is connected if there are no isolated regions of a par-ticular label that contains no seed-points. It is usually desirable for a delin-eation method to produce connected segmentations. Unfortunately, however,the segmentations produced by the RIFT are not guaranteed to be connected.An example of this situation is shown in Figure 6.9. Despite the fact that theRIFT may produce non-connected segmentation, the main conclusion of thepaper still holds: applying a few iterations of the proposed relaxation proce-
45
(a) (b)
Figure 6.8: Segmentation of the liver in an MR volume image, using the relaxed IFTproposed in Paper IV. Seed-points representing liver and background were placedinteractively by a human operator. (a) Due to noise and low contrast between the liverand adjacent organs, e.g., the heart, the IFT produces a highly irregular segmentation.(b) A smoother segmentation, obtained by applying ten iterations of the relaxationprocedure proposed in Paper IV.
A B C
D E F
1 1
5 5 5
4 4
(a)
A B C
D E F
1 1
5 5 5
4 4
(b)
A B C
D E F
1 1
5 5 5
4 4
(c)
Figure 6.9: An example of the RIFT method producing non-connected segmentations.See Paper IV for notation. (a) A graph with two seeds, A and D, drawn as rectangles.(b) Initial segmentation, L0, obtained by the IFT using an additive path cost function.(c) Defuzzified segmentation after one relaxation step, with β = 1. The node F is anisolated region, not containing any seed-points.
dure tends to produce much more predictable results in image regions withnoise and weak edges.
6.5 Fast computation of boundary verticesIn Papers III and IV, we use the IFT for seeded segmentation, and show thatthe segmentation results obtained by using the IFT can be improved in var-ious ways by modifying the labels of vertices close to the boundary of thesegmented regions. We define the boundary vertices of a labeling L as the set
V \{v | L(v) = L(w) for all w ∈N (v)} .
46
(a) (b) (c)
Figure 6.10: (a) A slice from an MR volume image. (b) Segmentation of the spleen,obtained by using the IFT as a seeded segmentation method. Seed-points representingobject and background are shown in gray. (c) The boundary vertices of the segmenta-tion. In Papers III and IV, we show that the segmentations obtained with the IFT canbe improved in various ways by modifying the labels of vertices close to the boundary.In Paper VI, we show that the set of boundary vertices may be obtained on-the-fly, asa by-product of the DIFT algorithm. This facilitates very efficient implementations ofthe methods proposed in Papers III and IV.
See Figure 6.10. The set of boundary vertices is usually much smaller thanthe total number of vertices, |V |. Since the methods proposed in Paper IIIand IV operate only in a small region around the boundary vertices, they maybe computed efficiently if the set of boundary vertices is known. 1
An efficient implementation of the IFT requires O(|V |) operations to com-pute an optimal path cost forest for the entire graph. For large data sets, suchas volume images produced by standard MRI or CT scanners in medical appli-cations, this is not fast enough for interactive feedback with today’s hardware.To achieve interactive feedback, we need a differential implementation of theIFT, as proposed by Falcão et al. [8].
Returning to the problem of computing boundary vertices, we note that forany given vertex, it is easy to check if it belongs to the set of boundary ver-tices by comparing the label of the vertex to the labels of its neighbors. Thus,a trivial algorithm for obtaining the boundary is to iterate over all verticesand check whether they belong to the set of boundary vertices. This however,requires O(|V |) operations, and thus the advantage of the differential imple-mentation is lost. In Paper VI, we show that the boundary vertices may becomputed as a by-product of the DIFT algorithm, at virtually no additionalcost. This allows the methods in Papers III and IV to be implemented effi-ciently in conjunction with the DIFT, thereby making these methods muchmore attractive for interactive segmentation.
1Note that the notation in Paper VI differs slightly from the notation in this thesis summary. InPaper VI, ∂L is defined as the set of boundary vertices, rather than the set of boundary edges.
47
6.6 Generalized hard constraints for graph partitioningAs mentioned in Chapter 3, hard constraints for interactive segmentation aretypically given in one of two forms. In the context of graph based segmenta-tion, these may be defined as follows:
Boundary constraints The cut is required to include a specified subset of thegraph edges.
Regional constraints The cut is required to separate all elements in a speci-fied subset of the graph vertices.
Both types of constraints have found wide-spread use. However, differentcomputational strategies have usually been employed for these two cases, i.e.,methods developed for finding segmentations that satisfy boundary constraintshave typically not been applicable to problems with regional constraints, andvice versa.
In Paper VII, we introduce a new type of hard constraints for image seg-mentation, which we call generalized constraints. We define a constraint as apair of vertices, that must be separated by any feasible cut. Additionally, werequire a feasible cut to be free from over-segmentation. We say that a cut isover-segmented with respect to a set of constraints if it is possible to removeone or more edges from the cut without violating any of the constraints. Weshow that both regional and boundary constraints can be seen as special casesof the proposed generalized constraints. Thus, the work in Paper VII unifiesand generalizes these two paradigms, which previously have been seen as un-related.
Additionally, we present an efficient method for computing a cut that satis-fies a set of constraints. This method is summarized briefly in the following.Let S be a cut on G, let e ∈ S, and let G′ = (V,E \ (S \ {e})). We define thesegment Se of S corresponding to e as
Se = {ev,w | ev,w ∈ S, v∼G′
w} . (6.2)
If we remove a segment from a cut, then the resulting set of edges is still a cut(in the sense of Definition 2 in Section 4.3.2). To compute a cut that satisfiesa set of generalized hard constraints C, we may start from the cut S = E, i.e.,a complete over-segmentation where every vertex in the graph is an isolatedcomponent. From this initial cut, we then repeatedly identify segments thatcan be removed from the cut without violating any of the constraints, andremove them. We show that when no more segments can be removed, theremaining edges S form a cut that satisfies the constraints.
At each step of this algorithm, there are usually several segments that arepotential candidates for removal. The order in which the segments are re-moved affect the final segmentation result, and so we are interested in finding
48
(a) (b) (c)
Figure 6.11: Interactive segmentation of the liver in a slice from a CT volume image,using three different interaction paradigms. All segmentations were computed usingthe algorithm proposed in Paper VII. (a) Segmentation using boundary constraints.The black dots indicate graph edges that must be included in the segmentation bound-ary. (b) Segmentation using regional constraints. Black and white dots indicate back-ground and object seeds, respectively. (c) Segmentation using generalized constraints.Each constraint is displayed as two black dots connected by a line.
an ordering that leads to cuts that are “good” in some sense. In the proposedalgorithm, we remove, at each step, the segment corresponding to an edgefor which the edge weight is maximal. While this strategy is based on greedychoices, we show that it leads to cuts that are globally optimal in the sensethat they minimize
maxe∈S
(W (e)) (6.3)
among all cuts that satisfy C.
49
7. Conclusions
In this chapter, the work in this thesis is concluded with a summary of theresults and some suggestions for future work.
7.1 Summary of contributionsThe work presented in this thesis contributes to the field of graph based inter-active segmentation in a number of ways:
• By modifying existing delineation methods to improve their performanceon images with noisy or missing data.• By utilizing fuzzy concepts to compute segmentations from which feature
estimates can be made with increased precision.• By unifying and generalizing two common paradigms for user interaction:
regional constraints and boundary constraints.
7.2 Future workThe outcome of the experiments in Paper IV demonstrates the importance ofsmoothness as a criterion for delineation. To obtain a smooth segmentation,we may either incorporate some form of smoothness condition in the delin-eation method itself, or perform smoothing as a post-process. While the for-mer approach is appealing from a theoretical perspective, the latter approachcan yield good segmentation results with only a small penalty in computationtime, as shown in Paper IV. The method in Paper IV, however, does not nec-essarily preserve the connectedness of the segmentation. An interesting chal-lenge for future work is to formulate an efficient smoothing procedure thatguarantees connected results or, more generally, preserves a set of generalizedconstraints.
In Papers III and V, located cuts are used as an intermediate representationin the process of generating a vertex coverage segmentation. It appears fruit-ful, however, to consider located cuts as a graph-based object representationin its own right. An interesting direction for future work is to formulate tools,e.g., feature estimators or morphological operators, that operate directly onlocated cuts.
51
In the author’s opinion, the most important contribution in this thesis is theintroduction of generalized hard constraints, which unify and generalize thetwo most common forms of user input. As pointed out in Section 3.2, thisfacilitates the development of general purpose methods for graph partitioningthat are not restricted to a particular paradigm for user input. In Paper VII, wepresent one method for computing cuts that satisfy a set of generalized con-straints. The field of possible such algorithms, however, is wide and remainsto be explored.
52
Summary in Swedish
Grafbaserade metoder för interaktiv bildsegmenteringDatoriserad bildanalys handlar om att utvinna information ur bilder, lagradei digital form i en dator. När vi pratar om digitala bilder menar vi oftastden typ av bilder som vi tar med vanliga digitalkameror. I takt med atttekniken för bildtagning har utvecklats, har dock begreppet digital bildblivit mer generellt. Ett bra exempel på detta är utvecklingen av metoderför att skapa tredimensionella volymbilder. Metoder som datortomografi(CT) och magnetresonanstomografi (MRI) används idag rutinmässigt påsjukhus över hela världen för att skapa högupplösta tredimensionella bilderav människokroppens inre.
Ett grundläggande problem inom bildanalys är segmentering, d.v.s, att iden-tifiera och separera relevanta föremål och strukturer i en bild. Segmenteringär ofta ett tidigt steg i bildanalysprocessen och ligger till grund för fortsatt be-handling och informationsutvinning. Trots att segmentering är ett ämne somhar studerats intensivt under många år, finns det fortfarande inga helt automa-tiska metoder som ger tillfredsställande resultat på godtyckliga bilder. Ett sättatt lösa problemet är att använda interaktiva segmenteringsmetoder, där en an-vändare styr och övervakar segmenteringsprocessen. Målet för dessa metoderär att minimera den tid som användaren måste lägga på att åstadkomma ettönskat segmenteringsresultat. Samtidigt är det viktigt att användaren har fullkontroll över segmenteringen, då det är användaren som ansvarar för att resul-tatet blir korrekt.
I denna avhandling presenteras ett antal metoder för interaktiv segmenter-ing. Dessa metoder baseras på grafteori. En bild representeras då av en graf,där varje bildelement representeras av en nod, och där angränsande bildele-ment kopplas samman av bågar. Bågarna i grafen tilldelas skalärvärda vikter,som beräknas från bildinnehållet. Denna diskreta bildrepresentation är rela-tivt enkel att hantera matematiskt och lämpar sig därför väl för att formuleraeffektiva algoritmer. Fokus i avhandlingen ligger på generell metodutveck-ling. För att illustrera den praktiska nyttan med de metoder som presenteras iavhandlingen används exempel från medicinska tillämpningar.
53
Sammanfattning av bidragNedan ges kortfattade sammanfattningar av de artiklar som ingår i avhandlin-gen.
Modeller för styrning av segmenteringsprocessenDet finns flera olika modeller för hur användaren kan styra segmenterings-processen. I det enklaste fallet kan det handla om att ange ett antal parame-trar, som styr en automatisk segmenteringsmetod. Denna typ av interaktionger dock i allmänhet inte tillräcklig kontroll över resultatet. I den här avhan-dlingen har vi istället fokuserat på metoder där användaren gör olika typerav markeringar i bilden. Två olika typer av markeringar är vanligt förekom-mande:
1. Användaren markerar delar av gränsen mellan olika föremål i bilden.2. Användaren anger, för ett litet antal bildelement (fröpunkter), en etikett
som anger vilket föremål bildelementet tillhör.
De flesta befintliga segmenteringsmetoder kan bara hantera den ena av dessatyper av markeringar. Ett viktigt resultat i avhandlingen är att dessa två typerav markeringar kan hanteras som specialfall av en mer generell form av mark-eringar. Detta gör det möjligt att utveckla generella segmenteringsmetoder,som inte är begränsade till en enda typ av inmatning från användaren. Ett ex-empel på en sådan segmenteringsmetod presenteras i Artikel VII.
Segmentering med vägbaserade avståndsmåttAvståndet mellan två noder i en graf kan definieras som längden, eller kost-naden, för den kortaste vägen genom grafen mellan dessa noder. Kostnadenför en väg genom grafen kan definieras på flera olika sätt, t.ex. som summanav bågvikterna, eller den högsta bågvikten, längs vägen. Avståndet mellan tvånoder i grafen beror i dessa fall inte enbart på nodernas position i bilden, utanäven på bildinnehållet. Sådana avstånd kan beräknas effektivt med t.ex. Dijk-stras algoritm.
Många kraftfulla segmenteringsmetoder bygger på beräkningar av avståndmellan noderna i en graf. Ett exempel är live-wire-metoden, där användarenmarkerar en rad punkter på föremålets kontur. Den fullständiga konturen ska-pas sedan genom att beräkna den kortaste vägen genom de punkter använ-daren markerat. Genom att definiera ett lämpligt avståndsmått kan konturentvingas att följa till exempel skarpa kanter i bilden. Live-wire-metoden ärursprungligen formulerad för segmentering av tvådimensionella bilder. I Ar-tikel I föreslår vi en vidareutveckling av metoden, som identifierar ytor avföremål i volymbilder.
Vägbaserade avståndsmått kan också användas för interaktiv segmenter-ing med fröpunkter. Varje nod tilldelas då samma etikett som den närmaste
54
fröpunkten, enligt något avståndsmått. Sådana segmenteringar kan beräknaseffektivt, och har visat sig ge bra resultat i många tillämpningar. För brusigabilder, och bilder med dålig kontrast, resulterar dock metoden ofta i seg-menteringar med ojämna kanter. I Artikel IV föreslår vi en metod som min-skar dessa problem, genom att efterbehandla segmenteringsresultatet. I Ar-tikel VI redovisas tekniska detaljer, som gör det möjligt att beräkna dettaefterbehandlingssteg på ett effektivt sätt.
I vanlig, euklidisk geometri är den kortaste vägen mellan två punkter unik– det är en rak linje mellan punkterna. Detta gäller dock i allmänhet inteför vägbaserade avståndsmått, där det kan finnas många vägar med sammakostnad. Detta innebär att segmenteringsmetoder som använder vägbaseradeavståndsmått inte alltid har en unik lösning. I Artikel II undersöker vi hur välolika vägbaserade avståndsmått approximerar den unika euklidiska lösningen.Resultaten visar att avståndsmått baserade på sekvenser av grannrelationerhar goda egenskaper i detta avseende.
Noggranna mätningar med oskarpa avbildningarMatematiskt kan en segmentering beskrivas som en avbildning från mängdenav bildelement till en mängd av föremålsklasser som finns i bilden (till exem-pel {förgrund, bakgrund}). Vanligtvis är denna avbildning skarp, d.v.s., varjebildelement avbildas på exakt ett element i mängden av föremål. Det finnsdock forskning som visar att oskarpa avbildningar, där varje bildelement kanavbildas med olika styrka på flera olika föremålsklasser, kan vara fördelak-tiga. Sådana avbildningar kan ge förbättrad precision när det gäller att mätageometriska egenskaper hos de segmenterade föremålen. I synnerhet har detvisats att avbildningar baserade på täckningsgrad, där styrkan av sambandetmellan ett bildelement och en föremålskategori bestäms av hur stor del avbildelementet som täcks av detta föremål, är särskilt lämpade för sådana mät-ningar. Artiklarna III och V handlar om att översätta dessa begrepp till bilderrepresenterade som grafer.
55
Errata
In Paper IV, page 7, the statement “If the segmentations within in Γ are com-pletely disjoint, then the fuzziness if Γ̂ is 1” is incorrect. In this case the fuzzi-ness is 1/|Γ|. The correct statement is “If each image element belongs to theforeground in exactly half of the segmentations in Γ, then the fuzziness of Γ̂
is 1”.
57
Bibliography
[1] R. Audigier and R. A. Lotufo. Seed-relative segmentation robustness of wa-tershed and fuzzy connectedness approaches. In A. X. Falcão and H. Lopes,editors, Proceedings of the 20th Brazilian Symposium on Computer Graphicsand Image Processing, pages 61–68. IEEE Computer Society, 2007.
[2] R. Bellman. On a routing problem. Quarterly of Applied Mathematics,16(1):87–90, 1958.
[3] Y. Boykov and G. Funka-Lea. Graph cuts and efficient N-D image segmentation.International Journal of Computer Vision, 70(2):109–131, 2006.
[4] C. Couprie, L. Grady, L. Najman, and H. Talbot. Power watersheds: A unifyinggraph-based optimization framework. IEEE Transactions on Pattern Analysisand Machine Intelligence, 2010. doi: 10.1109/TPAMI.2010.200.
[5] J. Cousty, G. Bertrand, L. Najman, and M. Couprie. Watershed cuts: Thinnings,shortest path forests, and topological watersheds. IEEE Transactions on PatternAnalysis and Machine Intelligence, 32(5):925–939, 2010.
[6] E. W. Dijkstra. A note on two problems in connexion with graphs. NumerischeMathematik, 1:269–271, 1959.
[7] A. Eriksson, C. Olsson, and F. Kahl. Normalized cuts revisited: A reformulationfor segmentation with linear grouping constraints. Journal of MathematicalImaging and Vision, 39(1):45–61, 2010.
[8] A. X. Falcão and F. P. G. Bergo. Interactive volume segmentation with dif-ferential image foresting transforms. IEEE Transactions on Medical Imaging,23(9):1100–1108, 2004.
[9] A. X. Falcão, J. Stolfi, and R. A. Lotufo. The image foresting transform: The-ory, algorithms, and applications. IEEE Transactions on Pattern Analysis andMachine Intelligence, 26(1):19–29, 2004.
[10] A. X. Falcão and J. K. Udupa. A 3D generalization of user-steered live-wiresegmentation. Medical Image Analysis, 4(4):389–402, 2000.
[11] A. X. Falcão, J. K. Udupa, and F. K. Miyazawa. An ultra-fast user-steered imagesegmentation paradigm: Live wire on the fly. IEEE Transactions on MedicalImaging, 19(1):55–62, 2000.
59
[12] A. X. Falcão, J. K. Udupa, S. Samarasekera, S. Sharma, B. E. Hirsch, and R. A.Lotufo. User-steered image segmentation paradigms: Live wire and Live lane.Graphical Models and Image Processing, 60(4):233–260, 1998.
[13] L. R. Ford and D. R: Fulkerson. Flows in Networks. Princeton University Press,1962.
[14] L. Grady. Space-Variant Machine Vision - A Graph Theoretic Approach. PhDthesis, Boston University, 2004.
[15] L. Grady. Random walks for image segmentation. IEEE Transactions on PatternAnalysis and Machine Intelligence, 28(11):1768–1783, 2006.
[16] L. Grady. Minimal surfaces extend shortest path segmentation methods to 3D.IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2):321–334, 2010.
[17] L. Grady and M. P. Jolly. Weights and topology: A study of the effects of graphconstruction on 3D image segmentation. In D. Metaxas et al., editors, Pro-ceedings of MICCAI 2008, volume 1 of LNCS, pages 153–161. Springer-Verlag,2008.
[18] M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. Inter-national Journal of Computer Vision, 1(4):321–331, 1988.
[19] C. Kauffman and N. Piché. Seeded ND medical image segmentation by cellularautomaton on GPU. International Journal of Computer Assisted Radiology andSurgery, 5(3):251–262, 2009.
[20] Y. Li, J. Sun, C. K. Tang, and H. Y. Shum. Lazy snapping. ACM Transaction onGraphics, 23:303–308, 2004.
[21] W. E. Lorensen and H. E. Cline. Marching cubes: A high resolution 3D surfaceconstruction algorithm. Computer Graphics, 21(4):163–169, 1987.
[22] P. A. Miranda and A. X. Falcão. Links between image segmentation basedon optimum-path forest and minimum cut in graph. Journal of MathematicalImaging and Vision, 35(2):128–142, 2009.
[23] S. D. Olabarriaga and A. W. Smeulders. Interaction in the segmentation ofmedical images: A survey. Medical Image Analysis, 5(2):127–142, 2001.
[24] M. Poon, G. Hamarneh, and R. Abugharbieh. Efficient interactive 3D livewiresegmentation of objects with arbitrarily topologies. Computerized MedicalImaging and Graphics, 32(8):639–650, 2008.
[25] A. Rosenfeld. Picture Processing by Computer. Academic Press, New York,1969.
[26] J. A. Sethian. Level set methods and fast marching methods. Cambridge Uni-versity Press, 1999.
60
[27] J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 22(8):888–905, 2000.
[28] A. K. Sinop and L. Grady. A seeded image segmentation framework unifyinggraph cuts and random walker which yields a new algorithm. In Proceedings ofthe 11th International Conference on Computer Vision (ICCV). IEEE ComputerSociety, IEEE, 2007.
[29] N. Sladoje and J. Lindblad. Estimation of moments of digitized objects withfuzzy borders. In F. Roli and S. Vitulano, editors, Proceedings of the 13th Inter-national Conference on Image Analysis and Processing (ICIAP), volume 3617of LNCS, pages 188–195. Springer-Verlag, 2005.
[30] N. Sladoje and J. Lindblad. High-precision boundary length estimation by utiliz-ing gray-level information. IEEE Transactions on Pattern Analysis and MachineIntelligence, 31(2):357–363, 2009.
[31] M. Sonka, V. Hlavac, and R. Boyle. Image Processing, Analysis, and MachineVision. International Thomson Publishing, 1999.
[32] R. Strand. Weighted distances based on neighbourhood sequences. PatternRecognition Letters, 28(15):2029 – 2036, 2007.
[33] R. Strand. Distance Functions and Image Processing on Point-Lattices. PhDthesis, Uppsala University, Centre for Image Analysis, 2008.
[34] J. K. Udupa, P. K. Saha, and R. A. Lotufo. Relative fuzzy connectedness andobject definition: Theory, algorithms, and applications in image segmentation.IEEE Transactions on Pattern Anaysis and Machine Intelligence, 24(11):1485–1500, 2002.
[35] E. Vidholm. Visualization and Haptics for Interactive Medical Image Analysis.PhD thesis, Uppsala University, 2008.
[36] C. Zahn. Graph theoretical methods for detecting and describing gestalt clusters.IEEE Transactions on Computers, 20(1):68–86, 1971.
61
![A robust graph-based segmentation method for breast tumors ... Robust Graph-Based... · Region based segmentation methods based graph theory have also been proposed [21,22]. It is](https://static.documents.pub/doc/80x56/601d8da62474fc7d0a5941f9/a-robust-graph-based-segmentation-method-for-breast-tumors-robust-graph-based.jpg)
