A Hierarchical Approach for Fast and Robust Ellipse Extraction

Pattern Recognition 41 (2008) 25122524www.elsevier.com/locate/pr

A hierarchical approach for fast and robust ellipse extractionF. Mai, Y.S. Hung, H. Zhong, W.F. Sze

Department of Electrical and Electronic Engineering, The University of Hong Kong, Pokfulam Road, Hong KongReceived 16 June 2007; received in revised form 5 December 2007; accepted 28 January 2008

Abstract

This paper presents a hierarchical approach for fast and robust ellipse extraction from images. At the lowest level, the image is described as aset of edge pixels, from which line segments are extracted. Then, line segments that are potential candidates of elliptic arcs are linked to form arcsegments according to connectivity and curvature conditions. Next, arc segments that belong to the same ellipse are grouped together. Finally,a robust statistical method, namely RANSAC, is applied to t ellipses to groups of arc segments. Unlike Hough Transform based algorithms,this method does not need a high dimensional parameter space, and so it reduces the computation and storage requirements. Experiments onboth synthetic and real images demonstrate that the proposed method has excellent performance in handling occlusion and overlapping ellipses. 2008 Elsevier Ltd. All rights reserved.

Keywords: Ellipse extraction; Elliptic arcs; Arc segments; RANSAC

1. Introduction

Circular and elliptical objects are frequently seen in the realworld.An ellipse or a circle in 3D space is projected to an ellipsein an image. Therefore, the extraction of ellipses from imagesis a key problem in computer vision and pattern recognition.Applications include determination of the 3D position of theobject or the camera pose [1], and recognition of trafc signsor manufactured objects in industrial environments [24].

All existing approaches for detecting ellipses in an imagestart from extracting edge pixels from the image, and thenellipses are tted to the edge pixels. Generally they can begrouped into three classes: Hough Transform (HT) based algo-rithms [513], GeneticAlgorithm (GA) based methods [1418],and edge-following algorithms [1923].

HT is one of the most widely used approaches for detectingparametric curves or other predened shapes from a planar setof points. The Duda and Harts formulation of the HT [5] iscommonly referred to as the Standard Hough Transform (SHT),in which each of the edge points votes for the parameters,followed by a search for peaks in the quantized parameter space.

Corresponding author. Tel.: +852 2859 2711; fax: +852 2559 8738.E-mail addresses: [email protected] (F. Mai), [email protected]

(Y.S. Hung), [email protected] (H. Zhong), [email protected] (W.F. Sze).

0031-3203/$30.00 2008 Elsevier Ltd. All rights reserved.doi:10.1016/j.patcog.2008.01.027

In spite of its popularity owing to its simple theory of operationand its robustness to occlusion, the SHT is largely impracticalfor ellipse extraction due to its heavy computational burden andlarge storage requirement.

In order to detect ellipses effectively and quickly using HT,many variations have been suggested. One way is to use a sub-set of the edge points for voting rather than all of them, such asthe Randomized Hough Transform (RHT) [6] and Probabilis-tic Hough Transform (PHT) [7]. In Ref. [8], the RHT shows itsimprovements over SHT on accuracy and computational com-plexity, but it is easy to nd examples where the RHT cannotdetect all obvious ellipses in the scene, and as the number ofellipses increases, the accuracy gets poorer. The PHT workswell with the assumption that the pixels of the edge points aredistributed evenly throughout the image, but this condition ishard to be satised in real images. Furthermore, PHT needs acontext-dependent criterion to stop sampling.

Another way to overcome the excessive time and spacerequirements for HT-based ellipse extraction methods is todecompose the ve-dimensional parameter space into severalsub-spaces of lower dimensions. The decomposition is achievedby exploiting geometric characteristics of the ellipses, such asslope [9,10] and symmetry [11,12]. These properties help toaccelerate the process, but they constrain the application atthe same time. For example, it is quite often the case that the

F. Mai et al. / Pattern Recognition 41 (2008) 25122524 2513

ellipses in an image are occluded so that the symmetry charac-teristic is no longer available.

The second group, namely GA-based methods, extracts el-lipses by solving an optimization problem [1418]. AlthoughGA-based techniques might be effective for detecting ellipseswith partial occlusion, unfortunately, they consume excessivecomputational time. Furthermore, an inherent weak point of GAis that there is always a risk of nding a suboptimal solution.

The third group is the edge-following methods. They requirean input of lists of chained edge points produced from the orig-inal edge image after edge thinning and linking operations. TheUpWrite method [19] belongs to this group, and is robust forimages containing perturbation noise and not computationallydemanding, but it is sensitive to the resolution radius. Further-more, according to the experiments reported in Ref. [20], itmay have difculties for overlapping features that obscure theellipses. Kim et al. [20] proposed a fast and robust ellipse ex-traction method. Although the method can extract ellipses fromcomplicated images, it depends on the tangent information,which is quite sensitive to noise in digital images. In Ref. [21],Rosin proposed a nonparametric technique which segments adigital arc into short line segments and elliptical arc segments,and Ji et al. [3] merged these short segments by a statisticalmethod. However, they limit the candidate segments for merg-ing to those who are close together at least at one side. Thislimitation causes difculties when there is occlusion separat-ing two arc segments far apart at both sides. Furthermore, theassumption underlying the statistical model may not always bevalid for real images. In Ref. [22], the edge-following techniqueis combined with HT. They detect a circle osculating to an edgesegment by the HT, nd edge pixels tangent to it, ret an ellipseto the detected pixels, and repeat this process. This approachworks well for an image containing a single ellipse, but it mayhave difculties for scenes containing multiple ellipses.

Note that the edge-following based methods extract ellipsesin a bottom-up way. Contrary to the HT-based methods whichview the whole image globally, these bottom-up methods arebased on local features and pay little attention to global infor-mation.

The work presented here is an extension and improvementon that presented in Ref. [24]. It is based on the idea of edgefollowing, but the difculty due to the lack of global view inexisting methods is addressed by means of a new hierarchicalapproach. In this framework, the image is described as a setof edge pixels at the lowest level, from which line segmentsare extracted. Then line segments that are potential candidatesof elliptic arcs are linked to form arc segments according toconnectivity and curvature conditions. Next, arc segments thatbelong to the same ellipse are grouped together. Finally, a mod-ied RANSAC (RANdom SAmple Consensus) approach [25]is applied to t ellipses robustly. Experimental results demon-strate that our method is capable of accurately extracting mul-tiple ellipses from an image with complex background, evenwhen the ellipses are partially overlapped or occluded.

This paper is organized as follows. In Section 2, we de-scribe the proposed hierarchical approach consisting of fourdifferent stages. In Section 3, experimental results on both syn-

thetic data and real images are provided to illustrate the perfor-mance of our method in comparison with ve existing methods.Section 4 contains some concluding remarks.

2. The hierarchical approachfrom edge points to ellipses

In this paper, we adopt a hierarchical approach to ellipsedetection, generating progressively higher level descriptions ofthe data. The hierarchical algorithm, as shown in Fig. 1, consistsof four stages:

(i) Line segment extractionat the lowest level, the image isdescribed as a set of edge pixels, fromwhich line segmentsare extracted.

(ii) Line segment linkingline segments that are potentialcandidates of elliptic arcs are linked to form arc segments.

(iii) Arc segment groupingarc segments that belong to thesame ellipse are grouped together.

(iv) Ellipse ttingmodied RANSAC [25] is used to ro-bustly t ellipses to arc segment groups.

The detailed operations in each stage are described in thefollowing subsections.

2.1. Line segment extraction

The rst stage, line segment extraction, is to represent theboundaries of the objects in the images by piecewise linearsegments. There are many methods for line extraction [2629].

Ellipse fitting

Input image

Edge points

Edge detection

Line segments

Line segment extraction

Elliptic arc segments

Line segment linking

Groups of arc segments belonging to thesame ellipse

Arc segment grouping

Extracted ellipses

Fig. 1. The four stages in the proposed hierarchical approach for ellipseextraction method.

2514 F. Mai et al. / Pattern Recognition 41 (2008) 25122524

Wewill employ a pixel-connectivity edge-linking based method[26]. First, Canny edge detector [30] is used to nd edges withhysteresis thresholds of 0.1. Secondly, edge pixels are linkedtogether into lists of sequential edge points with one list foreach edge contour segment of at least 10 pixels long. A contoursegment starts (or stops) at an open end or a junction of morethan two contours. Thirdly, each contour segment is approxi-mated by a series of piecewise linear segments by the iterativeend-point t algorithm [31]. Finally, a line segment cleaningprocess is performed to remove line segments shorter than aminimum length (of 10 pixels by default).

In our software implementation, we use Kovesis code(available from http://www.csse.uwa.edu.au/pk/research/matlabfns) for line segment extraction with a slight modica-tion to output not only the extracted line segments, but alsotheir corresponding supporters. Here a supporter of a certainfeature (line segment, arc segment, or ellipse) is generally de-ned as an edge point sufciently close to the feature wherecloseness is measured by the Euclidean distance of the edgepoint to the feature.

2.2. Line segments linking

2.2.1. Two conditions for linkingAfter line segment extraction, the boundaries of the objects

in the images are represented by line segments. The purposeof the line segment linking stage is to link line segments intoarc segments, where an arc segment is a segment of an ellipse,represented by a sequence of n(n3) connected line segmentssatisfying the following connectivity and curvature conditions.

2.2.1.1. Connectivity condition. Given two line segments l1and l2 each with two end points, the proximity d of the two linesegments is measured by the smallest of the four (Euclidean)distances between their end points.

The connectivity condition is

dd0, (1)

where d0 is a predened threshold for closeness.Wewill refer totwo line segments satisfying the above condition as connectedline segments. A sequence of n connected line segments isdened to be an ordered n-tuple of line segments in whicheach intermediate line segment is connected to its predecessorin the n-tuple at one end and to its successor at the other endand is otherwise not connected to any other line segment ofthe n-tuple, whereas the rst and the last line segments may beconnected to each other at their free ends.

2.2.1.2. Curvature condition. If a sequence of connected linesegments traces an arc segment, the angle between two con-nected line segments represents the curvature of the arc. Thisangle should not be too small in order for the sequence ofline segments to represent a genuine curve, but yet not toolarge so that the curve do not have a sharp turning point. Letli (i=1, . . . , n) be a sequence of connected line segments, and

let i (i = 1, . . . , n 1) be the angle measured from li to li+1.Then, the above curvature condition can be stated as

min |i |max (i = 1, . . . , n 1). (2a)Furthermore, if the sequence of connected line segments rep-resents a convex arc (which is the case for an elliptic arc), thecurvature should be of the same sign throughout the sequence.Hence, a second condition for curvature is

1, . . . , n1 are of the same sign. (2b)

The two curvature conditions mentioned above dene only aloose criterion for a sequence of line segments to represent aconvex arc. To screen out convex arc that does not resemblean elliptic arc, we t an ellipse to all the supporters of all theline segments in the sequence li (i = 1, . . . , n), and impose anadditional supporter ratio criterion:

r > r0, (2c)

where

r = mM

(3)

is the ratio of the number of supporters supporting both thetted ellipse and the line segments in the sequence (denotedm) to the total number of supporters of all line segments in thesequence (denoted M), and r0 is a predened threshold.

If r attains its maximum value of 1, then all the line segmentsin the sequence t an ellipse closely, and thus they should belinked together as an elliptic arc segment. In practice, if r isgreater than a given threshold r0 (=0.9 by default), we shallaccept the sequence li (i = 1, . . . , n) as a candidate elliptic arcfor further processing in the ellipse extraction process.

Note that by means of Eqs. (2a)(2c), we are applying in-creasingly strict conditions on the curvature of the sequence ofline segments for it to represent an elliptic arc segment. Thisstrategy of progressively tightening the curvature condition en-ables sequences that are unlikely to come from an ellipse to bescreened out in a computationally efcient manner.

2.2.2. The linking algorithmThe algorithm implementing the above process of linking a

sequence of line segments into an elliptical arc segment con-sists of two steps: arc initialization and arc growing. In the arcinitialization step, a line segment pair satisfying the connectiv-ity condition (1) and the curvature condition (2a) is identied.Starting from this initial line segment pair, the arc segment isgrown in the second step by taking in line segments satisfy-ing conditions (1) and (2a)(2c). This two-step process will berepeated to get the arc segments one by one, as described inAlgorithm 1 below, with the following notations:

L the set of all line segments obtained in the line segmentextraction stage.

Ao a set of line segments from which connected line segmentpairs are drawn for arc initialization.


Ae a set of elliptic arc segments to be found satisfyingthe connectivity condition (1) and curvature conditions(2a)(2c).

Algorithm 1.1. Set Ao = L, Ae = .2. While Ao = , repeat Steps 2.1 and 2.2:

(2.1) Arc initialization:Choose any line segment l1 Ao. Find a line seg-ment l2 L connected to one end of l1satisfying (1) and (2a). If no such line segment exists,remove l1 from Ao. Repeat this processuntil s = (l1, l2) is found, or else stop if Ao = .(2.2) Arc growing:(i) Given sequence s = (l1, . . . , li ) (i2), identify

from L the set of line segments {li+1}connected to li at the tail end (i.e., the end of li notconnected to li1) and satisfyingconditions (2a,2b,2c).If {li+1} = , select the segment with the highestsupporter ratio (3) as li+1, and updates = (l1, . . . , li , li+1).Repeat this process until s cannot be grown anyfurther in length at the tail end.

(ii) Reverse the indices of s so that l1 becomes the tailend of the sequence, and repeat Step2.2(i).

(iii) If the length of s3, enter s into Ae. In any case,remove all line segments in s fromAo.

end while3. Output the set of elliptic arc segments Ae.end of Algorithm 1

In step 2.2.(i), the highest supporter ratio condition is meant tohandle the case of a junction where more than one line segmentqualies for arc growing. Locking on the highest supporter ratiomay have the risk of nding suboptimal solution. An alternativeapproach is to allow multiple solutions for arc growing whenwe come to a junction point, but at the expense of a morecomplex procedure for arc growing.

2.3. Arc segment grouping

Algorithm 1 detects all arc segments represented by se-quences of line segments that are locally linked together hav-ing the form of an elliptical arc. In this section, we look forglobal grouping of arc segments that are not connected (e.g.due to occlusion) but may come from the same ellipse. Weagain apply the idea of supporter ratio for the purpose of arcsegment grouping. Specically, given a pair of arc segments,s1 = (l11 , . . . , l1n1) and s2 = (l21 , . . . , l2n2), an ellipse is ttedto all the supporters of line segments of both s1 and s2 usingleast-squares technique. Suppose the line segments of each ofthe sequence si (i = 1, 2) has a total of Mi supporters, amongwhich mi are supporters of the tted ellipse . Dene the

supporter ratio of each arc segment for the ellipse as

ri = miMi

(i = 1, 2). (4)

If both r1 and r2 are larger than the threshold r0, we considerthat the two arc segments belong to the same ellipse and mergethem into a group. Observe that the use of two supporter ratios(one for each arc segment) is to avoid mistakenly merging ashort arc segment with a long one dominant in length even ifthey do not belong to the same ellipse.

This process can be continued to merge further arc segmentsinto a group of arc segments as if the group is a single arcsegment. The extension is obvious.

2.4. Ellipse tting by modied RANSAC

Up to now we have grouped arc segments (with their sup-porters) that presumably belong to the same ellipse. However,due to noise and extraneous features in the image, some sup-porters of the arc segments which do not belong to the ellipsemay have been included in the group. Hence, the supportersof the arc segments can be classied as inliers that t the el-lipse and outliers that do not. In order to extract the ellipse ac-curately using only the inliers, we apply RANSAC [25] (withsome modications) to remove the outliers and recover the el-lipse model.

Let D be the set of supporters of individual line segmentsbelonging to a group of arc segments presumably coming fromthe same ellipse. Our problem is to t an ellipse robustly tothe points of D. Let DI be the set of inlier points that supportsthe ellipse . In order to take into account line segments thatare part of but may have been missed out in our processso far (e.g. due to occlusion, or the sequence of line segmentsbeing too short to qualify as an arc segment), we expand oursearch space and look for inliers for among the set E of allsupporters of all line segments in the image. Let EI be theset of inlier points in E that supports the ellipse . Ideally, wewish to maximize both EI and DI . However, when we searchfor an EI with a larger size, it is possible that in practice thesize of DI may decrease slightly due to noise in the image. Tocater for this, we will use a modied RANSAC algorithm tomaximize the size of the inlier set EI , subject to the constraintthat the size of the inlier set DI is within a small margin 0 ofits maximum size. It is clear that this margin should be small,so that RANSAC does not ignore points in D (supporting arcsegments presumably belonging to the same ellipse) in avourof including more points from the global image. We will usethe following notations in Algorithm 2 below.

D ,E denote the estimated percentages of inliers in D andE, respectively

n() denotes the number of points in a set ()Ntrial denotes the number of trials required to obtain a set

of n inlier points from D with probability p (set to0.95 in our experiments), given by

Ntrial = log(1 p)/ log(1 nD). (5)


l2 l3l4

l5

l6l7

l1

Fig. 2. Example illustrating the reason for using two inlier percentages inthe modied RANSAC Algorithm 2.

Algorithm 2Initialize Ntrial = , k = 0, DI = , EI = , D = 0, E = 0.While kNtrial, repeat(a) Select n(=5) edge points randomly from D and t an

ellipse to the chosen points.(b) Determine the sets of supportersDI D andEI E

for the ellipse , and compute

D =n(DI )

n(D)and E =

n(EI )

n(E)

(c) If DD 0 and EE , then if DD , update D = D; update E = E ; update Ntrial by (5).(d) increase k by 1.Re-estimate the ellipse using all the points in EI ,output and stop.end of Algorithm 2

The main difference between Algorithm 2 and standardRANSAC [25] is that we consider the inlier percentages (i.e.,D and E) in the local set D as well as the global image E,whereas standard RANSAC would look for supporters of amodel only in the global image. Hence, condition (c) in Algo-rithm 2 requires the global inlier percentage E to exceed thecurrent level of support, possibly at the expense of a slight re-duction (by no more than 0) in the local inlier percentage Din order for the new solution to be accepted as a better one. Thereasons for doing so can be illustrated by an example shown inFig. 2, where an ellipse represented by black dotted line existsin the image. However, due to occlusion or deciencies of theedge detector or line segment extractor, only line segments l1to l4 are detected as an elliptic arc segment and line segmentsl5 to l7 are detected as isolated line segments. As a result, l5to l7 will not be linked to any group and (l1, l2, l3, l4) is theonly arc segment available for tting an ellipse. Therefore, anellipse will be tted to the arc segment in accordance withAlgorithm 2, where D includes supporters of l1 to l4 and Econtains supporters of all line segments in the image (includingl5 to l7). If we consider maximizing only the inlier percentage

D , then the green ellipse may be the optimal solution. How-ever, if we take into account the inlier percentage E in theglobal scene, then the dotted ellipse would be a better solution.

3. Experimental results

We perform experiments to evaluate the performance of theproposed algorithm, which is implemented using Matlab. Thechoice of the main parameters used in our method, their inu-ence and the robustness of the method to these parameters areexplained below.

In the linking stage, the threshold for the connectivity condi-tion is set at d0 = 5 pixels. If it is set too small, some line seg-ments that should be linked together may be missed, whereasif it is too large, there is a risk of including line segments thatare not really connected. Our experience is that the choice ofd0 is not critical (e.g. any value in the range of 510 will dofor most of images we tested).

The thresholds for the lower and upper bounds of the cur-vature condition (2a) are min = 0 and max = 0.5 rad. Theseare reasonable choices as min should be small to link line seg-ments representing elliptical arc with low curvature, and maxshould not be too large (say, max [0.5, 0.8]) to allow poly-gons to be mistaken as ellipses. The threshold for detectingsupporters is 2 pixels and the threshold for supporting ratio incurvature condition (2c) is r0 = 0.9.

The threshold 0 = 2% and p = 0.95 are used in Algorithm2. The reason for choosing a small 0 is already explained inSection 2.4 before Algorithm 2. A discussion of the inuenceof the choice of p can be found in Ref. [32]. The above valuesare used as default parameter values in all the examples shownin this paper, and they work well without the need of tuning,which suggests that the proposed method is robust to the choiceof parameters.

To compare the accuracy and computation time of ourmethod with existing techniques, potential candidates that maybe used as reference include:

HT-based methods, such as SHT, RHT, PHT. Edge-following methods, such as UpWrite and Fast Ellipse

Extraction (FEE) method of Ref. [20]. GA-based method (e.g. Ref. [17]), rule-based method (e.g.

Ref. [10]) and other methods (e.g. Ref. [4]).

Since HT-based methods are widely cited in the literature, wewill include SHT, RHT and PHT as reference methods forcomparison. As our method is based on edge-following andit bears some similarity to FEE, we will also adopt UpWriteand FEE as reference methods. Judging from the results givenin Ref. [17], the GA-based method is computationally muchmore demanding requiring computation times of an order ofmagnitude higher than our method. The rule-based techniqueof Ref. [10] is computationally efcient aimed at real-timeapplications, but the underlying four-points-two-directions-one-intersection process of the algorithm tends to detectextraneous ellipses. There are yet other methods (e.g. Ref.[4]) that detect approximate elliptic shapes rather than exactly


Fig. 3. (color online) Testing results for synthetic image 1 (500 400, threeellipses): The original image (denoted black ellipses) superimposed withextraction results (denoted either as red or colored ellipses) obtained using(a) SHT; (b) RHT; (c) PHT; (d) UpWrite; (e) FEE and (f) our method.

tted ellipses and therefore the target of the detection problemis not the same as ours. Hence, we shall employ only SHT,RHT, PHT, UpWrite and FEE as reference methods in ourexperiments for synthetic and real images.

When testing these reference algorithms, we run the sourcecode for the HT-based algorithms and UpWrite (in Linux C)kindly provided by the authors of Ref. [19], directly afteredge detection. We have tried to tune the parameters in thesource code but found that the default values perform the best,and therefore the default values are adopted in the compar-isons. For FEE, we adopt an open-source implementation formLTI-lib (in C + +, available from http://ltilib.sourceforge.net/doc/html/classlti_1_1fastEllipseExtraction.html). The experi-ments are performed on a 2.4GHz Pentium 4 PC.

When comparing the time cost of our method against thereference algorithms, it should be noted that Matlab code runsslower than the corresponding C/C+ + code in iterative algo-rithms (which is the case for our method). Hence, it is expectedthat the computation times for our method would be signi-cantly reduced if the algorithm is re-coded in C or C + +.

Fig. 4. (color online) Testing results for synthetic image 2 (500 400 fourincomplete ellipses): (a) original image and extraction results (denoted as redellipses) obtained using (b) SHT; (c) RHT; (d) PHT; (e) FEE and (f) ourmethod.

3.1. Synthetic images

Synthetic edge maps are generated and tested, four of whichare shown in Figs. 36. The rst image (Fig. 3) contains threecomplete ellipses separated from each other, the second image(Fig. 4) contains four incomplete ellipses to simulate occlusionthe third image (Fig. 5) is more complex with 12 ellipses, someof which are overlapping and four are incomplete, and thereare overlapping lines in the image, and the fourth image (Fig.6) is a corrupted version of Fig. 5 with salt and pepper noise(at noise level = 5%) added to imitate extraneous features inthe image.

To assess the extraction accuracy, we dene an error mea-sure in terms of the area ratio = Sn/Sg , where Sn is the non-overlapping area of the extracted ellipse and the ground truthellipse, and Sg is the area of the ground truth ellipse. Note thatSn includes both the areas bounded by detected ellipse not in-side the ground truth ellipse and vice versa, and therefore mayexceed a value of 1. The lower the error measure, the higherthe extraction accuracy, with = 0 in the case of an exact t.


Fig. 5. (color online) Testing results for synthetic image 3 (800 700 complicated 12 ellipses, with four incomplete). The original image (denoted blackellipses) superimposed with extraction results (denoted either as red or colored ellipses) obtained using (a) RHT; (b) UpWrite; (c) FEE and (d) our method.

Fig. 6. (color online) Testing results for synthetic image 3 with 5% salt and pepper noise (800 700 complicated 12 ellipses, with four incomplete). Theoriginal image (denoted black ellipses) superimposed with extraction results (denoted either as red or colored ellipses) obtained using (a) RHT; (b) UpWrite;(c) FEE and (d) our method.


Table 1Test results for synthetic image 1

Algorithm SHT RHT PHT UpWrite FEE Ours

Time(s) 36.76 0.62 1.64 0.39 1.28 0.55Accuracy#Detected ellipses 2 3 1 3 3 3 F, F 0.12, 0.20, 0.16 F NA 0.05, 0.03, 0.07 0.01, 0.02, 0.01

Note: Accuracies of erroneously extracted ellipses with large (> 0.8) will be denoted by F (for failed extraction). NA means that the accuracy cannot becomputed because the ellipse parameters are not available.

Table 2Testing results for synthetic image 2


Time(s) 38.93 1.05 2.31 0.43 1.33 0.67Accuracy#Detected ellipses 3 1 3 0 6 4 0.04, 0.05, 0.14 0.07 0.04, 0.05, 0.09 0.38, 0.02, 0.69, 0.67, 0.13, 0.04 0.01, 0.03, 0.01, 0.01



Time(s) 1616.03 8.72 26.79 2.03 6.89 2.89Accuracy#Detected ellipses 0 2 0 0 12 12 0.3, F Mean = 0.04 Mean = 0.01Note: For SHT and PHT, the accuracies for all extracted ellipses are poor (> 0.8), so we just use F to denote the extraction failure. For the two extractedellipses by RHT, one has a smaller value (> 0.3), the other one is quite poor (> 0.8), denoted by F. For the accuracy of our method, we show the meanvalue of here for space considerations.



Time(s) 13521.49 44.60 201.94 29.00 6.89 4.64Accuracy#Detected ellipses 0 1 0 0 14 12 F Mean = 0.11 Mean = 0.01

The original images and ellipses extracted by the ve ref-erence methods (i.e., SHT, RHT, PHT, UpWrite and FEE) incomparison with the proposed method are shown in Figs. 36,and the computation times (excluding that for edge detectionfor all methods) and the accuracy assessments are given inTables 14. Note that for UpWrite, we are unable to recoverthe ellipse parameters from the source code, and so cannotbe computed (marked as NA in the tables) and we can onlymake visual judgements on its accuracy based on the displaysgenerated using the source code.

From these results, we observe that: (i) The computationalcost of SHT algorithm is the most expensive and its accuracy ispoor. The other two HT-based algorithms reduce the computa-tion time signicantly, but the ellipse extraction accuracies re-main poor. (ii) For a simple image such as Fig. 3(a), UpWrite,

FEE and our method extract all three ellipses accurately, butUpWrite is a bit faster than ours. RHT also extracts all threeellipses, but with poor accuracy. This is because RHT is in-tended for single ellipse extraction and does not work well formultiple ellipses [17,33]. (iii) For occluded ellipses shown inFig. 4, the proposed algorithm is capable of extracting all thefour incomplete ellipses accurately, whereas all the other meth-ods fail with missing and/or erroneously extracted ellipses. (iv)With increased scene complexity (Figs. 5 and 6), the HT-basedmethods and UpWrite fail to detect any ellipse correctly, whilethe proposed algorithm can still extract all the ellipses accu-rately, even in the noisy image (Fig. 6). In this case, FEE is theonly competitor among the reference methods capable of de-tecting most of the ellipses, but with some erroneously detectedellipses (for Fig. 6) and more computational time.


Fig. 7. Testing results for real image 1 (no-feeding sign) with size 306 281: (a) original image; (b) the extracted line segments and extracted ellipses by (c)RHT; (d) UpWrite; (e) FEE and (f) our method.

Fig. 8. Testing results for real image 2 (car) with size 640 480: (a) original image; (b) the extracted line segments and extracted ellipses by (c) FEE and(d) our method.


Fig. 9. Testing results for real image 3 (spool) with size 640 480: (a) original image and the extracted ellipses by (b) UpWrite; (c) FEE and (d) our method.

Fig. 10. Testing results for real image 4 (ball) with size 640 480: (a) original image; (b) the extracted line segments and the extracted ellipses by (c) FEEand (d) our method.

3.2. Real images

We also test the proposed method on six real images, asshown in Figs. 712. To illustrate the effectiveness of the

method under different scenarios, the images selected includeconcentric ellipses, ellipses with occlusion and overlappingellipses. The elliptical objects may also be embedded in com-plex backgrounds such as leaves or a keyboard. Some ellipses


Fig. 11. Testing results for real image 5 (globe) with size 640 480: (a) original image; (b) the extracted line segments and the extracted ellipses by (c) FEEand (d) our method.

Fig. 12. Testing results for real image 6 (trafc sign) with size 640 480: (a) original image; (b) the extracted line segments; (c) the extracted ellipses byour method.

are intersected by linear features (e.g. edges of a keyboard ormonitor in Figs. 9 and 11, and supporting pole in Fig. 12). Wetested the images with the proposed algorithm and the refer-ence methods as well. From the experimental results, we ndthat: (i) The proposed method can extract the ellipses success-fully. (ii) The reference HT-based methods have difculty withreal images and fail in all our example images. (iii) UpWritecan detect elliptical objects in simple images (for example, itdetects the complete ellipse in Fig. 7 with clean background),but it fails to detect ellipses that are occluded or embeddedin a complex background (it nds one elliptical segment in

Fig. 9, but fails to detect any ellipse in Figs. 8, 1012). (iv) FEEis capable of extracting some ellipses in the images shown inFigs. 711, but it also misses and/or erroneously detects someellipses in the images. A comparison of the computation ef-ciency given in Table 5 shows that our method outperforms thereference algorithms not only in accuracy but also in speed. (v)The detailed time cost of the proposed method in Table 6 liststhe time costs in every stages of the proposed algorithm, fromwhich we can see that the ellipse tting by modied RANSACis quite efcient. The reason is that after the grouping process,the outlier ratio in each group is quite low (usually less than


Table 5Comparison of time cost for real images (s)

Method Image

No-feeding sign ( Fig. 7) Car ( Fig. 8) Spool ( Fig. 9) Ball ( Fig. 10) Globe ( Fig. 11) Trafc sign ( Fig. 12)

SHT RHT 3.9 PHT UpWrite 1.34 6.27 FEE 2.03 7.03 5.70 6.58 5.97 Ours 1.00 7.05 4.02 3.65 3.46 7.23

Note: (1) means the method fails to detect any ellipse.(2) The time costs for SHT, RHT, PHT and UpWrite do not include the time for edge detection, while the time costs for FEE and our method include it.

Table 6Time cost of our method for real images (s)

Stage Image

No-feeding sign ( Fig. 7) Car ( Fig. 8) Spool ( Fig. 9) Ball ( Fig. 10) Globe ( Fig. 11) Trafc sign ( Fig. 12)

Edge detection 0.26 0.95 0.94 0.98 0.96 0.99Line segment extraction 0.47 3.82 1.82 1.70 1.72 5.12Line segment linking+arc segment grouping+ellipse tting by RANSAC

0.27 2.28 1.26 0.97 0.78 1.12

Total time 1.00 7.05 4.02 3.65 3.46 7.23

10%). Therefore, RANSAC performed in each group will nottake much time (less than six iterations) [32].

4. Conclusions

This paper presents a fast and robust method to detect ellip-tical objects in images. We detect the ellipses using a hierarchi-cal approach, whereby the features of an ellipse are detectedprogressively from a local to the global level. The main contri-butions of the proposed method are:

Although each of the four stages of our algorithm is well-established method, choosing appropriate conditions to useat different stages and tting them together into a robusthierarchical procedure for ellipse extraction is by no meansobvious.

A distinctive characteristic of our method is that despite theextraction of progressively higher levels of data descriptionin a hierarchical approach, we do not discard low-level in-formation at later stages of ellipse extraction, but make useof the supporters of high-level features throughout all stagesof the process

The modication to RANSAC in order to maximize the inlierpercentages for both the local and global sets is novel.

Furthermore, the proposed method does not need a high di-mensional parameter space like HT based algorithms, and soit reduces the computation and memory requirements. Thealgorithm is easy to implement. Generally, it is capable of

extracting multiple ellipses with complex background in the im-ages quickly and correctly, even when the ellipse is occluded byother objects. Experiments on synthetic and real images showthat the proposed method is superior to the existing methods,especially when the testing images are complicated.

However, our method may have difculties when the ellipseto be extracted is relatively small in the image, in which casethe ellipse cannot be reasonably approximated by a sequenceof line segments. One way to overcome this problem is to takea multi-scale approach.

Acknowledgement

The work described in this paper was supported by a grantfrom the Research Grants Council of Hong Kong SpecialAdministrative Region, China (Project No. HKU7135/05E).References

[1] C.A. Rothwell, A. Zisserman, C.I. Marinos, D.A. Forsyth, J.L. Mundy,Relative motion and pose from arbitrary plane curves, Image VisionComput. 10 (4) (1992) 250262.

[2] A. Soetedjo, K. Yamada, Fast and robust trafc sign detection, in:Proceedings of IEEE International Conference on Systems, Man andCybernetics, Waikoloa, Hawaii, October 2005, pp. 13411346.

[3] Q. Ji, R.M. Haralick, A statistically efcient method for ellipse detection,in: Proceedings of IEEE International Conference on Image Processing,Kobe, Japan, October 1999, pp. 730734.

[4] C. Teutsch, D. Berndt, E. Trostmann, M. Weber, Real-time detectionof elliptic shapes for automated object recognition and object tracking,in: Proceedings of Machine Vision Applications in Industrial InspectionXIV, San Jose, CA, USA, January 2006, pp. 171179.


[5] R.O. Duda, P.E. Hart, Use of the Hough transform to detect lines andcurves in pictures, Commun. ACM 15 (1) (1972) 1115.

[6] L. Xu, E. Oja, P. Kultanen, A new curve detection method: randomizedHough transform (RHT), Pattern Recognition Lett. 11 (5) (1990)331338.

[7] N. Kiryati, Y. Eldar, A.M. Bruckstein, A probabilistic Hough transform,Pattern Recognition 24 (4) (1991) 303316.

[8] R.A. McLaughlin, Randomized Hough transform: improved ellipsedetection with comparison, Pattern Recognition Lett. 19 (3) (1998)299305.

[9] N. Guil, E.L. Zapata, Lower order circle and ellipse Hough transform,Pattern Recognition 30 (10) (1997) 17291744.

[10] S.C. Zhang, Z.Q. Liu, A robust, real-time ellipse detector, PatternRecognition 38 (2) (2005) 273287.

[11] C.T. Ho, K.H. Chen, A fast ellipse/circle detector using geometricsymmetry, Pattern Recognition 28 (1) (1995) 117124.

[12] Y. Lei, K.C. Wong, Ellipse detection based on symmetry, PatternRecognition Lett. 20 (1) (1999) 4147.

[13] Y.C. Cheng, The distinctiveness of a curve in a parameterizedneighborhood: extraction and applications, IEEE Trans. Pattern Anal.Mach. Intell. 28 (8) (2006) 12151222.

[14] T. Mainzer, Genetic algorithm for shape detection, Technical Report no.DCSE/TR-2002-06, University of West Bohemia, 2002.

[15] S. Procter, J. Illingworth, A comparison of the randomized Houghtransform and a genetic algorithm for ellipse detection, in: PatternRecognition in Practice IV: Multiple Paradigms, Comparative Studiesand Hybrid Systems, Elsevier, Amsterdam, 1994, pp. 449460.

[16] K.U. Kasemir, K. Betzler, Detecting ellipses of limited eccentricity inimages with high noise levels, Image Vision Comput. 21 (2) (2003)221227.

[17] J. Yao, N. Kharma, P. Grogono, A multi-population genetic algorithmfor robust and fast ellipse detection, Pattern Anal. Appl. 8 (1) (2005)149162.

[18] T. Kawaguchi, R.-I.Nagata, Ellipse detection using a genetic algorithm,in: Proceedings of the 14th International Conference on PatternRecognition, USA, 1998, pp. 141145.

[19] R.A. McLaughlin, M.D. Alder, The Hough transform versus the UpWrite,IEEE Trans. Pattern Anal. Mach. Intell. 20 (4) (1998) 396400.

[20] E. Kim, M. Haseyama, H. Kitajima, Fast and robust ellipse extractionfrom complicated images, in: Proceedings of IEEE InternationalConference on Information Technology & Applications (ICITA), 2002.

[21] P.L. Rosin, G.A.W. West, Nonparametric segmentation of curves intovarious representations, IEEE Trans. Pattern Anal. Mach. Intell. 17 (12)(1995) 11401153.

[22] K. Kanatani, N. Ohta, Automatic detection of circular objects by ellipsegrowing, Int. J. Image Graphics 4 (1) (2004) 3550.

[23] E. Kim, M. Haseyama, H. Kitajima, The extraction of circles from arcsrepresented by extended digital lines, IEICE Trans. 88-D (2) (2005)252267.

[24] F. Mai, Y.S. Hung, H. Zhong, W.F. Sze, A hierarchical approach forfast and robust ellipse extraction, in: Proceedings of IEEE InternationalConference on Image Processing, San Antonio, TX, USA, September2007.

[25] M.A. Fischler, R.C. Bolles, Random sample consensus: a paradigmfor model tting with applications to image analysis and automatedcartography, Commun. ACM 24 (6) (1981) 381395.

[26] R. Nevatia, K.R. Babu, Linear feature extraction and description,Comput. Vision Graphics Image Process. 13 (2) (1980) 257269.

[27] J. Princen, J. Illingworth, J. Kittler, A hierarchical approach to lineextraction based on the Hough transform, Comput. Vision GraphicsImage Process. 52 (1) (1990) 5777.

[28] J. Chanussot, G. Mauris, P. Lambert, Fuzzy fusion techniques for linearfeatures detection in multi-temporal SAR images, IEEE Trans. Geosci.Remote Sensing 37 (3) (1999) 12921305.

[29] J. Chanussot, P. Lambert, An application of mathematical morphology toroad network extraction on SAR images, in: ISMM984th InternationalSymposium on Mathematical Morphology and its Applications,Amsterdam, The Netherlands, June 1998, pp. 399406.

[30] J. Canny, A computational approach to edge detection, IEEE Trans.Pattern Anal. Mach. Intell. 8 (6) (1986) 679714.

[31] R.O. Duda, P.E. Hart, Pattern Classication and Scene Analysis, Wiley,New York, 1973.

[32] R. Hartley, A. Zisserman, Multiple View Geometry in Computer Vision,Cambridge University Press, Cambridge, 2000, pp. 103-104.

[33] R.A. McLaughlin, Intelligent algorithms for nding curves and surfacesin real world data, Ph.D. Thesis, Department of Electrical and ElectronicEngineering, The University of Western Australia, 2000.

About the AuthorFEI MAI received her B.Eng. and M.Eng. degrees in Control Theory and Control Engineering in 2000 and 2003, respectively fromTianjin University, China. She is currently a Ph.D. student in the Department of Electrical and Electronic Engineering at the University of Hong Kong. Herresearch interests include feature extraction, feature matching and 3D reconstruction.

About the AuthorY.S. HUNG received his B.Sc. (Eng) degree in Electrical Engineering and B.Sc. degree in Mathematics, both from the University ofHong Kong, and his M.Phil. and Ph.D. degrees from the University of Cambridge. He has worked as a research associate at the University of Cambridge,and as a lecturer at the University of Surrey before he joined the University of Hong Kong in 1989, where he is now an associate professor and the head ofthe department. Dr. Hung is a recipient of the Best Teaching Awards in 1991 from the Hong Kong University Students Union. He is a chartered engineer, afellow of IEE and a senior member of IEEE. He has authored and co-authored over 100 publications in books, international journals and conferences.

About the AuthorHUANG ZHONG received his B.Eng. and M.Eng. degrees in Control Theory and Control Engineering in 1996 and 1999, respectivelyfrom South China University of Technology, China, and his Ph.D. degree from the University of Hong Kong in 2006. He is now a research associate in theDepartment of Electrical and Electronic Engineering at the University of Hong Kong. His research interests include camera calibration and 3D reconstruction.

About the AuthorWUI-FUNG SZE received B.Eng. (1st class honour) and M.Phil. degrees in Electronic and Communications Engineering from theUniversity of Hong Kong in 2004 and 2006. His research interests are in the area of computer vision.

A hierarchical approach for fast and robust ellipse extractionIntroductionThe hierarchical approach---from edge points to ellipsesLine segment extractionLine segments linkingTwo conditions for linkingThe linking algorithm

Arc segment groupingEllipse fitting by modified RANSAC

Experimental resultsSynthetic imagesReal images

ConclusionsAcknowledgementReferences

Date post:	14-Oct-2015
Category:	Documents
Upload:	deriounbound
View:	22 times
Download:	0 times

A Hierarchical Approach for Fast and Robust Ellipse Extraction

Documents