The world perceived by the human eye is rich anddiverse. Images observed in the course of time varysignificantly and are rarely identical, yet there is aconsiderable resemblance between different images. De-tecting this similarity is key to the interpretation ofimages. In this paper we consider the problem of judg-ing similarity for the restricted, but still significant andchallenging domain of comparing the boundaries of 2Dshapes.
People are able to recognize objects even thoughtheir boundaries undergo a great many possible distor-tions due to changes in viewpoint, deformations orarticulations in shape, or the variations that occurbetween different objects of the same class. This sug-gests that we have a very general and powerful abilityto judge the similarity of shapes. While prior knowledgeof specific objects may influence overall judgements ofsimilarity, in this paper we focus on the problem ofgeneric shape similarity. We consider methods for de-
termining the similarity of two shapes without referenceto prior knowledge of other specific shapes. While it isstill an open question as to whether such a generalshape similarity capacity exists, we feel that it does fortwo reasons. First, as we will attempt to show, peopleform clear similarity judgements for seemingly arbitrarypairs of shapes. Second, even if overall similarity judge-ments are based upon previously seen shapes, the mostsimilar previous shapes must first be accessed. A gen-eral facility must tell us which previous shapes are mostrelevant to help us compare two new shapes.
Our paper attempts to make two contributions to theunderstanding of generic shape similarity processes.First, we attempt to clarify which properties may beimportant in human similarity judgements. We providesimple examples which suggest the role that part struc-ture may play in similarity judgements, that indicatehow human perception compares smooth shapes withdiscontinuous shapes, and that raises questions aboutthe relative importance of boundary-based comparisonsand comparisons of regions. This work is only sugges-tive; we delineate the space of significant issues thatseem most relevant to building models of similarity, we* Corresponding author. E-mail: [email protected].
R. Basri et al. / Vision Research 38 (1998) 2365–23852366
do not present psychophysics that will settle theseissues.
Second, we examine the extent to which these proper-ties can be captured by measures of similarity that arebased on local comparisons of the contours of shapes.To do this, we consider elastic matching, which is oneof the most powerful computational approaches tomeasuring similarity. Elastic matching is popular be-cause it can be computationally efficient and because itcan allow for non-rigid deformations in objects undercomparison. However, it is typically applied to specificapplications. The goal of this paper is to better under-stand the potential of elastic matching as a model ofhuman vision.
In elastic matching one searches for a correspon-dence between portions of two contours that minimizessome comparison cost between them. The key issue indeveloping the cost function is to determine how localdifferences between the contours should effect theirperceived similarity. Other questions involve structuringthe set of possible correspondences, and performing anefficient search for the best correspondence.
We particularly focus on the ability of this approachto explain the role that part structure appears to play inhuman similarity judgements. We show that a methodwhich compares shapes by measuring purely local de-formations can still account for some of the effects ofpart structure. In particular, we show how to modelobjects with parts that can undergo articulated motion,and to weight shape changes at part boundaries differ-ently from changes in the middle of parts. This allowsus to compare shapes in a way that respects partboundaries, without ever explicitly computing partstructure, or committing to a single part decompositionof a shape.
We also consider a number of other issues in shapesimilarity. We show how to derive our similaritymethod based on a simple model of contours as elastic1D objects. This allows us to relate the similaritycomputed by elastic matching to a physical energymeasuring the amount of deformation needed to ex-plain the differences in two contours. We also showhow one may develop a similarity method that cancompare both smooth and polygonal shapes; previousmethods based on differential properties do not applyto shapes with discontinuities. We also describe amethod of comparing partially occluded shapes. Fi-nally, we discuss the implications of requiring a com-parison method to provide a metric space for shapes. Inorder to make this very difficult problem tractable wehave limited the scope of our inquiry. Clearly, there areglobal properties of a shape, such as symmetry, thatplay an important role in shape similarity and thatshould be considered by methods that do not makepurely local comparisons. However, knowing that localcomparisons of shape will not tell the whole story
about similarity, our goal has been to provide betterinsight into the capabilities of such methods.
Although our focus is on what can be computedrather than on issues of how best to compute it, wehave also developed an efficient experimental system.The comparison methods that we consider are imple-mented using shortest path algorithms. This provides uswith an algorithm that is guaranteed to find the corre-spondence between the two contours that minimizes theenergy function under consideration. Therefore we mayevaluate the practical performance of our ideas withoutmuddying the waters with computational heuristics thatmight effect system output. These experiments help toillustrate some of the points that we also make theoret-ically, and also demonstrate some of the potentialutility of our ideas. A more abbreviated version of theseresults has appeared (see ref. [4]).
In sum, we feel that our work makes several potentialcontributions to the understanding of human shapesimilarity judgements. First, we provide examples thatsuggest some properties of generic similarity judge-ments; that they seem to respect the part structure ofobjects, and that they seem to allow comparisons be-tween smooth and polygonal shapes. Second, we dis-cuss how these properties can be encoded in a methodthat makes local comparisons of contours, withoutexplicitly computing part structure. Finally, we provideexamples to indicate some of the limitations of purelycontour-based approaches to similarity.
2. Background
In this section we provide a brief overview of theextensive body of work that is relevant to the problemof shape similarity. We focus on reviewing two streamsof prior work. First we look at work on matchingdeformable shapes using energy minimization methodsthat compare local portions of the contours. Next weconsider work that demonstrates the role played bypart structure in visual object recognition; our workwill attempt to combine these two approaches. We willdescribe more briefly other approaches to shape similar-ity, including feature based methods and functionalmethods. See Mumford [37] for another overview ofmethods of judging similarity.
2.1. Methods that measure deformations
Elastic matching methods measure the difference be-tween two shapes based on a specific correspondencebetween points on the shapes. A cost function weightsthe similarity of matched points on the two curves onthe basis of local properties of the points, such as thedistance between them, or the difference in the tangentor curvature of the contour at those points. The overall
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2367
similarity of the contours is then measured as the sumof these local costs. We illustrate this in Fig. 1. Typi-cally the cost function itself is used to find the corre-spondence between the curves; that is, some search iscarried out for the matching between the curves thatminimizes this cost. This matching allows for stretchingof contours, so that a larger portion of one contourmay match a smaller portion of the other. Such meth-ods produce both a measure of similarity and a corre-spondence between contours; the goal of the algorithmmay be either one or both of these. The two maincomponents of an elastic matching method, then, arethe its cost function and its search mechanism.
A number of different cost functions have been pro-posed for elastic matching. For example, Yoshida andSakoe [56] propose comparing hand- printed charactersby measuring the difference in angle and distance be-tween corresponding points along the two contours.Tappert [48] compares the difference between the angleand height of corresponding points for character recog-nition also. In related work, Burr [11] compares con-tours based on the distance and angle between thecontours. Related approaches are also taken by Tsaiand Yu [49], and Mehrota and Grosky [36], the latterfocusing on the problem of indexing into a data base ofshapes.
Similar approaches have been used in work on mo-tion tracking. In some cases this work also incorporatesa cost that measures the amount of ‘stretching’ of acontour. Hildreth [22] proposed computing motion flowbetween two contours with a cost function that penal-izes for the variation in the magnitude of motion.Cohen, Ayache, and Sulger [12] propose matching con-tours extracted from moving images of biological struc-tures (such as beating hearts) with a cost function basedon the difference in curvature between the two curves.
Tagare, O’Shea and Rangarajan [47] also compare thecurvature of two contours, taking care to propose acost function that is symmetric, as we did [4] in ourinitial work (that is, the distance from contour A tocontour B is the same as the distance from B to A).McConnell et al. [35] use a cost function based on theEuclidean distance between contour points to compareSAR images of ice floes that may deform due tomelting.
Related work compares shapes by measuring thedeformations needed to relate the regions bounded bythe shapes’ contours, rather than the contours them-selves (eg. Amit, Grenander and Piccioni [1] and Jain,Zhong, and Lakshmanan [28]). In Section 7 we brieflydiscuss some of the advantages and disadvantages ofcomparisons based on regions rather than the contoursthat bound them.
Another approach is to develop a general model ofshape that captures the properties of a class of objectsand then to compare a new shape to this generic model.Kass, Witkin, and Terzopoulos [31] match a genericshape model to intensity images. They track contours inimages by minimizing a cost function that favorssmooth contours that fit the gradients of the image aswell as possible. Yuille, Cohen, and Hallinan [57] recog-nize parts of a face using deformable models that theyhave hand-crafted, while Cootes et al. [14], Baumbergand Hogg [5], and Hinton, Williams and Revow [23]suggest learning deformable models of objects fromexamples. Finally, we should note that elastic matchinghas been widely used outside of vision to compare 1Dobjects, and especially to compare speech signals. Areview of some of this work can be found in Sankoffand Kruskal [43].
Many of these methods simply propose an intuitivelyappealing cost function, which may be crafted for aparticular application domain. However, some papersalso attempt to determine general properties of shapematching methods that are based on the cost functionsthey propose. For example, Hildreth [22] stresses thatthe proposed cost function leads to unique solutions inmotion tracking. Arkin et al. [2] stress that theirmethod leads to a cost function that is invariant whencontours undergo similarity transformations, and placescontours in a metric space. Tagare, O’Shea and Ran-garajan [47] show the symmetry of their measure and itsinvariance to parameterizations of the contour. Ourgoal will be to provide a more complete description ofthe properties that an elastic matching method mayhave in terms of the possible relevance of these proper-ties to modeling human vision. We will show how theseproperties depend on the choice of the cost functionused in shape comparison.
Given a choice of cost functions, many methods havebeen used to find the correspondence between contoursthat minimizes this cost. When the two shapes being
Fig. 1. In elastic matching, a correspondence is hypothesized betweenpoints on two contours. Then a cost is computed for each pair ofmatched points, based on the similarity of their local properties (e.g.tangent and curvature) and properties of the match (e.g. amount ofstretching one contour undergoes to match the other). The total costis the integral of this local cost over the whole correspondence.
R. Basri et al. / Vision Research 38 (1998) 2365–23852368
compared are ordered strings, dynamic programmingcan be used to find an optimal correspondence (eg.[35,48,49,56]). Recently, more efficient shortest pathalgorithms have been used on related problems byGeiger et al. [17], which we use in our implementation.Arkin et al. [2] use a more complex, geometrical al-gorithm specific to their formulation of the problem.Also popular have been various methods based ongradient descent (eg. [12,31,57]). These methods havethe disadvantage that they may converge to a locallyoptimal match that is not the best overall match. How-ever, they may be preferable when dynamic program-ming is not possible because the contour points are notordered in a 1D string, or when a good starting pointfor the method is available. Elastic matching of 2Dimages has also been considered. Tsukumo [50] andTsukumo and Tanaka [51] describe methods of register-ing images of Kanji by using dynamic programming toregister a 1D projection of the 2D image. The costfunction is based on features of 1D slices of the 2Dimage. Levin and Pieraccini [33] also describe a methodof warping 2D images by registering 1D slices of them,using dynamic programming. Our focus in this paper,however, is on how similarity is encoded in the costfunction rather than on solution methods.
2.2. Parts-based methods
A second influential computational approach to ob-ject classification stresses descriptions of shapes interms of their part structure. Parts generally are definedto be convex or nearly convex shapes separated fromthe rest of the object at concavity extreme, as in Hoff-man and Richards [25], or at inflections, as in Koen-derink and van Doorn [30]. Many methods have beensuggested for providing geometric descriptions of theseparts, such as generalized cylinders [7] and su-perquadrics [3,39]. Some methods have been proposedfor judging whether two images come from the sameclass of objects by describing object classes using partsthat have parameterized descriptions [10,20,21]. It hasalso been suggested that object parts be described usinga small set of qualitative or topological properties. Inthis approach parts with different shapes may be judgedsimilar when they contain the same qualitative proper-ties. Biederman [6] proposes describing parts in terms oftheir non-accidental properties. Marr and Nishihara[34] make qualitative comparisons between part struc-tures described as generalized cylinders. Connell andBrady [13] also propose a qualitative, parts-based de-scription of shape. A number of other proposals havealso been made for representing objects in terms oftheir possible decomposition into parts and the rela-tionships between these parts [32,44,58].
Biederman [6] has produced a good deal of evidencefor the contention that human object recognition is
Fig. 2. A pair of spectacles (left) and an object that has the sameparts, but does not look like spectacles (right).
sensitive to the part structure of objects (see also ref.[45]). However, as is shown in Fig. 2, part structure isinsufficient to solely determine the category of objects.Metric properties are essential to determine the class ofobjects. The methods that use parts to recognize objectslack a theory specifying how the metric propertiesshould be used and how they should interact with theparts. Such a theory should for example determinewhen objects that share the same part structure shouldbelong to different classes. Other problems with meth-ods that recognize objects by looking at their partstructure are that some objects lack distinctive partstructure (e.g. a shoe, as shown in Fig. 3), and thatmethods to extract parts from images tend to be un-stable and degrade significantly in the presence of noiseand occlusion. One reason for these problems is thatthere is no clear definition as to what constitutes thestable ‘parts’ of an object. It is one of the goals of ourpaper to show that some of these problems can beovercome by integrating parts-based and precise metricjudgements into a single framework, which attends tothe part structure of objects without requiring the ex-plicit computation of parts.
2.3. Other work
There has been a great deal of other work on shapesimilarity and object recognition which we can onlybriefly touch on here. Some proposals for comparingshape evaluate global, rather than local, deformations.Pentland and Sclaroff [40] construct mass and stiffnessmatrices for the given shapes, and deform the shapes byaligning the principal modes of their mass and stiffnessmatrices. Huttenlocher et al. [26,27] compare twoshapes by applying the rigid (or affine) transformationthat minimizes the Hausdorf measure between them. Toallow for occlusion only a certain fixed fraction of thecontour points are used. Global approaches tend to be
Fig. 3. A shoe. It appears to be difficult to divide this shape intomeaningful parts.
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2369
sensitive to occlusion. Huttenlocher et al.’s method forhandling occlusion allows only for a fixed amount ofocclusion. In addition, these methods fail to account forlocal deformations (such as articulations) since thesedeformations may change the global appearance ofobjects considerably while the entire deformation isconcentrated in specific points. Intermediates betweenlocal and global methods, Ullman [53] proposed com-paring shapes after deforming them with piece-wiseaffine transformations, although this suggestion is notfleshed out.
Another common method of shape comparison is todescribe shapes with a list of properties (features).These properties may be global (e.g. ‘the object ispolygonal’) or local. When global properties are used(see reviews refs. [9,16]) objects are represented aspoints in feature space. This representation is obtainedby arranging the list of properties of an object as thecomponents of a vector associated with the object.Similarity between objects is determined by the distancein feature space between the vectors associated with theobjects. When local features are used [41] an object isrepresented by a graph with nodes representing thefeature values and edges representing the spatial ar-rangement of the features. Objects are considered simi-lar if their graphs are isomorphic.
Methods that use features critically depend on the setof features extracted. A small change in the shape of anobject may sometimes cause a significant change in itsfeatures. It is difficult to find a coherent set of featuresthat can faithfully reflect all possible shapes. In addi-tion, global features tend to be sensitive to occlusion,whereas local features tend to be sensitive to noise andto small variations in shape.
Another class of methods attempts to assign ‘seman-tic interpretation’ to shapes. The most common seman-tic attribute is function. For example, a chair may bedefined as an object that has a sittable surface andprovides a stable support [46]. Two objects are consid-ered similar if they share the same set of semanticattributes. Methods that use function have attractedresearchers from both fields of computer vision andartificial intelligence [24,42,46,55].
The replacement of geometric structure with seman-tics is potentially powerful. In particular, eccentricallydesigned objects may be recognized using this methodeven when they differ visually from conventional de-signs. Methods that use function, however, suffer fromseveral problems. First, function is difficult to extract,especially under partial occlusion. Second, the use offunction is suitable for man-made objects, but is moredifficult to extend to natural objects. Third, methodsthat use function tend to over generalize. For example,a wide flat rock may be recognized as a chair because itis sittable. Finally, it seems useful to be able to judgethe purely visual similarity of shapes, regardless of their
possible functionality, and humans appear to have thisability.
3. Constraints on a similarity function
We now consider elastic matching as a potentialmodel of human shape similarity judgements. We feelthat currently, it is not well understood what propertiesperceptual similarity has. In fact, it is not clear whetherthere is a single similarity system, or whether differentsimilarity measures may apply in different circum-stances. For example, it is possible that different simi-larity methods operate between classes than thoseoperating within classes. It is also possible that ourprior knowledge of objects plays a significant role inour similarity judgements, a role which may vary con-siderably depending on the shapes we view. Therefore,our main goal is to describe the possibly desirableproperties of perceptual similarity, and then to see howthese might be obtained by elastic matching. Below weenumerate the desirable properties (see Section 3.1) andthen show that not all these properties can be satisfiedsimultaneously by elastic matching.
3.1. Desired properties of a similarity function
We define a cost by finding a mapping between twocontours, G1 and G2, and then summing the cost oflocal deformations that reflect the differences betweenthe two contours. Let G1 be parameterized by arclengths, so that G1(s) indicates a point on G1. Then a corre-spondence between two curves is given by a function,t(s), from arclength to arclength, so that the point G1(s)is mapped to the point G2[t(s)] (we will sometimes writet(s) as just t, when s is clear). Then we consider costfunctions that take this correspondence into account,by integrating over the curves a measure of the localdifference between the curves, given this correspon-dence. This similarity must be computed based on somelocal property of the two contours. As we will see, amethod based on local comparisons has the advantagethat it can model articulations and local changes ofobject shape. For example, if a person raises her arm,we want the cost of this change in shape to be concen-trated in a comparison of arm angle, while noticing thatthe rest of the person remains the same.
We will focus on costs that are based on the lowestorder differential properties of the curves that can stillmeet our objectives. In particular, we will consider costfunctions of the form:
C(G1, G2)=mint(s)
&G1
F�
k1, k2,dtds�
ds.
Here, C is shorthand for the overall similarity be-tween the two contours. K1 is the curvature of G1 at
R. Basri et al. / Vision Research 38 (1998) 2365–23852370
Fig. 4. Our work uses the intuition that as l2 decreases, the twofigures on the right each become more similar to the figure on the left.
Fig. 4 shows two examples which demonstrate therequirements for continuity. We let G1 be a curveconsisting of two straight lines each of length L1,connected by a corner with an exterior angle of p/2. LetG2 have three straight lines of length L1, L2, and L1−L2. Further, let the two corners connecting the interme-diate line of G2 have exterior angles of p/4. G3 is like G2,except that the two straight lines are connected by acircular arc of length L2, and curvature p/2L2, so thatthe angle between the two lines is also p/2. So, the threecurves have the same length, and they all turn by a totalangle of p/2. As L2�0, the two curves on the rightbecome more and more similar to the curve on the left.In the remainder of the paper, we will rely on theintuition that as L2�0, the cost of comparing G1 withG3 should also go to zero. Some assumption such asthis is necessary to define a cost for corners, whoselocal derivatives are not defined. In the case of G2, weonly rely on the intuition that the cost of comparing thetwo curves should be greater for some large value of L2
than for some small value of L2, even if we are notprepared to specify the exact form of this change incost, or its asymptotic behavior as L2�0.
3.1.2. MetricC is a metric. This implies:1. F(k1, k2, t %)\0.2. F(k1, k1, 1)=0; F(k1, k2, t %)\0 for k1"k2 or
t %"1.3. F(k1, k2, dt/ds)ds=F(k2, k1, ds/dt)dt4.
F(k1, k2, dt/ds)ds+F(k2, k3, du/dt)dt
]F(k1, k3, du/ds)ds, Ök1, k2, k3,where k3 indi-cates the curvature of a corresponding point on G3,which is parameterized by u.
It will be very useful for C to be a metric if we intendto use C in a machine recognition system. For example,if C is a metric, and we compare Gl to G2, and find themvery dissimilar, and compare G2 to G3 and find themsimilar, we do not need to explicitly compare Gl to G3;we know these two curves will be dissimilar. For thisreason, virtually all efficient methods of finding nearestneighbours rely on the metric properties of a compari-son function.
On the other hand, it is not clear that human com-parisons of shape obey metric properties. In particular,the triangle inequality may not hold for human com-parisons of shape. For example, it is quite possible thatpeople perceive a horse and a man to have very differ-ent appearances, while a centaur may be quite similarto both. Also, the symmetry constraint may not beobeyed [37,52]. For example, an ellipse may be per-ceived as more similar to a circle (it may be the imageof a slanted circle) than a circle is to an ellipse (it is veryuncommon for a circle to be the image of an ellipse).
point G1(s) [i.e. shorthand for k1(s)], and similarly k2 isthe curvature of G2 at point G2(t(s)). F is some stillunspecified function. The minimum is taken over allpossible continuous correspondences between the twocontours. This says that we judge the cost between twocurves based on the local differences of curvature andrelative speed of progression along the curves. Moreintuitively, we can think of this as judging cost basedon the minimum amount of bending (change in curva-ture) and stretching (dt/ds, or t %) required to make thecurves identical. Note that curvature is the lowest orderlocal property of curves that can meet our goals. If webased our cost on tangents, for example [48,49], thenthe articulation of a part would change our local de-scription of the entire part, not just the point at whichit bends. A cost function based on the stretching andbending needed to make the contours the same hasconsiderable intuitive appeal as well. One of the poten-tial values of a shape similarity judgement is that it mayallow one to identify an object that is stretching andbending in an unknown way. While it is possible tobase elastic matching on different local properties of acurve, the class of functions we consider encompassmost of those that we have reviewed.
We now enumerate some possible properties of ashape similarity method, and consider the implicationsof each for the cost function F that will fully specify oursimilarity method. We will then briefly discuss thedesirability of each property.
3.1.1. ContinuityC is continuous. That is, as the curves change
smoothly, so does C. This implies that if a sequence ofcurves, G2,i converges to G2 then C(G1, G2,i) convergesto C(G1, G2). This will be true if and only if F iscontinuous.
Perceptually, human comparisons between shapesmay not be smooth. For example, a deformation thateliminates some Gestalt property such as symmetry orcollinearity may have a discontinuous effect on humancomparisons of shapes. While it is beyond the scope ofthis paper to consider the effects of global properties,such as symmetry, on shape similarity, they are clearlyquite significant.
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2371
3.1.3. In6arianceWe may wish our cost to have some invariant prop-
erties under some classes of transformations. For exam-ple, we might wish the cost to be zero when comparingtwo shapes related by a Euclidean, similarity or affinetransformation, or we might wish the relative nearnessof curves to be fixed under some class oftransformations.
By taking F as a function of curvature, we guaranteethat it is invariant under Euclidean transformations.Should we wish to make it scale invariant, this wouldimply: F(k1, k2, t %)=F(k1, k2/a, at %). A weaker versionof this condition is that the cost of scaling is indepen-dent of the curvature, i.e. that we divide F into twocosts, one of which is scale-independent and the otherof which only measures the cost of scaling. This wouldmean that: F(k1, k2, t %)= f(k1, k2, t %)+g(t %) wheref(k1, k2, t %)= f(k1, k2/a, at %). This is easily achieved bymaking f a function only of k1 and k2t %. k2t % is scaleindependent since k2 and t % scale precisely inversely asG2 is scaled.
In cases in which our cost function is not scaleinvariant, we might prefer that scaling three curves doesnot change their relative similarity. This implies that:
F(k1, k2, t %)BF(k1, k3, u %)[F(ak1, ak2, t %)
BF(ak1, ak3, u %)
where a is any scale factor.
3.1.4. Handling polygonsC should not diverge for discontinuous curves. That
is, any two finite length curves should be related by afinite cost. In particular, this means that C provides ameaningful comparison between polygons, or betweena polygon and a smooth curve. The consequences ofthis property are complex, and we postpone discussionof them to the next section.
That the cost function handles both smooth andpolygonal shapes, also seems quite important. In apractical system we will be comparing discretized ver-sions of continuous or discrete contours; it thereforeseems essential that a close polygonal approximation toa shape be considered quite similar to that shape, andnot have a divergent cost. Humans also seem to sensi-bly compare polygonal and curved shapes; for examplea regular octagon may appear more similar to a circlethan does a regular triangle. This condition is a prereq-uisite to making such judgements of ideal, non-dis-cretized contours.
3.1.5. Handling partsWe may want the cost function to reflect the part
structure of objects. This has two main implications.First, objects often articulate at part boundaries. So if apart of an object rotates, the cost function should
capture the fact that the relationship between the partand the rest of the object has changed while the internalshape of the part has not. Second, articulations of ashape at the likely part boundaries should cost less thanbending that occurs in the middle of a part boundary.
The first condition is captured by our use of a localcost function that compares the curvature of contours.As a part articulates, the curvatures of the contour onlychange at the points where the part joins the body. Thesecond condition is captured if deforming a shape bybending it will cost less if the bending is done at pointsof high absolute curvature, given that part boundariestypically occur at points of high curvature.
This implies:
F(k1, k2, 1)\F(k1+a, k2+a, 1),
Öa, k1, k2\0, Öa, k1, k2B0.
It is less clear what this constraint should apply whent %"1. For example, if our function were scale invari-ant, this would imply that:
F(2, 1, 2)=0BF(3, 2, 2)
We make no assumptions for the case where onecurvature is positive and one curvature is negative,since this seems less clear intuitively.
This condition reflects the perceptual phenomenonthat contours appear more ‘bendable’ at places wherethere is already a high curvature than at points of lowcurvature. This is illustrated in Fig. 5. This reflects thepart nature of objects; part boundaries tend to occur atpoints of high absolute curvature. Articulated or de-formable objects tend to bend more easily at partboundaries.
3.1.6. MonotonicityIf one deforms a shape by bending it or stretching it,
the cost between the deformed and original shapeshould grow as the bending or stretching increases.
This means that: k1Bk2Bk3[F(k1, k2, 1)BF(k1, k3, 1), and 1B t %1B t %2, k1=k2[F(k1, k2, t %1)BF(k1, k2, t %2). Again, it is less clear how the cost functionshould behave if bending increases in the presence ofstretching, or if stretching occurs when the bending isunequal. This condition reflects the intuition that in-
Fig. 5. The middle figure shows the figure on the left, bent at pointsof high curvature. On the right, the figure is bent by equal amountsat points of low curvature. By penalizing the bending on the rightmore than the bending in the middle, we produce comparisonssensitive to the part structure of objects.
R. Basri et al. / Vision Research 38 (1998) 2365–23852372
Fig. 6. A circle, regular triangle, and regular octagon, with the sameperimeter. We may require a cost function to judge the octagon andcircle more similar than the triangle and circle.
identical, and may be matched perfectly. Therefore, thecost between the two curves will not change as l2changes, contrary to our assumptions about thisexample.
To overcome this problem, it seems intuitively thatwe must match a finite portion of G2 or G3 to a point orsmall portion of G1. This is most easily done by assum-ing that the curves are first smoothed by some fixedamount. A purely local comparison after smoothing isequivalent to a comparison with a small base of sup-port. Notice that if we smooth the curves by some fixedamount then as l2�0, the smoothed corners of G2 willbecome identical to the smoothed corner of G1 and thetwo curves will become identical. Note that our reason-ing applies to an abstract, continuous formulation ofthe cost function. In practice, any implementation mustbe discrete, which implies discrete comparisons andsome implicit smoothing. However, we see that we mustrely on these sorts of effects to handle corners.
We now show that problems arise even when ourcomparison is made after a small amount of smoothing.First, we use the comparison between G1 and G3 in Fig.4 to define the cost function at discontinuities. Wedefine a corner of angle u as two straight lines con-nected at a point of infinite curvature, such that theintegral of the curve’s curvature is u. As in our exam-ple, we assume that a corner of angle u can be thoughtof as the limit of a sequence consisting of two linesjoined by a circular arc of length l and curvature u/l, asthe length l goes to zero. Ignoring the effects ofsmoothing, and supposing that the cost of comparingtwo such sequences, which converge to corners, shouldbe similar to the cost of comparing two corners, wefind:
liml�0
lF�u1
l,
u2
l, 1��c(u1, u2)
where c(u1, u2) is a finite-valued function, that increasesmonotonically as u2 increases, for a fixed u1 and u1Bu2
In order for the corner cost to converge to a non-infi-nite value, then, it must be the case asymptotically that:
limk 2��
F(k1, k2, 1)=aF�
k1,k2
a, 1�
for any constant a. That is, asymptotically, F must belinear in k2. Hence, the cost of transforming a point offinite curvature into a corner will depend linearly on theangle of the corner. This creates a problem for condi-tion 3.1.7. In particular, this implies that when compar-ing a polygon to a circle, the cost will depend only onthe sum of the angles of the polygon, as these angles aredeformed to match portions of the circle that havefinite curvature. Since the exterior angles of any convexpolygon sum to 2p, this means that all convex polygonswill be judged equally similar to a circle. Again, we may
creasing deformations of a shape should make themless and less similar to the original shape.
3.1.7. Small 6ersus large deformationsTwo shapes that are related by many small deforma-
tions should be considered more similar than twoshapes that are related by a single deformation of equaltotal magnitude. This constraint implies that:nF(k1, k1+e, 1)BF(k1, k1+ne, 1),
Ö0Bk1, 0Be, 1Bn.It is generally desirable when dealing with noisy data
to count a single change of a given magnitude moreheavily than many small changes whose magnitude sumto the same amount. With Gaussian noise, for example,it is optimal to weight changes by the square of theirmagnitude. This principle may also be present psycho-physically, as when comparing a circle to a regulartriangle, and to a regular octagon (see Fig. 6). It is ouruntested hypothesis that the octagon is typically viewedas more similar to the circle. One possible reason forthis is because, while the octagon has more corners tobe smoothed out, these corners need be smoothed lessto match them to the circle. Both the triangle andoctagon, of course, have changes in angle totalling 2p.We will discuss this example in more detail later, withregard to a specific type of cost function. Overall, wefeel that this and the other properties we have describedmight be desirable in a model of human similarityjudgements, although their necessity is not yet proven.
3.2. The constraints cannot all be satisfied
We now discuss two difficulties that arise in trying tosatisfy these constraints with an elastic matchingmethod. First, we show that properly handling cornerspresents problems for a cost function based on purelylocal comparisons. Of course, curvature is not definedat the corners of a polygon. We can overcome thisproblem by considering a polygon as the limit of aseries of smooth curves that converge to the polygon.However, other problems still remain. Consider theexample comparing G1 and G2 (Fig. 4), as described inSection 3.1.1. For any l2\0, a cost function based ona one-to-one correspondence between the curves willmatch the corner of G1 to one of the corners of G2,while matching G2’s second corner to a point on one ofthe straight lines in G1. The remaining points are locally
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2373
rely on smoothing to overcome this problem. Asmoothed regular octagon will have curvatures muchmore similar to a circle than will a smoothed regulartriangle.
Although we must clearly rely on smoothing to pro-duce desirable results in the case of corners, this discus-sion clarifies our choices about the cost function’sbehavior at corners. For when smoothing is small rela-tive to the scale of a corner, the behavior of the costfunction will approximate its behavior on unsmoothedcorners. So we may choose a cost function that isasymptotically linear in curvature. This will tend tokeep the cost function from growing very large in thepresence of corners, but will also tend to produce alinear cost when comparing corners, which can beundesirable, as for example, in the case of comparingpolygons and circles. Alternately, we can choose a costthat is asymptotically non-linear for corners, which willproduce a cost for comparing corners that goes eitherto zero or infinity as the amount of smoothing is small.See Weiss [54] for a related discussion on an approachto handling corners in functions describing the energyof a curve.
We now demonstrate that another contradictionarises when we try to simultaneously satisfy all desiredconstraints on the function F. We then consider variouspossible tradeoffs in partially satisfying theseconstraints.
Let us consider what relative values F should havefor F(0, 1, 1), F(0, 2, 1), F(1, 2, 1). Constraint 3.1.6 tellus that we should have:
F(0, 1, 1)\F(1, 2, 1)
following the intuition that it becomes easier to bend ashape at higher curvature points. The metric constraint(Condition 3.1.2) tell us:
F(0, 2, 1)5F(0, 1, 1)+F(1, 2, 1)
Finally, Condition 3.1.7 tells us that:
2F(0, 1, 1)5F(0, 2, 1)
From the above equations, it follows that:
F(0, 2, 1)5F(0, 1, 1)+F(1, 2, 1)B2F(0, 1, 1)
5F(0, 2, 1) (1)
a contradiction.There are several possible responses to this situation.
First, if we replace the constraint:
F(k1, k2, t %)\F(k1+a, k2+a, t %),
Öa, k1, k2\0, Öa, k1, k2B0.
with:
F(k1, k2, t %)]F(k1+a, k2+a, t %),
Öa, k1, k2\0, Öa, k1, k2B0.
then all the constraints may be satisfied at equality.However, while a metric may be useful when the trian-gle constraint is satisfied by equality, the other twoconstraints are less useful at equality. That is, it is nottoo satisfying to allow all curve points to be equallyeasy to bend, or to allow many small changes to havean equal effect to one big change. A second possibilityis to consider functions for F that are not metricsbecause they do not satisfy the triangle inequality. Aswe have pointed out, this may be reasonable becausehuman perceptions of shape similarity may be non-met-ric, and because we may be able to design a total costfunction that is a metric based on a function F that isnot a metric. Third, we can rely on smoothing to satisfyconstraint 3.1.7 by smoothing out many small changesmore than one big change.
4. A model of cost
We have considered a few potential cost functionssince, as is noted above, it is not possible to find an Fwith all desirable properties. These functions offer arange of possible tradeoffs. Below we introduce a costfunction that is derived from a simple model of theformation of the contours (shapes) based on concatena-tions of springs. We show that this function satisfies allthe properties except that it violates the triangle in-equality. However, we are able to show that by takingthe q ’th root of the total cost function we obtain, forparticular values of q we are able to obtain a metric.This would imply that the cost function is no longerpurely the integral of a local cost, but is a function ofthis integral. In Appendix B we give another example ofa function which relies on smoothing to satisfy theconstraint that many small deformations should costless than one large deformation. This second function isintuitively appealing because it models a deformationas the sum of a series of small deformations. We showbelow, however, that this function has problems withhandling polygons properly.
In our model a deformation of shapes is decomposedinto bending and stretching. The energy used forstretching contours is the energy of the concatenatedstretched springs. The energy used for bending is alsoassociated with a spring-like model. First we analyzethe discrete case, in which the contours are made ofpoints, and then we take the limit where the size of thesprings become infinitesimal to consider continuousmodels.
4.1. Stretching
The energy associated with stretching a spring withstiffness a is given by Hooke’s law, E=1
2a(Dx)2, whereDx is the displacement from equilibrium. In our model
R. Basri et al. / Vision Research 38 (1998) 2365–23852374
we generalize Hooke’s law. We assume that the energygets increased by a factor (Dx)p with displacements Dx,where p\1, i.e. we assume
E=1pa(Dx)p.
If two springs of stiffness al and a2 are connectedtogether then the resulting spring has stiffness
aeq=a1a2
a11/(p−1)+a2
1/(p−1) (2)
This can be shown by requiring that when a displace-ment Dx is applied to the resulting spring then, thecorresponding displacements, Dx1 and Dx2 on eachspring must satisfy F1=F2, where F=a(Dx)(p−1) is theforce exerted by each spring. Thus, the three equationsthat yield Eq. (2) are
1paeq(Dx)p=
1p
a1(Dx)1p+
1pa2(Dx)2
p
energy conservation,
Dx=Dx1+Dx2 length conservation,
a1(Dx)1p−1=a2(Dx)2
p−1 force equilibrium.
The main assumption we make is that the stiffness ofeach spring element is inversely proportional to itslength to the power p−1, i.e. we assume that aninfinitesimal element dl will have stiffness
adl=a
dl p−1,
which is very large. It is easy to check that two con-nected elements, dl1 and dl2, will yield an equivalentstiffness
aeq=adl 1
adl 2
adl 1
1/p−1+adl 2
1/p−1=a
(dl1+dl2)p−1,
and the total contour will have an equivalent stiffnessacontour=a/Lp−1 where L is the length of the contour.
4.2. Matching contours
In order to compare two contours, Gs and Gt (seeFig. 7), via the stretched energy, we assume that bothare stretched versions of a ‘true’ relaxed contour. Thisassumption is necessary to guarantee that the cost todeform contour Gs into Gt is the same as deforming Gt
into Gs.Let us assume that the rest (relaxed) length of the
‘true’ contour element is dependent upon the two con-tours. So we can choose for instance the infinitesimal‘true’ relaxed length to be (dt+ds)/2 (when matchingelement ds, from Gs into element dt, from Gt). Thestretching energy becomes the sum of stretching therelaxed element (ds+dt)/2 once into ds and once intodt. More precisely,
dEstretching(ds, dt)
=1p
2p−1
(ds+dt)p−1�dt− (ds+dt)/2�p+1p
2p−1a
(ds+dt)p−1�ds
− (ds+dt)/2�p=a
p(t %−1)p
p(t %+1)p−1ds,
where t %=dt/ds. Thus, the total energy of stretching Gs
into Gt is
Estretching(Gs, Gt, dt, ds)=a
p&
Gs
�dt/ds−1�p(dt/ds+1)p−1ds.
4.3. Bending
For bending we assume the same spring model asabove, but instead of springs of length ds or dt we havesprings associated with the angles. More preciselydus=ks ds and dut=kt dt are the infinitesimal elementsthat are stretched (bended). Thus, to bend spring ele-ments dus into dut it costs
dEbending(ks, kt, ds, dt)=a
p�ktdt−ksds �p
(�ksds �+ �ktdt �)p−1ds
=a
p�ktdt %−ksds �p
(�ktt %�+ �ktdt �)p−1
and the total energy is
Ebending(Gs, Gt, dt/ds)=&
Gs
a
p�ktt %−ks �p
(�ktt %�+ �ks �)p−1ds
Thus, this model has total energy
E(Gs, Gt, dt/ds)
=a
p&
Gs
� �ktt %−ks �(�ktt %�+ �ks �)p−1+l
�dt/ds−1p�(dt/ds+1)p−1
nds.
Fig. 7. (A.) and (B.) Two elements of contour 1 (2) forming an angleus (ut). The contour is parametrized by arc length s (t). (C.) The twocontours are superimposed after being aligned according to their firstelement. The cost of bringing them together is then defined as the‘physical’ effort of bringing the element dt into ds, which requiresbending and stretching.
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2375
where l weights the relative contributions of stretchingand bending.
In Appendix A we demonstrate that this cost functionmeets the previously discussed constraints. In additionto all the other constraints, we show that by taking theappropriate qth root of the total cost over the entirecontour, the bending portion of this cost preserves thetriangle inequality.
Before we end this section we would like to mentionseveral related approaches to shape description andcomparison. An alternative cost function for shapecomparison that explicitly allows for articulations is afunction that identifies the position of extreme bendingand assigns a constant cost to these bendings. A costfunction that achieves that, in the spirit of Geman andGeman [18], Mumford and Shah [38], and Blake andZisserman [8], can be written (in a discretized form) asEbending=�s=1
S (ktt %−ks)2(1− ls)+gls, where ls=0, 1.ls=1 occurs if the difference in matched curvature ishigh compared to g, a parameter to be estimated. Oneproblem with this approach is in comparing polygons tosmooth curves. For example, with this cost function atriangle would be more similar to a circle than is asquare (or any high order polygon), since the similaritybetween the polygonal straight lines and the circles isthe same in both cases, but there are four corners in asquare (we activate ls=1 four times) and only threecorners in the triangle. That is, condition 3.1.7 would beviolated. This is clearly not desirable and our approachovercomes this problem.
Finally, many methods of curve description have beenproposed that are based on curve evolution. For exam-ple, recently Kimia et al. [29,44] have proposed curveevolution methods based on reaction-diffusion equa-tions, using the results of this evolution to describe theshape of an object. It is natural to consider a diffusionprocess to model the similarity between two shapes aswell. Kimia et al. propose simulating the reaction-diffu-sion equations on two shapes and comparing thembased on the similarity of their curve evolutions.
In the spirit of our work, one might also imaginecomparing shapes by measuring the probability that oneshape, undergoing a diffusion process, could evolve intothe second shape. However, diffusion processes are notreversible and information is lost. Thus, it is not alwayspossible to obtain one shape by evolving from one intoany other shape. Moreover, it is not symmetric. Theselimitations of the diffusion equations are overcome byour method.
4.4. Linear cost function
A special case of this model is obtained when p=1.As we have noted above, it is possible to satisfy bothconstraints 3.1.6 and 3.1.7 at equality, while still pro-ducing a metric cost. Specifically, we may let:
Fig. 8. We allow for the possibility of ignoring the differencesbetween the two figures as due to an occlusion, and focusing insteadon the similarities.
F(k1, k2, t %)= �k2t %−k1�+l �t %−1�As we have already discussed, this function satisfies
all other constraints. Furthermore, as with the springmodel, the bending cost is scale invariant.
We may also interpret this cost function physically.For t %=1 we may take this as the cost of bending a stiffmaterial, such as wire, from one shape to another, wherethe cost of bending is uniform throughout the wire (doesnot depend on curvature). The cost also reflects auniform, linear cost for stretching.
5. Occlusions
As Fig. 8 illustrates, when the bulk of two figures aresimilar, but a portion of the figures are quite different,our interpretation may be based on ignoring the dissim-ilar portions, possibly viewing these as due to an extra-neous, occluding object. We therefore allow for thepossibility that portions of the two contours remainunmatched, under the assumption that the true corre-sponding portions of the contour are not visible. Wenow consider how a proposed cost function should dealwith an hypothesized occlusion. (see ref. [19] for anotherapproach to occlusions in a dynamic programmingframework).
The cost function for occlusions should reflect threefactors. First, it is important to take account of thesimilarity between the relative positions of the beginningand ending of the hypothesized occlusion in each con-tour. Fig. 9, shows a rectangle with gaps that has beenstretched in two different ways. The gaps signify thelocation of an hypothesized occlusion. We can see
Fig. 9. Figures containing gaps to indicate the location of hypothe-sized occlusions. The relative angle and distance between the startand end of the gap may influence the similarity of the shapes.
R. Basri et al. / Vision Research 38 (1998) 2365–23852376
that the relative position of the gap endpoints influ-ences the cost of matching that would occur for anypossible missing contour. By ‘filling in’ these gaps withstraight lines, and matching these lines with our previ-ously developed cost functions, we account for thestretching and bending that must occur along anycontour fragment that connects the gaps.
Second, the cost function must take account of thelength of contour which we do not match. One way todo this is to measure the difference between two con-tours less the difference that one expects for poorlymatching contours, over the contour lengths that havebeen matched. This evaluates a match relative to one’sexpectations about how well similar shapes shouldmatch, and ensures that we will hypothesize an occlu-sion only when the contours cannot be matched well.This approach is equivalent to just adding a penalty forocclusions proportional to the length of the occludedcontours.
Third, the cost function should reflect the fact thatocclusions are relatively unlikely occurrences. All otherfactors being equal, one much prefers to match all ofthe contour rather than having to resort to an hypothe-sized occlusion. This cost should therefore reflect theexpected likelihood of occlusions occurring, and mayalso reflect the likelihood of occlusions of various sizesoccurring (i.e. given a probability distribution on thelengths of occlusions, we may determine the likelihoodof any particular hypothesized occlusion).
Finally, we note that when gaps occur in contours,these may be treated as known occlusions. For exam-ple, if the beginning of a contour does not meet theending, the gap between the two can be thought of as acontour fragment that is known to be missing. In thiscase, only the first of the above factors is applied. Thatis, we insist that the beginnings and endings of thecontours match, but also consider the bending andstretching cost of matching the gaps between the endsand the beginnings of the contours. In sum, becauseour measure of shape similarity is based on local com-parisons, it is easy to adapt it to the possible presenceof occlusion.
6. Algorithm and experiments
We now describe an algorithm that implements thecost functions that we have discussed, along with exper-iments. The input to our cost functions has been thecurvature at each point. To build a computationalsystem, we need to be able to reliably compute thecurvature at every pixel. The curvature for each point iis computed as the difference between the tangent atthat point and the tangent at point i−1, as shown inFig. 10.
Fig. 10. We compute curvature by taking the difference of adjacenttangents.
We compute the tangent at i as a weighted average ofthe unit vectors defined by the consecutive points sur-rounding i say, [i− j, i− j+1,…, i, i+1,…, i+ j ]. Inany discretization of the curvature there will be asmoothing effect dependent on the scale j. This is aparameter in the program. The weight each vector has,wn, inversely proportional to the distance from theclosest point of the vector to the point i.
Two contours are collected and listed in two chainsS={x(s); s=0, 1,…, �S �} and T={y(t); t=0, 1,…, �T �} where x(s) are the coordinates of the firstcontour, parameterized by s, and y(t) are the coordi-nates of the other contour, parametrized by t. Each onehas different sizes, denoted by �S � and �T �.
The goal is to find the match that minimizes the costfunction. A standard way to achieve this involves usingdynamic programming. When considering a match[x(s), y(t)], we can consider the total cost to reach allpossible predecessors [x(ps), y(pt)] and the cost to gofrom each predecessor to the current hypothesizedmatch [x(s), y(t)]. The set of possible predecessors isrestricted by only considering the ones that are eitherone unit before t, t−1, or one unit before s, s−1,allowing the other coordinate to vary by more thanone, i.e. we allow stretches (jumps) on either coordinate(contour) but not on both simultaneously.
Rather than using dynamic programming directly,however, we can instead formulate our problem as ashortest path problem. We let each match, [x(s), y(t)]be a node in a graph, which is linked with a weightededge to a predecessor state. Then, finding the lowestcost matching becomes equivalent to finding the short-est path from a start to an end state in this graph.While shortest path algorithms rely on dynamic pro-gramming, they may in some cases be more efficientthan the dynamic programming method we have de-scribed, and we find this to be true on our problem (seeref. [15] for more on this difference).
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2377
Table 1Experiments with regular polygons
6.1. Stretch cost
To complete the algorithm we need to consider thecost to go from a predecessor to the current hypothesismatch, the stretch cost. Let us consider the match[x(s), y(t)] and suppose the predecessor to be[x(ps), y(pt)]= [x(s−Stretch), y(t−1)]. Then, we inter-polate the curvature between t−1 and t by simplyrepeating (copying)the curvature at t−1, i.e. we extendkt−1 to the new coordinate t. Thus, the stretch costbecomes:
Stretchcost[(s−Stretch, t−1); (s, t)]
= %Stretch
l=1
F(ks− l, kt−1, 1/Stretch),
where F(ks− l, kt−1, 1/Stretch) is one of the cost func-tions discussed in the previous sections.
To simplify the experiments we have constrained thefirst points of each contour to match each other. This isnot necessary, for closed contours we can consider allpossible initial matches, but this does speed up thealgorithm. For contours with a length of about 100pixels, the algorithm’s run time is on the order of a fewseconds.
6.2. Occlusions
When occlusions are considered we can relax theprevious restriction to consider predecessors of theform (s−Stretchs, t−Stretcht). In this case the costmust be computed as the cost of an occlusion. Thealgorithm structure, however, remains the same.
We have experimented with a number of differentshapes to help us understand the advantages and disad-vantages of our cost function. In the experiments, cur-vature was computed by fitting tangent lines to sevenpoints along the contour. The cost function had theform:
�k2t %−k1�2�k2t %�+ �k1�+l
(t %−1)2
t %+1Spring model (3)
We begin with two simple experiments which weredesigned to demonstrate basic properties of this costfunction and to compare it with two other functions,the linear cost function (Section 4) and a functionwhich is based on a continuous deformation model (seeAppendix B). These two functions were defined asfollows.
�k2t %−k1�+l �t %−1� Linear model (4)
(t %+1)�e−m�k1�−e−m�k2��+l �t %−1�Continuous deformation model (5)
To standardize l between the different cost func-tions, we considered the performance of each functionon two ‘L’ shapes with different length sides. We setl so that each method would produce identical least-cost correspondences for all such ‘L’s. In this way,although l is a free parameter, it is set to a compara-ble value for each function. In function 5 we set p=1.
First, Table 1 shows the cost of comparing a circleto various regular polygons. ‘Synthetic’ indicates thatwe used ideal, unsmoothed curvatures, while ‘real’shows the results with curvatures computed as de-scribed above. As we expect, smoothing allows thespring and linear functions to prefer the polygons withmore sides. Even without smoothing there is a slightpreference in this direction due to discretization effects.Also as expected, function 5 actually prefers thepolygons with fewer sides. This demonstrates the sig-nificance of constraint 3.1.7, which this function doesnot satisfy.
Table 2 shows a simple experiment in which wecompare a shape after bending it either at a partboundary, or at points of zero curvature. This illus-trates the fact that the linear cost function does not givepreference to either sort of bending, while the other twocosts prefer bending at part boundaries.
The next figures show the performance of our func-tion (Eq. (3)) on several examples of hand-drawn nu-merals and on table-like shapes. Fig. 6 shows nine
R. Basri et al. / Vision Research 38 (1998) 2365–23852378
Table 2An experiment comparing bending at a part boundary, with bending in the middle of a part
examples of hand-drawn numerals. While we stressthat our work is meant to capture shape similarityjudgements in a way that is more general than workthat is specific OCR, this provides a simple, interest-ing example. The characters are somewhat challeng-ing, as they deliberately show quite different writingstyles. In this example we also consider the cost ofthe gap between the beginning and the end of eachnumeral, as described in Section 5.
Table 3 and Fig. 11 shows the cost of comparingall pairs of characters, using the spring model. Notethat the function finds a lower cost in matching allinstances of the same character than they do for anypair of different characters.
Next, Fig. 12 shows six table-like shapes, andTable 4 shows the result of comparing these shapeswith our cost function. While it is difficult to sayexactly how the functions should perform on this ex-ample, it is interesting to note that the function con-siders tables 4 and 5 to be the most similar, andtables 2 and 3 to be the most different. In general,performance seems in accord with our intuitions.
Finally, Figs. 13 and 14 show the outlines of fiveanimals, and the relative similarity of each pair.Again, this provides a simple example in which theresults accord with our intuitions.
While the focus of our paper has been on judgingthe similarity of contours, it is also interesting to un-derstand how to find correspondences between pointson two contours. Because elastic matching methodsproduce the minimum cost correspondence betweentwo contours, they can also be used to find a corre-spondence in cases in which two contours are knownto come from similar objects. For example, Fig. 15shows correspondences found between the outlines ofa dog and a horse. Such correspondences can provideclues to the common structure of the two shapes. Fig.16 shows correspondences matching the images of aperson, which can be used for the analysis of non-rigid motion.
In summary, the experiments demonstrate the rela-tive advantages of our cost function in different situa-tions. Overall, we see that this function performs wellin a wide variety of situations.
7. Some remaining challenges
This paper has examined the potential effectivenessof a cost function based on local shape comparison.Obviously, we have not considered the role of globalproperties of shapes such as whether two shapes areboth symmetric, or related by a single affine transfor-mation. Nor have we considered other contour prop-erties, such as contour texture. It is clear that ourwork addresses only some of the issues in shape simi-larity. In this section, we raise three, somewhat moresubtle issues in shape similarity that we feel have notbeen well addressed by our work, or by any otherwork that has come to our attention.
First we consider the question of whether weshould compare shapes by comparing their contoursor their internal regions. Consider Fig. 17. The localdistortion one must apply to the figure on the left toproduce the figures in the middle and on the right isidentical. In each case, two vertical lines are stretchedby a factor of two. And yet, we feel that the middlefigure clearly resembles the figure on the left morethan does the figure on the right. While this might bedue to effects of symmetry and collinearity, one otherpossible explanation for this phenomena can be foundby interpreting each countour as the boundary of a2D material. Although the contours are equally dis-torted in the two figures, the internal 2D regions aredistorted in very different ways. This suggests thatour general approach might be improved by consider-ing a cost based on a mapping between 2D regionsbounded by the contours that we compare. A relatedexample is also discussed by Mumford [37].
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2379
Table 3Results using the spring model (Eq. (3))
One can imagine an approach, analogous to ours, inwhich one seeks the continuous one-to-one mappingfrom one 2D region to another that minimizes thestretching and bending in the 2D mapping. Such anapproach faces several obstacles, including determiningthe appropriate cost function for such a mapping,finding efficient methods of optimizing such a costfunction (dynamic programming does not easily extendto 2D mappings), and understanding the implicationsof such an approach in modeling human perceptions.However, it is possible that such an approach could befruitful, since the example in Fig. 17 suggests thathuman perception is sensitive to the amount of distor-tion of the 2D regions enclosed by a contour, not justto the amount of distortion in the contour when viewedas a 1D object.
However, for a possible counterexample to this argu-ment, consider Fig. 18. When comparing the shapes onthe extreme left and the extreme right, the regionsenclosed by the contours appear to be hardly distortedat all. There is only a bending of the thin middle regionconnecting the regions on either side of the object. Onthe other hand, the middle shape seems to have a highlydistorted interior. None-the-less, the authors’ intuitionsare that the middle figure seems more similar to thefigure on the left than does the right-most figure. At
least it does not appear that the middle figure is muchmore distorted than the right-most one. This suggeststhat for this figure, at least, comparison based on theamount of distortion of the region bounded by thecontour would not lead to good results.
Both of these figures are meant merely to be sugges-tive; the desired behavior of any system may not beobvious. These figures merely raise the issue of whetherany method based either on deforming contours orinternal regions can handle all cases. We suggest alsothat these examples may prove challenging to manyother approaches to judging shape similarity, as well asto our own.
Second, Fig. 19 shows another example that will bechallenging to our approach, and perhaps to others.Despite the obvious similarity between the two shapes,there does not appear to be any good way of compar-ing them in terms of a correspondence between theircontours, regions, or parts. Perhaps this example isbetter thought of as a problem in texture, rather thanshape. However, it points out the difficulty of applyinga single, general approach to judging shape similarity.
Third, in Fig. 20 we show a complex shape, with onepart shifted to a new position. While our approachcould ignore the shifting limb as occluded, and matchthe remainder of the shape, this seems to ignore usefulinformation. It seems more desirable if one can notethat these two limbs are identical, but shifted to differ-ent positions. Providing a specific mechanism for doingthis, however, appears difficult.
These three examples are meant to illustrate thechallenging nature of the general shape similarity prob-lem. In our work, we have attempted to develop oneapproach, based on local deformations, as completelyas possible. However, it is clear that this approach canonly provide one piece of the solution to the generalshape similarity problem.Fig. 11. The numerals used in experiments.
R. Basri et al. / Vision Research 38 (1998) 2365–23852380
Fig. 12. Six table-like shapes used in experiments.
8. Conclusions
This paper has considered the problem of capturinghuman intuitions about shape similarity in a costfunctions based on local deformations. In doing sowe have identified novel constraints on the form ofsuch cost functions, in particular considering howsuch costs can model the part-based nature of ob-jects, and studying the behavior of cost functions thatcan be applied to both smooth and polygonal shapes.We have derived three novel cost functions fromphysical models of contours, and described experi-ments that demonstrate some of the strengths andweaknesses of each cost function. Hopefully, we alsolay the groundwork for alternate approaches to judg-ing shape similarity based on local deformations. Forexample, our work may be extended by consideringthe deformation of 2D regions rather than their 1Dbounding contours, or by considering the significanceof other local contour properties, such as curvatureextreme.
Primarily, we have attempted to demonstrate thatcomputational models of shape similarity should payclose attention to the quantitative, metric propertiesof shapes. These quantitative properties should beconsidered in ways that relate to the more qualitative,part-based nature of objects. However, we feel that apromising approach to part-based analysis is one thatconsiders parts as a continuous property, rather thancommitting to an all-or-nothing decomposition of ashape into parts, since this latter approach does notdegrade gracefully for noisy, occluded or mildly dis-torted shapes. In sum, if one feels that as a contouris distorted it becomes less and less similar to itsoriginal shape in a way that depends on the type ofdistortion, then one of the key problems of shapesimilarity is to understand the relationship betweenthe type of distortion and the change in similarity.Our work systematically addresses this problem.
Acknowledgements
The authors would like to thank Warren Smith forseveral helpful suggestions. The vision group at theWeizmann Institute is supported in part by the IsraeliMinistry of Science, Grant No. 8504. R. Basri is an
incumbent of Arye Dissentshik Career DevelopmentChair at the Weizmann Institute. D. Geiger was sup-ported by AFOSR under F 49620-96-1-0159 and F49620-96-1-0028 and a CAREER award from theNSF.
Appendix A. Properties of the spring model
In this appendix we show how the spring modelsatisfies the desired properties described above. Inparticular it is interesting to note that for this modelto become a metric we need to consider a q rootwhere q\p(p−1) (as we will show). To satisfy allthe properties we obtain that p]1.
Let us be more precise. We will concentrate on thebending cost and assume that k2\k1\0 and t %=1.To obtain the formulae for the other cases we per-form the same type of manipulations. Thus, we startwith the cost function F(k2, k1)= (k2−k1)p(k2t %+k1)1−p.
(1.) F is continuous. That is by construction.(2.) Metric properties:
(a) F(k1, k2, t %)]0. This is clearly satisfied.(b) F(k1, k1, l)=0; F(k1, k2, t %)\0 for k1"k2 or
t %"1. Both are clearly satisfied.(c) The triangle inequality is not satisfied for this
function. However, in the end of this appendix weshow that by taking the total bending cost as Ebending
1/q ,with q\p(p−1) we can satisfy the triangle inequal-ity.
(3.) To guarantee that our cost is scale invariantwe need F(k1, k2, t %)=F(k1, k2/a, at %). This is satisfiedsince
F(k1, k2, t %)= (k2t %−k1)p(k2t %+k1)1−p
=�k2
aat %−k1
�p�k1
aat %+k1
�1−p
=F�
k1,k2
a, at %
�.
Note that this is the case when the stretching term isnot considered. This is to say that our measure is notscale invariant, but is instead divided into a scale-in-variant bending cost and a stretching cost that penal-izes scaling.
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2381
Table 4Results for table-like shapes in Fig. 12 using the spring model.
(4.) Finite corner cost.
limk 2��
F(k2, k1)=k2
(5.) The constraint that it is easier to change thecurvature of a contour at points where curvature isalready high implies that:
(F(k1
+(F(k2
50 for k2\k1
We consider this in conjunction with the nextconstraint.
(6.) The constraint that increased bending also in-creases dissimilarity is equivalent to:
(F(k1
50 and(F(k2
]0 t %=1, k2\k1,
A similar constraint holds for stretching. We will notexplicitly show that the constraint on stretchingholds, since the cost function and constraint that weget by allowing t % to vary with k1=k2=1 are identi-cal to the case where t %=k1=1 and k2 is allowed tovary. We can show this constraint holds for p]1, asfollows:
(F(k1
= (k2−k1)p−1(k2+k1)−p
× [−p(k2+k1)+ (1−p)(k2−k1)]
(F(k1
= (k2−k1)p−1(k2+k1)−p[(1−2p)k2−k1]50 [
p]12−
k1
2k2
,
(F(k2
= (k2−k1)p−1(k2+k1)−p
× [p(k2+k1)+ (1−p)(k2−k1)]
(F(k2
= (k2−k1)p−1(k2+k1)−p[k2+ (2p−1)k1]50 [
p]0,
(F(k1
+(F(k2
=2(1−p)(k2−k1)p(k2+k1)−p50 [
p]1
(7.) We need to show:
nF(k1, k1+o, 1)BF(k1, k1+no, 1)
assuming WLOG that k1\0 and assuming also thateB0, nB1. For the function under consideration,this means:
no
2k1+oB
n2o2
2k1+no
This follows from straight-forward algebraicmanipulation.
We next show that the cost based on a springmodel can satisfy the triangle inequality if the qthroot of the cost function taken. We do this only forthe case of bending, with no stretching.
A.1. Triangle Inequality
Let us consider an infinitesimal contour element ds.The cost to match ds to dt in the other contour isgiven by:
dEbend= (k2t %−k1)p(k2t %+k1)1−pds
= (k2dt−k1ds)p(k2dt−k1ds)1−p.
Thus, we can rename the variables as u2=k2dt u1=k1ds, and C(ui, uj)= (dEbend)1/q, and
C(1, 2)= (dEbend)1/q= (u2−u1)p/q(u2+u1)(1−p)/q.
We would like to show that (dEbend)1/q satisfies thetriangle inequality. Note that if (dEbend)1/q satisfies thetriangle inequality, so does (Ebend)1/q (where Ebend isthe cost of bending for the entire contour) since
Ai1/q+Bi
1/q]Ci1/q Öi
[�%
i
Ai
�1/q
+�%
i
Bi
�1/q
]�%
i
Ci
�1/q
.
Thus, it is enough to show that
C(1, 2)+C(2, 3)]C(1, 3), (6)
For any choice of u1, u2, u3. Without any loss of gen-erality let us assume 05u15u25u3. We first note
R. Basri et al. / Vision Research 38 (1998) 2365–23852382
Fig. 13. Line drawings of five animals. All the contours used have almost the same size (up to 5% difference).
that for u1=u2 or u2=u3 the inequality becomes theequality since, C(A, A)=0. We then investigate theconditions such that by decreasing u1, starting fromu1=u2, the inequality is always preserved (for anyvalue of u2 and u3). More precisely,
−(C(1, 2)(u1
]−(C(1, 3)(u1
and
C(1, 2)+C(2, 3)]C(1, 2) [
C(1−o, 2)+C(2, 3)]C(1−o, 3),
oBB1, and if this is true for any u1, u2, u3 we can startat u1=u2 (where the equality is satisfied) and repeat theprocess of decreasing u1 until u1=0 (the bottom).Thus, we need to determine when
(C(1, 2)(u1
](C(1, 3)(u1
.
Again, we can guarantee this to be true for any u2 and
u3]u2, if(2C(1, 2)(u1 (u2
]0, since
(C(1, 3)(u1
�u3=u2+o−(C(1, 2)(u1
�u2]0 for any u2
[(2C(1, 2)(u1 (u2
]0.
Then, by starting at any u2 and incrementing it, one can
reach any u3 and the property −(C(1, 2)(u1
]−(C(1, 3)(u1
will be satisfied. We have reduced the problem to oneof showing under what conditions on q
(2C(1, 2)(u1 (u2
= −1q(u2−u1)(p/q)−2(u2+u1)[(1−p)/q]−2
×{(1−q)u22+ (2p−1)(1−q)u1
2
+2[2p(p−1)+ (1−q)]u1u2}]0.
Thus, for what values of q is
(1−q)u22+ (2p−1)(1−q)u1
2
+2[2p(p−1)+ (1−q)]u1u250 ?
It is clear that (1−q)+2p(p−1)50�q]1+2p(p−1) does satisfy this inequality, since the first two termsare negative (for q]1). This implies, for example p=1�q]1 (which is saying that the linear cost is ametric, obviously) and p=2�q]5 which is quite aloose bound. One can examine this equation moreclosely, and ask the condition for the functionE(u1, u2)= (q−1)u2
2+ (2p−1)u12
+2[(q−1)−2p(p− l)]u1u2 to be positive.We can show that for the two eigenvalues of theHessian matrix to be positive (convex function) we needq−1]p(2p−1−1) and they are the same as toguarantee that the minimum of the function is positive.Thus, we obtain a tighter bound on q, q\1+p(2p−1−1) which gives approximately for p=2�q]2.4.
Fig. 14. Applying our similarity measure to the five animals. Thefour-legged animals have similar matching scores (costs) and muchless than when matched to the person or to the bird. Within thefours-legged group, the most distinct is the rhinoceros, whose thicklegs and horn increased the cost significantly.
Fig. 15. The matching helps to visualize how the local computationstend to preserve object parts (e.g. legs go with legs, head with head).
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2383
Fig. 16. This figure shows the correspondence obtained between twocontours produced by a person with their limbs in different positions.
Fig. 18. A figure, with two possible distortions.
ture. This is equivalent to assuming that an object iseasier to bend as the magnitude of curvature in-creases.
One reason for choosing a cost function of thistype is the underlying, intuitive physical model of onecontour gradually changing into the other. A secondreason follows from our constraints. If constraint3.1.6 and the triangle inequality hold, we have, for0Bk1Bk2:
F�
k1,k1+k2
2, 1�
+F�k1+k2
2, k2, 1
�]F(k1, k2, 1)
F�
k1,k1+k2
2, 1�\F
�k1+k2
2, k2, 1
�This implies that:
2F�
k1,k1+k2
2, 1�]F(k1, k2, 1)
violating constraint 3.1.7. However, if we wish to vio-late this constraint to the smallest possible degree, weshould minimize:
2F�
k1,k1+k2
2, 1�
−F(k1, k2, 1)
This can be done in two ways, by minimizing:
F�
k1,k1+k2
2, 1�
−F�k1+k2
2, k2, 1
�or by setting:
F�
k1,k1+k2
2, 1�
+F�k1+k2
2, k2, 1
�=F(k1, k2, 1)
The first approach simply weakens the effect of con-straint 3.1.6, but the second approach has no appar-ent disadvantage. Therefore, we desire a cost functionfor which the triangle inequality is at equality whenchanges in curvature are all in the same direction,that is:
F(k1, k2,1)+F(k2, k3, 1)=F(k1, k3, 1)
for k15k25k3
Appendix B. Cost based on continuous deformation
Below we consider a cost function based on theintuitive idea that we slowly bend and stretch onecontour into the other, with the total cost equal tothe sum of an infinite number of infinitesimal defor-mations. That is, we assume that G1 and G2, are con-nected by a continuous series of contours, and thatthe cost of deforming G1 to G2 is the sum of the costsof deforming each contour into the next one in theseries.
To begin, we consider the form such a cost func-tion takes in the absence of stretching (i.e. t %=1).Here we assume that the cost of deforming one cur-vature into another is the sum of a series of smalldeformations, that is:
F(k1, k2, 1)=)& k2
k 1
g(k)dk)
for some function g. Intuitively, g is a function thatdescribes how difficult it is to deform a contour pointinfinitesimally, as a function of the curvature at thatpoint. This implies that we may write:
F(k1,k2, 1)= �f(k2)− f(k1)�for some f, which is the integral of g.
F will satisfy constraint 3.1.6 if and only if g de-creases monotonically as the curvature increases, forpositive curvature, or decreases for negative curva-
Fig. 17. The figure on the left is distorted into the other two figuresby stretching each of the dashed lines by a factor of 2.
Fig. 19. Two figures that appear similar, although their contours,regions and part structure seem quite different.
R. Basri et al. / Vision Research 38 (1998) 2365–23852384
Fig. 20. A figure, with one limb moved to a new position.
function will be bounded at corners, even for arbitrarilysmall amounts of smoothing, as discussed in Section 3.Added to this is a term that causes the difficulty ofbending a contour to decrease exponentially with cur-vature. The parameter c controls the relative weight ofthese terms, while a determines the rate at which theexponential term falls off. Fig. 21 illustrates thefunction.
It is readily verified that this function meets ourconstraints with the following exceptions. First, its be-havior under uniform scaling is not considered, exceptin that we can readily see that when matching two linesof different length, the cost will be the same for allcorrespondences. Second, as previously noted, thisfunction does not satisfy condition 3.1.7.
References
[1] Amit Y, Grenander U, Piccioni M. Structural image restorationthrough deformable templates. J Am Statis Assoc1991;86(414):376–87.
[2] Arkin E, Chew P, Huttenlocher D, Kedem K, Mitchell J. Anefficiently computable metric for comparing polygonal shapes.IEEE Trans on PA MI 1991;13(3):209–16.
[3] Bajcsy R, Solina F. Three dimensional object representationrevisited. Proc First ICCV Conf Lond 1987;231–240.
[4] Basri R, Costa L, Geiger D, Jacobs D. Determining the similar-ity of deformable objects. IEEE Workshop on Physics-basedModeling in Computer Vision 1995;135–143.
[5] Baumberg A, Hogg D. Learning flexible models from imagesequences. ECCV 1995;299–308.
[6] Biederman I. Human image understanding: recent research and atheory. Comp Graph Vis Image Process 1985;32:29–73.
[7] Binford TO. Visual perception by computer. IEEE Conf SysControl 1971.
[8] Blake A, Zisserman A. Visual Reconstruction. Cambridge, MA:MIT Press, 1987.
[9] Bolles R, Cain R. Recognizing and locating partially visibleobjects: the local feature-focus method. Int J Robot Res1982;1(3):57–82.
[10] Brooks R. Symbolic reasoning among 3D models and 2D im-ages. Artif Intell 1981;17:285–348.
[11] Burr D. Elastic matching of line drawings. IEEE Trans PatternAnal Mach Intell 1981;3(6):708–13.
[12] Cohen I, Ayache N, Sulger P. Tracking points on deformableobjects using curvature information. Eur Conf on Comp Vis1992;458–466.
[13] Connell JH, Brady M. Generating and generalizing models ofvisual objects. Artif Intell 1987;31:159–83.
[14] Cootes TF, Taylor CJ, Cooper DH, Graham J. Training modelsof shape from sets of examples. Proc Br Mach Vis ConfSpringer-Verlag 1992;9–18.
[15] Cormen T, Leiserson C, Rivest R. Introduction to Algorithms.Cambridge, MA: MIT Press, 1990.
[16] Duda RO, Hart PE. Pattern Classification and Scene Analysis.New York: Wiley-Interscience, 1973.
[17] Geiger D, Gupta A, Costa L, Vlontzos J. Dynamic program-ming for detecting, tracking and matching deformable contours.IEEE Trans Pattern Anal Mach Intell 1995;17(3):294–302.
[18] Geman S, Geman D. Stochastic relaxation, Gibbs distributions,and the Bayesian restoration of images. IEEE Trans PatternAnal Mach Intell 1984;6(7):721–41.
This implies in fact that F(k1, k3, 1) is exactly the sumof a series of intermediate changes in curvature. So ourmodel, which measures the cost of gradually deformingone contour into another, produces cost functions thatwill least violate constraint 3.1.7, so that smoothing willhave the best chance of overcoming this effect to satisfyconstraint 3.1.7.
We now wish to add the effect of stretching (i.e. t %) toour cost function. We can make our function symmet-ric, and provide an additive stretching penalty by alter-ing it so that:
F(k1, k2, t %)= [(t %+1)�f(k2)− f(k1)�+l �t %%−1�]ds
The right side represents a stretching cost when there isno bending, the left side gives a bending cost in theabsence of stretching. Notice that this is symmetricbecause:
F(k1, k2, t %)=��dt
ds+1
��f(k2)− f(k1)�+l)dtds
−1)n
ds
F(k1, k2, t %)= [(dt+ds)�f(k2)dt− f(k1)ds �+l �dt−ds �]
F(k1, k2, t %)=F�
k2, k1,dsdt�
l is a positive parameter that reflects the relative weightof stretching and bending costs. Note that this stretch-ing meets the metric constraint.
We now propose a particular form for the function,with which we will experiment. We let:
g(k)=e−a�k�+c
and therefore:
f(k)=ck−e−ak, 0Bk
f(k)=ck+eak, kB0
This is a simple, intuitive function that fulfils our needs.It is asymptotically linear in k, and therefore the cost
Fig. 21. The function e−a�k�+c, for a=1, c=0.1.
R. Basri et al. / Vision Research 38 (1998) 2365–2385 2385
[19] Gorman J, Mitchell R, Kuhl F. Partial shape recognition usingdynamic programming. IEEE Trans Pattern Anal Mach Intell1988;10(2):257–66.
[20] Grimson W. On the Recognition of Parameterized Objects. AIMemo 1108. Cambridge, MA: MIT 1987.
[21] Hel-Or Y, Werman M. Constraint-fusion for interpretation ofarticulated objects. IEEE Conf on Comp Vis Pattern Recog1994:39–45.
[22] Hildreth E. The Measurement Of Visual Motion. Cambridge,MA: MIT, 1983.
[23] Hinton G, Williams C, Revow M. Adaptive elastic models forhand-printed character recognition. NIPS 4:512–519.
[24] Ho S. Representing and using functional definitions for visualrecognition, PhD dissertation. University of Wisconsin,Madison 1987.
[25] Hoffman D, Richards W. Parts of Recognition. In: Pinker S,editor. Visual Cognition. Cambridge, MA: MIT Press, 1984.
[26] Huttenlocher D, Klanderman G, Rucklidge W. Comparing im-ages using the Hausdorff distance. IEEE Trans Pattern AnalMach Intell 1993;15(9):850–63.
[27] Huttenlocher D, Noh J, Rucklidge W. Tracking non-rigid ob-jects in complex scenes. Fourth Int Conf Comp Vis 1993;93–101.
[28] Jain A, Zhong Y, Lakshmanan S. Object matching using de-formable templates. IEEE Trans Pattern Anal Mach Intell1996;18(3):267–78.
[29] Kimia B, Tannenbaum A, Zucker S. Shapes, shocks and defor-mations. Int J Comp Vis 1995;15:189–224.
[30] Koenderink JJ, van Doom AJ. The shape of smooth objects andthe way contours end. Perception 1981;11:129–37.
[31] Kass M, Witkin A, Terzopoulos D. Snakes: active contourmodels. Int J Comp Vis 1988;1(4):321–31.
[32] Kupeev K, Wolfson H. On shape similarity. Proc Int ConfPattern Recog 1994;227–237.
[33] Levin E, Pieraccini R. Dynamic planar warping for opticalcharacter recognition. ICASSP III 1992;149–152.
[34] Marr D, Nishihara H. Representation and recognition of thespatial organization of three dimensional structure. Proc R SocLond B 1978;200:269–94.
[35] McConnell R, Kwok R, Curlander J, Kober W, Pang S. Scorrelation and dynamic time warping: two methods for track-ing ice floes in SAR images. IEEE Trans Geo Rem Sensing1991;29(6):1004–12.
[36] Mehrotra R, Grosky W. Shape matching utilizing indexed hy-potheses generation and testing. IEEE Trans Robot Autom1989;5(1):70–7.
[37] Mumford D. Mathematical theories of shape: do they modelperception? SPIE Vol 1570. Geom Meth Comp Vis 1991;2–10.
[38] Mumford D, Shah J. Boundary detection by minimizing func-tionals, I. Proc IEEE Conf Comp Vis Pattern Recog 1985.
[39] Pentland A. Recognition by parts. Proc First Int Conf CompVis 1987;612–620.
[40] Pentland A, Sclaroff S. Closed-form solutions for physicallybased shape modeling and recognition. IEEE Trans PatternAnal Mach Intell 1991;13(7):715–29.
[41] Pope A, Lowe D. Learning object recognition models fromimages. Fourth Int Conf Comp Vis 1993;296–301.
[42] Rivlin E, Dickenson S, Rosenfeld A. Recognition by functionalparts. IEEE Conf Comp Vis Pattern Recog 1994;267–275.
[43] Sankoff D, Kruskal J,(editors). Time Warps, String Edits andMacromolecules: The Theory and Practice of Sequence Com-parison. Reading Ma: Addison-Wesley, 1983.
[44] Siddiqi K, Kimia B.A shock grammar for recognition. ProcIEEE Conf Comp Vis Pattern Recog 1996;507–513.
[45] Siddiqi K, Tresness K, Kimia B. Parts of visual form: ecologicaland psychophysical aspects. Perception 1996;25:399–424.
[46] Stark L, Bowyer K. Achieving generalized object recognitionthrough reasoning about association of function to structure.IEEE Trans Pattern Anal Mach Intell 1991;13(10):992–1006.
[47] Tagare H, O’Shea D, Rangarajan A. A geometric criterion forshape-based non-rigid correspondence Fifth Int Conf Comp Vis1995;434–439.
[48] Tappert C. Cursive script recognition by elastic matching. IBMJ Res Develop 1982;26(6):765–71.
[49] Tsai W, Yu S. IEEE Trans Pattern Anal Mach Intell1985;7:453–462.
[50] Tsukumo J. Handprinted Kanji character recognition based onflexible template matching. Eleventh ICPR 1992;483–486.
[51] Tsukumo J, Tanaka H. Classification of handprinted chinesecharacters using non-linear normalization and correlation meth-ods. Ninth ICPR 1988;168–171.
[52] Tversky A. Features of similarity. Psycholog Rev1977;84(4):327–52.
[53] Ullman S. Aligning pictorial descriptions: an approach to objectrecognition. Cognition 1989;32(3):193–254.
[54] Weiss I. 3D shape representation by contours. CVGIP1988;41(1):80–100.
[55] Winston PH, Binford TO, Katz B, Lowry M. Learning PhysicalDescription from Functional Definitions, Examples and Prece-dents. AI Memo 679. Cambridge,MA: MIT, 1984.
[56] Yoshida K, Sakoe H. Online handwritten character recognitionfor a personal computer system. IEEE Trans Consum ElectronCE 1982;28(3):202–9.
[57] Yuille A, Cohen D, Hallinan P. Feature extraction from facesusing deformable templates. CVPR 1989;104–109.
[58] Zhu S, Yuille A. FORMS: A flexible object recognition andmodelling system. Harvard, TR 1994;94–1.
![Variational Context-Deformable ConvNets for Indoor Scene ... Variational Context-Deformable... · Deformable ConvNets v2 [56] reformulated DCN with mask weights, which alleviated](https://static.documents.pub/doc/80x56/5f26bf72421c4b2b0840bb0e/variational-context-deformable-convnets-for-indoor-scene-variational-context-deformable.jpg)
![[Ahmad Ar-Rowi] Rabiah Al Basri the Sufi Women](https://static.documents.pub/doc/80x56/55cf9846550346d03396acca/ahmad-ar-rowi-rabiah-al-basri-the-sufi-women.jpg)
