Pattern Recognition 45 (2012) 3780–3794

Contents lists available at SciVerse ScienceDirect

Pattern Recognition

0031-32

http://d

n Corr

0200, A

E-m1 Cu2 N

Departm

Australi

journal homepage: www.elsevier.com/locate/pr

Geometric graph comparison from an alignment viewpoint

Surya Prakash b,1, Antonio Robles-Kelly a,b,c,n

a NICTA2 Locked Bag 8001, Canberra ACT 2601, Australiab The Australian National University, Canberra ACT 0200, Australiac UNSW@ADFA, Canberra, ACT 2600, Australia

a r t i c l e i n f o

Article history:

Received 1 August 2010

Received in revised form

9 March 2012

Accepted 13 March 2012Available online 4 April 2012

Keywords:

Graph comparison and retrieval

Graph algorithms

Graph theory

03/$ - see front matter & 2012 Elsevier Ltd. A

x.doi.org/10.1016/j.patcog.2012.03.018

esponding author at: The Australian Nation

ustralia. Tel.: þ612 6267 6268; fax: þ612 62

ail address: antonio.robles-kelly@anu.edu.au

rrent address: IP Australia, Canberra ACT 26

ICTA is funded by the Australian Governm

ent of Broadband, Communications and th

an Research Council through the ICT Centre

a b s t r a c t

In this paper we propose a new approach for the comparison and retrieval of geometric graphs

formulated from an alignment perspective. The algorithm presented here is quite general in nature and

applies to geometric graphs of any dimension. The method involves two major steps. Firstly graph

alignment is effected making use of an optimisation approach whose target function arises from a

diffusion process over the graphs under study. This provides, from the theoretical viewpoint, a link

between stochastic processes on graphs and the heat kernel. The second step involves using a

probabilistic approach to recover the transformation parameters that map the graph-vertices to one

another so as to permit the computation of a similarity measure based on the goodness of fit between

the two graphs under study. Here, we view the transformation parameters as random variables and aim

at minimising the Kullback–Liebler divergence between the two graphical structures under study. We

provide a sensitivity analysis on synthetic data and illustrate the utility of the method for purposes of

comparison and retrieval of CAD objects and binary shape categorisation. We also compare our results

to those yielded by alternatives elsewhere in the literature.

& 2012 Elsevier Ltd. All rights reserved.

1. Introduction

The quest for a robust means to comparison and retrieval ofgraphs has been the focus of sustained activity in the areas ofcomputer vision and structural pattern recognition for over twodecades [44,42,16,19,49,33]. More generally, in computer vision,graph registration and embedding have found application in facerecognition approaches [41,28,27,46], mesh stitching [23,32] and,in general, object recognition [13] and CAD model matching andretrieval [22]. Nonetheless its importance, comparison and retrie-val of graphs is not a straightforward task. The reason being that,given two objects represented by graphical structures, the corre-spondences between them are not, in general, known a priori.Moreover, the complexity of the problem is greatly increased inthose cases when the graphs under study are incomplete or notan exact match to one another.

Here, we focus on the comparison and retrieval of geometricgraphs by viewing the problem from an alignment viewpoint.

ll rights reserved.

al University, Canberra ACT

67 6210.

(A. Robles-Kelly).

06, Australia.

ent as represented by the

e Digital Economy and the

of Excellence program.

Geometric graphs are those which can be realised in terms of ageometric configuration. These appear in a wide variety ofpractical problems [37], spanning from 3D mesh comparisonand retrieval, (where the graphs to be compared are flips) toshock graphs, 2D shape comparison (where the graph is aprojection on a plane), intersection graphs, chordal graphs [9]and levi graphs to name a few.

1.1. Previous work

The problem of finding a general measure of similaritybetween two relational structures is one that arises in a numberof areas in computer vision and pattern recognition. Furthermore,much of the early work on structural pattern recognitionattempted to solve the problem of matching structural represen-tations by optimising a measure of relational similarity. Some ofthe earliest work in the area was undertaken by Shapiro andHaralick [45] and Fu and his co-workers [42,16], who showedhow string edit distance could be extended to relational struc-tures. The idea was to measure the similarity of graphs bycounting the number of graph edit operations, i.e. node, edgeinsertions and deletions, required to transform a graph intoanother. More recently, Sebastian and Kimia [43] have used adistance metric analogous to the string edit distance to performobject recognition from a dataset of shock graphs. More recently,Robles-Kelly and Hancock [40] have recovered a metric between

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3781

pairs of graphs by converting them into string sequences andusing existing string edit distance methods to recover the dis-tance between them.

Despite aimed at more general graphs that the ones examinedhere, the main argument leveled against the work mentionedabove is that it adopts a heuristic approach to the relationalmatching problem by making use of a goal-directed graphsimilarity measure. To overcome this problem, several authorshave adopted a more general approach using ideas from informa-tion and probability theory. For instance, Wong and You [50]defined an entropic graph-distance for structural graph matching.Christmas et al. [11] have shown how a relaxation labelingapproach can be employed to perform matching using pairwiseattributes whose distribution is modeled by a Gaussian. Wilsonand Hancock [49] have used an MAP (maximum a posteriori)estimation framework to accomplish purely structural graphmatching.

These approaches are closely related to the problem ofmeasuring structural affinity using the maximum common sub-graph. Furthermore, Rice et al. [38] showed the relationshipbetween a particular set of cost functions and the longestcommon subsequence for a pair of strings. The maximum com-mon subgraph can be used to depict common topology and, as aresult, it is a powerful tool for graph matching [20] and patternrecognition [25]. Unfortunately, it is not unique and does notdepend on the index of the nodes of the graph. Moreover, thecomputation of the maximum common subgraph is anNP-complete problem. As a result, conventional algorithms forcomputing the maximum common subgraph have an exponentialcomputational cost.

1.2. Overview and motivation

The algorithms above are, in general, optimisation approacheswhich are either computationally costly or prone to distortions.Further, whereas graph comparison methods can be constrainedto be linear in nature, they require a number of constraints on thegraph topology that may be unrealistic in practical applications.Here we exploit the links between registration, retrieval andclassification in terms of their inherent use of measures ofsimilarity for structured data.

Here, we opt to focus our attention in the problem ofcomparison and retrieval of geometric graphs. As mentionedearlier, a large number of graphs examined elsewhere in theliterature are, in fact, geometric [37]. Moreover, geometric graphshave become an increasing focus of attention of those researcherworking on the bridge between geometry and metric graphtheory [3].

We note that, for geometric data, fine registration oftenintends to recover one-to-one correspondences between pointson the space rather than a rough alignment between the graphsunder study. The typical example of these approaches is theiterative closest point algorithm (ICP) [5] and signed distancefields [30,31]. ICP aims at minimising the distance between pointcorrespondences in a two-step process. Firstly, it determinescorresponding point pairs in the two sets of data points. Secondly,it computes the displacement that best aligns the pairs recoveredin the first step. These two steps are interleaved until conver-gence. The first step above often involves a search method such asthe weighted least squares approach proposed by Dorai et al. [14].The method in [14] is reminiscent of that in [10], where thesearch operation is based upon the direction normal to thesurface of reference. Extending the method of Dorai et al. [14],Zhang [52] introduced a thresholding technique aimed at limitingthe maximum distance between point-pairs. Zinfler [53] incorpo-rated a scaling parameter into the ICP algorithm formulation. In a

recent development, Amberg et al. [2] have extended the ICPframework to nonrigid registration by making use of a locallyaffine regularisation operation.

This suggests graphs and correspondences may be recoveredthrough a registration operation. It is somewhat surprising thatthe link between the recovery of a similarity measure betweenrelational structures and their registration or alignment in a givenembedding space has been somewhat under researched in thepattern analysis community. In the data mining community,graph alignment has been seldom used as a generalisation ofgraph isomorphism. For instance, in [48], Weskamp et al. have‘‘aligned’’ graphs representing proteins making use of physico-chemical descriptors.

Thus, here we do not view the computation of a measure ofsimilarity between pairs of relational structures as an integralpart of a graph matching task or graph isomorphism analysis, butrather as a quantity derived from the topological differencesbetween those graphs under consideration. In this paper, wepropose a graph comparison and retrieval method based upon therecovery of the geometric transformation parameters correspond-ing to the vertex-set transformation so as to map the graphsunder study to one another. We can view this mapping as a formof registration so as to compare two graphical structures whichmay not be exact instances of one another, but rather belong todifferent categories or classes. Our approach delivers a measure ofsimilarity between the two graphs which is based upon thegoodness of fit between them.

The method presented here is a two-step one. Firstly, graphalignment is effected making use of an optimisation approachwhose cost function arises from a diffusion process between thevertices in the graphs under study. Secondly, we make use of aprobabilistic approach to recover the transformation parametersthat map the vertices on the model graph onto those on the datagraph. Here, we employ a maximum-likelihood approach so as toestimate the transformation parameters between the two graphnode-sets. Thus, the algorithm is effective in matching two graphsbelonging to the same class. Moreover, nonetheless our focus inthis paper is graph comparison and retrieval, the method pre-sented here is quite general in nature and can be applied tomorphing and re-sampling tasks pertaining graphical structures.

The rest of the paper is organised as follows. In Section 2 weintroduce our graph alignment approach. With the aligned graphsat hand, we elaborate on the computation of a similarity measurebetween geometric graphical structures and provide the stepsequence of the comparison method in Section 3. Results arepresented in Section 4 and conclusions provided in Section 5.

2. Graph comparison as a registration task

Recall that, here, we align graphs so as to transform them toone another. This is as a means to the recovery of a similaritymeasure between two relational structures. This measure ofsimilarity can be used for purposes of comparison or databaseindexing and retrieval. The rationale is that, with a suitabletransformation between two graphs in the dataset, the similaritybetween two relational structures can be recovered using thegoodness of fit between them.

We commence by introducing a method to align the twographs under study. The alignment operation is devoid of corre-spondence recovery operations or pose normalisation steps. As aresult, it permits the alignment of graphs whose order, scale,orientation and coordinate systems are largely disparate. Withthe aligned graphs at hand, we proceed to use a probabilisticapproach to recover the transformation parameters to fit the twographs under study to one another. Thus, we aim at estimating

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943782

the transformation parameters that map the vertex v in the modelgraph GM onto the vertex u in the data graph GD such that thevertex w be given by the following rule:

w¼s�R½Tþu� if uAGD

v if vAGM

(ð1Þ

where s is an n-dimensional shear vector, R is a rotation matrix,T is a translation vector and � denotes the Hadamard, i.e. entry-wise, product between the scale vector and R[Tþu].

Here, the goodness of fit between two graphs is recoveredmaking use of the transformation parameters between the ver-tices in the relational structure under comparison. We do this byusing a kernel-based algorithm which, at output, delivers one ofthe graphical structures, i.e. the data graph, transformed to themodel graph. Hence, the section is organised as follows. Wecommence by introducing our graph alignment algorithm. Wethen proceed to introduce our transformation parameter recoveryprocedure.

2.1. Graph alignment

2.1.1. Diffusion processes

To tackle the alignment task, we commence by defining afunction over the vertex-set for the model and the transformedvertices in the data graph. Let the stochastic process to beWt ¼ f tðvÞ, where t is a real-valued, non-negative parameter. Notethat, so as to imply the parameterisation of the transformedvertex in the graph GD with respect to t. Accordingly, we canrewrite the rule in Eq. (1) as follows:

w¼st�Rt½Ttþu� if uAGD

v if vAGM

(ð2Þ

where st , Rt and Tt are the parameterised analogues of the shearvector, rotation matrix and translation vector introduced before.

To render the stochastic process under study a tractable one,we impose the constraint

DWt ¼�@Wt

@t¼@f tðvÞ

@tð3Þ

In other words, we view the alignment process as one governedby a transformation parameter-set which arises from a diffusionprocess.

The formulation above not only constrains the stochasticprocess so as to allow the recovery of the transformation para-meters, but opens-up the possibility of using the properties of theLaplace–Beltrami operator D for purposes of analysis. The impor-tance of this resides in the fact that the operator D permits theintroduction of the apparatus of the heat kernel K : Ga � Ga �

Rþ/R, a¼ fM,Dg to relate two node-coordinates v and w overthe domain parameterised with respect to t. Thus, we write

f tðvÞ ¼

ZGa

Kðv,w,tÞf t0ðwÞ dw ð4Þ

where f t0ðwÞ are the initial conditions for the function f tð�Þ at the

vertex w and

Kðv,w,tÞ ¼X1k ¼ 1

fkðvÞfkðwÞexpð�lktÞ ð5Þ

In the expression above, fkðvÞ are the eigenfunctions of ft(v) suchthat

Df tðvÞ ¼ lkf tðvÞ ð6Þ

and lk is the kth eigenvalue of ft(v).Hence, the kernel Kðv,w,tÞ is a linear combination over the

eigenpairs of the function ft(v) defined over the vertex-pairs of thegraphs under study. Moreover, it can be shown that, for Euclidean

spaces, the solution of the heat kernel above is given by

Kðv,w,tÞ ¼1ffiffiffiffiffiffiffiffiffiffiffið4ptÞ

p exp�9v�w92

4t

!ð7Þ

In other words, the kernel is an exponential function of thedistance between the vertices v and w whose decay is governedby the parameter t.

2.1.2. Cost function

Now, we turn our attention to the recovery of the transforma-tion parameter-set between the two graphs. To this end, we makeuse of Eq. (7) so as to formulate a cost function which we can thenoptimise accordingly. Recall that the kernel Kðv,w,tÞ is an expo-nential function whose maximum is attained when the term9v�w92

is zero. Moreover, we can view the alignment of themodel graph onto itself as the upper bound of the kernel. This is,the upper bound of the kernel occurs when the data and themodel graph are identical and their vertices are perfectly alignedto one another. This is somewhat intuitive in the Euclidean case,which yields

Kðv,w¼ v,tÞn ¼1ffiffiffiffiffiffiffiffiffiffiffið4ptÞ

p exp�9v�v92

4t

!ð8Þ

where GM ¼ GD. Moreover, this implies that f ðw0Þ � 1. Thus, wecan define the upper bound of the stochastic process under studyto be given by

f tðvÞn¼

ZGM

Kðv,w¼ v,t0Þ dvt ð9Þ

where t0 is the initial value of the parameter t. Note that this isconsistent with the case in which the model and the data graphsare equivalent, which, in turn, reflects the notion that thestochastic process is extremised when the vertices v and w aremembers of identical graphs.

With the upper bounds at hand, we can define the costfunction as the L-2 norm expressed as follows:

D¼X

vAGM

9f nðvÞ�f tðvÞ92

¼X

vAGM

XwAGM

Kðv,w,t0Þ�X

wAGD

Kðu,w,tÞ

����������2

ð10Þ

In the expressions above, we have substituted the integral in theprevious equations for a sum due to the discrete nature of thegraphs under study. Note that, in Eq. (10), ft(v) is defined over thedata graph, whereas f ðvÞn corresponds to the model.

2.1.3. Optimisation process

With the cost function at hand, we now turn our attention toits extremisation. For the minimisation of the target function wehave used a Levenberg Marquardt [29] approach. The Levenberg–Marquardt algorithm (LMA) is an iterative procedure whichprovides a numerical solution to the problem of minimising afunction over a space of parameters.

For purposes of minimising the cost function via the LMA, wecommence by writing the kernel Kðv,w,tÞ in terms of the para-meter set, parameterised with respect to t. Moreover, we notethat, at the commencement of the alignment task, the stochasticprocess under study is in equilibrium. That is, t-1. This is due tothe fact that, as diffusion is a transport process, we view thesystem as being in a state such that the transformation para-meters are extrinsic to the initial arrangement of the meshesunder study. Thus, the extremisation of the function above iseffected via a time inversion of a one-dimensional diffusionprocess [47]. This process is an invariant one with respect to

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3783

time, i.e. t*1=t. As a result, the kernel in Eq. (7) becomes

Kðv,w,tÞ ¼

ffiffiffiffiffiffit

4p

rexpð�4t9v�w92

Þ ð11Þ

The time inversion above is not only theoretically importantdue to the fact that it permits the evolution of the parameter setfrom an extrinsic nature to an intrinsic one as the optimisationprocess progresses, but its also practically useful since it allows usto view t as the iteration index in the LMA.

Let x be the parameter vector at iteration t. At each iteration,the new estimate of the parameter set is defined as xþd, where dis an increment in the parameter space and x is the currentestimate of the transformation parameters. To determine thevalue of d, let gðv,xþdÞ ¼ f nðvÞ�f tðvÞ be approximated using aTaylor series such that

gðv,xþdÞ � gðv,xÞþ JðvÞd ð12Þ

where J(v) is the Jacobian of @gðv,xþdÞ=@x.The set of equations that need to be solved for d is obtained by

equating to zero the derivative with respect to d of the expressionresulting from substituting Eq. (12) into the cost function D. Makinguse of the vector J whose ith entry is given by @gðv,xþdÞ=@x we canwrite the resulting equations in compact form as follows:

ðJT JÞd¼ JT½Gn�GðxÞ� ð13Þ

where Gn is a vector whose elements correspond to the optimumvalues of gð�Þ for every v in Ga,a¼ fM,Dg. Analogously, GðxÞ is a vectorwhose elements correspond to the values gðv,xÞ.

Levenberg [24] introduced a ‘‘damped’’ analogue of the equa-tion above given by

ðJT JþZIÞd¼ JT½Gn�GðxÞ� ð14Þ

where Z is a damping factor adjusted at each iteration and I is theidentity matrix. In [29], the identity matrix I above is substitutedwith the diagonal of the Hessian matrix JT J. We also note thatthe values of Gn should be zero as we aim at minimising Eq. (10).This yields

ðJT JþZJT JÞd¼�JT½GðxÞ� ð15Þ

2.2. Transformation parameter recovery

With the aligned model and data graphs at hand, we now turnour attention to the transformation parameter recovery task. Tothis end, we recover the set of random variables which are relatedto the transformation parameters by making use of a maximumlikelihood formulation. Here, we use a kernel-based approach tocompute the posterior probability as a means to the recovery ofthe rotation matrices, scaling factors and translation vectors,which are our goal of computation.

To do this, consider the set of N-nearest neigbourhood F forthe vertex u in the graph GD. We denote the N-nearest neighbourset for the vertex vAGM by C. Moreover, by viewing the set ofpoints in the neigbourhood under consideration as a distributionwhose probability density function is Gaussian in nature, we canrecover the pairwise transformation parameters between theneighbourhoods as follows. By viewing the translation T v,u asthe mean of the Gaussian distributions for both F and C, we canmake use of the vertex vAGM as a reference for the translationoperation with respect to the aligned candidate vertex u andwrite

T v,u ¼ v�u ð16Þ

Following the rationale above, the scaling factor av,u can beobtained from the ratio of variances for the neighbourhood sets C

and F. This follows from the notion that the two distributions areequivalent if both the mean and the variance are equal. Thus, wecan express the scaling factor av,u as follows:

av,u ¼

PwAC9v�w9PwAF9u�w9

ð17Þ

With the translation and scaling factor at hand, we now turnour attention to the rotation matrix Rv,u. To recover the rotationmatrix Rv,u, we commence by constructing the matrices M and Dwhich represent, in compact form, the coordinates for theN-nearest neighbours in F and C.

Recall that a Procrustes transformation is of the formQ ¼Rv,uD which minimises JM�QJ2. It is known that minimisingJM�QJ2 is equivalent to maximising Tr½DMTRv,u�. This is effectedby using Kristof’s inequality, which states that, if S is a diagonalmatrix with non-negative entries and T is orthogonal, we haveTr½TS�rTr½S�.

Let the singular value decomposition (SVD) of DMT be USVT .Using the invariance of the trace function over cyclic permutation,and drawing on Kristof’s inequality, we can write

Tr½DMTRv,u� ¼ Tr½USVTRv,u� ¼ Tr½VT RUS�rTr½S� ð18Þ

It can be shown that the maximum of Tr½DMTRv,u� is achievedwhen VT RU ¼ I [7]. As a result, the optimal rotation matrix Rv,u isgiven by Rv,u ¼VUT .

2.2.1. Maximum likelihood

Note that, the transformation parameters above are pairwisein nature. As a result, for every vertex vAGM there will be morethan one set of transformation parameters. In fact, there will be9GM9� 9GD9 transformation parameter sets that arise from everypair of vertices in the model and the data graphs. Thus, we requirea means to recover a single rotation matrix R, scaling factor s andtranslation vector T that correspond to the vertex uAGD.

Here, we adopt the same estimation process for each of thethree transformation parameters under consideration, i.e. therotation matrices, scaling factors or translation vectors. Further,we view each of the pairwise transformation parameters asrandom variables. We denote these random variables j and, asa result of our general treatment of the problem, we deal withthem in a general setting.

For the recovery of the transformation parameters, we makeuse of a maximum likelihood method hinging in the minimisationof the Kullback–Liebler divergence between the probability dis-tributions corresponding to both, the vertices in the model anddata graphs. The problem is hence that of approximating anarbitrary distribution PðGÞ by a distribution PðGÞ, where G is theset of random variables j, i.e. pairwise transformation para-meters, each of which correspond to a vertex uAF.

Recall that both, PðGÞ and PðGÞ are unknown. To render theestimation problem tractable, we formulate the problem as thatof recovering the transformation parameters such that bothdistributions are identical up to the first order. In other words,we require the first moments for both distributions of randomvariables to be equal to one another. This is reminiscent ofM-estimator problems [26], where the aim is that of estimatinga hyperparameter such that the residual error is minimum fortwo underlying distributions which are symmetric, identical andindependent. Moreover, for symmetric, identical, independentdistributions, we can show that the minimum of the Kullback–Liebler divergence

KLðPðGÞ,PðGÞÞ ¼ZF

PðjÞlogPðjÞPðjÞ

dj ð19Þ

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943784

with respect to the first moment over the density PðGÞ is given bythe expectation over the neighbourhood F such that

y¼ EwAF½j� ð20Þ

where EwAF½�� is the expectation operator.The fact that the expectation values over the density PðGÞ are

the first-order minimisers of Eq. (19) is relevant to our develop-ment since it suggests the use of a maximum likelihood frame-work to recover the values of the hyperparameter y. Furthermore,following the M-estimator viewpoint, we can consider the hyper-parameter y to be the robust estimate of each of the transforma-tion parameters for the vertex uAGD, i.e. the rotation matrix,scaling factor and translation vector.

Following this rationale, the problem reduces itself to that ofrecovering the hyperparameter y. To do this, we proceed asfollows. Let the likelihood of the variable set j be, by definition,given by the joint density

PðG9yÞ ¼Y

wAFPðj9yÞ ð21Þ

By introducing the posterior probability

Pðu9j,yÞ ¼Pðj9u,yÞPðuÞ

Pðj9yÞð22Þ

we can write the gradient of the log-likelihood for the randomvariables jAG given the hyperparameter y making use of theposterior probability as follows:

ryLðG9yÞ ¼X

wAFPðu9j,yÞry log Pðj9u,yÞ ð23Þ

where ry log Pðj9u,yÞ is the gradient of the posterior probabilitywith respect to the hyperparameter y.

Recall that the maximum likelihood corresponds to the valuesof y for which ryLðG9yÞ ¼ 0. Further, let the residual squarederror of the hyperparameter y with respect to the randomvariables j be given by e¼ ðj�yÞ2. By setting Pðj9u,yÞpexpð�eÞ and, after some algebra, we can recover the maximumlikelihood estimate for the hyperparameter y given by

yn¼

PwAFPðu9j,yÞjP

wAFPðu9j,yÞð24Þ

2.2.2. Kernel-based posterior computation

Note that, following the expressions above, we are left withthe task of computing Pðu9j,yÞ. To this end, we consider the classof kernels given by

Kðu,vÞ ¼ Pðu9v,jÞPðyÞ ð25Þ

where Pð�Þ is a probability density function. We can expand thesekernels by using the chain rule over the conditionals. We get

Kðu,vÞ ¼ Pðu9v,jÞPðv9jÞPðjÞPðyÞ ð26Þ

Further, we can extend these kernels by taking sums overproducts of weighted probability distributions [6]. In this manner,the kernel Kð�Þ can be viewed as a function which is proportionalto a mixture distribution of the form

Pðu9j,yÞ ¼X

wAC

rPðu9w,jÞPðyÞ ð27Þ

where r is the mixture weight given by Pðv9jÞPðjÞ. Note that rrelates the vertices in the model graph, i.e. vAGM , to those in thedata graph, which are represented by the random variables jcorresponding to the vertices uAGD. This is an important obser-vation since it permits the use of the marginal for the distributionabove with respect to the vertex vAGM so as to recover the

posteriors for the nodes in the data graph. This yields

Pðu9j,yÞ ¼rPðu9v,jÞPðyÞP

wACrPðv9w,jÞPðyÞð28Þ

Moreover, making use of the equation above and Eq. (26),we can rewrite Eq. (24) as follows:

Pðu9j,yÞ ¼Kðu,vÞP

wACKðu,wÞð29Þ

The expression above is important since it permits the recov-ery of the translation, scaling and rotation parameters for each ofthe vertices in the data graph. This is as we can substitute Eq. (29)into Eq. (24). By using the appropriate shorthand of j for each ofthe transformation parameters, we have

s¼ W�1X

wAC

Kðu,wÞau,w

T¼ W�1X

wAC

Kðu,wÞT u,w

R¼ W�1X

wAC

Kðu,wÞRu,w ð30Þ

where W¼P

wACKðu,wÞ and s, T and R are the transformationparameters that minimise the Kullback–Liebler divergence asshown in the previous section. As a result, and following Eq. (1),the vertex-coordinates for the data graph GD in the embeddingspace are given by

w¼ s�R½Tþu� ð31Þ

3. Similarity measure computation and algorithm description

With the theoretical background of the graph alignmentmethod provided above, in this section, we elaborate further onthe step sequence for our approach. We also present the Procrus-tean goodness of fit [21] used here for purposes of comparisonand retrieval. In Algorithm 1, we show the step sequence of ourmethod. Note that, in the pseudocode, we have used a number ofsubroutines. These correspond to the two main steps in thealgorithm, i.e. graph alignment and transformation parameterrecovery. The pseudocode also includes an additional step com-prising an ICP application whose importance will become appar-ent later on in the section.

The algorithm takes at input the two graphs under study and atolerance value. At output, delivers the matrix of transformedvertex-set for the candidate and the goodness of fit between themand the model graph. In the algorithm, we have denoted Q andQ n as the matrices whose ith column correspond to the verticesindexed i in the model and data graphs. Note that, since the modelgraph vertices are not transformed, we have denoted, throughoutthe pseudocode, Mn as the matrix whose jth row in correspondsto the model vertex indexed j.

Algorithm 1. MainðGM ,GD,EÞ.

Require: GM ,GD,E

GM: Model graph GD: Data graph E: Tolerance for the optimisation process

1:

Mn,Q’ AlignGraphs ðGM ,GD,EÞ 2: V’ ICP ðMn,Q Þ 3: Q n’ GraphTransformation(Mn,Q ,V,9GD9). ! 4: E’exp �

JMn�Q nJ2

JMnJ2

–100 –80 –60 –40 –20 0 20 40 60 80 100

–100

0

100

200

–150

–100

–50

0

50

100

150

y

x

z

archetype pointscandidate points

–80 –60 –40 –20 0 20 40 60 80 100 120

–100

0

100–150

–100

–50

0

50

100

150

x

z

y

archetype points

candidate points

–60 –40 –20 0 20 40 60 80 100 120–1000100–150

–100

–50

0

50

100

xy

z

archetype pointscandidate points

–60 –40 –20 0 20 40 60 80 100 120–1000

100–150

–100

–50

0

50

100

xy

z

candidate pointsarchetype points

Fig. 1. (a) Edges corresponding to two 3D meshes; (b) Initialisation using probabilistic relaxation; (c) Alignment yielded by the AlignGraphsð�Þ algorithm; (d) Final

alignment delivered by the GraphTransformationð�Þ algorithm.

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3785

5:

return Q n,E Q n: Matrix of transformed vertex coordinates for the datagraph E: Goodness of fit, i.e. similarity measure, between themodel and the data graphs.

In Algorithm 2, we show the pseudocode for the AlignGraphsroutine on Line 1 of Algorithm 1. In the algorithm, we havedenoted Q ðiÞ and Mn

ðiÞ as the ith rows of the matrices Q and Mn,respectively. Also, note that the pseudocode requires, in Line 6,the solution of the LMA equation presented earlier.

After the call to Algorithm 2, the main routine applies the ICPapproach in [2] at Line 2. The method can be generalised to higherdimensions in a straightforward manner and delivers at output aset of one-to-one correspondences in V that are then passed-on tothe transformation parameter recovery method, which is shownin Algorithm 3. In our pseudocode, V is a matrix, whose entryindexed i,j is unity if and only if the ith row in Q corresponds tothe jth row in Mn.

It is worth noting that, in our experiments, we have used anexponential kernel to recover the value of Kðv,uÞ. This kernel givessmall weights to vertices that are far apart and favours verticeswhich are close to one another. The kernel is defined as follows:

Kðv,uÞ ¼b expð�gðv,uÞ2Þ if Jgðv,uÞJok0 otherwise

(ð32Þ

where gðv,uÞ is a function of the vertices vAGM and uAGD given by

gðv,uÞ2 ¼Jv�uJ2

2 h2ð33Þ

where b¼ 1=2ph2, h is a bandwidth parameter and k is a cut-offvalue.

Algorithm 2. AlignGraphsðGM ,GD,EÞ.

Require: GM ,GD,E

GM: Model graph GD: Data graph E: Tolerance for the optimisation process

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943786

1:

x’ TransformationParameterInitialisation(GM ,GD) 2: t’1 3: d’1 4: while dZE do 5: d’ Solution of ðJT JþZIÞd¼ JT

½Fn�GðxÞ�

6:

x’xþd 7: t’tþ1 8: end while 9: for uAGD do 10: Q ðiÞ ¼ ½st�Rt ½Ttþu��T

11:

end for 12: for vAGM do 13: Mn

ðiÞ ¼ ½v�T

14:

end for 15: return Mn,Q

Q : Matrix of transformed vertex coordinates for the datagraph

Mn:Matrix of vertex coordinates for the model graph.

At output, the GraphTransformation routine delivers the matrixof transformed vertex coordinates Q n, which we then use, in Line4 or the Algorithm 1 to recover the measure of similarity betweenthe two graphs. Note that, at output, the vertex order in the matrixQ n is in accordance with the correspondence information in matrixV. This is important, since it permits the direct computation of thesimilarity in Line 4 of Algorithm 1.

Recall that Procrustes analysis determines a linear transforma-tion between the vertex matrix for the model graph and thenormalised, transformed data graph-vertex coordinate matrix suchthat the normalised sum of squared errors is minimised. Thus, wecan define the goodness-of-fit for the Procrustean transformation as

E ¼ exp �JMn�Q nJ2

JMnJ2

!ð34Þ

which is the normalised sum of squared errors between thetransformed data graph vertices, as recovered by our algorithm,and the model. Note that, the expression above is unity if the graphsfit perfectly, whereas it tend to zero if the graphs are grosslydissimilar.

Algorithm 3. GraphTransformationðMn,Q ,V,9GM9Þ.

Require: Mn,Q ,V,9GM9Þ

Q : Matrix of aligned vertex coordinates for the data graph

Mn: Matrix of vertex coordinates for the model graph.

V: Matrix of correspondences between vertex coordinates

in Mn and Q .

9SC9: Number of vertices in the candidate mesh.

1:

for j¼ f1;2,3, . . . ,9GM9g do

2:

for i¼ f1;2,3, . . . ,9GM9g do

3:

T i,j’MnðiÞ�Q ðjÞP

4:

ai,j’

k9MnðiÞ�Mn

ðkÞ9Pk9Q ðjÞ�Q ðkÞ9

5:

Ri,j’VUT

6:

end for 7: end for 8: for j¼ f1;2,3, . . . ,9GM9g do

9:

W’P

iKðQ ðjÞ,Q ðkÞÞ

10: T n

j ’W�1PiKðQ ðjÞ,M

nðiÞÞT i,j

11:

aj’W�1PiKðQ ðjÞ,M

nðiÞÞai,j

12:

Rj’W�1PiKðQ ðjÞ,M

nðiÞÞRi,j

13:

end for 14: for Vði,jÞ ¼ 1 do 15: Q n

ðiÞ ¼ ½aj½Q ðjÞTRjþT n

j ��T

16:

end for 17: return Q n

Q n: Matrix of transformed vertex coordinates for the datagraph

In Algorithm 2, we call the TransformationParameterInitialisa-tion routine. It is worth noting in passing that our method is,indeed, sensitive to initialisation. This is since poor initial para-meters may lead to the LMA falling into local minima [34]. Thiscan be greatly alleviated by making use of a well suited initialapproach. As a result, initialisation can be, in general, an applica-tion-dependant task. In the following section we illustrate theeffects on the method of random initialisation and show how theparameter set can be initialised.

To illustrate the behaviour of the algorithm, we show, in Fig. 1the outputs of Algorithms 2 and 3 for two sample geometricgraphs. Here, we have used two distinct meshes which can beviewed as a triangulation of a point set, i.e. a flip graph. In thefigure, the top row corresponds to the input data and modelgraphs before and after the transformation parameter initialisa-tion. In this case, we have used probabilistic relaxation [49] toinitialise our method. Note that the output of Algorithm 2 yields amuch improved alignment as compared with the initialisation.We then recover one-to-one correspondences via the applicationof ICP. These correspondences are then used as input to Algorithm 3,whose output is shown in the bottom left-hand panel. Note thatAlgorithm 2 applies the LMA, whereas the ICP is used so as to recoverthe correspondence used to compute the transformed vertex coordi-nates for the data graph. That is, Algorithm 2 aligns the graph, whileAlgorithm 3 computes the per-vertex transformation parameters forthe non-rigid fitting operation.

4. Experiments

Now, we turn our attention to a number of experiments so asto illustrate the effectiveness of our method for purposes of graphcomparison and retrieval. As experimental vehicles, we have usedsynthetic graphs, a database of 3D meshes and a set of binaryshapes. In all our experiments we have used the method ofChui and Rangarajan [12] and the transformation matching in[1] as alternatives. For the transformation matching method in [1]we have followed the reported settings (KNN¼5). For ourmethod, in all our experiments we have set the nearest neighboursize N to 8, the tolerance E for the optimisation step to 1� 10�4

and the bandwidth parameter h in Eq. (33) to unity.Our reasons for using the method of Chui and Rangarajan [12] as

an alternative to ours for comparison and retrieval tasks are twofold.Firstly, this is a point matching algorithm that can estimate non-rigid transformations between two vertex-sets. Secondly, it is adeterministic annealing method based upon a thin plate splinetreatment of the problem, which is robust to outliers, graphdecimation and variation. We have used the method in [1] since itis a robust point-matching method which iteratively eliminatesmatches in order to obtain a consensus graph.

4.1. Sensitivity study on synthetic data

Firstly, we examine the effects of graph decimation andvariation in the fitting results yielded by our algorithm. To this

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3787

end, we used the head in Fig. 1 and set of synthetic graphs. Thehead consists of 5770 nodes, whereas the set of synthetic graphsconsists of 10 groups of 10 graphs which have been randomlygenerated as Delaunay triangulations of point clouds in a twodimensional space. Each group accounts for a different graph size,ranging from 50 to 150 nodes in increments of 10 vertices ingraph order. As mentioned earlier, the initialisation for both, ourmethod and the alternative, are identical.

Firstly, we probe the sensitivity of the method to graphdecimation. We commence by randomly decimating each of the100 graphs in our dataset as follows. For each graph, we randomlyselect a predetermined proportion of vertices for deletion fromthe graphical structure. Note that, as a vertex is deleted, itscorresponding edges will also be excised. For each graph, wehave done this 10 times ranging from 5% to 50% of deleted nodesin the graph. As a result, our node-deleted dataset comprised10 000 graphs. For purposes of our analysis, we have compared

Fig. 2. From left to right: Goodness of fit as a function of proportion of nodes deleted an

method in [1].

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

0.2

0.4

0.6

0.8

1

1.2

Fraction of Deleted Nodes

Goo

dnes

s of

Fit

Goodness of Fit as a Function of Graph–nodePercentage Deleted

Our methodMethod of Chui and RangarajanMethod of Aguilar et al.

Fig. 3. Left-hand panel: Goodness of fit as a function of the percentage of nodes exc

transformation matching method in [1]. Right-hand panel: Goodness of fit as a func

Rangarajan.

these graphs against the original relational structures from whichthey were generated. The intuition here is that the goodness of fitE in Algorithm 1 should be unity if, regardless of the nodedeletions, the graphs are well aligned and the transformationparameters recovered accurately. Thus, a good ‘‘fit’’ implies avalue of E close to unity. This is a convenient means of compar-ison since the alternative provides at output one-to-one corre-spondences, and, thus, the corresponding goodness of fit E can becomputed using Eq. (34). For our experiments, we have randomlyinitialised our parameter set.

Fig. 2 shows the plot of the average E as a function ofboth, graph size and proportion of nodes deleted for our method(left-hand panel), the method of Chui and Rangarajan [12](middle panel) and the transformation matching alternative in[1] (right-hand panel). From the figure, its clear that our methodis quite robust to perturbation due to node deletion. Moreover,the alternative in [12] is prone to error due to decimation and

d graph size for our algorithm the method in [12] and the transformation matching

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.2

0.4

0.6

0.8

1

Noise Variance

Goo

dnes

s of

Fit

Goodness of fit as a function of noise variance forthe initialisation values

Our method

Method of Chui and Rangarajan

ised from the head shape in Fig. 1 for our method, the method in [12] and the

tion of noise variance on the initialisation of our method and that of Chui and

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943788

variation. Our method yields a value of E close to unity even incases where the proportion of nodes deleted is over 30%, whichcan induce considerable structural differences between the node-deleted data graphs and the models.

Note that the random decimation process above is not aimedat partial shape comparison but rather evaluating the sensitivityof the methods under consideration to graph size difference. Toprovide a sensitivity study on partial shape comparison, we usethe head shape in Fig. 1 and randomly select a single node. Withthis point in hand, we delete an increasing number of closestneighbours, starting with 5% of the overall graph node-set up to50% in 5% increments. As a result, as the percentage of deletednodes increases, the greater the fraction of the head that isexcised from further consideration. In the left-hand panel ofFig. 3, we show the plot for the goodness of fit E as a functionof the fraction of nodes deleted for our method and the alter-natives. Since we have done this twentyfold, in the plots, the errorbars account for the corresponding variance. Note that these plotsfollow the trend shown earlier, where our method deliveredbetter performance.

Next, we turn our attention to the effects of initialisation.To this end, we have computed the goodness of fit E for our 100graphs of different sizes and added Gaussian noise of zero meanand increasing standard deviation to the initial vector x used bythe Levenberg–Marquardt method in Section 2.1.3. We have donethis for variances ranging from 0.1 to 1 in regular intervals of 0.1.In Fig. 3, we show the plot for the goodness of fit E as a function ofnoise variance for our method and the alternative in [12]. Here,we have omitted the alternative in [1] since this is a sampling-based robust matching algorithm devoid of specific initialisationprocedures. For each graph and variance value we have repeatedthe trial 10 times. Thus, in the figure, the trace corresponds to themean of E, whereas the error bar accounts for its variance. Fromthe error plots, we conclude that our method provides a margin ofimprovement over the alternative. This is particularly evident asthe noise variance increases, in which case the error in the fitdelivered by the alternative in [12] grows rapidly.

4.2. Comparison and retrieval of CAD objects

Next we demonstrate the applicability of our method to theproblem of 3D object comparison and retrieval. To this end, wehave used a data set of 170 3D objects from the 3D MeshesResearch Database by INRIA GAMMA Group. The objects corre-spond to 10 classes, with 17 meshes per category. Sample objectsin the data set used are shown in Fig. 4. In the figure, for the sakeof convenience, we show one sample per class, where eachcategory has been denoted accordingly.

For the CAD objects, the graph vertices are the nodes in themeshes, whereas the arcs in the models constitute the graphedge-set. Note that the objects used in our experiments have alarge variation in terms of mesh-vertex densities. As a result, ourgraphs can vary greatly with respect to the number of nodes ofwhich they are comprised, typically being from approximately400 up to 1200 nodes.

For our retrieval and comparison experiments, we have sepa-rated the data set into two groups. The first of these comprises150 meshes (15 objects per class) for purposes of databasebuilding. The other 20 meshes (2 per class) are used as query

Car Plane Fish Human head Cow

Fig. 4. Sample objects for the object catego

objects. For the 150 meshes in the testing set we have computedthe complete set of similarities E between each pair of distinctobjects. We have done this using, as before, our method, the onein [12] and, additionally, the Chebychev distances for the objectshape distributions in [36].

We have used a search method based upon the shape signa-ture in [18] so as to recover the initial values of the transforma-tion parameter-set. Our search approach is a simple one whichcompares shape functions relative to a number of points on a pairof reference spheres, which osculate the data graph (candidatemesh) and the model, i.e. archetype shape. Note that, this is aquite general procedure that can be extended to further dimen-sions due to the fact that, for geometric graphs, we can alwaysconsider a Darboux frame [35] and constrain the basis on eachsphere to be an outer product of an orthogonal set. Moreover, theinitialisation used here can be viewed as a global search and,therefore, prevents our method from being affected by localoptima.

Our initialisation proceeds as follows. Let the osculatingsphere for the data graph be SD. Analogously, the sphere for themodel graph is denoted SM . Let the reference points on the datagraph osculating sphere be pðiÞn, i¼ f1;2,3g, where n is thedimensionality of the graph-vertex coordinates. Consider a num-ber of uniformly distributed points on the osculating sphere SD.We commence by randomly sampling pð1Þn out of the sphere SM .The point pð2Þn is then randomly sampled out of the greater circleorthogonal to pð1Þn and pð3Þn ¼ pð2Þn � pð2Þn, where � denotes theouter product. With the points on the sphere SM at hand, wesearch over the SD for the point pð1Þ such that the differencebetween the shape functions for the points pð1Þ and pð1Þn isminimised. With the point pð1Þ at hand, we can then focus ourattention on the great circle orthogonal to the vector defined bythe line passing through the center of the sphere under con-sideration and the point pð1Þ. By selecting the point pð2Þ, forwhich the difference in its shape function with respect to pnð2Þ isminimum, we can recover the point pð3Þ ¼ pð1Þ � pð2Þ.

The above search approach is a simple one which hinges in thenotion that corresponding points should correspond to minimumdifferences in the shape function. In other words, the initial valuesof the transformation parameters should correspond to those forwhich the shape functions are similar to one another. Forpurposes of comparison, the shape signature is converted to aprobability density function using the kernel density estimationmethod in [8]. Note that a number of density basis functions canbe used for this purpose. Here, we have used the Epanechnikovkernel [15]. For purposes of computing the differences betweenshape functions, we use Chebychev distances [17]. Since thearchetype and the candidate may greatly differ in scale andresolution, we normalise the shape functions by scaling themsuch that they add to unity. To provide fair grounds for compar-ison, both our method and the alternative in [12] are provided thesame initialisation.

For visualisation purposes, we have preformed metric multi-dimensional scaling (MDS) [7] on the pairwise distances. Broadlyspeaking, this is a method for visualising objects characterised bypairwise distances rather than by ordinal values. It hinges aroundcomputing the eigenvectors of a dissimilarity matrix. The leadingcomponents of the eigenvectors are the co-ordinates associatedwith the meshes under study. The method can be viewed as

Cup Donkey Shoe Teapot Wine glass

ries used in our retrieval experiments.

Fig. 8. MDS plot for the shape distributions presented in [36].

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3789

embedding the objects in a pattern space using a measure of theirpairwise dissimilarity to one another. Thus, for accurate dissim-ilarity measures between objects, it is expected that the MDSembeddings capture the class or cluster structure accordingly.

In Fig. 5, we plot the positions of the objects in the testing seton the plane corresponding to the two leading dimensions of theeigenspace resulting from MDS. In Figs. 6–8 we display the MDSembeddings for the alternatives. Of the four distance measures,the clusters resulting from the use of the goodness of fit for ourmethod produce the clearest cluster structure for the distinctobjects in our experimental test set. Moreover, nonetheless theembedding for the methods in [1,36] provide a better clusterstructure than the alternative in [12], the shoes and the cars havebeen scattered across a single cluster. The same applies to the fishand the airplanes and the donkeys and cows. Note that, for ourmethod, these very same object categories are close to each other

Fig. 5. MDS plot for the distances yielded by our method.

Fig. 6. MDS plot for the distances yielded by the method in [12].

Fig. 7. MDS plot for the distances yielded by the method in [1].

Fig. 9. Clusters extracted using the goodness of fit for our method.

Fig. 10. Clusters extracted using the goodness of fit for the method in [12].

on the eigenspace but, in contrast with the alternatives, they arenot scattered or mixed with one another. Hence, the dissimilaritymeasure delivered by our method appears to be effective in bothdistinguishing between different object classes and for measuringfine changes in shape structure for meshes of the same class.

Now, we turn our attention to the retrieval task. This is asfollows. We commence by applying the pairwise clusteringalgorithm described in [39] to the dissimilarity measures at hand.The pairwise clustering algorithm requires distances to be repre-sented by a matrix of pairwise affinity weights. Ideally, thesmaller the distance, the stronger the weight, and hence themutual affinity to a cluster. The affinity weights are required to bein the interval [0, 1]. Hence, for the pair of meshes indexed i and j

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943790

the affinity weight is taken to be

Wi,j ¼ exp �cdði,jÞ

maxðDÞ

� �ð35Þ

where c is a constant and di,j is the element indexed i,j of thematrix D¼ 1�E of pairwise dissimilarities between distinct pairsof objects. The clustering algorithm maintains two sets of vari-ables. The first of these is a set of cluster membership indicatorswhich measure the affinity of the graph indexed i to the cluster ofinterest. The second, is an estimate of the affinity matrix based onthe cluster-membership indicators. At output, the algorithmdelivers the maximum likelihood estimates for the cluster-mem-bership indicator values and the cluster indexes corresponding toeach of the meshes in the testing set.

Fig. 11. Clusters extracted using the shape distributions in [1].

Fig. 12. Clusters extracted using the shape distributions in [36].

Fig. 13. Retrieval results

With the cluster membership indicators and cluster indexes athand, we can perform retrieval by comparing a query object withthe mesh in each cluster whose within-class affinity is largest.This mesh is that whose membership indicator is largest in rankamongst those for the objects in its respective cluster. We canview this mesh as the cluster center and, hence, by searching overthese centers, we reduce the complexity of the retrieval opera-tion. Once the comparison with the cluster centers is effected, wesearch within the class whose center is most similar to the queryobject. We can then display the results, in order of relevance, asthose objects whose goodness of fit makes them most similar tothe query mesh in the cluster of reference.

In Figs. 9–12 we show the clustering results and the clustercenters for our method, the alternatives in [12,1] and thedistances between shape distributions in [36]. In the figures, thetop section shows the cluster elements, where distinct columnscorrespond to separate clusters. The bottom section shows thecluster center. Note that the recovered clusters reflect the scatterin the MDS plots. Furthermore, the classes recovered using thegoodness of fit for the alternative in [12] are fragmented, withsome object categories spread across more than three clusters.Note that, for our method, the alternative in [1] and the distancesbetween shape distributions in [36], the clusters are in accor-dance with the object categories. Nonetheless, the clusters corre-sponding to our method are more balanced across classes. This is,the number of elements per class is more evenly distributed. Thiscan be attributed to classes being more compact and, therefore,objects for the same category having greater affinity between oneanother.

In Figs. 13–16 we show the retrieval results for our method,the alternative in [12], the robust matching method in [1] and theshape distributions in [36]. In the figures, we show, on the toprow, the query object. The second row shows the cluster centerwhich is most similar amongst those in the dataset. The fourbottom rows show, in order of relevance, from top-to-bottom, thequery results. From the query results, we can conclude that ourmethod delivers the best performance, with no misclassificationsfor the object retrieved as the most relevant to the queryoperation. In the other hand, the methods in [36,1] yield correctcluster centers with some misclassification at the class level.In contrast, the goodness of fit for the alternative in [12] yields anumber of misclassifications where some of the cluster centersare incorrectly assigned.

Finally, we provide a more quantitative evaluation of ourretrieval results in Table 1. In the table, we show the number ofcorrectly retrieved objects as a function of relevance rank. That is,for each order of relevance, we have counted how many of theretrieved objects correspond to the same category as the query mesh.

using our method.

Fig. 14. Retrieval results using the goodness of fit for the method in [12].

Fig. 15. Retrieval results using the goodness of fit for the method in [1].

Fig. 16. Retrieval results using the method in [36].

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3791

Thus, the fractions in the table correspond to the retrieval perfor-mance, where the denominator is always 20, i.e. the amount of queryobjects, and the numerator accounts for the number of correctlyretrieved meshes for each relevance rank. The results in the table arein accordance with our earlier observations. Our method provides amargin of improvement over the use of shape distributions and themethod in [1], with the goodness of fit for the method in [12]delivering the most adverse results.

Furthermore, our method is quite computationally efficient ascompared with the graph transformation method in [1]. For ourmethod, the average per-graph processing time (the time it takes

for the Algorithm 1 to execute) is approximately 0.65 s. Incontrast, the method in [1] takes 4.56 s in average to deliver thematches so as to compute the goodness of fit. The method of Chuiand Rangarajan [12] processes graphs in an average of 0.72 s.All our experiments were effected on a Quad Core i7, 3.4 GHz desktopwith 8 GB of RAM, with all the methods being implemented on Cþþ.It is worth noting that the times above should be taken with cautionsince execution times can vary greatly between systems and imple-mentations. Nonetheless, these results are somewhat expected sincethe graph transformation in [1] requires the computation of ak-nearest neighbour graph at each iteration.

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943792

4.3. Binary shape categorisation

Now, we turn our attention to the categorisation of binaryshapes. To this end, we have used the MPEG7 CE-Shape-1 shapedatabase, which contains 1400 binary shapes of 70 differentclasses with 20 images in each category. In Fig. 17, we showsome example shapes for the MPEG-7 dataset. We have repre-sented each shape as a graph in G whose vertices correspond tocontour pixels sampled in regular intervals. Here, we havesampled 1 in every 10 pixels on the shape contours. With thesample contour pixels at hand, we build a fully connected graphwhose edge-weights are given by the Euclidean distances on theimage plane between each pair of pixel-sites.

Thus, the entries of the weight matrix for the graph correspond tothe pairwise distances between the image-coordinates for every pairof vertices in the graph. The weigh matrix is then normalised to unityso as to have every weight in the graph in the interval [0, 1]. With thegraphs in hand, we perform classification in a manner akin to theretrieval operation effected earlier on the CAD models by applying anearest-neighbour classifier. In our experiments, we have clusteredthe database so as to obtain 10 centers.

For all our shape categorisation experiments, we have divided thegraphs in the dataset into a training and a testing set. Each of thesecontains half of the graphs in each dataset. That is, we have used 700randomly selected graphs for training and the remaining 700for testing. We have done this 10 times, applying our submersion

Table 1Retrieval results as a function of relevance rank for our method and the

alternatives.

Method Relevance rank

1 2 3 4

Our approach 20/20 19/20 19/20 18/20

Goodness of fit using the method in [12] 15/20 8/2 13/20 12/20

Goodness of fit using the method in [1] 20/20 19/20 19/20 15/20

Shape distributions [36] 17/20 15/20 17/20 15/20

Fig. 17. Sample shapes in the M

Table 2Shape categorisation results on the MPEG-7 dataset.

Categorisation method Our method

Classification rate 82.6370.67%

Categorisation method Shape context [4]

Classification rate 77.5572.39%

method and the alternatives to each of these training and testingsets.

For all the three structural matching methods, i.e. the one in [1],the one in [12] and ours, we use the goodness of fit for structuring thedatabase and to retrieve the best candidate amongst those graphswithin the cluster whose center is most similar to the testing (data)graph. For purposes of initialisation, we have used a method akin tothat presented in the previous section, where the search is onlyeffected on the circle which osculates the data and the model graphunder consideration.

For the sake of comparing our results with specialised alter-natives elsewhere in the literature, we also show classificationrates obtained by making use of two methods specificallydesigned for purposes of shape classification. These are the shapeand skeletal contexts in [4,51], respectively. Once the shape andskeletal contexts are at hand, we train two one-versus-all SVMclassifiers whose parameters have been selected through ten-foldcross validation.

The categorisation results are shown in Table 2. In the table, weshow the mean and variance for the percentage of correctly classifiedshapes in the dataset. Despite the basic strategy taken for theconstruction of our graphs, which contrasts with the specialiseddevelopments in the alternatives, our method and the alternative in[1] still provide a margin of improvement with respect to the methodin [4] and that in [51]. Moreover, our method delivers a lowervariance, which indicates that the algorithm is more stable to trainingset variation than the alternatives.

Moreover, the timing of our method, again, is quite compar-able to those of the alternatives with the exception of the graphtransformation method in [1]. For the method in [1], the per-graph testing time is of approximately 6.57 s. This is comparedwith the 0.9 s for our method, 1.1 s for the method in [12] andapproximately 0.7 s for the shape and skeletal contexts.

5. Conclusions

In this paper, we have proposed a graph comparison methodbased upon stochastic processes on graphs by casting the graph

PEG7 CE-Shape-1 database.

Method in [12] Graph transformation in [1]

74.4374.38% 80.8671.97%

Skeletal context [51]

79.9171.78%

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–3794 3793

comparison task into an alignment setting. The method presentedhere is a two-step one in which the first step comprised theoptimisation of a target function arising from a diffusion process.The second step in our method hinges in the use of a statisticalapproach to recover the node-wise transformation parametersbetween vertices in the data and model graphs. We have shownhow these transformation parameters can be recovered using akernel-based method. We have done this by viewing the trans-formation parameters as random variables and minimising theKullback–Liebler divergence between the two graphs under con-sideration. As a result of this formulation, two graphs can becompared through the use of the Procrustean goodness of fit. Wehave provided a sensitivity study on synthetic data and illustratedthe utility of the method for purposes of comparison and retrievalof CAD objects and classification of binary shapes. It is worthnoting that the comparison method presented here is quitegeneral and may be used for other tasks, such as morphing, re-meshing or comparison of structured data in biochemistry orbioinformatics.

References

[1] W. Aguilar, Y. Frauel, F. Escolano, M.E. Martınez-Perez, A. Espinosa-Romero,M.A. Lozano, A robust graph transformation matching for non-rigid registra-tion, Image Vision Computing 27 (7) (2009) 897–910.

[2] B. Amberg, S. Romdhani, T. Vetter, Optimal step nonrigid icp algorithms forsurface registration, in: IEEE Computer Society Conference on ComputerVision and Pattern Recognition (CVPR), USA, June 2007, pp. 1–8.

[3] H.-J. Bandelt, V. Chepoi, Metric graph theory and geometry: a survey,Contemporary Mathematics, vol. 453, American Mathematical Society, 2008.

[4] S. Belongie, J. Malik, J. Puzicha, Shape matching and object recognition usingshape contexts, IEEE Transactions on Pattern Analysis and Machine Intelli-gence 24 (4) (2002) 509–522.

[5] P.J. Besl, N.D. McKay, A method for registration of 3-d shapes, IEEE Transac-tions on Pattern Analysis and Machine Intelligence 14 (2) (1992) 239–256.

[6] C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.[7] I. Borg, P. Groenen, Modern Multidimensional Scaling, Theory and Applica-

tions, Springer Series in Statistics, Springer, 1997.[8] A. W Bowman, A. Azzalini, Applied Smoothing Techniques for Data Analysis,

Oxford University Press, 1997.[9] T. Caetano, T. Caelli, D. Schuurmans, D.A.C. Barone, Graphical models and

point pattern matching, IEEE Transactions on Pattern Analysis and MachineIntelligence 28 (10) (2006) 1646–1663.

[10] Y. Chen, G. Medioni, Object modelling by registration of multiple rangeimages, Image and Vision Computing 10 (3) (1992) 145–155.

[11] W.J. Christmas, J. Kittler, M. Petrou, Structural matching in computer visionusing probabilistic relaxation, IEEE Transactions on Pattern Analysis andMachine Intelligence 17 (8) (1995) 749–764.

[12] H. Chui, A. Rangarajan, A new point matching algorithm for non-rigid registration,Computer Vision and Image Understanding 89 (2–3) (2003) 114–141.

[13] W. Dickson, Feature grouping in a hierarchical probabilistic network, Imageand Vision Computing 9 (1) (1991) 51–57.

[14] C. Dorai, A.K. Jain, Shape spectrum-based view grouping and matching of 3dfree-form objects, IEEE Transactions on Pattern Analysis and MachineIntelligence 19 (10) (1997) 1139–1145.

[15] V. Epanechnikov, Nonparametric estimates of a multivariate probabilitydensity, Theory of Probability and its Applications 14 (1969) 153–158.

[16] M.A. Eshera, K.S. Fu, A graph distance measure for image analysis, IEEETransactions on Systems, Man and Cybernetics 14 (1984) 398–407.

[17] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed., Aca-demic Press, 1990.

[18] R. Gal, A. Shamir, D. Or-Cohen, Pose oblivious shape signature, IEEE Transac-tions on Visualization and Computer Graphics 13 (2007) 261–271.

[19] S. Gold, A. Rangarajan, A graduated assignment algorithm for graph match-ing, Pattern Analysis and Machine Intelligence 18 (4) (1996) 377–388.

[20] R. Horaud, T. Skordas, Stereo correspondence through feature grouping andmaximal cliques, IEEE Transactions on Pattern Analysis and Machine Intelli-gence 11 (11) (1989) 1168–1180.

[21] R.A. Horn, C.R. Johnson, Topics in Matrix Analysis, Cambridge UniversityPress, 1991.

[22] C.Y. Ip, S.K. Gupta, Retrieving matching cad models by using partial 3d pointclouds, Computer Aided Design and Applications 4 (5) (2007) 629–638.

[23] Kim, S.H., Hwang, Y.H., Hong, H.K., Choi, M.H., An improved icp algorithmbased on the sensor projection for automatic 3d registration, MICAI 2004,pp. 642–651.

[24] K. Levenberg, A method for the solution of certain non-linear problems inleast squares, The Quarterly of Applied Mathematics 2 (1944) 164–168.

[25] R. Levinson, Pattern associativity and the retrieval of semantic networks,Computer and Mathematics with Applications 23 (1992) 573–600.

[26] S.Z. Li, Markov Random Field Modeling in Image Analysis, Springer, 2001.[27] X. Lu, A.K. Jain, Deformation modeling for robust 3d face matching, Proceed-

ings of the IEEE Computer Society Conference on Computer Vision andPattern Recognition (2006) 1377–1383.

[28] X. Lu, A.K. Jain, D. Colbry, Matching 2.5d face scans to 3d models, IEEETransactions on Pattern Analysis and Machine Intelligence (PAMI) 28 (1)(2006) 31–43.

[29] D. Marquardt, An algorithm for least-squares estimation of nonlinear para-meters, SIAM Journal on Applied Mathematics 11 (1963) 431–441.

[30] T. Masuda, Generation of geometric model by registration and integration ofmultiple range images, in: 3rd International Conference on 3D DigitalImaging and Modeling, Canada, 2001, pp. 254–261.

[31] T. Masuda, Object shape modelling from multiple range images by matchingsigned distance fields, in: 1st International Symposium on 3D Data Proces-sing Visualization and Transmission (3DPVT), Padova, Italy, 2002, pp.439–448.

[32] N.J. Mitra, N. Gelfand, H. Pottmann, L. Guibas, Registration of point cloud datafrom a geometric optimization perspective, in: SGP ’04: Proceedings of the2004 Eurographics/ACM SIGGRAPH Symposium on Geometry Processing,ACM Press, New York, NY, USA, 2004, pp. 22–31.

[33] R. Myers, R.C. Wilson, E.R. Hancock, Bayesian graph edit distance, PAMI 22 (6)(2000) 628–635.

[34] J. Nocedal, S. Wright, Numerical Optimization, Springer, 2000.[35] B. O’Neill, Elementary Differential Geometry, Academic Press, 1997.[36] R. Osada, T.A. Funkhouser, B. Chazelle, D.P. Dobkin, Shape distributions, ACM

Transactions on Graphics 21 (4) (2002) 807–832.[37] J. Pach (Ed.), Towards a Theory of Geometric Graphs, Contemporary Mathe-

matics, vol. 342, American Mathematical Society, 2004.[38] S.V. Rice, H. Bunke, T.A. Nartker, Classes of cost functions for string edit

distance, Algorithmica 18 (1997) 271–280.[39] A. Robles-Kelly, E.R. Hancock, A probabilistic spectral framework for spectral

clustering and grouping, Pattern Recognition 37 (7) (2004) 1387–1405.[40] A. Robles-Kelly, E.R. Hancock, Graph edit distance from spectral seriation,

Transactions on Pattern Analysis and Machine Intelligence 27 (3) (2005)365–378.

[41] T. Russ, C. Boehnen, T. Peters, 3d face recognition using 3d alignment for pca,Proceedings of the IEEE Computer Society Conference on Computer Visionand Pattern Recognition (2006) 1391–1398.

[42] A. Sanfeliu, K.S. Fu, A distance measure between attributed relational graphsfor pattern recognition, IEEE Transactions on Systems, Man and Cybernetics13 (1983) 353–362.

[43] T.B. Sebastian, P.N. Klein, B.B. Kimia, Shock-based indexing into large shapedatabases, in: European Conference on Computer Vision, vol. 3, 2002,pp. 731–746.

[44] L.G. Shapiro, R.M. Haralick, Relational models for scene analysis, IEEETransactions on Pattern Analysis and Machine Intelligence 4 (1982) 595–602.

[45] L.G. Shapiro, R.M. Haralick, A metric for comparing relational descriptions,IEEE Transactions on Pattern Analysis and Machine Intelligence (1985)90–94.

[46] H. Tanaka, M. Ikeda, H. Chiaki, Curvature based face surface recognition usingspherical correlation, Proceedings of the International Conference in Auto-matic Face and Gesture Recognition (1998) 372–377.

[47] S. Watanabe, On time inversion of one-dimensional diffusion processes,Probability Theory and Related Fields 31 (2) (1975) 115–124.

[48] N. Weskamp, E. Hullermeier, D. Kuhn, G. Klebe, Multiple graph alignment forthe structural analysis of protein active sites, ACM/IEEE Transactions onComputational Biology and Bioinformatics 4 (2) (2007) 310–320.

[49] R.C. Wilson, E.R. Hancock, Structural matching by discrete relaxation, IEEETransactions on Pattern Analysis and Machine Intelligence 19 (6) (1997)634–648.

[50] A.K.C. Wong, M. You, Entropy and distance of random graphs with applica-tion to structural pattern recognition, IEEE Transactions on Pattern Analysisand Machine Intelligence 7 (1985) 599–609.

[51] J. Xie, M. Shah, P. Heng, Shape matching and modeling using skeletal context,Pattern Recognition 41 (5) (2008) 1756–1767.

[52] Z. Zhang, Iterative point matching for registration of free-form curves andsurfaces, International Journal of Computer Vision 13 (2) (1994) 119–152.

[53] T. Zinßer, J. Schmidt, H. Niemann, Point Set Registration with Integrated ScaleEstimation, in: R. Sadykhov, S. Ablameiko, A. Doudkin, L. Podenok (Eds.),Proceedings of the Eighth International Conference on Pattern Recognitionand Image Processing (PRIP 2005), Minsk, Republic of Belarus, 2005,pp. 116–119.

Surya Prakash received the Graduate Diploma in Computer Science from the University of Auckland, New Zealand, the MSc Degree in Applied Mathematics from theUniversity of New England, Australia and the PhD Degree in Computer Science from the Australian National University, Australia. He is a Patent Examiner at IP Australia.His research interests include computer vision, 3D reconstruction and pattern recognition.

S. Prakash, A. Robles-Kelly / Pattern Recognition 45 (2012) 3780–37943794

Antonio Robles-Kelly received his BEng degree in Electronics and Telecommunicat

ions from the Inst. Tecnologico y de Estudios Superiores de Monterrey with honours in1998. In 2001, being a graduate student at York, he visited the University of South Florida as part of the William Gibbs/Plessey Award to the best research proposal to visitan overseas research lab. He received his PhD in Computer Science from the University of York in 2003. After receiving his doctorate, Dr. Robles-Kelly remained in Yorkuntil 2004 as a Research Associate under the MathFit-EPSRC framework. In 2005, he took a research scientist appointment with National ICT Australia (NICTA) at theCanberra Laboratories. After working on surveillance systems with query capabilities, in 2006 he was appointed the project leader of the Spectral Imaging project and, in2011, promoted to Principal Researcher. He is a Senior Member of the IEEE, an Adjunct Associate Professor of the ANU and a Conjoint Senior Lecturer of the University ofNew South Wales at the Australian Defense Force Academy (UNSW@ADFA). His research interests are in the areas of Computer Vision, Pattern Recognition, SpectralImaging and Computer Graphics. Along these lines, he has done work on segmentation and grouping, tracking, graph-matching, shape-from-X, hyperspectral imageunderstanding and reflectance models. He is also interested in the differential structure of surfaces and graphical models for recognition and classification.