GRAPHICAL MODELS AND IMAGE PROCESSING

Vol. 59, No. 6, November, pp. 373–387, 1997ARTICLE NO. IP970447

Multidimensional Alignment Using theEuclidean Distance Transform

Dorota Kozinska

Nencki Institute of Experimental Biology, Warsaw, Poland

and

Oleh J. Tretiak, Jonathan Nissanov, and Cengizhan Ozturk

Drexel University, Philadelphia

Received January 29, 1996; revised March 21, 1997; accepted July 23, 1997

one individual or from different individuals. Such combina-tion or comparison requires the registration of images orWe present a methodology for alignment of multidimensional

data sets that is based on the Euclidean distance transform volumetric data sets. This involves the determination of theand the Marquardt–Levenberg optimization algorithm. The geometric transformation between images and subsequentproposed approach operates on pixel or voxel descriptions of transformation of one or both data sets into a commonobjects to be matched and estimates the parameters of a space coordinate system. This paper deals with the determinationtransformation for optimal alignment of objects. The computa- of the geometrical transformation required to align twotional cost of an algorithm developed with this method is esti-

data sets.mated. The methodology is tested by developing an algorithmImage registration techniques have recently been re-for rigid body transformation alignment of three-dimensional

viewed by van der Elsen et al. [1] who give a comprehensivedata sets. Tests with synthetic and real objects indicate thatand insightful overview of the work in this field. Amongthe method is accurate, reliable, and robust. 1997 Academic Press

the categories used for classifying image registrationmethods it is most cogent to concentrate on the typeof data and the type of transformation used for image1. INTRODUCTIONmatching.

The types of data range from extrinsic features, such asImage alignment or registration is a common task inimage processing. It is encountered in many fields such as fiducial objects introduced into the field of view to provide

coordinates of alignment, to intrinsic features, namelyremote sensing, where two-dimensional data sets must beregistered, and biomedical imaging, where two- or three- structures which are normally present in objects being

matched. While extrinsic features are preferable since theydimensional images are collated. This problem arises whendata collected at different times or with different modalities can provide unambiguous and controlled fiducial informa-

tion, the introduction of these structures may complicateare to be correlated. For example, in medicine a body canbe imaged through computer tomography (CT), magnetic the data acquisition process adding cost and time. In many

cases, such as when viewing the interior of an intact organ-resonance imaging (MRI), positron emission tomography(PET), single photon emission computed tomography ism or in remote sensing, it is just not feasible to use

extrinsic features. Another issue is the type of intrinsic(SPECT), and ultrasound. Since these methods generallyprovide complementary information, synthesis of data feature to be used. A traditional approach to image align-

ment relies on correlation, where the gray value distribu-gathered by more than one method can give synergisticinformation. Such synthesis allows the combination of ana- tions of the images or data sets are compared. This, how-

ever, is not feasible if there is a qualitative differencetomical (e.g., CT and MRI) and functional (PET, SPECT)information in two- or three-dimensional images. Such vi- between the gray scales, such as in multimodality imaging.

Under these conditions a viable strategy is to preprocesssualization can be very helpful to evaluate the locationand type of pathology under examination. Another task the images to extract homologous geometric features in

the data sets to be registered, such as points, lines, orfacilitated by combining images is the comparison of stud-ies in one modality but collected at different times from surfaces. These features may then be registered and the

3731077-3169/97 $25.00

Copyright 1997 by Academic PressAll rights of reproduction in any form reserved.

374 KOZINSKA ET AL.

gray-scale images transformed by the mapping obtainede(P) 5 ON

i51d 2(T(pi , P)). (2)from this alignment.

The types of transformations used range from rigid bodymotions [2–7] to more general classes such as affine map- The parameters P0 of the best fit transformation are deter-pings [8], spline equations, and general ‘‘elastic’’ transfor- mined by minimizing this quantity over transformationmations [9, 10]. In general, we may write the transforma- parameters Ption as p9 5 T(p, P), where p stands for the coordinatesof an image point before alignment, p9 is the point after P0 5 arg min e(P). (3)alignment, and P 5 (f1, f2, . . . , fP) are the parameters ofthe transformation. For example, in a two-dimensional An exemplary study of this technique is reported by

Pellizari et al. [7], where the accuracy with which MRI,rigid body transformation there are three parameters,namely the rotation angle and two translations. In a general CT, and PET data sets are aligned is validated with extrin-

sic fiducial marks. In this study the ‘‘head’’ surface is ap-elastic transformation the number of parameters is on thesame order as the number of pixels (or voxels) in the image. proximated with a set of plane patches (i.e., a polyhedron),

and the search for the best fit is performed with a Newton–This paper presents a methodology for designing algo-rithms for multidimensional image alignment on the basis Muller method. For 1500 vertices of ‘‘head’’ and 200–250

points of ‘‘hat,’’ the time of matching was 5–10 min on aof geometric features obtained from intrinsic image data.The methodology is appropriate for finding transforma- VAX-11/750. With pixel sizes 0.91, 1.17, and 2.5 mm for

CT, MRI, and PET, respectively, the alignment error wastions specified by a moderate number of parameters. Weillustrate this methodology by developing an algorithm for in the range of 0.73–1.83 mm.

The speed of Pellizari’s algorithm is limited by the timerigid body transformation matching of three-dimensionaldata sets on the basis of intrinsic data. The approach we required to compute the shortest distance between ‘‘hat’’

points and ‘‘head’’ polyhedra. Furthermore, the evaluationpresent offers a number of advantages over other solutionsto this problem. The algorithm is simple, accurate, and of the polyhedral approximation to the ‘‘head’’ object is

a complex and time consuming task. We propose to evalu-robust. It operates with voxel-based descriptions of theobjects to be aligned, so that the preprocessing is simple ate the distance function directly from a voxel description

of the object’s surface. Furthermore, we propose to use aand can be performed with standard image processing algo-rithms. It is based on a combination of two ideas: a vector distance transform, that is, we will evaluate the distance

function for a voxel array. Distance transforms were firstEuclidean distance transform and the Marquardt–Levenberg optimization algorithm. These two ideas com- proposed by Rosenfeld and Pfalz [11], who developed a

rapid algorithm for their evaluation. The original algorithmplement each other, leading to an efficient and effectivealignment technique. computed a ‘‘city block’’ distance, and Danielsson [12]

developed an algorithm for two-dimensional EuclideanOur research is most closely related to the method intro-duced by Levin et al. [6], who describe it with the ‘‘hat and distance functions. This algorithm was extended to three

dimensions by Mohr and Bajcsy in 1983 [13]. Subsequently,head’’ simile. A similar idea was introduced, independentlyand in a different context, by Besl and McKay [3]. Two Borgefors [14] classified theories and methods for arbitrary

dimension distance functions, and Ragnemalm [15] devel-3D data sets, e.g., brain surfaces in MRI and PET, are tobe aligned. A set of points in one of these, the ‘‘hat,’’ is oped a method for evaluating the sufficiency of algorithms

for Euclidean distance calculation. Distance functions havealigned with the entire surface in the other data set, the‘‘head.’’ The alignment is performed as follows. Let pi , been used for many image processing operations such as

computing the skeleton, morphological dilation and con-i 5 1, 2, . . . N, be the points in the ‘‘hat,’’ e.g., points onthe PET scan of the scalp. These are transformed into the traction, construction of shortest paths between points, and

shape factor computation. Borgefors [16] suggested the useMRI coordinates with a rigid body transformationof distance map for matching and described a hierarchical(multiresolution) chamfer matching algorithm which em-bodies ideas similar to the statistical principle of outlierp9i 5 T(pi , P), (1)elimination. This method was used by Jiang et al. [5] toregister 3D surfaces. In contrast to Borgefors and Jiang,who use a chamfer distance (an approximation to Euclid-where P is the vector of six parameters (translation and

rotation) that specify the transformation. One then com- ean distance), our work is based on the use of a vectorvalued Euclidean distance function. This distance functionputes the shortest distances d(p) between the transformed

points and the ‘‘head’’ (e.g., the scalp surface in the MRI is well adapted to the Marquardt–Levenberg (M–L) [17]algorithm for minimization, which is robust and convergesdata set). The sum of the squares of distances is a measure

of how well the ‘‘hat’’ fits the ‘‘head.’’ rapidly to the optimum point.

MULTIDIMENSIONAL ALIGNMENT 375

In Section 2, we present a general methodology for dis- of the reference section. We choose boundaries (contours)because they are geometrical objects that are easy to ex-tance-based alignment. We discuss the general principles

of geometric alignment based on penalty functions, state tract from an image.If St and Sr are obtained experimentally, then lack ofsome of the properties that follow from this principle, such

as subset matching, and propose the possibility that the precision or measurement errors may perturb one or bothsets so that it is not possible to align the objects exactly. Wealignment may not be unique. We then propose the use

of the distance function as the measure of disparity and propose to deal with this difficulty by using a set disparityfunctional e(U, V) which is a real-valued function thatdescribe some of the properties of the Euclidean distance

function such as the relation between the gradient and the measures the disparity between sets U and V. The specificfunctional we propose is constructed as follows: choose avector distance (13) and the gradient theorem (18). This

general methodology is illustrated with a two-dimensional finite subset of points Sa , St and evaluateexample presented through pictures. In Section 3 we de-scribe in detail how our distance-based alignment method- e(T(Sa), Sr) 5 O

p[Sa

d 2(T(p), Sr), (4)ology is used to develop a three-dimensional rigid-bodyalignment algorithm. Numerical trials of this algorithm,both with synthetic and experimental objects, are pre- where d 2(p, S) is a real-valued penalty function which is

equal to zero if p [ S, and is positive otherwise. In Fig.sented in Section 4. We summarize our findings in Section5 and present suggestions for further research. 1c Sa is indicated by white squares.

The disparity functional (4) has the following attribute,which we call the subset matching property: If T aligns St2. METHODOLOGYwith Sr, then e(T(Sa), Sr) 5 0. That is, if St . Sa and if

We propose a methodology to solve the following there exists a T such that T(St) 5 Sr, then e(T(Sa), Sr) 5 0.problem: The subset matching property is both a strength and a

weakness. It is attractive because it suggests that we canGiven: Two multidimensional objects, a reference objectfind a match on the basis of a small subset of St. Thus, itand a test object.allows one to find the transformation that matches twoFind: A space transformation T that aligns the test objectobjects even if one of them is partially occluded. It is awith the reference.weakness because the property is only a necessary and nota sufficient condition for alignment. The fact that e(T(Sa),Figure 1 shows an example: Fig. 1a is an image (rat brain

section, horizontal plane) that we take as the reference Sr) 5 0 does not guarantee that St and Sr are homologous.Furthermore, even if the two objects are homologous, butobject, and 1b is the test image to be aligned with the

image in 1a. if Sa is not a representative subset of St, then zero disparitymay not ensure that T aligns the two full sets. Nonetheless,In this section we first propose the general principle of

geometrical matching on which our work is based and then we expect that if Sa is large enough, if it is a good enoughapproximation of St, then the proposed disparity measureintroduce the form of set disparity functional which is the

tool for measuring the degree of alignment. We propose will be a good test of whether the sets are homologousand will find the transformation that aligns St with Sr.to use the M–L algorithm to minimize this functional, and

this requires the computation of certain functions and their We mentioned earlier that under experimental condi-tions both Sa and Sr may be inaccurate. It is thereforederivatives. These calculations can be performed in an

efficient way when the disparity is based on the Euclidean desirable that the penalty function d 2(p, S) should havecertain analytic properties: it should be continuous, differ-distance transform which we subsequently introduce and

whose properties we present. entiable, etc. Under these conditions we expect that whenthe test object suffers small perturbations, when St PThe general principle for alignment followed in this pa-

per is geometrical matching. Objects are represented by T210 (Sr), and if we find the transformation T that minimizes

the disparity criterion (4), then T P T0. There is, however,point sets in a Euclidean space (rather than by images).Suppose we have two such objects, Sr the reference object a general caveat about geometric matching: the aligning

transformation may not be unique. Specifically, if the refer-and St the test object, and there is a transformation T0

(in some class T , e.g., rigid body motions) for which ence set is symmetric with respect to the class of transfor-mations, that is, if there is a transformation T1 such thatT0(St) 5 Sr. Then objects Sr and St are homologous (have

the same shape) for class T , and we say that T0 aligns Sr 5 T1(Sr), and if T aligns St with Sr, then so will T1 n T.Under these conditions the set disparity functional willobject St with object Sr. In Fig. 1c we show the section

outlines which are the geometric objects to be used to align have multiple minima. It follows from this that if Sr is‘‘approximately’’ symmetric, then the disparity functionalimage 1b with 1a. The white outline is St, the boundary of

the test section, and the black outline is Sr, the boundary may have more than one local minimum. These multiple

376 KOZINSKA ET AL.

FIG. 1. Example of geometrical alignment. (a) Reference image (horizontal rat brain section). (b) Test image, an adjacent section to be alignedwith (a). (c) Point sets to be used for alignment. Test image boundary is white, reference image boundary is black, and alignment points arewhite squares. (d) A step in the alignment process. Gray values—Euclidean distance transform of the reference set, large distances are white.Squares—alignment points before and circles—after iteration. (e) Same as (d) except circles denote the final position of test boundary points. (f )Test image transformed with rotation and translation found in (e). Reference image boundary is in black.

minima are both a practical problem and a theoretical correct values of P relies on the M–L algorithm whichproduces a sequence of parameter estimates P0, P1, . . . ,challenge to alignment methodology.

In our work we use parametrized transformations. That Pk, . . . . To compute successive estimates of the parametersthe disparity function d(T(p, P), Sr) is expanded in a first-is, we write T(p) 5 T(p, P), where the equations T(p, P)

specify a family of space mappings and P 5 (f1, order Taylor series over P around the point Pk,f2, . . . fP) are parameters that instantiate a transformation

d(T(p, P), Sr) P d(T(p, Pk), Sr) 1 DT gradP d, (5)in the family. The methodology we propose for finding the

MULTIDIMENSIONAL ALIGNMENT 377

where D 5 P 2 Pk and the gradient of d with respect to in the course of his research on morphological skeletons.We make essential use of the following: If S is a closedP is evaluated at P 5 Pk. This Taylor series is substituted

into (4) to develop an approximation to e(P). The approxi- set, then for each p there is a point a(p) [ S such thatd(p, S) 5 ip 2 a(p)i. This is a point in S closest to p.mation to the disparity functional isObserve that a(p) contains the same information as d(p),since the latter can be computed from the former. Wee(P) 5 O

p[Sa

d 2(T(p, P), Sr) P e(Pk) 1 DTG 1 As DTAD,define the vector distance function for set S as

(6)v(p, S) 5 p 2 a(p). (11)

where G, the gradient of e(P), is a (column) vector whosecomponents are given by This function was introduced into the image processing

literature by Qin-Zhong Ye in 1988 [21], who called it thesigned distance. Observe that

Gi 5 2 ONl51

d(T(pl , Pk))­d(T(pl , Pk))

­fi, (7)

d(p, S) 5 iv(p, S)i. (12)

N is the number of points in Sa, and matrix A has coeffi-The Euclidean distance function has the following im-cients

portant property: If d(p, S) is differentiable at p, then

Ai, j 5 2 ONl51

­d(T(pl , Pk))­fi

­d(T(pl , Pk))­fj

. (8)gradp d(p, S) 5 1v(p,S) 5

v(p, S)iv(p, S)i

. (13)

This approximation ek(P) is used to generate a set of linearThat is, the gradient is a unit vector in the direction of the

equations that are solved to find Dk, which then is used tovector distance function. This is a property of the Euclidean

find the next estimate Pk11 5 Pk 1 Dk. A possible candidatedistance function and does not apply to other norms. To

for Dk is the minimizer of the quadratic form (6), Dk 5prove (13), note that if d(p) is differentiable at p, then

2A21G. The strategy for computing Dk is given in Mar-there is a vector g such that, for all vectors u,

quardt’s paper, as well as in Section 3.3 below. Figure 1dshows an iteration of this process: the squares representT(Sa, Pk) and the circles T(Sa, Pk11). Both sets are super- lim

tR0

d(p 1 tu) 2 d(p)t

5 g · u, (14)posed over an image of the distance transform of the refer-ence set. This iterative process is continued until a stopping

where ? is the inner product, and g is the gradient of d atcriterion is met. In Fig. 1e we show the final state of align-p. Matheron [20] proves thatment: the circles indicate the final locations of the trans-

formed test set points. Figure 1f shows the test imageud(p, S) 2 d(q, S)u # ip 2 qi. (15)shifted and rotated in accordance to the transformation

found through the matching process. The black outline isFrom this and (14) it follows that, for all u, ug · uu # iuiSr, the boundary of the reference image.which implies that igi # 1. Matheron’s argument aboutWe have not yet specified the penalty function d(p, S)the downstream direction of p states that if a is the closestin Eq. (4). We propose the definitionpoint to p in S, then a is the closest points for all q 5 a 1a1v(p), where 0 # a # d(p). Therefore, d(a 1 a1v(p)) 5d(p, S) 5 inf

q[Sip 2 qi, (9)

a, at least for 0 # a # d(p). We evaluate the derivativeof d at p in the direction 1v(p) and find that

where i?i is the Euclidean distance; that is, if x 5 (x1,x2, . . . , xn), where xi are real numbers, then Dtd(p 1 1v(p)t) 5 Dad(a 1 a1v(p))ua5d(p) 5 Daa 5 1; (16)

i.e., the component of g in the direction 1v(p,S) is equal toixi 5 !Oni51

x 2i . (10)

one. Therefore, g 5 1v(p,S) 1 w, where w is orthogonal tov(p, S). But since

Function d(p, S) is the Euclidean distance transform ofigi 5 Ï1 1 iwi2 # 1, (17)set S. The distance function plays an important role in

mathematical morphology [18, 19]. Matheron [20] has de-rived a number of interesting properties of this function we conclude that w 5 0, and g 5 1v(p,S). Q.E.D.

378 KOZINSKA ET AL.

In summary, Eqs. (11) and (12) state that (whenever d(p) 3. AN ALGORITHM FOR THREE-DIMENSIONALRIGID BODY ALIGNMENTis differentiable) the vector distance function encodes both

magnitude and the gradient of the scalar distance function.We illustrate our methodology by implementing and test-As given in Eqs. (7) and (8), the computation of vector

ing an algorithm for three-dimensional rigid body align-G and matrix A for the Marquardt–Levenberg procedurement. The method for computing the distance map and itsrequire the values of ­d/­f. These may be found by usinggradient is described in Section 3.1, the theory and formulasthe following result:used for computing ­y/­f are given in Section 3.2, and the

Gradient Theorem for Euclidean Distance Function. M–L iteration procedure is described in Section 3.3. Numer-Let y(P) 5 T(p, P) and let d(y(P)) be a Euclidean dis- ical trials of the algorithm are presented in Section 4.tance function evaluated at this point. Then, whenever d(?)

3.1. Distance Transformis differentiable at y(P), the partial derivative of d withrespect to f, one of the parameters, is given by In this section we describe an algorithm for computing

the vector three-dimensional Euclidean distance functionof an object described by a voxel array. The iteration and­d(y(P))

­f5 1v(y(P)) ·

­y(P)­f

. (18)updating process we propose is similar to Borgefors [14].Our algorithm computes the vector-valued distance trans-

Here ? denotes the scalar product. Observe that ­d 2/­f, form, first introduced by Ye [21]. This implementation isthe quantity needed for evaluating G in Eq. (7) is found not as accurate as those suggested by Mohr and Bajcsymore simply from [13] or Ragnemalm [15], but requires less computer time.

The object is specified by a voxel array, and the distancemap is in the form of a three-dimensional array whose

d(y(P))­d(y(P))

­f5 v(y(P)) ·

­y(P)­f

. (19) entries are three-dimensional vectors from the closest pointin the object to the given voxel. The algorithm operatesas follows: we are given an array containing the referenceEquation (18) follows from the chain rule of differentiation,object. That is, the reference object is represented by a setof points located on a three-dimensional lattice. The array­d(y(P))

­f5 Od

i51

­d­yi

­yi

­f5 grady d ·

­y(P)­f

5 1v(y(P)) ·­p(P)

­f. has dimensions W 3 H 3 D. For each voxel (x, y, z) the

algorithm computes a three-dimensional vector v(x, y, z).(20) This array is initialized to

Equation (19) follows from (18) and (12). v(x, y, z)In summary, the methodology we propose for aligning

multidimensional images is to minimize the sum of squared5 H(0, 0, 0) if (x, y, z) is in object

(W 2 1, H 2 1, D 2 1) if (x, y, z) is not in object.Euclidean distances between points on the test set and thereference set. The minimization, performed with the Mar-quardt–Levenberg algorithm, requires the computation of That is, we initially store a zero vector for points in the

object and vectors that overbound v(x, y, z) for all otherdistances and their gradients, which can conveniently befound from the vector-valued distance transform. points. Subsequently we perform a forward pass:

for(z51;z,5D21;z11)for(y51;y,5H21;y11)h

for(x51;x<5W-1;x11) /*right*/v(x,y,z)5argmin (uuv(x,y,z)uu2,

uuv(x21,y,z) 1 (1,0,0)uu2,uuv(x,y21,z) 1 (0,1,0)uu2,uuv(x,y,z21) 1 (0,0,1)uu2);

for(x5W22;x.50;x22) /*left*/v(x,y,z)5argmin (uuv(x,y,z)uu2,

uuv(x11,y,z) 1 (21,0,0)uu2);j

j

MULTIDIMENSIONAL ALIGNMENT 379

FIG. 4. Distance calculation for a point located outside of the cuboid.In this case, qx 5 0, qy 5 py, and qz 5 0.

FIG. 2. Three-dimensional Euclidean distance algorithm. First pass: During the optimization step, vector distance functionpropagation forward, up, right, then left. values are evaluated by array access. To find the vector

distance at x, y, z we compute p 5 (round(x), round(y),round(z)) and fetch v(p) from an array. Subsequently,

(This pseudocode is simplified: it does not compute the d(p) is computed from the formuladistance on planes such as z 5 0.) We use the shorthand

d(p) 5 Ïv2x 1 v2

y 1 v2z . (21)

argmin(ixi2, iyi2) 5 Hx if ixi2 # iyi2

y otherwise.Occasionally, the matching algorithm may require values

This propagates the distance in the increasing z and y of the vector distance for points outside the cuboid. Thesedirections, as shown in Fig. 2. For each line with constant are not computed exactly, but are approximated by they, z values we make forward and reverse passes along x, equationpropagating the displacement vectors in the direction thatproduces the smallest distance. Note that the minimum in

v(p) 5 v(q(p)) 1 (p 2 q(p)), (22)the innermost loop above is based on the Euclidean normof the vectors. After the forward pass is completed, we gothrough a second pass, moving in the decreasing z and y

where p is outside the cuboid and q(p) is the point on thedirections, as shown in Fig. 3.face of the cuboid closest to p (see Fig. 4). The x coordinateof q(p) is computed by the rule

qx 5 50, px # 0

W 21, px $ W

px , otherwise,

(23)

where px is the x coordinate of p (similarly for the y andz coordinates). The approximation has no effect on theaccuracy of the final estimate of P, since once the objectsare aligned the test points will be inside the cuboid. Thisevaluation is slower than direct access for points in thecuboid, but it conserves storage and is much faster thanexact evaluation of the distance function. Relatively fewaccesses are made outside the cuboid: this typically occursFIG. 3. Three-dimensional Euclidean distance algorithm. Second

pass: propagation back, down, left, then right. only for early iterations when alignment is poor.

380 KOZINSKA ET AL.

where p 5 [x, y, z]T is the original point, p9 5 [x9, y9, z9]T

is the transformed point, R is a rotation matrix, the vectorC 5 [Cx, Cy, Cz]T is the center of rotation, and [Sx, Sy, Sz]T

is the translation. The rotation matrix R is represented bythe product of three elementary rotation matrices

R 5 XYZ, (25)

where X, Y, and Z are defined by

X 531 0 0

0 cos ux sin ux

0 2sin ux cos ux

4, Y 53cos uy 0 sin uy

0 1 0

2sin uy 0 cos uy

4,

Z 53cos uz sin uz 0

2sin uz cos uz 0

0 0 14. (26)

FIG. 5. Mathematically defined surface used for testing the algorithm.

The parameters of the transformation are the three rota-Figure 5 shows a surface used to test the method. The tion angles and the three translation components,

image of a 3D distance map calculated for this object isshown in Fig. 6. Darker areas represent shorter distances P 5 (f1, f2, f3, f4, f5, f6) 5 (Sx, Sy, Sz, ux, uy, uz). (27)and the surface itself is black, while the brighter valuescorrespond to larger distances. The cut away view in Fig. These parameters are varied to align the test object with6b shows the distance map inside the object. the reference object. The coordinates of the center of rota-

tion, (Cx, Cy, Cz), are constant during the alignment3.2. Rigid Body Transformation process.

A rigid body transformation p9 5 T(p, P) may be repre-3.3. Gradient Equations

sented by the matrix equationThe general equations for the coefficients of vector G

and matrix A are given in (7), (8), and (18). The partialp9 5 R(p 2 C) 1 C 1 S, (24)

FIG. 6. Plots of distance function of the object in Fig. 5. (a) The face is near the end of the object. White denotes large distance. (b) Cutawayview showing distance map inside the surface.

MULTIDIMENSIONAL ALIGNMENT 381

derivatives of d with respect to the translation parameters Equations (28) and (32)–(34) are substituted into (7) and(8) to compute vector G and matrix A.have a very simple form:

From the computational perspective, all of the partialderivatives with respect to angle have the form­d(p9)

­f15

­d(p9)­Sx

5v(p9)x

d(p9)­d­ui

51

d(p9)v(p9)TMi(p 2 C), (36)­d(p9)

­f25

­d(p9)­Sy

5v(p9)y

d(p9)(28)

where p is the test point, C is the center of rotation, p9 is­d(p9)­f3

5­d(p9)

­Sz5

v(p9)z

d(p9). the transformed test point, d(?) is the scalar Euclidean

distance function, and v(?) is the vector distance function.The matrices Mi for the three angles are given, implicitly,

To obtain the partial derivatives with respect to the in Eqs. (31) through (35).angles, we observe from the general equation for the trans-formation (20) that 3.4. Optimization

As stated in Section 2, in each cycle of the M–L methodwe approximate the function e(P) by­p9

­ux5

­

­uxR(p 2 C) 5 S ­

­uxXD YZ(p 2 C). (29)

e(P) P ek(P) 5 e(Pk) 1 DTG 1 DTAD, (37)We compute that

where G and A are defined in Eqs. (7) and (8), respectively.G is the exact gradient of e(?) at Pk and A is an approxima-tion to the Hessian. If ek(P) is a good approximation of­

­uxX 53

0 0 0

0 2sin ux cos ux

0 2cos ux 2sin ux

45 QxX, (30)e(P), we can find the value of D that minimizes e(P)by solving

where Qx is a matrix that rotates 908 around the x axis and AD 1 G 5 0. (38)projects all points on the y–z plane,

This is akin to the Newton–Raphson method for findingroots of a nonlinear equation. If, however, the approxima-tion is not good, the solution of (38) may not converge toQx 53

0 0 0

0 0 1

0 21 04. (31)

the minimum. In the M–L method we instead estimate Dby solving

Similar expressions are obtained for partial derivatives [A 1 lkI]D 1 G 5 0, (39)with respect to the other angles. Applying Eq. (15) wefinally obtain where I is the identity matrix and lk is a positive scalar.

When l is small this approximates the Newton method,while for large l the value of D is in the direction of­d

­f45

­d­ux

51

d(y9)vTQxXYZ(p 2 C), (32)

negative gradient, so that with the appropriate value of lwe can assure that e(Pk11) , e(Pk). The parameter l ismodified in each iteration, depending on whether the value

­d­f5

5­d­uy

51

d(y9)vTXQyYZ(p 2 C), (33)

of D computed from (39) causes e(P) to decrease or in-crease. The initial value of l is found empirically: In our­d

­f65

­d­uz

51

d(y9)vTXYQzZ(p 2 C), (34) trials, l in the range from 2 to 4 gave satisfactory results.

If e(Pk11) , e(Pk), then l is divided by constant v whichis greater than 1, causing subsequent trials to be morewhere Qy and Qz are defined asNewton-like. If e(Pk11) $ e(Pk), then l is multiplied by vand Eq. (39) is solved again. In our experience, n in therange from 8 to 10 worked well. This approach ensures that:

Qy 530 0 1

0 0 0

21 0 04, Qz 53

0 1 0

21 0 0

0 0 04. (35)

(1) The neighborhood of solutions (the size of vectorDk) is controlled.

382 KOZINSKA ET AL.

(2) The test object always moves in the direction ofdecreasing e(P).

(3) In the Newton equation AD 1 G 5 0, if A is asingular (or ill conditioned), the value of D is unreliable.The M–L Eq. (34), however, can be stabilized by choosinga sufficiently large value of l.

We used the following implementation of the M-Lalgorithm:

1. Initialize: l 5 3, n 5 10. The initial transformationparameters are all equal to zero. C, the centerof rotation in (24), is set to the centroid of thetest object. Compute d0 5 e(P0).

2. Compute matrices A, G.3. Solve equations (A 1 lI)D1 5 2G and (A 1

(l/v)I)D2 5 2G.4. Compute d1 5 e(P 1 D1) and d2 5 e(P 1 D2).5. If iD1i , t FIG. 7. Grid defining the computer generated surface used in 3-D

Assign P 5 P 1 D1; done. alignment trials.6. If d2 , d0

Assign l 5 l/n, P 5 P 1 D2, d0 5 d2, go to step 2.7. Else if d2 $ d0 and d1 , d0 eter estimation time, reduce the number of iterations andAssign P 5 P 1 D1, d0 5 d1, go to step 2.

the number of test points. Implications of these choices8. Elseare explored in Section 4.Assign l 5 ln, solve for D, and continue doing

so until e(P 1 D) , d0. Once this happens,4. RESULTSassign d0 5 e(P 1 D), P 5 P 1 D, and

go to step 3. A program for rigid body alignment, based on the formu-lation given in Section 3, was written in MPW–C and run

3.5. Time Estimates on a Macintosh 7100/80 computer with 40 Megabytes ofRAM. The program consisted of two modules: one forWe use the following symbols: Q is the number of voxeldistance function calculation and a second for objectin the cuboid over which the distance function is computed,matching. The algorithm was tested by registering bothN is the number of test points used for alignment, and Icomputer generated and natural 3D surfaces.is the number of minimization iterations.

The proposed algorithm consists of two phases, distance4.1. Synthetic Objectcalculation and error minimization, so that the running

time is equal to T 5 D 1 M, where D is the time for the The computer generated surface (see Fig. 7) consistedvector distance calculation and M is the time for minimiza- of two ellipsoids: the first had major axes (along x, y, andtion. Distance calculation is proportional to Q, the number z) equal to 110, 90, and 110, while the second ellipsoidsof voxels in the evaluation cuboid, D 5 TDQ, and is per- axes were 125, 105, and 125, respectively. The two ellipsoidsformed only once for a given reference set. We assume were displaced along the z coordinate and conjoined sothat each iteration takes the same time, so that M 5 TSI, that the whole object was 140 pixels long. 3D coordinateswhere TS is the time for the minimization step. The minimi- of vertices of a grid covering the surface were computed,zation step consists of calculations repeated over N test the rectangular grid patches were split into pairs of trian-points, such as evaluating matrices G and A, and of other gles, and plane surfaces were interpolated. The final surfacecalculations done only a fixed number of times per itera- was stored in a 256 3 256 3 100 binary voxel array. Thistion, such as solving for D. Therefore, TS 5 T1 1 NT2. was too small to contain the full object, so that part of theThus the total time for the algorithm is second ellipsoid was truncated. A 3D distance map of this

surface was computed by the algorithm described in Sec-T 5 QTD 1 I(T1 1 NT2). (40) tion 3.1. The time for this calculation was 23 min.

The test object was found by taking N points from thesurface of the reference object, transforming them withThis equation leads to some obvious conclusions: to reduce

distance map calculation time, reduce Q. To reduce param- a specified translation and rotation, and converting the

MULTIDIMENSIONAL ALIGNMENT 383

TABLE 1coordinates of the transformed points to integer (voxel)Alignment Results, Varying the Number of Test Pointsvalues. The alignment algorithm was then used to deter-

mine the translation and rotation that best matched the Actual Residualtest object point set to the reference surface. N error error Iterations Time (s)

The accuracy of alignment was measured in two ways:(all) 20,536 0.495 0.231 18 25 : 40actual error and residual error. The actual error is de-

10,268 0.444 0.206 18 13 : 05fined as 5134 0.482 0.223 18 6 : 38

2567 0.66 0.282 18 3 : 241284 0.565 0.257 16 1 : 39

«m 51N O

iip9i 2 aii, (41) 642 0.779 0.315 19 1 : 04

321 0.601 0.184 25 0 : 53161 2.062 0.509 33 0 : 54

81 7.273 1.206 17 0 : 26where p9i is the location of the ith test point after alignment41 10.642 1.380 10 0 : 15and ai is the actual location of this point in the reference21 12.705 2.401 9 0 : 12

object. Note that the locations of these points are known 11 18.081 3.139 12 0 : 15because the test object was generated on the computer.

Note. In these trials the transformation between test and referenceSince we start with test points at voxel coordinates,objects was Tx 5 20, Ty 5 230, Tz 5 10, ux 5 0.3, uy 5 0.4, uz 5 20.5.rounding introduced in this step will produce error even

under perfect alignment. If these displacements are equallylikely over a (20.5, 0.5) cube, the expected Euclidean dis-

somewhat better than for N 5 642. The residual error (42)tance due to round off error is 0.466.

is always smaller than the actual error (41). In some cases,Actual error can only be evaluated when the ‘‘true’’

the difference is very striking: for example, for N 5 41 thelocations of the test points are known, as when the object

residual error is 1.38 but the actual error is 10.642. Therotation and translation is performed on a computer. A

two values are however highly correlated (r2 5 0.973), andmore practical measure is the residual error, defined as

the residual error can serve as an indication of errors inthe average value of the distance function, evaluated over

alignment when actual error cannot be computed. Notethe transformed test points

that the number of iterations is largely independent of thenumber of test points. The dependence of running timeon number of test points and iterations is in good«r 5

1N O

id(p9i ), (42)

agreement with the prediction, Eq. (35), with Td 5 210 es,T1 5 5.7 ms, and T2 5 66 es.

Performance measures such as accuracy and runningwhere p9i is the location of the ith test point after matchingand d(?) is the distance function of the reference object. time are not simple continuous functions of experimental

parameters such as N, the number of test points, or of theNote that this error measures the disparity between thetransformed test points and the reference surface, not the initial transformation coordinates. We feel that this is due

to the presence of local minima in the function used foraccuracy with which the alignment matches its true value.The residual error can, however, be computed even when alignment. This phenomenon was investigated with Monte

Carlo multiple alignment experiments with the same N,we are aligning unknown objects. It is interesting to com-pare the actual error with residual error, because the latter but with various random values of initial test object trans-

formations (Tx, Ty, . . . , uz). The range of translations wasmeasures the apparent accuracy while the former measuresthe actual accuracy. 220 # Tx, Ty, Tz # 20 pixels. The range of angles was

20.4 # ux, uy, uz # 0.4 radians. The experiment for eachAs may be expected, the accuracy and running timedepended on the stopping criteria used by the algorithm. value of N was performed 96 times. All performance indi-

ces given on Table 1 were collected. Statistical measuresMore stringent stopping criteria led to greater accuracyand to longer time for convergence. The alignment algo- of actual error obtained in these experiments are given in

Table 2, and histograms are shown in Fig. 8.rithm was stopped whenever the greatest translationalchange in x, y, or z during an iteration was less than 0.1 We see that the average actual error for N $ 642 points

is about 0.8 pixels, with standard deviation (over trials) ofpixel spacing, and the greatest rotation change was lessthan 0.003 radians. These criteria provided a good compro- about 0.2 pixels. Over this range of N accuracy differences

are not statistically significant. Maximum error is aboutmise between achievable accuracy and running time.A typical set of results is presented in Table 1. From 1.5 pixels for N 5 642. Under these conditions the perfor-

mance is satisfactory. If N is too small, we see from thethese we see that, in general, accuracy is higher when moretest points are used. There are, however, some exceptions: histograms that accuracy varies from case to case with

many instances of poor performance. This is probably duefor example, the accuracy with N 5 321 test points is

384 KOZINSKA ET AL.

TABLE 2 acquired with a camera attached to the cryostat and theStatistical Results on Actual Errors Obtained through brain surface was traced. A three-dimensional data set of

Multiple (Monte Carlo) Alignment Trials with Varying the surface was assembled and scaled so that voxel sizeNumbers of Test Points was 80 em. This was used as the reference object, and a

distance map over a 365 3 365 3 117 pixel array wasNumber of Average Minimum Maximum Standardtest points error error error deviation computed. Points from the structured light technique were

used as the test set, and the algorithm was used to align10,268 0.806 0.390 1.141 0.209

these objects. Figure 9a shows the reference surface, 9b5134 0.792 0.381 1.135 0.202the test point set, 9c the coaligned objects, and 9d an2567 0.788 0.399 1.164 0.212

1284 0.777 0.257 1.239 0.222 exemplary horizontal section at the middle of the brain.642 0.745 0.297 1.541 0.253 Numerical alignment accuracy is reported in Table 3.321 0.867 0.407 2.132 0.320 Numbers in this table are based on residual errors or the161 1.890 0.570 22.164 2.886

distance between test object points and the reference ob-81 3.812 0.850 9.586 2.298ject surface. This experiment shows that our algorithm caneffectively align complex objects, since it is evident fromFig. 9a that the surface of a rat brain is quite complex,

to local minima in the disparity functional. Other results with many indentations and irregularities.observed from these trials are that the number of iterations,accuracy, and running time depend on the initial position.

5. DISCUSSIONFor example, the accuracy for the initial position used inTable 1 is substantially better than for the average accuracy We have developed a methodology for distance-basedin Table 2. alignment and demonstrated the effectiveness of this ap-

proach in rigid body alignment of three-dimensional ob-4.2. Natural Object

jects. The algorithm was tested with both synthetic andnatural data. Monte Carlo trials with a computer-generatedOur method was used to evaluate the performance of a

structured light technique for imaging brain surfaces [22]. object indicate that the algorithm is accurate if enough testobject points were used. Alignment tests of natural objectsThe surface of a frozen rat brain was measured in two

ways: from block-face images and with structured light. (frozen rat brains) showed that the algorithm can alignquite complex and irregular surfaces. In our laboratory,Points of the frozen brain surface were measured by placing

the brain on a turntable and acquiring structured light we have made extensive use of a two-dimensional versionof this algorithm for aligning histological sections andimages at 108 rotation increments. Subsequent cryosection-

ing at 80 em intervals in the horizontal plane revealed found that this version is also stable and rapid.The algorithm has two key ingredients: the vector-valuedsuccessive outlines of the brain. Block-face images were

FIG. 8. Histograms of actual error with varying numbers of test points. (a) N 5 10268, (b) 5134, (c) 2567, (d) 1284, (e) 642, (f ) 321, (g) 161,and (h) 81.

MULTIDIMENSIONAL ALIGNMENT 385

TABLE 3 efficient for minimizing a sum of squares, and the Euclid-Accuracy in Matching Structured Light Point Sets ean vector distance encodes both the magnitude and the

to Brain Surfaces gradient of the objective function. The M–L algorithm isknown to be robust and efficient: it can find minima ofTotal Average Maximum SD of

number residual residual residual functions with discontinuous derivatives, and if the func-Brain of points error error error tion is sufficiently smooth, the M–L algorithm exhibits

order 2 convergence. However, this algorithm demands1 1163 4.29 32.7 4.26many evaluations of distance and derivatives. This is done2 1050 2.04 51.7 2.43efficiently by precomputing the Euclidean distance trans-3 807 2.26 12.04 1.94form over a three-dimensional grid. While this solution is

Note. Distances are in pixels, with pixel 5 80 em. memory intensive, it leads to a fast algorithm.The geometry of the transformation enters the minimiza-

tion algorithm through the parameter gradient formula(18). This formula applies to any transformation that isEuclidean distance transform, as implemented by a two-

pass propagation method, and the use of the M–L algo- differentiable with respect to the parameters. For example,if we wish to align objects with an affine transform, we canrithm for parameter search (matching). These two ideas

are complementary: the M–L algorithm is effective and use the same computer programs as those used for the

FIG. 9. Alignment of block-face and structured light data sets. (a) Rendered surface of frozen rat brain obtained from block-face images, lateralview. (b) Points from rat brain surface, measured with structured light technique, bottom view. (c) Coaligned surface from block-face and structuredlight data. (d) Exemplary section through aligned data sets, showing the outline from a block-face image and points of the test object that, afteralignment, fell into the given section plane (80 em thick).

386 KOZINSKA ET AL.

rigid body transformation, only replacing the formulas for the number of voxels, which in turn is proportional to thecube of linear resolution. Should it be possible to devisethe derivatives of distance with respect to the affine trans-

formation parameters. This method was used to write a an algorithm that can achieve accuracy better than voxeldimensions, one could compute the distance map over afamily of programs for both rigid body and affine alignment

in two and three dimensions: these may be examined at coarser grid realizing both time and storage savings. Theapproach proposed by Lavanllee and Szeliski [23] showsthe web site http:/ /cbis.ece.drexel.edu/ICVC/.

It is interesting to contrast the effectiveness of the algo- promise for achieving this aim.rithm with the requirements imposed by mathematical op-timization theory. Sufficient conditions for convergence of ACKNOWLEDGMENTSa gradient-based optimization method (such as the M–L

This work was supported by NIH Grant P41–RR01638. D. Kozinskaalgorithm) require that the objective function have contin-was supported by Junior Fullbright Research Grant 33817.uous first derivatives. The distance function, as imple-

mented in this research, fails in several ways. Though theREFERENCESEuclidean distance function is continuous, d(x, Sr) is not

differentiable at the boundary of the set nor at points 1. P. A. van den Elsen, E. J. D. Pol, and M. A. Viergever, Medicalequidistant to more than one point is Sr (points on the image matching—a review with classification, IEEE Eng. Med. Biol.

3, 1993, 26–39.morphological skeleton). Furthermore, the function com-2. N. M. Alpert, J. F. Bradshaw, D. Kennedy, and J. A. Correia, Theputed by the Danielsson algorithm may differ from the

principal axes transformation—a method for image registration. J.true distance transform for some (very few) points. MoreNucl. Med. 31, 1990, 1717–1722.accurate algorithms have been proposed (e.g., [13, 15]),

3. P. Besl and N. D. McKay, A method for registration of 3-D shapes,but even these algorithms are not perfect. Finally, we eval-IEEE Trans. Pattern Anal. Mach. Intell., 14(2), 1992, 239–256.

uate the distance by accessing values in an array, and this4. L. S. Hibbard and R. A. Hawkins, Objective image alignment for

function is piecewise constant, not even continuous! We three-dimensional reconstruction of digital autoradiograms, J.feel that the algorithm is effective in spite of these limita- Neurosci. Meth. 26, 1988, 55–74.tions because of the strong information conveyed in the 5. H. Jiang, R. A. Robb, and K. S. Holton, A new approach to 3-D

registration of multimodality medical images by surface matching,vector distance function and because of the robustness ofSPIE [Visualization in Biomedical Computing] 1808, 1992, 334–341.the Marquardt–Levenberg procedure.

6. D. N. Levin, C. A. Pelizzari, G. T. Y. Chen, C. T. Chen, and M. D.An important issue in implementing the proposed meth-Cooper, Retrospective geometric correlation of MR, CT, and PETodology is how to choose the alignment point set, both theimages, Radiology 169, 1988, 817–823.

size of the set and which points to use. This matter certainly7. C. A. Pellizari, G. T. Y. Chen, D. R. Spelbring, R. R. Weichselbaum,deserves more study. In principle, if the two objects are and C. T. Chen, Accurate three-dimensional registration of CT, PET,

truly homologous, very few points should be required (a and/or MR images of the brain, J. Comput. Assist. Tomog. 13(1),minimum of four points for rigid body alignment in three 1989, 20–26.

dimensions). If, however, the object boundaries are impre- 8. A. C. Evans, S. Marrett, J. Torrescorzo, S. Ku, and L. Collins, MRI-PET correlation in three dimensions using a volume of interest (VOI)cise, additional points improve accuracy by averaging er-atlas, J. Cereb. Blood Flow Metab. 11, 1991, 69–78.rors. Some points, such as extrema or corners, should con-

9. R. Bajcsy and S. Kovacic, Multiresolution elastic matching, Comput.vey more information. We devoted considerable thoughtVision Graphics Image Process. 46, 1989, 1–21.to this matter and explored several ideas. In the reported

10. M. I. Miller, G. E. Christensen, Y. Amit, and U. Grenander, Mathe-trials, the alignment points were chosen to approximate matical textbook of deformable neuroanatomies, Proc. Natl. Acad.uniform coverage of the object boundary: this was the best Sci. USA 90, 1993, 11,944–11,948.of the schemes that we explored. 11. A. Rosenfeld and J. Pfaltz, Distance functions on digital pictures,

Our tests suggest that the accuracy of the alignment Pattern Recog. 1, 1968, 33–61.procedure is limited by voxel size. Consider the trials re- 12. P. E. Danielsson, Euclidean distance mapping, Comput. Graphics

Image Process. 14, 1980, 227–248.ported in Table 2: once enough test points are used, neither13. R. Mohr and R. Bajcsy, Packing volumes with spheres, IEEE Trans.the average actual error (about 0.8) nor its standard devia-

Pattern Anal. Mach. Intell. 5, 1983, 111–116.tion (about 0.2) depends on N. The portion of error that14. G. Borgefors, Distance transformations in arbitrary dimensions,can be attributed to roundoff of test point coordinates is

Comput. Vision Graphics Image Process. 27, 1984, 321–345.0.466, so that there is a statistically significant contribution15. I. Ragnemalm, The Euclidean distance transform in arbitrary dimen-of error due to some yet unknown source. We conjecture

sions, Pattern Recog. Lett. 14, 1993, 883–888.that this has to do with the discrete representation for the

16. G. Borgefors, Hierarchical chamfer matching: A parametric edgedistance function. If so, then the voxel resolution estab- matching algorithm, IEEE Trans. Pattern Anal. Mach. Intell. 10(6),lishes a limit on alignment accuracy. In most applications, 1988, 849–865.an accuracy of 0.8 voxels is adequate. However, the time 17. D. W. Marquardt, An algorithm for least-squares estimation of non-

linear parameters, J. SIAM 11, 1963, 431–441.(and storage) for the distance function is proportional to

MULTIDIMENSIONAL ALIGNMENT 387

18. J. Serra, Image Analysis and Mathematical Morphology, Academic applications, in Proceedings, 9th International Conference on PatternPress, New York, 1982. Recognition, 1988, 405–499.

19. J. Serra (Ed.), Image Analysis and Mathematical Morphology. Vol. 22. J. Nissanov, C. Ozturk, and D. Kozinska, Brain surface scanning2: Theoretical Advances, Academic Press, New York, 1992. using structured light, Proc. SPIE [Videometrics] 2598, 1995, 196–213.

20. A. Matheron, in Examples of Topological Properties of Skeletons, (J. 23. S. Lavanllee and R. Szeliski, Recovering the position and orientationSerra, Ed.), pp. 217–238, 1992. of free form objects from image contours using 3D distance map,

IEEE Trans. Pattern Anal. Mach. Intell. 17(4), 1995, 378–390.21. Qin-Zhong Ye, The signed Euclidean distance transform and its