Diffeomorphic Registration of Imageswith Variable Contrast Enhancement

Guillaume Janssens1, Laurent Jacques1, Jonathan Orban de Xivry1, Xavier Geets2, and Benoit Macq1

1 Information and Communication Technologies, Electronics andApplied Mathematics (ICTEAM),

2 Department of Radiation Oncology, Laboratory of Radiobiology and Radiation Protection (RBNT),

Université catholique de Louvain (UCL), Belgium.

September 28, 2010

Abstract

Non-rigid image registration is widely used to estimate tissuedeformations in highly deformable anatomies. Among the ex-isting methods, non-parametric registration algorithms such asoptical flow, or Demons, usually have the advantage of beingfast and easy-to-use. Recently, a diffeomorphic version oftheDemons algorithm was proposed. This provides the advantageof producing invertible displacement fields, which is a neces-sary condition for these to be physical. However, such methodsare based on the matching of intensities and are not suitablefor registering images with different contrast enhancement. Insuch cases, a registration method based on the local phase likethe Morphons has to be used. In this work, a diffeomorphicversion of the Morphons registration method is proposed andcompared to conventional Morphons, Demons and diffeomor-phic Demons. The method is validated in the context of ra-diotherapy for lung cancer patients, on several 4D respiratory-correlated CT of the thorax with and without variable contrastenhancement.

Keywords : Non-rigid image registration, diffeomor-phism, Morphons, contrast enhancement, lung cancer.

1 Introduction

In the context of image-based medical diagnostics and treat-ment, highly deformable anatomies are a problem for multipletime imaging analysis along the course of treatment. Indeed,a precise tracking of organs is made difficult because of shapeand position variations. Non-rigid registration may be used tocompute a displacement vector for each voxel of an image [1],enabling the estimation of the spatial variations of the anatomy.The displacement vectors are computed as pointing to the bestcorresponding location of the voxels in another image accord-ing to a metric which is a measure of the image matching andunder some constraints on global properties of the resulting de-formation, such as invertibility and smoothness.

Several registration methods have been used in the pastyears to estimate deformations in highly deformable anatomies[2, 3, 4, 5]. Many efforts have been made to improve the qual-ity of displacement estimates, but also to reduce the amountofrequired pre-processing or modeling and improve registration

speed [6, 7, 8]. Besides, the choice of a registration methodformedical application depends on the characteristics (e.g., modal-ity) of the images to be registered [1]. The existing methods[9]can be divided into parametric, or model-based, methods (B-splines[10], thin-plate splines[11], radial basis functions[12],linear elastic FEM [13], etc.) and non-parametric methods (vis-cous fluid[14], optical flow[15], etc.). In this second category,the algorithm calledDemons[16, 17] is fast, efficient and easy-to-use, as it requires no particular pre-processing nor patient-specific modeling. This method aims at calculating a regulardisplacement field which produces a good matching of the in-tensities in both images by minimizing a metric, such as thesum of squared differences (SSD) [18] or the mutual informa-tion (MI) [8], between images along with a measure of the fieldregularity.

In a growing number of applications, the displacementfields resulting from registration are used to deform imagesfrom other modalities or other spatial distribution maps (e.g.,the dose map associated to CT scans in radiotherapy [19, 20]).Therefore, the matching of structures in images based on theirintensities is not a sufficient constraint for producing realis-tic anatomical deformation estimations [21]. This is the rea-son why a priori information on the physical characteristicsof anatomical deformations have to be included in the regis-tration process. Diffeomorphism is a necessary condition fordisplacement fields to be physical [22]. Indeed, organs can becompressed and deformed, but cannot undergo non-invertiblespatial transformations,e.g., showing mirror effects. A methodhas been proposed in [23] to limit the displacement fields com-puted by the Demons to a set of diffeomorphic transformations,using diffeomorphic flows and Lie algebra.

In several medical protocols, contrast agents are used inorder to facilitate interpretation. This makes the registrationproblem incompatible with the hypothesis of intensity conser-vation. Furthermore, an histogram equalization is often notable to correct for contrast agent variability, as different regionswill be enhanced in different ways inside the image. Therefore,simple metrics, such as SSD or cross-correlation, are not suit-able for matching those images, and methods that are suitablefor registering variable contrast images have to be investigated[24, 25].

A method similar to Demons but using a phase-based ap-proach was first proposed in [26], and was calledMorphons.

1

The principle of the method is to match transitions (betweendark and bright zones) rather than intensities, by looking lo-cally at the spatial oscillations in intensities. This method usesGaussian smoothing as regularization of the displacement fieldand additive accumulation during the iterative process. This isnevertheless not sufficient to ensure the invertibility of the de-formation [29, 22].

In this paper, a Morphons registration using a diffeomor-phic accumulation step is proposed and its accuracy is assessedin the case of thorax image registration, also in presence ofdif-ferent contrast enhancements, and compared to the Demons.The paper is organized as follows. In Section 2, the mainmathematical concepts and definitions are presented. Then inSection 3 a generic non-parametric registration process ispre-sented and its particularization to Morphons and to diffeomor-phisms is proposed in Section 4. In Section 5, different reg-istrations are applied on images of the thorax, without contrastenhancement in the first experiment, and with contrast enhance-ment in the second. The results of these experiments are even-tually discussed in Section 6.

2 Mathematical Framework

For the sake of clarity, let us introduce some key mathematicalconcepts used throughout this paper.

2.1 Images and Deformation Fields

In this paper, we always denote 3D images by lower case let-ters. For instance, in the process of estimating a displacementfield, the fixed and the moving images are writtenf andm re-spectively. We consider them as real valued functions on thevolumeR3 of pointsx= (x1,x2,x3), i.e., f ,m∈ F = {g : R3→R : x 7→ g(x)}. Most of the time, these functions, but also thecontinuous operations performed on them, such as convolutionsor integrals, must be understood as approximated on the dis-cretevoxelgrid G = {(x1,x2,x3) ∈ Z

3}, omitting the treatmentof volume boundaries. In this study, image convolutions wereperformed using zero-padding outside the boundaries.

A displacement field onR3 is a vectorial fieldD ∈ V ={V : R3→R

3,x 7→V(x)}. It is associated to the “deformation”operation∆ , Id+D, i.e., ∆(x), x+D(x), with Id the identitydeformation: Id(x) , x. The operation∆, and by extension itsvector fieldD, is saiddiffeomorphicif it is invertible, differen-tiable, and its inverse is differentiable. For the transformation∆ to be invertible, its Jacobian must not vanish in any pointx,that is, ifdet(J )(x) 6= 0∀x, with J i j =

∂∆i∂x j

. Moreover, it has to be

positive (det(J )(x)> 0). Indeed, a transformation∆ with neg-ative Jacobians does not correspond to physical deformations(as themirror operation).

Mathematically, given the imagesf and m, we will seethat our global objective of our study is to estimateD such thatthe warping of m by D is “close” to f , i.e., f ≃ m◦∆ with ◦the common function composition. We will use sometimes thenotation

m⋄D = m◦∆,

to insist on thewarping action ofD on m. By extension, thiswarping symbol can also be used on vector fields themselves,

e.g., for two displacement fieldsD1 andD2, D1⋄D2 = D1◦∆2.In practice, the warping is applied on discrete images. The

transformation might therefore need to be truncated (on thevol-ume boundaries) to the closest point inside the volume in orderto avoid extrapolation of the images to be warped.

2.2 Compositive Accumulation

In this paper, we promote a particular way to combine, orac-cumulate, properly two displacement fieldsD1 andD2. Addingthem to formD1+D2 (as performed by many non-parametricregistration methods, see Section 3), is of course computation-ally efficient, but it breaks the consistency with the composi-tion of the corresponding spatial transformations, as illustratedin Fig. 1.

(a) D2 (b) D1 (c) D1 ⋄D2

(d) x+D2 (e) x+(D1+D2) (f) x+(D1⊕D2)

(g) moving imagem (h) m⋄ (D1+D2) (i) m⋄ (D1⊕D2)

Figure 1: Comparison between additive and compositive fieldaccumulations. Warping is implemented using linear interpo-lation. In (a) and (b), two different displacement fields arede-fined on the plane (for visual clarity). In (c), the fieldD1 warpedby D2, i.e., D1⋄D2 = D1◦∆2. In (d), the fieldD2 is applied onthe pixel grid. In (e), the grid is warped by the field resultingfrom an addition-based accumulation ofD1 andD2. In (f), thegrid is warped by the displacement fieldD1⊕D2 arising by thecomposition of∆1 and∆2, which is the sum of the dark blueand gray arrows (given by∆1 ◦∆2− Id). This composition isreally the accumulation that matters since it corresponds to theway displacement fields are iteratively applied to an image (seeSection 3). SinceD1⊕D2 = D2+D1⋄D2, the summed vectorsin (f) correspond to the vectors in (a) and (c). In (g), a movingobjectm, divided in 4 colors (regions between pixel centers).In (h), the result of the warping ofm by the sum of the fields.Clearly, the surfaces are inverted (mirror effect, visiblebecauseof the inversion of colors), leading to non-physical deforma-tions (negative Jacobians). In (i), the result of the warping of mby the composition of the fields. One can notice that, in spiteofthe deformation of the shape of the object, the location of thecolors is conserved.

2

Indeed, one can clearly see in Fig. 1(h) that the warpingof the image in Fig. 1(g) by the sum of two diffeomorphicfields,D1 andD2, does not correspond to the successive warp-ing of this image byD1 and then byD2, which is representedin Fig. 1(i).

However, thecompositiveoperation, denoted⊕, solvesthis issue. It is simply defined as

D1⊕D2 , ∆1◦∆2− Id .

By construction, the deformation operation linked to the defor-mation fieldD1⊕D2 is therefore∆1◦∆2. If both displacementfields are diffeomorphic, their composition is also diffeomor-phic [38].

The operation⊕ has some interesting and useful proper-ties. First, the neutral accumulation is of course obtainedwiththe null displacement field,i.e., D ⊕ 0 = 0⊕D = D. Second,it is easy to prove theassociativerelations(D1⊕D2)⊕D3 =D1⊕ (D2⊕D3) = D1⊕D2⊕D3 for three displacement fieldsD1, D2 andD3. And finally, ⊕ and⋄ are linked through thesimple relation:

D1⊕D2 = D2+D1⋄D2,

meaning that the displacement fieldD1⊕D2 is equivalent tosumming the fieldD2 with the fieldD1 warped byD2. This isillustrated in Fig. 1: the vectors in Fig. 1(i), corresponding tothe successive warping byD1 and thenD2, are the sum of thevectors in Fig. 1(a) and Fig. 1(c), as shown in Fig. 1(f).

2.3 Diffeomorphic Flow and Exponentiation

An important notion used in Section 4.2 is the concept of (con-tinuous)diffeomorphic flow[27, 28, 29]. Given a pointx∈ R

3

and a smooth vector fieldD ∈ V , the flowϕD(x, t) is the dy-namic solutionu(t) ∈ R

3 of the following (autonomous) ordi-nary differential equation:

{ddt u(t) = D(u),

u(0) = x.(1)

At a given “time” t > 0, the positionϕD(x, t) is simply apoint on the trajectory followingD tangentially from the ini-tialization on x (see Fig. 2). Following [29], the expo-nential of a vector fieldD, i.e., exp(D) ∈ V , is the non-linear deformation operation obtained by the flow ofD at timet = 1, i.e., exp(D)(x) = ϕD(x,1). Interestingly, this expo-nential map acts as the common scalar valued exponential,i.e., exp(αD) ◦ exp(βD) = exp((α + β)D) for α,β ∈ R, andit is invertible by simply considering the inverted vector field,i.e., exp(−D) ◦exp(D) = Id. In addition, for differentiableD,exp(D) is also a diffeomorphism onR3. In other words, exp(D)modifies the 3D coordinates with no intersection between themotions of points. Indeed, such a possibility would induce apoint x with two different motion vectors, a situation that isforbidden by (1) sinceD(x) is uniquely defined.

2.4 Scaling and Squaring

A numerical scheme exists to compute approximately but effi-ciently exp(D)(x) whenx belongs to a regular grid of voxelsG .

x

DD(x)

ϕD(x, t)

∆(x)

exp(D)(x)

Figure 2: The diffeomorphic flow exp(D) associated to the vec-tor fieldD is the solution at timet = 1 on the trajectory tangentto D at each point (here represented in 2D). We see that the mo-tion of x induced by exp(D)(x) is more compatible withV thanthis produced by∆(x) = x+D(x).

Indeed, when the fieldD is close enough to zero (i.e., ∆ ≈ Id),the exponential of the field can be approximated using the first-order Taylor expansion exp(D)≈ Id+D = ∆, i.e., by the trans-formation itself. On the other hand, the solution of the flowequation (1) int = 1 can be approximated by “discretizing”tbetween 0 and 1. Indeed, as exp(D) = exp(2−kD)2k

(where theexponent 2k expresses the number of times the deformation op-eration is combined with itself), one can use thescaling andsquaringstrategy for computing the exponential [30]. If onechoosesk such that the field 2−kD is close enough to zero, thefirst-order approximation can be used to estimate exp(2−kD)(based on the Padé approximant near the origin). Then the so-lution of the flow equation is computed by performingk recur-sive compositions of the field by itself, given that such compo-sitions are computationally affordable. Notice that taking k= 0is equivalent to the simple first-order approximation. The scal-ing and squaring steps for field exponentiation [22] is depictedhereafter:

• Scaling: Divide D by a factor 2k such that 2−kD is smallenough,e.g., when ‖2−kD‖∞ = maxx‖2−kD(x)‖ < 0.5voxels.

• Exponentiation: Compute first-order explicit integrationof the flow: ∆(0)(x) = ϕD(x,2−k)≈ Id(x)+2−kD(x).

• Squaring: Performk recursive squarings (using field com-position) of the flow at time 2−k in order to obtain theflow at time 1, which is the field exponential. In otherwords, starting with∆∗ = ∆(0), do k times the computa-tion ∆∗← ∆∗ ◦∆∗, in order to get∆∗ ≃ exp(D).

We see that using this method, onlyk compositions (and there-forek interpolations) are needed for estimating the exponential.Compared to standard estimation of the flow over a regular dis-cretization of the time interval[0,1] in 2k steps, the scaling andsquaring method limits the numerical errors due to compositionof vector fields, but it does not decrease the amplification oftheerror due to the field estimation at timet = 2−k.

3

3 Generic Registration Pipeline

Non-rigid registration methods can be divided into paramet-ric and non-parametric methods. Parametric (or model-based)methods aim at calculating the parameters of a deformationmodel in a high-dimensional space in order to optimize a globalobjective function that takes into account image similarity andtransformation regularity [10]. In this case, the a priori infor-mation is included in the modelization and regularity criteriaof the non-rigid transformation. For example, the harmonicenergy of transformation can be explicitely included in theob-jective function [31].

On the other hand, non-parametric methods makes it pos-sible to decouple similarity optimization from regularization bydirectly acting on the displacement field. The a priori informa-tion has then to be included in the optimization process by us-ing proper regularization techniques. Decoupled optimizationmakes the registration computationally efficient [8], mainly be-cause the computation of each displacement vector is indepen-dant from others, but it prevents us from easily including morecomplex regularization constraints in the process,e.g., such asin volume preserving registrations [32, 33].

3.1 Multiscale Non-Parametric Registration

Most non-parametric registrations are based on an iterative pro-cess which is composed of 3 steps: (i) Field Computation, (ii)Field Accumulation and (iii) Field Regularization. The ideais to progressively build a proper displacement field by iter-atively improving the matching between the fixed image andthe moving image warped by this displacement field, accord-ing to a certain metric. Note that, depending on the nature ofthe displacement one tries to model, the regularization is ap-plied either on the increment field or on the accumulated field.Regularizing the field increment corresponds to a viscous fluidmodeling, while regularizing the global transformation corre-sponds to an elastic solid modeling [14]. Only the second isconsidered in this study.

In this paper, our general non-parametric registrationframework (e.g., valide for Demons and Morphons) adopts amulti-scaleapproach, that is, the displacement field estimationis stabilized by decomposing the fixed and the moving imagesin severalscales, e.g., using a simple smoothing and down-sampling procedure [34].

The three steps mentioned above are then applied a certainnumber of times (until the algorithm reaches a certain stoppingcriterion) to each scale separately from coarse to fine scales(Fig. 3). The general explanation of these three basic blocksare given hereafter. The way they are iteratively applied ateachscale is described in Section 3.2.

Field Computation

At each iteration of the registration process, an update displace-ment field (Du) is first computed as a function (Θ) of the fixedimage (f ) and the moving image (m) warped by the displace-ment field resulting from previous iterations (Da):

Du←Θ( f ,m◦∆a), (2)

Image warpingRegularization

AccumulationUpdate !eld

computation

Warped

image

Moving image

Fixed image

Coarsest

scale

Finest

scale

Figure 3: The non-parametric registration pipeline is composedof 3 main operations (Θ, Φ andΨ) and the warping of the mov-ing image. Those operations are performed from coarse to finescales. At each scale, the process is applied iteratively, until itreaches a stopping criterion.

where∆a and∆u denote the deformation operations linked toDa andDu respectively.

Depending on the nature of the images to be registered,this local displacement estimation can be based on different lo-cal image metrics, such as SSD [17], mutual information com-puted on blocks of voxels [35, 8], local phase [26], etc.

Field Accumulation

After the field computation, the total displacementDa must beincreased by the update field:

Da←Φ(Da,Du). (3)

This accumulation operationΦ is sometimes implementedas a simple addition of accumulated and update fields (as in[18], [36] and [37]). However, as explained in Section 2.2,this accumulation is perhaps computationally efficient butisnot consistent with the composition of the corresponding spa-tial transformations. The solution is therefore to replaceit bythe compositive accumulation⊕ introduced earlier. The ac-cumulationDa⊕Du of the displacement fieldsDa with Du isthen compatible with the wayDu is estimated. Indeed, sinceDu is computed fromm◦∆a, the accumulation ofDa andDu

must modifyDa by Du, a process intrinsically integrated bythe operation⊕. Moreover, the associativity of⊕ validates thecompositive accumulation of displacement fields over severaliterations, as illustrated in Fig. 3.

Field Regularization

Eventually, the field is regularized in order to get a smoothertransformation and reduce the impact of image noise on theregistration output:

Da←Ψ(Da). (4)

This operationΨ is achieved by applying a low-pass filter oneach components of the displacement field. We assume it to be

4

a Gaussian smoothing with a sizeσ2Ψ of a few voxels, which

tends to reduce the harmonic energy of the transformation [31].It is always possible to produce invertible fields by per-

forming a very strong Gaussian smoothing. This, however, mayreduce significantly the accuracy of the estimated displacementby limiting the solution to excessively smooth displacementfields. On the other hand, by preventing the displacement fieldfrom being non-invertible, the diffeomorphic accumulation actsin some way as a regularization, allowing the estimation of in-vertible fields while performing only moderate smoothing.

3.2 Registration Algorithm

Let us explain now the whole multi-scale non-parametric reg-istration algorithm relying on the three specific procedures{Θ,Φ,Ψ} defined in Section 3.1.

The algorithm takes as inputs the fixed and the movingimagesf andm, some parameters described below, and outputsthe estimated transformation∆a = Id+Da such thatf ≃m◦∆a.The whole procedure described in Table 1 and depicted in Fig.3 involves computations on different scalesj ∈ [0,J], fromcoarse (j = J) to fine (j = 0). Each scale is associated to asub-sampled grid of voxelsG j = κ j G , whereκ is the sub-sampling factor (e.g., κ=

√2 in this study) between scalej and

scale j +1. The functionsf andg, defined on the initial gridG = G0 = {(x1,x2,x3) ∈ Z

3}, are down-sampled (after anti-aliasing smoothing) at any scalej by the operationDown j().An up-sampling operatorUp(), implemented as a simple linearinterpolation, is used to transfer any displacement field definedon a gridG j+1 to the finer gridG j usingκ as up-sampling fac-tor. For each scalej ∈ [0,J], the accumulated displacementfield is iteratively updated until one reaches a particular stop-ping criterionS (e.g., based on the convergence ofDa or on theSSD, as precised in Section 5).

4 Diffeomorphic Morphons

Our paper adapts the global registration method explained inthe previous section to Morphons [26, 39] by taking care of theinvertibility of the accumulated displacement field, that is, byintroducing diffeomorphic field accumulations.

As already mentioned above, the particularity of Mor-phons, compared to other non-parametric methods, is that thefield computation (functionΘ in Equation 2) is based on thelocal phase rather than intensity difference. In other words,knowing the phase difference between periodic signals of thesame frequency allows the estimation of the spatial shift be-tween them. Therefore, under the assumption that images canlocally be considered as a sum of periodic signals, the computa-tion of the local phase difference is equivalent to the estimationof the local displacement between images. This procedure isstabilized by the multi-scale approach described in Section 3.2.Besides, Morphons combines the estimation of displacementvectors with a measure of the confidence we have in these esti-mations, resulting in acertainty map. Therefore, for Morphons,given two imagesf andw= m◦∆, the displacement field esti-

Table 1:Multi-scale Non-Parametric ProcedureInputs and parameters:

• Imagesf andmdefined onG .

• Number of scalesJ.

• A stopping criterionS .

• Gaussian kernel varianceσ2Ψ of Ψ.

Output: The displacement fieldDa.

Algorithm:

1. Initialization:

Set scale toj = J and initializeDa = 0 onGJ+1.

2. Transfer on gridG j :

Computemj = Down j (m), f j = Down j ( f ),

and assignDa← Up(Da).

3. WhileS is false, do:

(i) Warping: w= mj ◦∆a

(ii) Field computation: Du← Θ( f j ,w)

(iii) Accumulation: Da←Φ(Da,Du)

(iv) Regularization: Da←Ψ(Da)

4. If j = 0, stop and returnDa,else, setj ← j−1 and return to step2.

mationΘ is actually split into two quantities,

Du←ΘD( f ,w)

cu←Θc( f ,w),

that is, respectively, an update of the displacement field alongwith an update of the certainty map. A similar split is alsoperformed on subsequent operationsΦ andΨ.

Here are the details about the three steps{Θ,Φ,Ψ} of thepipeline of Section 3 for this specific registration, including ourcontribution to the field accumulation step.

4.1 Displacement Field Calculation

In Morphons, a displacement field is estimated thanks to the de-phasing between the local phases of the fixed and the movingimages. This local phase can be probed at a certain frequencyand in a particular direction using quadrature filters [40].Moreprecisely, Morphons method uses a quadrature filterhη of di-rectionη ∈ R

3 (also calledloglets [40]) defined in frequencyby the polar separable function

Hη(ω) = χ+(ηTω)(ηT ω̂)2R(‖ω‖),

whereω ∈ R3 is the frequency vector,χ+(λ) = 1 if λ > 0 and

0 else,‖ω‖2 = ωTω, ω̂ = ω/‖ω‖ is the unit vector supportingω andR is a radial function centered onρ > 0 and defined asR(r) = exp[− ln2(r/ρ)/ ln2] for r > 0.

Since their support corresponds to the half volume{ω ∈R

3 : ηTω > 0} and since(ηT ω̂)2 = cos2 φ (with φ the angleseparatingω andη), loglets can be seen as the analytic counter-parts of the steerable filters introduced by Freeman and Adelson

5

[41]. As a matter of fact, only a limited number of orientationsη are necessary to cover the whole frequency plane. Typically,in 2-D, these directions are taken asηk = (cosφk,sinφk) withφk = kπ/4 for 0≤ k≤ 3, and in 3-D,η is taken as the 6 nor-mal vectors{ηk : 0≤ k≤ 5} to the faces of a hemi-icosahedron[42, 43]. Notice also that each filterhk(x) in the spatial domainis centered around the origin with a typical width given by 1/ρ.

Morphons take advantage of the following behavior.Given an imagef , defining the filtering

qf (x;k) = ( f ∗hk)(x),

with ∗ the common convolution operation and the shorthandhk = hηk, we can writeqf (x;k) = Af (x;k)eiφ f (x;k) sinceqf ∈C. Therefore, by processing the warped imagew similarly, thelocal phase difference can be computed as

∆φk(x) = arg(qf (x;k)q∗w(x;k)

),

with (·)∗ the complex conjugation and∆φk(x) = φ f (x;k)−φw(x;k) the localdephasingbetweenf andw in directionηk.

An important observation is that the non-negative value

Af (x;k) = |( f ∗hk)(x)|=∣∣∫R3

f (x′) Txhk(x′) dx′

∣∣,

represents also the correlation betweenf (x′) and the translatedfilter Txhk(x′) = hk(x′− x), that is, the filterhk(x′) = hk(−x′)translated onx. If the image f was perfectly represented bythe latter, that is, if we had locallyf (x′) = chk(x′− x) for anyx′ ∈R

3 and some constantc∈R, a displacement off by a dis-placement fieldD(x) approximately constant over the supportof Txhk would induce a dephasing∆φk(x) = ρηT

k D(x) since thefrequency vector ofTxhk is −ρηk. An important implicit as-sumption is nevertheless that|ρηT

k D(x)| < π since the dephas-ing is known up to modulo 2π. Moreover, onlyηT

k D and notDcan be determined, as another manifestation of the blank wallproblem [44].

In practice, for most ofx, f (x) is not perfectly representedby one filter but by a linear combination of them where theamplitudeAf (x;k) measures the adequacy of the fit betweenf (x) and Txhk. Consequently, the local update displacementDu(x) linking f (x) andw(x) = f (x+Du(x)) in eachx∈ R

3 isestimated by solving the weighted least square optimization

ΘD( f ,w) = argmind∈R3

∑k

[ck(ρηT

k d − ∆φk)]2

, (5)

where theck(x) = Af (x;k)Am(x;k) are thecertainty mapof thefilter hk. As explained above,ck reflects for each voxel howreliable the field estimation is,i.e., how contrasted the band-pass filtered images are.

Numerically, the optimization in Equation 5 is a stan-dard weighted least square minimization, that is, it correspondsthe minimization of the energyE(d) = ‖C(Nd− Γ)‖22, us-ing the diagonal matrixC = diag(c1, · · · ,c6), the matrixN =

(η1, · · · ,η6)T and the vectorΓ =

(∆φ1, · · · ,∆φ6

)T. An easy

computation shows that the solution of (5) is then given by theMoore-Penrose pseudo inverse(CN)† = (NTC2N)−1NTC, thatis,

ΘD( f ,w) = (CN)†CΓ,

with ΘD( f ,w) arbitrary set to 0 when(NTC2N) is not invert-ible.

Jointly to the estimation (Equation 5), a global certaintymap associated to the quality of the estimation ofΘD is definedas [43]

Θc( f ,w) = ∑k

ck(x),

i.e., the sum of all certainty measures for each quadrature filter.This update of the certainty map must then be combined with anaccumulated certainty computed from previous iterations (seeSection 4.2).

In the multi-scale approach described in Section 3.2, usingthe same quadrature filters at decreasing scalesHηk is equiv-alent to estimating the phase of the band-pass filtered imagearound increasing cut-off frequencies, that is, withρ← 2ρ eachtime j ← j +1. This sustains the coarse-to-fine displacementestimation, that is, the computation ofΘD andΘc on differentscale bandsf j andmj of f andm.

Convolutions with quadrature filters can be implementedefficiently in the Fourier domain thanks to the FFT and the con-volution theorem. However, since the spatial extent of those fil-ters is small, it is also possible to use efficient spatial convolu-tions with truncated kernels, as done in this study. As the localphase is invariant to local intensity scaling, the Morphonspro-cedure is suitable for registering images with various contrastenhancements. Besides, some studies indicate that the phaseextraction allows a fast convergence and a sub-voxel precisionin displacement estimation (e.g., see [39]).

4.2 Field Accumulation

In the original Morphons method, the accumulated field is com-puted as a weighted sum of the update field and the previousaccumulated field, as used in damped optimization schemes.The weights are given by the certainty on the update field (cu,as computed fromΘc) and the accumulated certainty map (ca).As the certainty map must also be accumulated in order to re-flect the confidence in all previous displacement computations,the accumulation stepΦ must be divided into two operationsΦD (field accumulation) andΦc (certainty accumulation):

ΦD(Da,Du,ca,cu) = Da+cu

ca+cuDu, (6)

Φc(ca,cu) =c2a+c2

uca+cu

, (7)

where in the last formula, similarly to the field accumulation,the certainty map is updated by its own certainty [43].

However, as it was explained before, the addition of dis-placement fields is not really appropriate for accumulatingspa-tial transformations, in contrast to composition. The composi-tive accumulation may also be damped using the certainty as aweighting factor:

ΦD(Da,Du,ca,cu) = Da⊕ cuca+cu

Du.

The (SSD-based) Demons registration is a non-parametricalgorithm which performs the optimization of the SSD betweenimages. In [29], a diffeomorphic field accumulation is proposedas improvement of the Demons method. The idea is to use anadaptation of the optimization method to Lie groups [45] in or-der to limit the possible solutions to diffeomorphic transforma-tions. In practice, this is done by replacing the accumulation

6

step of the Demons by an accumulation using the diffeomor-phic flow exp() introduced in Section 2. This accumulationreads then

ΦD(Da,Du) = Da⊕(

exp(Du)− Id), (8)

where the field exponential exp(Du) can be efficiently es-timated using a small number of recursive compositionsof the field Du by itself. Consequently, the displacementfield ΦD(Da,Du) is linked to the deformation operation∆a ◦exp(Du).

In the case of the Morphons, the accumulation step can beachieved in the same way. This will produce smoother fieldsthan the traditional addition or composition. However, theac-cumulation step in the Morphons method involves a dampingbased on the certainty. Therefore, we propose the followingaccumulation step for diffeomorphic Morphons:

ΦD(Da,Du,ca,cu) = Da⊕(

exp( cuca+cu

Du)− Id). (9)

Since exp(0D) = Id for any vector fieldD, the accumulationfades away whenca≫ cu. The accumulation of the certaintymap remains as explained previously (Equation 7).

Notice that, because the field is discretized on a grid ofvoxels, interpolation is needed for computing the compositionof two diffeomorphisms. Therefore, errors due to successiveinterpolations could potentially lead to non-invertible transfor-mations. However, such problems were not observed in practi-cal experiments using reasonable smoothing of the field.

4.3 Field Regularization

During the displacement estimation step, the relevance of localphase computation is estimated and used as weight for the ac-cumulation. This certainty map may also be used for a smartregularization of the displacement field. Regularization is per-formed using anormalized convolution[46] of the field by aGaussian kernel, taking into account the certainty map in orderto put greater importance to high certainty locations. The cer-tainty is also regularized in the same way as displacement fieldcomponents in order to preserve the correspondence betweenthe displacement vectors and their corresponding certainty.

Mathematically, given a positive functionh and a filterg(typically a Gaussian kernel of varianceσΨ > 0), the normal-ized convolution of a (scalar) functionsby g as involved by thenormalizationh is

s∗h g,(hs)∗g

h∗g.

This operation does not increase the maximum amplitude of thefiltered function. Indeed, for a non-negative kernelg, we showeasily that‖s∗h g‖∞ ≤ ‖s‖∞, with ‖s‖∞ = maxx |s(x)|. The ac-cumulated displacement fieldDa and subsequently the certaintymap are therefore regularized thanks to this operation using fornormalization the certainty mapca, that is,

ΨD(Da,ca) = Da∗ca g,

Ψc(ca,ca) = ca∗ca g.

Notice that, for computingΨD, the normalized convolution isperformed separately on all components of the vector field.

This operation tends to propagate the displacement fieldfrom high certainty areas to areas which show less significantfilter responses. Besides, by setting to zero the certainty outsidethe volume boundaries, normalized convolution cancels thein-fluence of the padding strategy. This step produces a smoothversion of the accumulated field that may reduce the accuracyof image matching resulting from the displacement estimationstep, as it limits the possible solutions to smooth displacementfields.

However, if the iterative algorithm is to converge, the so-lution will be regular and invertible (except for large numericalerrors), thanks to accumulation and regularization constraints,but it will also be (at least locally) optimal in terms of lo-cal phase difference. Indeed, as the phase is monotonic andsmooth, a mismatch between local structures will automaticallylead to non-zero field update with a high certainty value, whichwill tend to improve the displacement estimate and fit the struc-tures together.

The Jacobian of the displacement field may be used as acriterion for validating the physical behavior of the deforma-tion. Indeed, the Jacobian gives for each voxel the change involume this voxel encounters during deformation. Jacobianin-dicates expansion when it is greater than 1, and compressionwhen it is smaller than 1. A negative Jacobian means that thevoxel is “inverted” (getting a negative volume), which is in-compatible with the mass-preservation principle.

In the following, the diffeomorphic version of Demonsand Morphons are denoted respectively D-Demons and D-Morphons.

5 Experiments and Results

The methods were first compared for several simple 2D virtualsituations in order to demonstrate the interest in chosing theaccurate registration method with respect to the images to beregistered.

For the clinical validation, Morphons and D-Morphonsregistrations were first validated on a 10-phases point-validatedpixel-based breathing thorax model (POPI-model) from theLéon Bérard Cancer Center, Lyon, France[4] in order to com-pare the D-Morphons to Morphons, Demons and D-Demons inthe case of intensity conservation between images. Then it wasapplied to lung images with different contrast enhancements, inorder to illustrate the benefit of a phase-based approach com-pared to traditional SSD-based registration methods in thecasewhere intensities are not conserved between the images to beregistered.

All simulations were performed using Linux, on a singleprocessor Intel Core 2 (2.4GHz). OurMATLAB R© implemen-tation used for the prototyping of the methods was also usedfor simulation. Notice that no efforts were made for achievinggood performances in terms of computational cost and memoryrequirements in the implementations used in this study. Thelo-cal phase estimation was performed using convolutions with9×9×9 quadrature filters. Less than 1 GB of RAM was re-quired for registering two volumes of 256×256×100 voxelsusing all registrations. The time required for registeringsuchimages, using the parameters presented hereafter, was around6 minutes for Demons, 42 for Morphons, 7 for D-Demons and

7

43 for D-Morphons. However, preliminary results based on aC++ implementation of the Morphons, which uses operationsin the Fourier domain instead of convolutions (as done in ourMATLAB implementation) and using 4 threads on a quad-coreCPU, allowed a division of the computation time by 50, leadingto Morphons registrations taking about one minute for such atypical image size.

5.1 Illustrative Virtual Experiments

Two 2D virtual experiments were performed. The first exper-iment, illustrated in Fig. 4, is based on a virtual disk imageafter blurring. Two images of the same disk were created, theonly difference being the scale of intensities (multiplication by0.75). This experiment shows the interest in using a phase-based method (conventional Morphons in this example) whileregistering identical shapes with different contrasts, comparedto an intensity-based method (conventional Demons).

The second virtual experiment is based on two images ofa disk (see Fig. 5). In the fixed image, a disk of radiusr1+ r2

was created, and a hole (disk of radiusr2) was added in its cen-ter. In the moving image, a disk of radiusr1 was created withthe same intensity scaling as in the fixed image. This exampleillustrates the case where a structure is missing in one imagecompared to the other, as it may occur in practice (e.g., theproblem of bowel gas in CT images of the abdomen). This ex-periment illustrates how the diffeomorphic version of the Mor-phons algorithm can prevent from producing negative volumesafter registration, without increasing the smoothing by using alarger Gaussian regularization kernel.

Moving image

Fixed image

Displacement !eld

from Demons

Displacement !eld

from Morphons

Deformed image

from Demons

Deformed image

from Morphons

Figure 4: Results of the registration between 2 identical-sizedblurred disks with different constrasts, using Demons and Mor-phons. In yellow: the contour of the disk. In red: the vectorfield resulting from the registration. The displacement field re-sulting from Morphons was very close to zero. Notice that theSSD is actually lower using Demons than Morphons. However,the SSD does not reflect the matching of the shapes, in opposi-tion to the disk contour after warping.

Moving image

Fixed image

Displacement !eld

from Morphons

Displacement !eld

from D-Morphons

Deformed image

from MorphonsDeformed image

from D-Morphons

minimum Jacobian = -680minimum Jacobian

close to 0

Figure 5: Results of the registration between 2 images usingMorphons and D-Morphons registrations, illustrating the casewere a structure (i.e., the bright hole at the center of the fixedimage) is missing in the moving image. Both methods leadto deformed images very similar to the fixed image except forthe central bright part (because it was not present in the mov-ing image). The diffeomorphic method produced very low butstill positive Jacobian values ((J) close to 0) in the center ofthe disk. Given that the field is defined on the pixel grid of thefixed image, this means that the surface of the central brightpart (which disappears in the moving image) corresponds, asexpected, almost to a singular point in the moving image. Theconventional method, however, produced highly negative Jaco-bians in the central part, leading to the creation of areas that are“mirrors” of areas in the other image.

5.2 Accuracy Assessment on a Breathing Tho-rax Model

The POPI-model [4] is composed of 10 volumes reconstructedfrom a 4D Respiration-Correlated CT scan (RCCT) of the tho-rax, each volume corresponding to a particular phase of an av-erage breathing cycle. 41 landmarks were identified by medicalexperts in each of the 10 images for registration validation.

Conventional Morphons, D-Demons and D-Morphonswere applied between a reference phase and the 9 others. Forall methods, the number of scales was set toJ = 8, with finalresolution of 2mm×2mm×2mm. The variance of the Gaussiankernel used for regularization was empirically set to twicethevoxel size (σ2

Ψ = 2 voxels). For this experiment, a minimumof 10 and a maximum of 20 iterations was used at each scale.In between, the iterative process was stopped if the changes,measured in terms of SSD, were inferior to 0.01%. Such aconvergence criterion was usually reached before the 20th it-eration, supporting the fact that both Demons and Morphonsbehave like optimization methods.

The results were then compared with each other and withthe results from a conventional Demons algorithm as used in[4]. The comparisons were achieved in terms of error in land-mark position, SSD between images, harmonic energy, andminimum Jacobian.

• The landmark position error evaluates the ability of theregistration in finding the physical motion of organs.

8

• The SSD between fixed and deformed images is a measureof the image matching according to the assumption of in-tensity conservation. It is computed as∑x ( f −m◦∆)2.

• The harmonic energy [31, 27] of the displacement fieldD indicates how regular the field is, and is computed as12 ∑x (‖∇D1‖2+‖∇D2‖2+‖∇D3‖2).

• The Jacobian of the field indicates the volume change ofeach voxel. Recall that negative values of the Jacobiancorrespond to inverted volumes, which is not acceptablein a physical point of view. The Jacobian is computed asdet(J ), with J i j =

∂∆i∂x j

= δi j +∂di∂x j

, whereδi j is the Kro-

necker’s delta (δi j = 1 if i = j, 0 else) anddi is the ith

component of the displacement field. In practice, the par-tial derivatives∂di

∂x jcan be computed using centered finite

difference approximations.

The comparisons of landmark position errors (expressedin mm) resulting from the different registrations can be seenin Table 2 with, from left to right, the error in landmark posi-tion (norm of the difference) before registration, using Demons(values from the POPI website), Morphons, D-Demons andD-Morphons. Position errors are noted as follows: mean /std (max). On average, for Morphons, D-Demons and D-Morphons, the error in landmark position was equal or inferiorto 1mm, which is half the size of the voxels at the finest scaleof the registration process.

Results showed that all registrations greatly improved thematching of intensities. The SSD between fixed and deformedimage was similar for Morphons, D-Demons and D-Morphons(see Fig. 6). The harmonic energy of the fields resulting fromthese registrations were also comparable (see Fig. 6).

The matching and the harmonic energy obtained byDemons (as presented by the authors of [4] on the POPI web-site) was slightly less good than for the 3 other methods. How-ever, this is most likely due to the parameters used for regis-tration (e.g., the number of scales, the variance for smoothing,etc.). In particular, for very similar images (first 2 phasesof theRCCT), the algorithm was not able to find a smooth displace-ment field that reduced the SSD.

The minimum Jacobian of the displacement fields result-ing from conventional methods gets down to -0.5 for bothDemons and Morphons (see Fig. 6), as respectively 67 and460 voxels were inverted for the corresponding phase when ap-plying the field on the moving image (which is composed ofalmost 6 mega voxels). However, when using diffeomorphicaccumulation, the minimum Jacobian was raised to 0.2 for theDemons and 0.1 for the Morphons, showing that the diffeo-morphic accumulation step prevented the field from invertingvoxels.

5.3 Application to Images of the Thorax withand without Iodine Contrast Agent

The breathing-correlated motion of tumor is a typical featureof lung cancer that has to be dealt with in radiotherapy plan-ning. RCCT images provide information about the tumor mo-tion throughout the breathing cycle. From the different respi-ratory phases, an adequate margin around the tumor (the ITV,

0

1

SSD

0

0.01

0.02

Harmonic energy

−0.5

0

0.5

Minimum jacobian

Figure 6: Results for the 9 registered phases of the POPI model.Left: boxplots of the SSD before registration (in yellow) andafter all 4 registrations . Center: boxplots of the energy ofdeformation after all 4 registrations. Right: boxplots of theminimum Jacobian after all 4 registrations. From left to right,these registrations are: Demons (light blue), Morphons (lightgreen), Diffeomorphic Demons (dark blue) and DiffeomorphicMorphons (dark green). For each box, the center horizontalline represents the median value, the box goes from the lowerquartile to the upper quartile, and the vertical lines representthe most extreme values within 1.5 inter quartile range. Thecrosses represent outlier values.

i.e., the Internal Target Volume) can be estimated, integratingthus all tumor positions through the respiratory cycle [20].

However, the lack of contrast-enhancement, as well as thehigh noise level and the presence of artifacts that characterize4D RCCT, may significantly impair the accurate delineation ofthe target volumes on these images. More particularly, the io-dine contrast agent is of prime importance to help at differen-tiating tumor extents from vascular structures in the centrally-located lung tumors. In this context, the acquisition of a con-ventional contrast-enhanced CT (CE-CT) acquired during freebreathing should be considered for the delineation task, whilethe 4D RCCT is used to estimate the motion range of the tumorduring breathing. To automatize this process, the delineatedtumor volume at the CE-CT can be deformed on the variousrespiratory phase images from the 4D RCCT using non-rigidregistration to finally get the ITV, as illustrated on Fig. 7.

The purpose of this experiment is to compare Demons andMorphons algorithms (conventional and diffeomorphic ver-sions) for the registration between images with and withoutcontrast-enhancement, while keeping the same setting as forthe POPI experiment.

A CE-CT scan of 3 lung cancer patients was acquired aswell as a 4D RCCT scan at another time point. The first CTscan was taken in free breathing using an iodine contrast agent.The 4D RCCT scan was acquired without any contrast agentand was reconstructed into 10 phases. Histogram equalizationwas not able to correct for localized contrast differences be-tween the CE-CT and RCCT phase images. For all 3 patients,Demons, Morphons, D-Demons and D-Morphons were appliedbetween each of the 10 RCCT images and the CE-CT, with thesame registration parameters as for the POPI simulation.

9

Table 2: Results for the POPI experiment: error in landmark positionRCCT phases Original Demons [4] Morphons D-Demons D-Morphons

Phase 1 0.5 / 0.5 (2.4) 1.3 / 0.3 (1.8) 0.7 / 0.3 (1.6) 0.7 / 0.3 (1.6) 0.7 / 0.3 (1.6)Phase 2 0.5 / 0.6 (2.6) 1.4 / 0.2 (2.1) 0.7 / 0.4 (2.1) 0.7 / 0.4 (1.6) 0.7 / 0.4 (2.1)Phase 3 2.2 / 1.8 (6.6) 1.4 / 0.4 (2.3) 1.2 / 0.6 (2.5) 1.2 / 0.6 (2.5) 1.2 / 0.6 (2.4)Phase 4 4.3 / 2.5 (10) 1.2 / 0.4 (2.3) 1.0 / 0.4 (2.2) 1.0 / 0.5 (2.5) 1.0 / 0.4 (2.2)Phase 5 5.8 / 2.6 (12) 1.3 / 0.5 (2.6) 1.1 / 0.5 (2.7) 1.1 / 0.5 (2.5) 1.1 / 0.5 (2.8)Phase 6 6.1 / 2.9 (14) 1.1 / 0.4 (2.0) 1.0 / 0.5 (2.1) 1.1 / 0.6 (2.8) 1.0 / 0.5 (2.1)Phase 7 5.0 / 2.3 (12) 1.3 / 0.5 (2.4) 1.1 / 0.6 (2.8) 1.2 / 0.6 (2.7) 1.1 / 0.6 (2.8)Phase 8 3.7 / 1.6 (6.2) 1.1 / 0.3 (1.7) 0.8 / 0.4 (1.9) 0.8 / 0.4 (1.8) 0.8 / 0.4 (1.8)Phase 9 2.1 / 1.1 (4.5) 1.1 / 0.3 (1.9) 0.8 / 0.4 (2.0) 0.8 / 0.4 (1.7) 0.8 / 0.4 (2.0)

All phases 3.3 / 2.0 (14) 1.2 / 0.4 (2.6) 0.9 / 0.5 (2.8) 1.0 / 0.5 (2.8) 0.9 / 0.5 (2.8)

Reference phase

D1

D2

D3

D4

Phase 1

Phase 2

Phase 3

Phase 4

Reference phase

Figure 7: Schematic representation of the ITV creation (withonly 4 phases). The CTV delineated on a reference image withcontrast enhancement (on the left) is deformed towards everyphases (middle) using displacement fields estimated by regis-tration and their union is taken as ITV (on the right).

The the displacement fields resulting from these registra-tions were compared in terms of harmonic energy and mini-mum Jacobian (see Fig. 8). The resulting image were com-pared in terms of SSD and mutual information.

The harmonic energy of displacement fields resulting fromDemons and D-Demons was quite higher than with the Mor-phons and D-Morphons, and the minimum Jacobian of the dis-placement fields were positive only for registrations usingthediffeomorphic accumulation. In the worst case, 7455 and 1114voxels were inverted using respectively Demons and Morphonswithout diffeomorphic accumulation (on an image of 5 megavoxels). An example of area leading to bad transformations(with negative Jacobians) using conventional methods is de-picted in Fig. 9. D-Morphons lead to the smoothest transfor-mation, with minimum Jacobian values around 0.2. These quitelow values, however, were very sporadic within the image vol-ume.

We noticed that, unlike the results obtained with thePOPI simulation, the SSD resulting from the Morphons andD-Morphons was a bit higher than the SSD resulting fromDemons and D-Demons. However, as illustrated in the exam-ple of Fig. 4, the SSD does not reflect the matching in variablecontrast areas. On the other hand, no significant differences

0

0.03

0.06

0.09

0.12

Harmonic energy

−1

−0.5

0

0.5

Minimum jacobian

Figure 8: Results for the variable contrast experiment on 30phases (3 patients with 10 phases each). Left: boxplots of theenergy of deformation after all 4 registrations. Right: boxplotsof the minimum Jacobian after all 4 registrations. From lefttoright, these registrations are: Demons (light blue), Morphons(light green), Diffeomorphic Demons (dark blue) and Diffeo-morphic Morphons (dark green). For each box, the center hor-izontal line represents the median value, the box goes from thelower quartile to the upper quartile, and the vertical linesrep-resent the most extreme values within 1.5 inter quartile range.The crosses represent outlier values.

in terms of mutual information were observed between imagesresulting from the different registrations. This is likelydue tothe very low contrasts in the non-contrasted images within theregions corresponding to contrast-enhanced tissues in theotherimage, whereas the main differences in terms of displacementfield were located in these regions, as illustrated in Fig. 10.

In order to illustrate the effect of the registration oncontrast-enhanced tissues, one phase of the RCCT scan of oneof the 3 patients was chosen as example. For this patient, thetu-mor was located close to contrasted tissues. The tumor and theblood vessels were delineated by a physician, on the contrast-enhanced scan and on one phase of the RCCT scan. The de-lineations on the phase image were deformed according to thefields resulting from the different registrations. The results areillustrated in Fig. 10.

The change in volume due to warping was computed, aswell as the harmonic energy inside the delineated stucturesandthe difference between the center of mass of the tumor with and

10

Moving image

Fixed image

Demons D-Demons

Morphons D-Morphons

Figure 9: Illustration of negative Jacobians resulting from non-diffeomorphic registrations. Left: moving and fixed images.Right: fields resulting from registrations (red arrows) andtheirJacobian (grayscale images). The negative Jacobians regionsare contoured in yellow.

without registration.

The change in volume was very small when using a phase-based field computation for both the vessels (around 0%) andfor the tumor delineations (around 1%), while it rose up to 23%for the vessels and to 6% for the tumor while using the Demons.In the same way, the harmonic energy and the error on the cen-ter of mass of the tumor were much smaller for the phase-basedregistration methods. These results are summarized in Table 3.One can notice that the diffeomorphic accumulation of the fieldin the Morphons did not change the results in terms of har-monic energy and volume changes compared to conventionalMorphons. This is due to the fact that the displacement of theconsidered organs is small and smooth.

6 Discussion

The first medical experiment showed that D-Morphons and D-Demons lead to similar matching of both image intensities andanatomical landmarks. This shows that for mono-modal reg-istration of lung CT scans, the phase difference has an effi-ciency comparable to the efficiency of the SSD metric. Further-more, the D-Morphons produced displacement fields as smoothas those obtained with D-Demons. In opposition to conven-tional Demons and Morphons, both diffeomporphic methodsproduced invertible displacement fields which are physicallymeaningful.

The second medical experiment illustrates the limitationsin registering images with various levels of contrast enhance-ment with the Demons method. Indeed, the intensity match-ing resulting from Demons was better than from Morphons,but the field was obviously wrong, as the Demons results ina global shrinking of the contrasted tissues (arteries) that doesnot reflect a proper anatomical behavior, but that is due to thefact that the Demons registration is based on the minimizationof the SSD, which produces an improper displacement estima-tion when the intensities of identical tissues are different in thefixed and moving images. This mismatch between registeredanatomical structures is clearly visible on Fig. 10. As illus-trated in the example of Fig. 4, the field produced by Demons

tries to match structures of same intensity, which do not cor-respond to identical anatomical structures because of the dif-ference in contrast agent concentration. Therefore, the fieldresulting from Demons (see the field on the left part of Fig. 10)is far less smooth than it should be, and can lead to wrong de-formation estimations as it illustrated in the example (seeTable3). In this case, the difference in intensity between the im-ages with and without contrast enhancement lead to importantvolume changes for vessels and tumor by using Demons or D-Demons, while almost no changes in volume were observedfor these tissues when using a phase-based approach. Besides,the harmonic energy inside these tissues shows that the fieldis much more smooth using the phase-based registration. Itis important to notice that these effects are mostly limitedbythe regularization of the displacement field during the Demonsand D-Demons registrations, and that they will still be worseif less regularization is used (smaller variance of the Gaussiankernel used for smoothing the displacement field). This is notthe case for the fields produced by the Morphons and diffeo-morphic Morphons, which are much smoother and preserve theanatomical topology even with contrast variations betweenim-ages (see Fig. 10). Notice that the reduction of the smallestsegmentation that can be observed in the Morphons results ismostly due to inter-slices motion, as confirmed by the Jacobianclose to 1 in this area that shows that there is no important vol-ume changes within this segmented region. Finally, one can seethat the invertibility of the displacement field is observedwithboth diffeomorphic registrations.

These results can be summarized by classifying the dif-ferent registration strategies according to the smoothness (har-monic energy) and the invertibility (minimum Jacobian) of theresulting displacement fields (see Table 4) for the variablecon-trast experiment.

Table 4: Classification of the registration algorithmsfor variable contrast enhancement.

Low Highharmonic energy harmonic energy

Invertible D-Morphons D-Demons(Jmin > 0)

Non-invertible Morphons Demons(Jmin < 0)

One can notice that the D-Morphons algorithm combinesboth advantages: the field is invertible and smooth, which sug-gests that it is likely a better estimation of the real transforma-tion which is known to be smooth in this area.

7 Conclusion

The D-Morphons is a multi-resolution registration algorithmwhich computes a diffeomorphic displacement field based on

11

Table 3: Comparison of volume change, harmonic energy and errors in centerof mass of the delineations of the vessels and tumor on a single phase.

Original Demons Morphons D-Demons D-Morphons

Volume change (vessels) [in %] 23 0 21 0Volume change (tumor) [in %] 6 1 6 1

Harmonic energy (vessels) [×10−3] 89 8 69 8Harmonic energy (tumor) [×10−3] 39 4 34 4

Error on COM (tumor) [inmm] 2.1 1.5 1.1 1.5 1.1

the minimization of the local intensity phase. The methodmanaged to estimate the deformations in a breathing thorax,with an accuracy comparable to the accuracy of the D-Demons,and leads to the same requisite property of invertibility ofthefield. Moreover, the D-Morphons managed to accurately esti-mate the deformations between images with variable contrast,while the conventional SSD-based methods lead to misalign-ment of anatomical structures affected by the contrast variation.

Acknowledgment

The authors thank the Medical Informatics laboratory ofLinköping University for sharing their implementation of theMorphons. GJ and JOX are funded by a FRIA grant. LJ is aPostdoctoral Researcher funded by the Belgian FRS-FNRS.

References

[1] J. B. Maintz and M. A. Viergever, “A survey of medicalimage registration.”Med Image Anal, vol. 2, no. 1, pp.1–36, Mar 1998.

[2] H. Wang, L. Dong, J. O’Daniel, R. Mohan, A. S. Gar-den, K. K. Ang, D. A. Kuban, M. Bonnen, J. Y. Chang,and R. Cheung, “Validation of an accelerated ’demons’algorithm for deformable image registration in radiationtherapy.”Phys Med Biol, vol. 50, no. 12, pp. 2887–2905,Jun 2005.

[3] A. Pevsner, B. Davis, S. Joshi, A. Hertanto, J. Mecha-lakos, E. Yorke, K. Rosenzweig, S. Nehmeh, Y. E. Erdi,J. L. Humm, S. Larson, C. C. Ling, and G. S. Mageras,“Evaluation of an automated deformable image match-ing method for quantifying lung motion in respiration-correlated ct images.”Med Phys, vol. 33, no. 2, pp. 369–376, Feb 2006.

[4] J. Vandemeulebroucke, D. Sarrut, and P. Clarysse, “Thepopi-model, a point-validated pixel-based breathing tho-rax model,” inXVth International Conference on the Useof Computers in Radiation Therapy (ICCR), 2007.

[5] E. Rietzel and G. T. Y. Chen, “Deformable registrationof 4d computed tomography data.”Med Phys, vol. 33,no. 11, pp. 4423–4430, Nov 2006.

[6] B. Zitova and J. Flusser, “Image registration methods: asurvey,” inImage and vision computing, vol. 21, pp. 977–1000, 2003.

[7] A. Klein, J. Andersson, B.A. Ardekani, J. Ashburner, B.Avants, M.-C. Chiang, G.E. Christensen, L.D. Collins, J.Gee, P. Hellier, J. Hyun Song, M. Jenkinson, C. Lepage,D. Rueckert, P. Thompson, T. Vercauteren, R.P. Woods, J.John Mann and R.V. Parsey, “Evaluation of 14 nonlineardeformation algorithms applied to human brain MRI reg-istration,” inNeuroimage, vol. 46, pp. 786–802, Jan 2009.

[8] M. Modat, T. Vercauteren, G.R. Ridgway, D.J. Hawkes,N.C. Fox and S. Ourselin, “Diffeomorphic demons usingnormalized mutual information, evaluation on multimodalbrain MR images,” inMedical Imaging 2010: Image Pro-cessing, vol. 7623, 2010.

[9] M. Holden,”A review of geometric transformations fornonrigid body registration.”,IEEE Transactions on Medi-cal Imaging, vol. 27, pp. 111-128, Jan 2008.

[10] D. Rueckert, L.I. Sonoda, C. Hayes, D.L.G. Hill, M.O.Leach and D.J. Hawkes, ”Nonrigid registration using free-form deformations: application to breast MR images,”IEEE Transactions on Medical Imaging, vol. 18, pp. 712-721, Aug 1999.

[11] K. Rohr, H. Stiehl, R. Sprengel, W. Beil, T. Buzug,J. Weese and M. Kuhn, ”Point-based elastic registra-tion of medical image data using approximating thin-plate splines,”Visualization in Biomedical Computing,vol. 1131, pp. 297–306, 1996.

[12] M. Fornefett, K. Rohr, H. Stiehl and Arbeitsbereich Kog-nitive Systeme, ”Radial basis functions with compactsupport for elastic registration of medical images,”IVC,vol.19, pp. 1–2, 1999.

[13] M. Ferrant, B. Macq, A. Nabavi and S.K. Warfield, ”De-formable modeling for characterizing biomedical shapechanges,”Discrete Geometry for Computer Imagery, pp.235-248, 2000.

[14] G. E. Christensen, R. D. Rabbitt and M. I. Miller, “De-formable templates using large deformation kinematics,”IEEE Transactions on Image Processing, vol. 5, no. 10,pp. 1435–1447, Oct. 1996.

[15] B. Horn and B. Schunck, ”Determining optical dlow,”MIT Tech. Rep., 1981.

[16] J.P. Thirion, “Fast Non-Rigid Matching of 3D MedicalImages,”HAL - CCSd - CNRS, 1995.

[17] J. P. Thirion, “Image matching as a diffusion process:an analogy with maxwell’s demons.”Med Image Anal,vol. 2, no. 3, pp. 243–260, Sep 1998.

12

[18] X. Pennec, P. Cachier, and N. Ayache, “Understand-ing the "demon’s algorithm": 3d non-rigid registrationby gradient descent,” inMedical Image Computing andComputer-Assisted Intervention. Springer, 1999, pp. 597–605.

[19] M. Rosu, I. J. Chetty, J. M. Balter, M. L. Kessler, D. L.McShan, and R. K. T. Haken, “Dose reconstruction in de-forming lung anatomy: dose grid size effects and clinicalimplications.”Med Phys, vol. 32, no. 8, pp. 2487–2495,Aug 2005.

[20] J. Orban de Xivry, G. Janssens, G. Bosmans, M. D.Craene, A. Dekker, J. Buijsen, A. van Baardwijk, D. D.Ruysscher, B. Macq, and P. Lambin, “Tumour delineationand cumulative dose computation in radiotherapy basedon deformable registration of respiratory correlated ct im-ages of lung cancer patients.”Radiother Oncol, vol. 85,no. 2, pp. 232–238, Nov 2007.

[21] T. Rohlfing, “Transformation model and constraints causebias in statistics on deformation fields.“Medical imagecomputing and computer-assisted intervention: MICCAI2006, vol. 9, pp. 207, 2006.

[22] V. Arsigny, O. Commowick, X. Pennec, and N. Ay-ache, “A log-euclidean framework for statistics on diffeo-morphisms.”Medical Image Computing and Computer-Assisted Intervention, pp. 924–931, 2006.

[23] T. Vercauteren, X. Pennec, A. Perchant and N. Ayache,”Non-parametric diffeomorphic image registration withthe demons algorithm,”Int Conf Med Image ComputComput Assist Interv, vol. 10, pp. 319–326, 2007.

[24] B. Rodríguez-Vila, J. Pettersson, M. Borga, F. García-Vicente, E. J. Gómez and H. Knutsson, “3D deformableregistration for monitoring radiotherapy treatment inprostate cancer,”Proceedings of the 15th Scandinavianconference on image analysis (SCIA’07), Jun 2007.

[25] A. Melbourne, D. Atkinson and D. Hawkes, “Influence oforgan motion and contrast enhancement on image regis-tration,” Med Image Comput Comput Assist Interv, vol.11, pp. 948–955, 2008.

[26] H. Knutsson, and M. Andersson, “Morphons: segmenta-tion using elastic canvas and Paint on Priors.”IEEE Inter-national Conference on Image Processing (ICIP’05), Sep2005.

[27] J. Jost: “Riemannian geometry and geometric analysis.”Springer Verlag, 2005.

[28] X. Pennec: “Probabilities and Statistics on RiemannianManifolds: A Geometric Approach.”IEEE Workshop onNonlinear Signal and Image Processing, pp. 194–198,2004.

[29] T. Vercauteren, X. Pennec, A. Perchant, and N. Ayache,“Diffeomorphic demons: efficient non-parametric imageregistration.” Neuroimage, vol. 45, pp. S61–S72, Mar2009

[30] N. J. Higham, “The scaling and squaring method for thematrix exponential revisited,” inSIAM Journal on MatrixAnalysis and Applications, vol. 26, pp. 1179–1193, 2005.

[31] B. Fischer and J. Modersitzki, “Fast diffusion registra-tion,” in AMS Contemporary Mathematics, Inverse Prob-lems, Image Analysis, and Medical Imaging, vol. 313, pp.117–129, 2002.

[32] T. Rohlfing, C.R. Maurer, D.A. Bluemke and M.A.Jacobs, “Volume-preserving nonrigid registration of MRbreast images using free-form deformation with an in-compressibility constraint,”IEEE Trans Med Imaging,vol. 22, pp. 730–741, Jun 2003.

[33] E. Haber and J. Modersitzki, “Image Registration withGuaranteed Displacement Regularity,”Int. J. Comput. Vi-sion, vol. 71, pp. 361–372, 2007.

[34] S. Mallat.A wavelet tour of signal processing: The sparseway. Academic Press., 3rd edition, Dec. 2008.

[35] E. Suarez, J.A. Santana, E. Rovaris, C.F. Westin and J.Ruiz-Alzola, “Fast entropy-based nonrigid registration,”Computer Aided Systems Theory - EUROCAST 2003, pp.607–615, 2003.

[36] L. Ibanez, W. Schroeder, L. Ng, and J. Cates,The ITKsoftware guide second edition updated for ITK version2.4, Kitware, Ed. Kitware, 2003.

[37] J. Plumat, M. Andersson, G. Janssens, J. O. de Xivry,H. Knutsson, and B. Macq, “Image registration using themorphon algorithm: an itk implementation,”Insight Jour-nal, 2009.

[38] J. Ashburner, “A fast diffeomorphic image registration al-gorithm,” Neuroimage, vol. 38, pp. 95–113, Oct 2007.

[39] A. Wrangsjo, J. Pettersson, and H. Knutsson, “Non-rigidregistration using morphons,” inProceedings of the 14thScandinavian conference on image analysis (SCIA 05).Springer, 2005.

[40] H. Knutsson and M. Andersson, “Implications of Invari-ance and Uncertainty for Local Structure Analysis Fil-ter Sets,” Signal Processing: Image Communications,vol. 20, pp. 569–581, jul 2005.

[41] W. T. Freeman and E. H. Adelson, “The design and use ofsteerable filters,”IEEE Transactions on Pattern Analysisand Machine Intelligence, vol. 13(9), pp. 891–906, 1991.

[42] H. Knutsson, “Representing local structure using tensors,”In The 6th Scandinavian Conference on Image Analysis,pp. 244–251, Oulu, Finland, June 1989.

[43] L. Bergnéhr, “Segmentation and alignment of 3-Dtransaxial myocardial perfusion images and automaticdopamin transporter quantification,” PhD Thesis, 2008,Institutionen för medicinsk teknik. Linköpings Univer-sitet, Sweden.

13

[44] B.D. Lucas and T. Kanade, “An iterative image registra-tion technique with an application to stereo vision”,Inter-national Joint Conference on Artificial Intelligence, vol.3, pp. 674–679, 1981.

[45] R. Mahony and J. Manton, “The geometry of the Newtonmethod on non-compact lie groups,”Journal of GlobalOptimization, vol. 23, no. 3, pp. 309–327, 2002.

[46] J. Pettersson, H. Knutsson, and M. Borga, “Normalisedconvolution and morphons for non-rigid registration,” inProceedings of the SSBA Symposium on Image Analysis.Umea, Sweden: SSBA, March 2006.

Figure 10: Illustration of the results for the variable contrastexperiment. 1rst row: fixed image (left) and moving image(right). 2nd row: deformed image with deformed contours anddisplacement field resulting from D-Demons (left) and from D-Morphons (right). 3rd row: Jacobian of the displacement fieldresulting from both registrations, represented using the samecolor scale. 4th row: harmonic energy of the displacement fieldresulting from both registrations, represented using the samecolor scale.

14