Medical Image Analysis 18 (2014) 314–329

Contents lists available at ScienceDirect

Medical Image Analysis

journal homepage: www.elsevier .com/locate /media

Ultrasound elastography using multiple images

1361-8415/$ - see front matter � 2013 Elsevier B.V. All rights reserved.http://dx.doi.org/10.1016/j.media.2013.11.002

⇑ Corresponding author. Tel.: +1 5143981573.E-mail addresses: rivaz@jhu.edu (H. Rivaz), eboctor1@jhmi.edu (E.M. Boctor),

michael.choti@utsouthwestern.edu (M.A. Choti), hager@cs.jhu.edu (G.D. Hager).URL: http://cs.jhu.edu/~rivaz/ (H. Rivaz).

Hassan Rivaz ⇑, Emad M. Boctor, Michael A. Choti, Gregory D. HagerDepartment of Computer Science, Johns Hopkins University, Baltimore, MD, USA

a r t i c l e i n f o a b s t r a c t

Article history:Received 7 March 2013Received in revised form 20 November 2013Accepted 25 November 2013Available online 4 December 2013

Keywords:Ultrasound elastographyElasticity imagingStrain imagingLiver ablationExpectation Maximization (EM)

Displacement estimation is an essential step for ultrasound elastography and numerous techniques havebeen proposed to improve its quality using two frames of ultrasound RF data. This paper introduces atechnique for calculating a displacement field from three (or multiple) frames of ultrasound RF data. Tocalculate a displacement field using three images, we first derive constraints on variations of the displace-ment field with time using mechanics of materials. These constraints are then used to generate a regu-larized cost function that incorporates amplitude similarity of three ultrasound images anddisplacement continuity. We optimize the cost function in an expectation maximization (EM) framework.Iteratively reweighted least squares (IRLS) is used to minimize the effect of outliers. An alternativeapproach for utilizing multiple images is to only consider two frames at any time and sequentially calcu-late the strains, which are then accumulated. We formally show that, compared to using two images oraccumulating strains, the new algorithm reduces the noise and eliminates ambiguities in displacementestimation. The displacement field is used to generate strain images for quasi-static elastography. Simu-lation, phantom experiments and in vivo patient trials of imaging liver tumors and monitoring ablationtherapy of liver cancer are presented for validation. We show that even with the challenging patient data,where it is likely to have one frame among the three that is not optimal for strain estimation, the intro-duction of physics-based prior as well as the simultaneous consideration of three images significantlyimproves the quality of strain images. Average values for strain images of two frames versus ElastMIare: 43 versus 73 for SNR (signal to noise ratio) in simulation data, 11 versus 15 for CNR (contrast to noiseratio) in phantom data, and 5.7 versus 7.3 for CNR in patient data. In addition, the improvement of Elas-tMI over both utilizing two images and accumulating strains is statistically significant in the patient data,with p-values of respectively 0.006 and 0.012.

� 2013 Elsevier B.V. All rights reserved.

1. Introduction

Displacement or time delay estimation in ultrasound images isan essential step in numerous medical imaging tasks including therapidly growing field of imaging the mechanical properties of tis-sue (Ophir et al., 1999; Greenleaf et al., 2003; Parker et al., 2005).In this work, we perform displacement estimation for quasi-staticultrasound elastography (Ophir et al., 1999), which involvesdeforming the tissue slowly with an external mechanical forceand imaging the tissue during the deformation. More specifically,we focus on real-time freehand palpation elastography (Hallet al., 2003; Hiltawsky et al., 2001; Doyley et al., 2001; Yamakawaet al., 2003; Zahiri and Salcudean, 2006; Deprez et al., 2009;Goenezen et al., 2012) where the external force is applied by sim-ply pressing the ultrasound probe against the tissue. Ease of use,real-time performance and providing invaluable elasticity images

for diagnosis and guidance/monitoring of surgical operations areinvaluable features of freehand palpation elastography.

A typical ultrasound frame rate is 20–60 fps. As a result, an en-tire series of ultrasound images are freely available during the tis-sue deformation. Multiple ultrasound images have been usedbefore to obtain strain images of highly compressed tissue by accu-mulating the intermediate strain images (O’Donnell et al., 1994;Varghese et al., 1996; Lubinski et al., 1999) and to obtain persis-tently high quality strain images by performing weighted averag-ing of the strain images (Hiltawsky et al., 2001; Jiang et al., 2007,2006; Chen et al., 2010; Foroughi et al., 2010). Accumulating andaveraging strain images increases their signal to noise ratio (SNR)and contrast to noise ratio (CNR) (calculated according to Eq.(35)). However, these techniques are susceptible to drift, a problemwith any sequential tracking system. We show that consideringthree images simultaneously to solve for displacement field signif-icantly improves the quality of the elasticity images compared tosequentially accumulating them. Multiple images have also beenused to obtain tissue non-linear parameters (Krouskop et al.,1998; Erkamp et al., 2004a; Oberai et al., 2009; Goenezen et al.,2012).

width (mm)

dept

h (m

m)

0 10 20 30

10

20

30

40

50

width (mm)

dept

h (m

m)

0 10 20 30

10

20

30

40

50

dept

h (m

m)

0 10 20 30

5

10

15

20

25

30

35

40

45

50

55

width (mm)

Fig. 1. Consecutive strain images are ‘‘similar’’ up to a scale factor. First and second (S1 and S2 from left) are two strain fields calculated from I1 and I2, and from I2 and I3

respectively (I1; I2 and I3 not shown here). S1 & S2 look similar. Third image is S1 � gS2 for g ¼ 1:1. The strain range in the first two images is 0–0.6%, and in the third image is�0:3%. Images are acquired freehand and in vivo during liver surgery.

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 315

Depth calculation from a trinocular-stereo system (Ayache andLustman, 1991; Mulligan et al., 2002; Brown et al., 2003) is a sim-ilar problem where more than two images are used to increase theaccuracy and robustness of the stereo system. The third image isused to introduce additional geometric constraints and to reducethe noise in the depth estimates. Unfortunately, these geometricconstraints do not hold in the elastography paradigm, and there-fore these methods cannot be applied to elastography.

Fig. 1 shows two consecutive strain images calculated from threeultrasound images using the 2D analytic minimization (AM) meth-od (Rivaz et al., 2011a).1 Our motivation is to utilize the similarity ofthese two images to calculate a low variance displacement field fromthree images. We derive physical constraints based on the mechan-ical properties of soft tissue, and incorporate them into a novel algo-rithm that we call ElastMI (Elastography using Multiple Images).ElastMI minimizes a cost function that incorporates data obtainedfrom three images and exploits the mechanical constraints. Like Pel-lot-Barakat et al. (2004); Jiang and Hall (2006); Sumi (2008); Sumiand Sato (2008); Brusseau et al. (2008); Rivaz et al., 2008a, 2009,2011a; McCormick et al. (2011), we use a regularized cost functionthat exploits tissue motion continuity to reduce the variance of thedisplacement estimates caused by ultrasound signal decorrelations.The cost function is optimized using an iterative algorithm basedon expectation maximization (EM) (Moon, 1996). Compared to ourprevious work (Rivaz et al., 2011b), we present significantly moredetails and in-depth analysis of ElastMI. We also provide extensiveresults for validation and more analysis of the results.

To formally study the advantage of using three images, we as-sume ultrasound noise is additive Gaussian and prove that exploit-ing three images not only reduces the noise in the displacementestimation, but also eliminates false matches due to possible peri-odic patterns in the tissue. We assume an additive Gaussian noisemodel in ultrasound images for two main reasons. First, most real-time motion estimation techniques use different forms of sum ofsquared differences (SSD) as a similarity metric. This includes win-dow-based methods2 and the sample based methods of 2D AM andElastMI. The fact that these similarity metrics have been shown togive low noise displacement estimates suggests that additive Gaussiannoise model is a good approximation for the true ultrasonic noise for

1 The 2D AM code is available online at www.cs.jhu.edu/�rivaz.2 Real-time window based methods generally use SSD, cross correlation or

normalized cross correlation as the similarity metric. Under certain normalityconditions, it can be shown that all of these methods are maximum likelihoodestimators if the ultrasound noise model can be assumed to be additive Gaussian.

small deformations. Second, using the additive Gaussian noise model inultrasound images allows us to analytically obtain the noise in the esti-mated displacement field as a function of the image noise for three dif-ferent algorithms: AM (Rivaz et al., 2011a), ElastMI, and a third methodthat we propose in the AppendixA.

We use simulation, phantom and in vivo patient trials to vali-date our results. The in vivo patient trials that we present in thiswork are related to imaging liver tumors and also imaging ablationlesions generated by thermal ablation. Thermal ablation is a lessinvasive alternative for tumor resection where the cancer tumoris coagulated at temperatures above 60 �C. To eliminate cancerrecurrence, the necrosis should cover the entire tumor in additionto some safety margin around it. Currently, both guidance andmonitoring of ablation are performed under ultrasound visualiza-tion. Unfortunately, many cancer tumors in liver have similar ech-ogenicity to normal tissue and are not discernible in ultrasoundimages. Regarding ablation monitoring, the hyperechoic region inthe ultrasound image caused by formation of gas bubbles duringablation does not represent tissue ablation and usually disappearswithin 1 h of ablation (Goldberg et al., 2000). To minimize the mis-classification of these hyperechoic regions with ablated lesion,ultrasound elastography has been proposed for monitoring abla-tion: HIFU probes (high intensity focused ultrasound) (Righettiet al., 1999), radio-frequency Cool-tip probes (Valleylab/TycoHealthcare Group, Boulder, CO) (Fahey et al., 2006; Jiang andVarghese, 2009; Jiang et al., 2010) and radio-frequency RITA probes(Rita Medical Systems, Fremont, CA) (Varghese et al., 2003, 2004;Boctor et al., 2004; Rivaz et al., 2008b) have been investigated.Electrode vibration elastography (Bharat et al., 2008; DeWallet al., 2012a) and shear wave imaging (Arnal et al., 2011) have alsobeen used to monitor ablation. Elastography in the presence of gasbubbles is challenging because they are a major source of noise inthe ultrasound signal and degrade the quality of both B-mode andstrain images.The noise associated to them is also not simply addi-tive Gaussian and depends strongly on both the spatial locationand time. We show that ElastMI generates high quality strainimages in such high noise environment in three patient trials.

The contributions of this work are: (1) introducing constraintson variation of the motion fields based on similarities of strainimages through time; (2) proposing ElastMI, an EM-based algo-rithm to solve for motion fields using three images; (3) formallyproving that the ElastMI algorithm reduces displacement estima-tion variance, and further illustrating that with simulation,phantom and patient data, and (4) reporting clinical studies ofablation guidance/monitoring, with data collection corresponding

316 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

to before, during and after ablation, which is to the best of ourknowledge, the first such study.

2. Displacement estimation error

Assume we have a set of ultrasound frames Jk; k ¼ 1; . . . ; p, eachof size m� n, and let x ¼ ði; jÞ; i ¼ 1; . . . ;m; j ¼ 1; . . . ;n be a 2D vec-tor denoting the coordinates of image samples (Fig. 2). The imagesare obtained during the freehand palpation of the tissue. From theoriginal sequence Jk, we pick a triple, and set I1 as the middleimage, and I2 and I3 as the first and third images. Let~dkðxÞ ¼ ð~akðxÞ;~lkðxÞÞ denote the ground truth axial and lateraldisplacements of the sample x between the 1st and kth image(see Fig. 2). Note that, by choice of reference, ~d1ðxÞ ¼ 0. For simplic-ity, we only look at a particular A-line and also assume that themotion ~dk is in the axial direction. Therefore, ~ak

i ; i ¼ 1; . . . ;m denotethe ground truth axial displacement of samples of the particularA-line. The subscript i shows the dependency of ~ak to x. Assuming thatultrasound noise is additive Gaussian, the image intensity at point i is

IkðiÞ ¼ ~Iði� ~aki Þ þ nkðiÞ; nkðiÞ � N ð0;r2Þ; k ¼ 1; . . . ;p ð1Þ

where N ðl;r2Þ denotes a Gaussian distribution with the mean land variance r2, and ~IðiÞ refers to an unknown ideal image thathas no noise and no deformation. The goal of ElastMI is to estimate~ak

i , i.e. a displacement for every sample. We make two comparisonsbetween ElastMI and companding (Chaturvedi et al., 1998): (1) incompanding, the scaling of the signal is directly computed andcan be used as a strain image, while ElastMI does not directly esti-mate scaling. (2) ElastMI allows the signal to be stretched since itallows every sample to have a different displacement. Therefore,like companding methods it can give accurate results for imageswith large displacements.

In Rivaz et al. (2011a), we proposed the following cost functionfor calculating the displacement field between I1 and Ik:

Cðak1; . . . ; ak

mÞ ¼ CD þ CR;

CD ¼Xm

i¼1

I1ðiÞ � Ikðiþ aki Þ

� �2;

CR ¼Xm

i¼2

aki � ak

i�1

� �2

ð2Þ

where CD and CR are respectively the data and regularization terms.We have assumed pure axial motion. Replacing I1 and Ik with ~I fromEq. (1) we have

CDðak1; . . . ; ak

mÞ ¼Xm

i¼1

~IðiÞ �~Iðiþ aki � ~ak

i Þ þ n1ðiÞ � nkðiþ aki Þ

� �2ð3Þ

Using Taylor series to linearize ~Iðiþ aki � ~ak

i Þ around i we have

CDðak1; . . . ; ak

mÞ ¼Xm

i¼1

�ðaki � ~ak

i ÞT �~I0aðiÞ þ n1ðiÞ � nkðiþ ak

i Þ� �2

ð4Þ

Fig. 2. Axial, lateral and out-of-plane directions. The coordinate system is attach

where ~I0a is the derivative of the image in the axial direction (sub-script a indicates that the derivative is performed in the axial direc-tion). The value of ak

i that minimizes CD can be easily found bysetting the @CD=@ak

i to zero:

aki ¼ ~ak

i � ~I0aðiÞh i�1

n1ðiÞ � nkðiþ aki Þ

� �ð5Þ

where ½���1denotes inversion. The expected value and variance ofthe ak

i are therefore

E½aki � ¼ ~ak

i ð6Þ

var½aki � ¼ ~I0aðiÞ

h i�2var½n1ðiÞ � nkðiþ ak

i Þ� ¼ 2r2 ~I0aðiÞh i�2

ð7Þ

where r2 is the noise in the images as presented in Eq. (1). Theseequations show that without regularization, the expected value ofthe displacement is the true displacement (i.e. there is no bias),and its variance increases with image noise r. The variance de-creases where image gradient is high, i.e. at the tissue boundariesand areas where speckle is present. This is why speckle trackingmethods do not work (i.e. have very high estimation variance) incysts, which do not have speckle.

We now investigate the redundancy in consecutive strainimages by looking at the mechanics of the tissue. We then intro-duce new priors into our displacement estimation technique basedon this redundancy.

3. Deriving physical-based constraints

In this Section, we assume quasi-static motion and derive con-straints on the variations of the tissue displacement with time. Weuse these constraints in the ElastMI algorithm, Section 4, to de-crease the error in the displacement estimation.

To calculate the deformations of a continuum, mechanical char-acteristics of the continuum and the external forces (i.e. boundaryconditions) are required. The mechanical characteristics of a con-tinuum itself can be described by the three properties of stress–strain relationship (linear or nonlinear), homogeneity and isotropy.Linear stress–strain behavior means that if we scale the stress (orforce) by a factor, the strain (or displacement) also gets scaled bythe same factor, i.e. the Hooke’s law. The stress–strain relation islinear for a large range ordinary objects. Many tissue types also dis-play linear stress–strain relation in the 0–5% strain range (Emelia-nov et al., 1998; Yeh et al., 2002; Greenleaf et al., 2003; Erkampet al., 2004a,b; Hall et al., 2007, 2009; Oberai et al., 2009). Homo-geneity means that the continuum has uniform mechanical prop-erties, i.e. its properties are spatially invariant. Isotropy meansthat at each point, the continuum has the same properties in differ-ent directions. Muscle for example is not an isotropic material dueto its fibers. For simplicity and for intuitive analysis, we only con-sider scalar fields and ignore anisotropy. We can therefore analyzehow a continuum deforms by selecting one of these three

ed to the ultrasound probe. The sample (i,j) marked by x moved by (ai;j; li;j).

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 317

properties: linear or non-linear continuum, homogeneous orinhomogeneous continuum, and external forces that result inuniform stress or nonuniform stress (resulting in 23 ¼ 8 cases).

We hypothesize that the ratio of two strain (or displacement)images obtained at different times from the same continuum hassmall spatial variations (as observed in Fig. 1). To illustrate this,we show that among the 8 total cases, this ratio is spatially invari-ant in the five cases shown in Fig. 3. The remaining three cases allshare tissue non-linearity, which we avoid by limiting the totalstrain to less than 5%. In this figure, image I1 is acquired at zerocompression (to simplify the figure), image I2 after compressionand image I3 after more compression. We assume the applied pres-sure in I2 and I3 has the same profile (i.e. the two external pressurefields are the same up to a scale factor). This means that in cases(a), (c) and (e) the applied pressure is always uniform and in (b)and (d) the applied pressure has the same profile. P1 and P2 aretwo arbitrary points whose strain values are �k

1 and �k2 and whose

axial displacement values are ak1 and ak

2 respectively, wherek ¼ 2;3 refers to strain value at image k. We prove that in the five

Fig. 3. Five cases for which Eq. (8) holds (i.e. different strain or displacement images aredisplacement and axial direction as before. Refer to the text for details.

cases shown in Fig. 3, the ratio of the strain images and the ratio ofthe displacement images are spatially invariant, i.e.

�21

�31

¼ �22

�32

anda2

1

a31

¼ a22

a32

: ð8Þ

An intuitive proof for this equation in the five cases shown in Fig. 3is as following:

(a) Linear, homogeneous, uniform stress. This is the simplestcase, and Eq. (8) can be proven because �2

1 ¼ �22 and �3

1 ¼ �32

(since the stress is uniform). The second part a21

a31¼ a2

2a3

2can also

be simply proven by noticing that two triangles OZ1P21 and

OZ2P22, as well as the two triangles OZ1P3

1 and OZ2P32 are

similar.(b) Linear, homogeneous, non-uniform stress. Either the hole in

the continuum or the non-uniform force applied to the topis enough to generate non uniform stress and strain fields.This case might be the hardest to prove Eq. (8). Considerthe finite element analysis of the continuum, which meshes

simply scaled version of each other). s; �; a and z are respectively stress, strain, axial

318 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

the continuum into small parts. Since the continuum is lin-ear, the final force–displacement equation becomes f ¼ Kawhere f is the force vector applied to the boundaries, K isthe stiffness matrix and a is the displacement of each nodein the mesh. Let the forces when I2 and I3 are acquired berespectively f2 and f3, and the displacements be respectivelya2 and a3. Since we have assumed the pressure keeps its pro-file, f2 and f3 are identical up to a scale, i.e. f2 ¼ gf3. Usingf ¼ Ka, we have a2 ¼ ga3 and therefore the second part ofEq. (8). Since the displacements are scaled version of eachother, so are the strains and therefore we have the first partof Eq. (8).

(c) Linear, inhomogeneous, uniform stress. Because of linearityand uniform stress, s2 ¼ E1�2

1 ¼ E2�22 and s3 ¼ E1�3

1 ¼ E2�32

(s2 and s3 are the stress values corresponding respectivelyto I2 and I3 and are not related to s2 which is variance else-where in the paper). Dividing two equations gives Eq. (8).The second part a2

1a3

1¼ a2

2a3

2can be proven as following. Because

both parts are linear, it can be shown that the extension ofthe two curves corresponding to the bottom part of theimage (the dashed lines) intersect at a ¼ 0 axis (if linearityis not met, they do not intersect on a ¼ 0 axis). Therefore,it can be shown that a2

1a3

1¼ a2

2a3

2holds exploiting similarity rela-

tionships between the six triangles generated in the dis-placement-depth curve. If linearity is not held, neither partof Eq. (8) holds.

(d) Linear, inhomogeneous, non-uniform stress. Since the tissue islinear, this case can be proven by superposition using cases(b) and (c).

(e) Non-linear, homogeneous, uniform stress. The proof is thesame as case (a) where linearity was not used.

Our analysis in (c) and (d) can be simply extended to an inho-mogeneous medium with n homogeneous parts, which is a goodapproximation for most inhomogeneous tissues. Although we as-sumed only axial displacement and strain, Eq. (8) can be similarlyproven for 2D strain and stress in the above five cases. For theremaining 8� 5 ¼ 3 cases Eq. (8) does not hold even in the 1D case.In addition, other simplifications such as assuming strain andstress to be scalars (rather than tensors), neglecting anisotropicbehavior of tissue, assuming that the pressure profile does notchange from I2 to I3, and biological motions inside the living tissuelimit the scope of Eq. (8). However many tissue types (linear ornonlinear, homogeneous or inhomogeneous and isotropic or aniso-tropic) combined with any applied pressure can be locally approx-imated with one of the above cases. Therefore, we impose theadditional constraint that the ratio between two displacement fieldsshould have limited spatial variations (instead of the more rigorousconstraint that it should be spatially invariant). Let gi (which hassmall spatial variations) be the scaling factor at each samplei : a3

i ¼ gia2i . In the 2D case, the scale factor is gi where

d3i ¼ gi: � d2

i where :� denotes element-wise multiplication.3 Inthe next Section, we present the algorithm that utilizes thisconstraint.

4. ElastMI: elastography using multiple images

We have a set of p ¼ 3 images Ik; k ¼ 1; . . . ;3, and would like tocalculate the two 2D displacement fields d2 ¼ ða2; l2Þ andd3 ¼ ða3; l3Þ as described in the beginning of Section 2. We assumed3 ¼ g: � d2 where g ¼ ðga;glÞ and ga and gl are the ratios between

3 Axial and lateral strains are related through the Poisson’s ratio m. For now wesimply assume they are independent and hence we use the point-wise operation. InSection 4 we take the relation between the axial and lateral strains into account.

respectively the axial and lateral displacement images. Followingthe discussion in Section 3, d2 and d3 have to result in strain valuesof less than 5% so that the tissue can be approximately linear. In afreehand palpation elastography setup with ultrasound acquisitionrate of 20 fps or more, taking three consecutive images as I2; I1; I3

guarantees this.Let h contain all the displacement unknowns d2 and d3. If we

know g, it is relatively easy to estimate h by maximizing its poster-ior probability. On the other hand, it is easy to estimate g if wehave h. Since we know neither, we iterate between the steps ofestimating h and g using an Expectation Maximization (EM) frame-work. Our proposed algorithm, shown in Fig. 4, is as follows.

1. Find an estimate for h by applying the 2D AM method (Rivazet al., 2011a) to two pairs of images (I1; I2) and (I1; I3)independently.

2. Find an estimate for g using the calculated h (details below).3. Using the estimated g, estimate h by maximizing its posterior

probability (details below). Note that unlike the traditionalEM where the likelihood of h is maximized, we maximize itsposterior probability.

4. Iterate between 2 and 3 until convergence.

Different stopping criteria can be used in step 4, such as termi-nating the iteration when the changes in the displacement field orthe cost function is smaller than a predefined threshold. We foundthat the convergence of the ElastMI algorithm is fast and iteratingit only once always generates strain images with high quality andCNR; we therefore use this simple criteria. Steps 2 and 3 are elab-orated below.

Calculating g from h using least squares: At each sample ði; jÞ inthe displacement field d2

i;j; i ¼ 1; . . . ;m; j ¼ 1; . . . ; n take a windowof size mw � nw centered at the sample (mw and nw are in the axialand lateral directions respectively and both are odd numbers).Stack the axial and lateral components of d2

i;j that are in the win-dow in two vectors a2

i;j and l2i;j, each of length mw � nw. Similarly,

generate a3i;j and l3

i;j using d3. Note that since both displacementfields d2

i;j and d3i;j are calculated with respect to samples on I1, the

displacements correspond to the same sample ði; jÞ. We first calcu-late the axial component gði;jÞ;a (gði;jÞ ¼ ðgði;jÞ;a;gði;jÞ;lÞ). Discarding thespatial information in a2

i;j and a3i;j, we can average the two vectors

into two scalers �a2i;j and �a3

i;j and simply calculate gði;jÞ;a ¼ �a3i;j=�a2

i;j.

Fig. 4. The ElastMI algorithm. The reference image I1 corresponds to an interme-diate deformation between I2 and I3.

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 319

However, a more elegant way which also takes into account thespatial information is by calculating the least squares solution tothe following over-determined problem

a2i;jgði;jÞ;a ¼ a3

i;j ð9Þ

which results in

gði;jÞ;a ¼a2T

i;j a3i;j

a2Ti;j a2

i;j

ð10Þ

where superscript T denotes transpose. This is however not sym-metric w.r.t. a2

i;j and a3i;j: if we define g0ði;jÞ;a to be the least square

solution to a3i;jg0ði;jÞ;a ¼ a2

i;j, it is easy to show g0ði;jÞ;a – 1=gði;jÞ;a (Fig. 5).A method for symmetric calculation of g is depicted in Fig. 5 whereboth vectors are projected into a2

i;j þ a3i;j. The ratio of the two projec-

tions is g, i.e.

gði;jÞ;a ¼a3T

i;j ða2i;j þ a3

i;jÞa2T

i;j ða2i;j þ a3

i;jÞð11Þ

To calculate the ratio of the lateral displacement fields gði;jÞ;l, wetake into account possible lateral slip of the probe, which resultsin a rigid-body-motion. The rigid-body-motion can be simplycalculated by averaging the lateral displacement in d2

i;j and d3i;j

in the entire image i ¼ 1; . . . ;m; j ¼ 1; . . . ; n, and then calculatingthe difference between these two average lateral displacements.The lateral scaling factor gði;jÞ;l can be calculated using an equationsimilar to (11) where the axial displacement ai;j is replaced withthe lateral displacements li;j. However, we use the following ap-proach which results in a better estimate for gði;jÞ;l. The lateralstrain �l (the gradient of the lateral displacement in the lateraldirection) is simply m�a where m is an unknown Poisson’s ratio.Since m has a small dynamic range in soft tissue (Konofagouand Ophir, 1998) and since the difference between the two dis-placement maps d2 and d3 is small, we can assume that m doesnot vary from d2 to d3. Therefore, gði;jÞ;l ¼ gði;jÞ;a. This gives betterestimate for gði;jÞ;l since axial displacement estimation is moreaccurate (Rivaz et al., 2011a).

Calculating h by maximizing its posterior probability. Knowing thevalue of the latent variable g, the posterior probability of h can bewritten as

PrðhjI1; I2; I3Þ / PrðI1; I2; I3jh;gÞPrðhjgÞ ð12Þ

where we have ignored the normalization denominator. The dataterm PrðI1; I2; I3jh;gÞ is the likelihood of h parametersLðhjI1; I2; I3;gÞ. We set the prior term PrðhjgÞ to a regularizationRðhjgÞ. The MAP estimate for h is

hMAP ¼ arg maxh

PrðhjI1; I2; I3Þ: ð13Þ

Fig. 5. Calculating the scale factor g from two strain images a2 and a3. Left showshow the calculation of g through Eq. (10) is not symmetric. It is trivial to show thatg0ði;jÞ;a ¼ 1=gði;jÞ;a holds if and only if u ¼ 0 or u ¼ p, a condition that is not generallyguaranteed. Right shows a symmetric approach for calculating g where both vectorsare projected into a2

i;j þ a3i;j. The ratio of the two projections is a symmetric measure

for g (Eq. (11)).

To be able to solve this equation analytically, we assume all thesamples in the three images are independent and identically dis-tributed and that their noise is Gaussian (Eq. (1)). The likelihoodof h can therefore be simply written as the product of Gaussian ran-dom variables:

LðhjI1; I2; I3;gÞ ¼Ymi¼1

1ffiffiffiffiffiffiffiffiffiffiffiffi2pr2p exp �ðI1ðxiÞ � I2ðxi þ d2

i ÞÞ2

2r2

!

�Ymi¼1

1ffiffiffiffiffiffiffiffiffiffiffiffi2pr2p exp �ðI1ðxiÞ � I3ðxi þ d3

i ÞÞ2

2r2

!ð14Þ

Note that we are calculating the displacements of the vertical col-umns (RF-line samples) simultaneously and therefore the multipli-

cation is performed from 1 to m. d3i can be replaced by gi: � d2

i . Sincethe prior PrðhjgÞ and the likelihood function are multiplied in theposterior probability Eq. (12)), we set the regularization to beGaussian so that the posterior probability can be easily minimized:

PrðhjgÞ ¼Ymi¼1

1

2p Aj j1=2 exp½�ðd2i � d2

i�1ÞTAðd2

i � d2i�1Þ�;

A ¼ diagðaðg;uÞ;bðg;uÞÞ ð15Þ

where A is a 2 � 2 diagonal matrix as indicated, j � j denotes thedeterminant operator and a and b are the axial and lateral regular-ization weights. a and b can be dependent on g and also on the an-gle u between a3

i;j and a2i;j (Fig. 5), but in this work we simply set

them to constant values. Inserting Eqs. (14) and (15) into Eq. (12)and taking its log followed by negation, we arrive at the costfunction

CðhÞ ¼ � log PrðhjI1; I2; I3Þ ¼Xm

i¼1

ðI1ðxiÞ � I2ðxi þ d2i ÞÞ

2

þXm

i¼1

ðI1ðxiÞ � I3ðxi þ gi: � d2i ÞÞ

2

þXm

i¼1

ðd2i � d2

i�1ÞTAðd2

i � d2i�1Þ þ f ðA;r2Þ ð16Þ

where f ðA;r2Þ contains all the terms that do not have d and there-fore can be ignored in finding the optimum d value. We can now

linearize I2ðxi þ d2i Þ and I3ðxi þ gi: � d2

i Þ respectively around

xi þ dAMi and xi þ gi: � dAM

i where dAMi is an estimate value for d2

i ,known by comparing I1 and I2 using 2D AM. This approach, how-

ever, is not symmetric and does not take d3 into account as the ini-

tial estimate (although d3 is used to estimate g). A symmetric initial

estimate for d2i and d3

i is

d2i ¼

gi;ad2i þ d3

i

2gi;a; d3

i ¼gi;ad2

i þ d3i

2¼ gi;ad2

i : ð17Þ

Note that we have only used gi;a since we have assumed gi;l ¼ gi;a.We have also dropped the subscript j since the cost function C is de-fined for a specific A-line at each time. Taylor expansion can now be

used to linearize I2ðxi þ d2i Þ and I3ðxi þ gi: � d2

i Þ in Eq. (16) respec-

tively around d2i around d3

i :

CðhÞ ¼Xm

i¼1

I1ðxiÞ � I2ðxi þ d2i Þ � Dd2T

i rI2ðxi þ d2i Þ

� �2

þXm

i¼1

I1ðxiÞ � I3ðxi þ gi;ad2i Þ � gi;aDd2T

i rI3ðxi þ gi;ad2i Þ

� �2

þXm

i¼1

ðd2i � d2

i�1ÞTAðd2

i � d2i�1Þ þ f ðA;r2Þ

ð18Þ

320 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

where Dd2i ¼ d2

i � d2i . Setting the derivative of C w.r.t. the axial

(Da2i ¼ Dd2

i;a) and lateral (Dl2i ¼ Dd2

i;l) components of Dd2i for

i ¼ 1; . . . ;m to zero and stacking the 2m unknowns inDd2 ¼ Da2

1 Dl21 Da2

2 Dl22; . . . ;Da2

m Dl2m

h iTand the 2m initial estimates

in d2 ¼ a21 l2

1 a22 l2

2 � � � a2m l2

m

h iTwe obtain the linear system of size 2m:

ðI0 þDÞDd2¼ r�Dd2; D¼

a 0 �a 0 0 0 � � � 00 b 0 �b 0 0 � � � 0�a 0 2a 0 �a 0 � � � 00 �b 0 2b 0 �b � � � 00 0 �a 0 2a 0 � � � 0... . .

.

0 0 0 � � � �a 0 a 00 0 0 � � � 0 �b 0 b

2666666666666664

3777777777777775

; ð19Þ

where I 0 is a symmetric tridiagonal matrix of size 2m� 2m with2� 2 matrices I 0 in its diagonal:

I0 ¼ diagðI 02ð1Þ � � � I 02ðmÞÞ;

I 02ðiÞ ¼I02;a

2 þ g2i;aI03;a

2 I02;aI02;l þ gi;agi;lI03;aI03;l

I02;aI02;l þ gi;agi;lI03;aI03;l I02;l

2 þ g2i;lI03;l

2

24

35 ð20Þ

where I02 and I03 are calculated respectively at ðxi þ d2i Þ and at

ðxi þ gi: � d2i Þ, superscript 0 indicates derivative and subscript a

and l determine whether the derivation is in the axial or lateraldirection, and r is a vector of length 2m with elements:

i odd : ri ¼ I02;aðxi þ d2i Þ I1ðxiÞ � I2ðxi þ d2

i Þh i

þ gi: � I03;aðxi þ gi: � d2i Þ I1ðxiÞ � I3ðxi þ gi: � d2

i Þh i

i even : ri ¼ I02;lðxi þ d2i Þ I1ðxiÞ � I2ðxi þ d2

i Þh i

þgi: � I03;lðxi þ gi: � d2i Þ I1ðxiÞ � I3ðxi þ gi: � d2

i Þh i

ð21Þ

The inverse gradient estimation method Rivaz et al. (2011a) can beused to make the method more computationally efficient: all thederivatives of I2 at ðxi þ d2

i Þ and derivatives of I3 at ðxi þ gi: � d2i Þwill

be simply replaced with the derivatives of I1 at xi. With this modi-fication, Eq. (20) becomes

I 02ðiÞ ¼ð1þ g2

i;aÞI01;a

2 ð1þ gi;agi;lÞI01;aI01;l

ð1þ gi;agi;lÞI01;aI01;l ð1þ g2

i;lÞI01;l

2

24

35

and Eq. (21) becomes

i even : ri ¼ I01;aðxiÞ I1ðxiÞ � I2ðxi þ d2i Þ

h iþ gi: � I01;aðxiÞ I1ðxiÞ � I3ðxi þ gi: � d2

i Þh i

i odd : ri ¼ I01;lðxiÞ I1ðxiÞ � I2ðxi þ d2i Þ

h iþ gi: � I01;lðxiÞ I1ðxiÞ � I3ðxi þ gi: � d2

i Þh i

ð22Þ

We minimize the effect of outliers via iterative reweighted leastsquares (IRLS) by giving a small weight to the outliers. Each imagepair in Eq. (18) is checked independently, i.e. for the same sample i,two different weights w12;i and w13;i are used:

CðhÞ ¼Xm

i¼1

w12;i I1ðxiÞ � I2ðxi þ d2i Þ � Dd2T

i rI2ðxi þ d2i Þ

� �2

þXm

i¼1

w13;i I1ðxiÞ � I3ðxi þ gi;ad2i Þ � gi;aDd2T

i rI3ðxi þ gi;ad2i Þ

� �2

þXm

i¼1

ðd2i � d2

i�1ÞTAðd2

i � d2i�1Þ þ f ðA;r2Þ

ð23Þ

where w12 and w13 are Huber (Hager and Belhumeur, 1998; Huber,1997) weights and are calculated as:

w12;i ¼ wðI1ðxiÞ � I2ðxi þ d2i ÞÞ

w13;i ¼ wðI1ðxiÞ � I3ðxi þ gi;ad2i ÞÞ

wðriÞ ¼1 jrij < TTjri jjrij > T

( ð24Þ

where T is a tunable parameter which determines the residual levelfor which the sample can be treated as outlier. A small T will treatmany samples as outliers. With these new weights, Eq. (19) stillholds with the following modifications:

I 02ðiÞ ¼ðw12;i þw13;ig2

i;aÞI01;a

2 ðw12;i þw13;igi;agi;lÞI01;aI01;l

ðw12;i þw13;igi;agi;lÞI01;aI01;l ðw12;i þw13;ig2

i;lÞI01;l

2

24

35ð25Þ

and Eq. (21) becomes

i even : ri ¼ w12;iI01;aðxiÞ I1ðxiÞ � I2ðxi þ d2

i Þh i

þw13;igi: � I01;aðxiÞ I1ðxiÞ � I3ðxi þ gi: � d2i Þ

h ii odd : ri ¼ w12;iI

01;lðxiÞ I1ðxiÞ � I2ðxi þ d2

i Þh i

þw13;igi: � I01;lðxiÞ I1ðxiÞ � I3ðxi þ gi: � d2i Þ

h i: ð26Þ

To obtain a displacement field from three images using the ElastMIalgorithm, Eq. (19) –with parameters defined in Eqs. (25) and (26) –is solved.

In the next two Sections we show that exploiting the third im-age reduces displacement variance and eliminates ambiguity.

4.1. Reducing variance in displacement estimation

Similar to Section 2, we assume the motion is only in the axialdirection. Adding the similarity metric between images 1 and 2and 1 and 3 we have

CDða21; . . . ; a2

m; g1 � � �gmÞ ¼Xm

i¼1

I1ðiÞ � I2ðiþ a2i Þ

� �2

þXm

i¼1

I1ðiÞ � I3ðiþ gia2i Þ

� �2 ð27Þ

and using the noise model of Eq. (1) we arrive at

CDða21; . . . ; a2

m; g1; . . . ;gmÞ ¼Xm

i¼1

~IðiÞ �~Iðiþ a2i � ~a2

i Þ�

þn1ðiÞ � n2ðiþ a2i Þ�2

þXm

i¼1

~IðiÞ �~Iðiþ gia2i � gi~a

2i Þ

�

þn1ðiÞ � n3ðiþ gia2i Þ�2: ð28Þ

The displacement can now be estimated by linearizing ~Iðiþ a2i � ~a2

i Þand ~Iðiþ gia

2i � gi

~a2i Þ around i and minimizing CD:

a2i ¼ ~a2

i � ~I0aðiÞh i�1�ðgi þ 1Þn1ðiÞ þ n2ðiþ a3

i Þ þ gin3ðiþ a3i Þ

g2 þ 1ð29Þ

and therefore

E½a2i � ¼ ~a2

i ð30Þ

var½a2i � ¼ r2 ~I0aðiÞ

h i�2 ðgi þ 1Þ2 þ g2i þ 1

ðg2i þ 1Þ2

: ð31Þ

Fig. 7. 8 simulated ultrasound image frames of a uniform phantom. The percentileunder each frame shows the value of the compression w.r.t. F1. We set I1 and I2 to F7

and F8 as shown and one of F1 to F6 frames as I3, resulting in different g valuesshown at the bottom. Note that we set the reference image I1 such that itsdeformation is between I2 and I3.

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 321

Let’s consider a case where gi ¼ �1, which indicates that the defor-mation from I1 to I2 is equal to the negative of the deformation fromI1 to I3 (i.e. one is compression and the other one is extension). Set-

ting g ¼ �1 we have var½a2i � ¼ 0:5r2 r~I

h i�2, which is 1/4th of the

var½a2i � when only two images are utilized (Eq. (7)). This reduction

in the noise is a result of using three images and also incorporatingthe prior that the displacement fields at different instances of thetissue deformation are not independent. Please note that in our for-mulation all images are compared to image 1, so that ElastMI for-mulation can be extended to more than three images. However,in our implementation we compare images with the middle image,i.e. we compare I1 with I2, and I2 with I3. Therefore, since the ultra-sound frame rate is much higher than the hand-held palpation fre-quency, gi is negative.

It is important to note that this equation does not provide an ex-act comparison between ElastMI and AM. It assumes zero regular-ization, while the regularization terms in both AM and ElastMImethods significantly reduce the displacement estimationvariance.

By way of comparison, we propose a method in the Appendixfor calculating two displacement fields from three ultrasound RFdata frames. Unlike ElastMI, this method does not impose con-straints based on mechanics of materials. Instead, it uses naturalconstraints among the three displacement fields defined by thethree images. We show that this method does not decrease the var-iance of displacement estimation.

4.2. Eliminating ambiguity in displacement estimation

Ambiguity has been reported before as a source of large errorsin the displacement estimation (Hall et al., 2003; Viola and Walker,2005). Periodic ultrasound patterns happen if the tissue scatterersare organized regularly on a scale comparable to ultrasoundwavelength, such as the lobules of the liver and the portal triads

Fig. 6. Eliminating ambiguity with three images. Left shows that it is impossible withunderlying ultrasound image is periodic. The O and X marks can both be the match of tdisambiguates the false displacement from the true displacement. Here, the X cannot bevalue. g is approximately 1.5.

(Fellingham and Sommer, 1983; Varghese et al., 1994). We showthat an ambiguity in displacement estimation using two imagescan be resolved with three images. Assume that the ground truthimage ~I of Eq. (1) has the same intensity at i and at iþ s, i.e.

IkðiÞ ¼ ~Iði� ~aki Þ þ nkðiÞ; k ¼ 1;2;3; ~IðiÞ ¼ ~Iðiþ sÞ ð32Þ

where nkðiÞ is Gaussian noise as defined in Eq. (1). Eq. (3) now canbe written as

CD ¼Xm

i¼1

~IðiÞ �~Iðiþ sþ aki � ~ak

i � sÞ þ n1ðiÞ � nkðiþ aki Þ

� �2ð33Þ

where we have added and subtracted s to the argument of~Iðiþ ak

i � ~aki Þ. Now it can be seen that CD has two local minima at

aki ¼ ~ak

i and at aki ¼ ~ak

i þ s. In addition, the expected value of CD atboth local minima is equal:

E CDð~ak1; . . . ; ~ak

m� �

¼ E CDð~ak1 þ s; . . . ; ~ak

m þ sÞ� �

¼ 2mr2 ð34Þ

where r2 is the variance from Eq. (1). Therefore, the false match~ak

i þ s cannot be eliminated. Now assume that we have threeimages I1; I2 and I3 for displacement estimation. Similar to the casefor two images, Eq. (28) can be modified by adding and subtractings to iþ ak

i � ~aki :

two images to differentiate true displacement from false displacement when thehe O in the top image. Right shows the addition of the third image (in the bottom)the match anymore since in the third image it corresponds to a different intensity

(a) 2D AM

(c) ElastMI

lateral, SNR = 16.7 axial, SNR = 43.3

10

20

40

30

10

20

40

30

10

20

40

30

10

20

40

30

10

20

40

30

dept

h (m

m)

10

20

40

30

5 10 15 20width (mm)

5 10 15 20width (mm)

5 10 15 20width (mm)

5 10 15 20width (mm)

5 10 15 20

width (mm)

5 10 15 20

width (mm)

dept

h (m

m)

dept

h (m

m)

lateral, SNR = 19.4 axial, SNR = 53.8

lateral, SNR = 34.1 axial, SNR = 73.4

2DAMaccum.ElastMI

(e) var.

2DAMaccum.ElastMI

-3 -2.5 -2 -1.5 -1 -0.5

-3 -2.5 -2 -1.5 -1 -0.5

axial

lateral

0.4

0.4

0.8

1

0.8

1

0.6

0.2

0.6

0.2

0.9

0.95

1

1.05

1.1

0.4

0.45

0.5

0.55

0.6

ax. lat.

% %

(b) Accumulate

(d) strain colormap

Fig. 8. Strain results of the simulated images of Fig. 7. (a–c) Show the axial and lateral strain images, with color-maps shown in (d). In (a) only F7 and F8 are used, while in (b)and (c) three frames of F5, F7 and F8 are utilized. (e) Shows the ratio of the variance of the strain of different methods compared to 2D AM with different g values.Accumulating strains and ElastMI both give lower than 1 ratios. ElastMI gives the smallest variance.

322 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

CDða21; . . . ; a2

m; g1; . . . ;gmÞ ¼Xm

i¼1

~IðiÞ �~Iðiþ sþ a2i � ~a2

i � s�

þn1ðiÞ � n2ðiþ a2i Þ�2

þXm

i¼1

~IðiÞ �~Iðiþ sþ ga2i � g~a2

i � s�

þn1ðiÞ � n3ðiþ ga2i Þ�2

It can now be easily seen that CD has two local minima at a2i ¼ ~a2

i

and at a2i ¼ ~a2

i þ s. However unlike the case for two images, the ex-pected value of CD at the incorrect match a2

i ¼ ~a2i þ s is more than

its expected value at a2i ¼ ~a2

i because:

E CDð~ak1; . . . ; ~ak

m� �

¼ 4mr2

E CDð~ak1 þ s � � � ~ak

m þ sÞ� �

¼ 4mr2 þ EXm

i¼1

ð~IðiÞ �~Iðiþ gsÞÞ2" #

In the other words, unlike the case for two images (Eq. (34)), thetrue match results in smaller average cost compared to the falsematch. Fig. 6 shows how with two periodic images it is not possibleto differentiate the true displacement (~a2, marked with a circle)

from the false displacement (~a2 þ s, marked with a cross) sinceI1ðiÞ � I2ðiþ ~a2Þ� �2 and I1ðiÞ � I2ðiþ ~a2 þ sÞ

� �2 are in average (i.e.ignoring the noise) equal. However, by adding a third image it ispossible to differentiate the true displacement ~a2 from the falsedisplacement ~a2 þ s since I1ðiÞ � I2ðiþ ~a2Þ

� �2 þ I1ðiÞ � I3ðiþ gi~a2Þ

� �2

is in average smaller than I1ðiÞ � I2ðiþ ~a2 þ sÞ� �2þ

I1ðiÞ � I3ðiþ gið~a2 þ sÞÞ� �2.

5. Results

We use data from simulation, phantom experiments and pa-tient trials to validate the performance of the ElastMI algorithm.All the ElastMI results are obtained using Eq. (19) with parametersdefined in Eqs. (25) and (26). The ElastMI algorithm is currentlyimplemented in Matlab mex functions and runs in real-time on aP4 3.6 GHz single core processor. In (Rivaz et al., 2011a), we pro-posed to estimate the strain from the displacement as following:we first applied a least square filtering in the axial direction to findan estimate for the strain. We then applied a Kalman filter in thelateral direction to remove the noise, while preventing blurring.

Fig. 9. Box plot of the variance of strain accumulation and ElastMI, compared to 2DAM. A ratio of smaller than 1 indicates a reduction in the variance achieved withusing 3 frames.

Table 1The SNR and CNR of the strain images of Fig. 10. The improvement % is w.r.t. 2D AM.The SNR is calculated for the background window. For B-mode, we calculate twovalues: one for the top (vertical) and another for the left (horizontal) backgroundwindows. Maximum values are in bold font.

B-mode 2D AM Accumulation ElastMI

Vert. Horiz. Axial Lateral Axial Lateral Axial Lateral

SNR 2.6 2.9 11.1 6.0 12.0 6.3 14.9 6.6SNR improv. % – – 0 0 8 5 34 10CNR 0.2 0.5 8.5 3.0 8.6 3.1 11.1 3.4CNR improv. % – – 0 0 3 5 31 13

2D AM accum. ElastMI

4

5

6

7

8

CN

R

Fig. 11. The CNR values of the lesion in axial strain images computed over 10 sets ofultrasound frames.

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 323

We use the same technique here, with 50 samples in the axialdirection to perform the least square fitting.

We compare ElastMI against the 2D AM strain and accumulatedstrain images. Two approaches are usually taken to utilize multipleimages: (1) Displacements are accumulated to increase the

axial

axia

l per

c. %

late

ral p

erc.

%

(a) B-mode

(c) Accumulate

axial lateral

5

10

25

20

15

5

10

25

20

15

5

10

25

20

15

5

10

25

20

15

5

10

25

20

15

0

0.5

1

1.5

1

2

3

4

5

6

0 10 20 30

0 10 20 30 0 10 20 30

Fig. 10. Axial and lateral strain images of the phantom with the target and background wspherical and has a diameter of 1 cm. The axial and lateral strain scales are identical for all7% and 2%. The difference between different methods is most visible at a 2x zoom.

displacement amplitudes, i.e. the Lagrangian particle tracking(e.g. for cardiac strain imaging over the cardiac cycle (Shi et al.,2008; Ma and Varghese, 2012)), and (2) strain images are averagedto reduce noise. In Lagrangian particle tracking, one should notethat the location of a particle keeps changing in the imagesequence, and therefore appropriate displacements must beaccumulated. In ElastMI, both displacements are calculated withrespect to the one reference image, i.e. I1. Therefore, we do notneed to perform any Lagrangian tracking and the displacementsare not accumulated in ElastMI.

axial lateral

(b) 2D AM

(d) ElastMI

axial lateral

5

10

25

20

15

0 10 20 30

0 10 20 30

0 10 20 30

0 10 20 30

5

10

25

20

15

indows (see Table 1 for SNR and CNR values). All axes are in mm. The hard lesion isimages and are shown in (a): the maximum axial and lateral strains are respectively

Fig. 12. Strain images of the 1st in vivo patient trial after ablation. The thermallesions is delineated and pointed to by an arrow. The lateral strain images does notimmediately carry anatomical information and will not be shown hereafter. All axesin mm. See also Table 2.

324 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

In all our results, we map the strain images of 2D AM, accumu-lated strain and ElastMI to the same range, so that they can be eas-ily compared.

5.1. Simulation results

Field II (Jensen, 1996) and ABAQUS (Providence, RI) software areused for ultrasound simulation and for finite element simulation.The specifications of the ultrasound probe and the uniform

28mm

2D AM B-mode

Bef

ore

abla

tion

B-mode

2D AM

Aft

er a

blat

ion

CT

10 20 30

30

30

20

10

30

20

10

3

2

1

10

20

40

50

20

40

50

10

20

40

30

10

10 20 30

10 20 30

30

10 20 30

50

Fig. 13. Strain images of the 2nd in vivo patient trial corresponding to before and afterpointed to by arrows. All axes in mm. See also Table 2.

phantom are in Rivaz et al. (2011a). 8 ultrasound image framesare simulated at different compression levels from 0% to 4%, asshown in Fig. 7. We set frame F7 as I1 and frame F8 as I2 as shownin the figure. I3 is set to one of the other frames, resulting indifferent g values shown in the bottom of Fig. 7.

The axial and lateral strain images obtained from F7 and F8using 2D AM are shown in Fig. 8a. Using the three frames of F5,F7 and F8, we calculate strains between consecutive frames, addthe strains, and divide it by two to have a 1% strain image similarto (a). The result is in (b). The ElastMI results using the same threeframes is shown in (c). Note the SNR values shown on top of eachstrain image, and how it increases from 2D AM to accumulatedstrain to ElastMI. The axial and lateral strains in (a–c) have thesame intensity scale (as shown in (d)) to ease comparison.

We repeat this experiment by setting I3 to frames F1 throughF6, and compute the ratio of the noise compared to the 2D AMstrain. The result is shown in Fig. 8e. Both ElastMI and accumula-tion of strain decrease the variance. However, this reduction is sig-nificantly more in ElastMI because it incorporates a powerfulphysical constraint into its cost function and considers all threeimages to estimate the displacement estimates.

Finally, we generate 10 different realizations of frame F1 inFig. 7 with 10 different simulated phantoms, and compress eachphantom to obtain 10 instances of frames F3, F5, F7 and F8. Wethen repeat the experiment of Fig. 8e for each phantom. Fig. 9shows the results. Please note that we do not perform Lagrangianspeckle tracking; we rather average the strain images to get theaccumulated strain values. We see that using three images, bothstrain accumulation and ElastMI result in a reduction in the vari-ance. Also, the variability in the lateral strain results in (b) is gen-erally more than that of the axial strain images in (a). This can beattributed to the lower resolution in the lateral direction, and thelack of phase information in this direction. In both axial and lateral

strain %

ElastMIAccumulate

ElastMIAccumulate

0

0

0

30

20

10

20

40

50

30

10

10 20 30 10 20

10 20 30 10 20 30

0.5

1

1.5

2

2.5

30

ablation. The tumor in the first row and the ablation lesion in the second row are

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 325

strains, ElastMI improves the results of strain accumulation by astatistically significant amount (p < 0:00002 for paired t-tests).This improvement is mainly due to imposing the physics-basedprior in ElastMI.

5.2. Phantom results

RF data is acquired from an Antares Siemens system (Issaquah,WA) at the center frequency of 6.67 MHz with a VF10-5 linear ar-ray at a sampling rate of 40 MHz. An elastography phantom (CIRSelastography phantom, Norfolk, VA) is compressed axially in twosteps using a linear stage, each step 0.1 in. The Young’s elasticitymodulus of the background and the lesion under compressionare respectively 33 kPa and 56 kPa. Three RF frames are acquiredcorresponding to before compression (F1), after the first compres-sion step (F2) and after the second compression step (F3). I1; I2 andI3 are respectively set to F2, F1 and F3. Two displacement maps,one between F1 and F2, and the second between F2 and F3 are esti-mated with 2D AM. They are then added to give the F1 to F3 dis-placement map. The unitless metrics signal to noise ratio (SNR)

B-mode 2D AM A

Bef

ore

abla

tion

Dur

ing

abla

tion

B-mode 2D AM

60mm

Aft

er a

blat

ion

2D AM

B-mode

CT

20

40

30

10

20

40

30

10

20

40

30

10

20

40

30

10

20

40

30

10

20

40

30

10

20

40

30

10 20

40

30

10

20

40

30

10

10 20 30 10 20 30

10 20 30 10 20 30

10 20 30

Fig. 14. Strain images of the 3rd in vivo patient trial corresponding to before (1st row), durow, and the ablation lesion in second and third row are pointed to by arrows. Four prongto by arrows. CT is acquired 3 weeks after ablation. All axes in mm. See also Table 2.

and contrast to noise ratio (CNR) are calculated to compare 2DAM, accumulated and ElastMI strains:

CNR ¼ CN¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi2ð�sb � �stÞ2

r2b þ r2

t

s; SNR ¼

�sr

ð35Þ

where �st and �sb are the spatial strain average of the target and back-ground, r2

t and r2b are the spatial strain variance of the target and

background, and �s and r are the spatial average and variance of awindow in the strain image respectively. Fig. 10 shows the axialand lateral strain images along with the target and background win-dows used for SNR and CNR calculation. The SNR is only calculatedfor the background window. The results are in Table 1. In compar-ison with 2D AM, both accumulating strain and ElastMI improvethe SNR and CNR. However, the improvement of ElastMI is signifi-cantly more which is due to the utilization of our novel mechanicalprior and the EM optimization technique.

Using the same ultrasound machine and probe, we collect RFdata from freehand palpation of a CIRS breast elastography phan-tom (CIRS, Norfolk, VA). The lesion is three times stiffer than thebackground. We select 10 set of ultrasound frames with 3 frames

0.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65

ccumulate ElastMI

0

0.1

0.2

0.3

0.4

0.5

Accumulate ElastMI

0.5

1

1.5

2

2.5

Accumulate ElastMI

20

40

30

10

20

40

30

10

20

40

30

10

10 20 30 10 20 30

10 20 30 10 20 30

10 20 30 10 20 30

ring (2nd row) and after (3rd row) ablation. All axes are in mm. The tumor in the tops of the ablation probe are visible in the US image of the second row and are pointed

Table 2The CNR of the strain images of Figs. 12–14. P1, P2 and P3 respectively correspond to patients 1, 2 and 3. Maximum values are in bold font.

Before ablation During ablation After ablation

2D AM Accum. ElastMI 2D AM Accum. ElastMI 2D AM Accum. ElastMI

P1 – - – – – – 4.2 4.9 5.3P2 12.2 11.8 15.3 – – – 2.7 2.4 3.5P3 8.2 7.7 9.2 1.9 2.4 3.8 5.1 5.0 6.8Average 10.2 9.8 12.3 1.9 2.4 3.8 4.0 4.1 5.2

326 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

per set. We then set I1 to the image with intermediate compres-sion, and I2 and I3 to maximum and minimum compression. Ineach set, we first compute the 2D AM strains between I1 and I2.We then use all the three frames to compute accumulated andElastMI strains, and compute the contrast to noise ratio betweenthe lesion and background in each set. Fig. 11 shows the results.The improvement of ElastMI over both 2D AM and strain accumu-lation is statistically significant with p-values of paired t-test lessthan 0.002.

5.3. Clinical study

RF data is acquired from ablation therapy of three patients withliver cancer using an Antares Siemens system (Issaquah, WA) ultra-sound machine. The patients underwent open surgical radiofre-quency (RF) thermal ablation for primary or secondary livercancer. All patients enrolled in the study had unresectable diseaseand were candidates for RF ablation following review at our insti-tutional multidisciplinary conference. Patients with cirrhosis orsuboptimal tumor location were excluded from the study. All pa-tients provided informed consent as part of the protocol, whichwas approved by the institutional review board. The RF data we ac-quired is as follows: for the first patient only after ablation, for thesecond patient before and after ablation, and for the third patientbefore, during and after ablation. A VF10-5 linear array at the cen-ter frequency of 6.67 MHz with a sampling rate of 40 MHz is usedfor RF data acquisition. The ablation is administered using the RITAModel 1500 XRF generator (Rita Medical Systems, Fremont, CA).Tissue is simply compressed freehand at a frequency of approxi-mately 1 compression per 2 s with the ultrasound probe withoutany attachment and the strain images are generated offline.

The strain–stress curve of liver is approximately linear for alarge strain range (Yeh et al., 2002), and therefore, the assumptionthat the tissue should remain linear in the three images of ElastMIis comfortably met. In addition, higher graded fibrotic liver tissue isabout four times stiffer than lower graded fibrotic tissue (DeWallet al., 2012b), and hence, elastography imaging can potentially beused to estimate the fibrotic grade.

The strain images obtained with 2D AM, accumulation of con-secutive strains and the ElastMI algorithm are shown in Figs. 12,13 and 14. Images before ablation show the tumor. Images corre-sponding to during ablation (second row, Fig. 14) are acquiredapproximately 3 min after start of the ablation while the ablationdevice is temporarily shut down, but remains in tissue, for ultra-sound data acquisition, but is still in the liver tissue. The prongsof the ablation probe are visible in the US image of the secondrow and are pointed to by blue arrows. Images after ablation showthe ablated lesion and are acquired approximately 3 min after theablation device is retracted from the tissue.

The severe attenuation in the B-mode image of Fig. 12 has not de-graded the strain images. The region with low strain in Fig. 14d–f iscaused by both ablation and by the ablation probe’s prongs holdingthe tissue, as also suggested by Varghese et al. (2004).

CNR values are calculated between target and background win-dows, each of size 10 mm � 10 mm. The target window is insidethe tumor (before ablation) or the ablation lesion, and the back-

ground window is outside. Table 2 shows the results. ElastMI sig-nificantly improves all the CNR values. The average values forbefore and after ablation are shown in the last row. The averageCNR over all values of this table are 5.7 for 2D AM, 5.7 for accumu-lating strains, and 7.3 for ElastMI. The improvements of ElastMIover both 2D AM and strain accumulation are statistically signifi-cant, with paired t-test p-values of respectively 0.006 and 0.012.

Accumulating strain images generally improves the results.However, it tends to blurs sharp boundaries and lower the contrastin our experience; the tumor/ablation lesion is significantly ‘‘lessdark’’ in Figs. 12 and 13 and 12. This is an inherent result of aver-aging/accumulating strain images. Another reason lies in the factthat as we add consecutive strains, the chances of having incorrectdisplacement estimates at any part of the image increases. TheElastMI algorithm however utilizes additional physics-based priorsand robust-to-outlier IRLS method to solve for displacement esti-mation using three images simultaneously. We see that these fea-tures enable it to continue generating low noise and sharpelasticity images in the challenging data of patient trials.

6. Discussions and conclusion

In this work, we focused on utilizing three images to calculatelow variance displacement fields. We first derived constraints onvariation of displacement fields with time using concepts frommechanics of materials. We then proposed ElastMI, an EM basedalgorithm that uses these constraints. We showed that ElastMIoutperforms our previous algorithm AM. We corroborated theseresults using simulation, phantom and in vivo experiments.

The advantages of ElastMI over accumulating displacementfields of the intermediate images are as follows. First, by displace-ment estimation using two images only a fraction of the availableinformation is utilized, making tracking prone to signal decorrela-tion and false matches. ElastMI uses all the three images in agroup-wise scheme to find displacement fields. Second, the phys-ics-based prior substantially reduces the estimation variance asshown formally and using simulation and experimental data. Final-ly, by accumulating displacement fields, errors are accumulated.This is in fact a well known problem of any sequential trackingor stereo system (Brown et al., 2003). Its disadvantage, however,is that it is computationally more expensive. In our implementa-tion, ElastMI takes 0.2 s to generate strain images of size1000 � 100 on a single core 3.8 GHz P4 CPU, compared to 0.04 sof 2D AM and 0.08 s for accumulating strains.

Both ElastMI and 2D AM assume displacement fields are contin-uous. This assumption breaks for vascular strain imaging wherethe two vessel walls can move in opposite directions (Shi and Var-ghese, 2007; Shi et al., 2008). This issue also has been addressed inmodel-based elasticity reconstruction problems by assigning softand hard constraints (Le Floc’h et al., 2009; Richards and Doyley,2011). Therefore an interesting avenue for future work would beto relax the displacement continuity in ElastMI for image regionswhere 2D AM predicts high variability in the direction of displace-ments. Discontinuity preserving ElastMI can then be used in non-invasive vascular elastography applications (Maurice et al., 2004,2007; Shi and Varghese, 2007; Shi et al., 2008; Hansen et al.,

x

z

I1

1

2

i

i+1

m

1 2 j j+1 n

x(i , j)

x

z

x

1

2

i

i+1

m

1 2 j j+1 n

a12i,j

l12i,j

(i+a12i,j , j+l12

i,j)

I2

x

z

x

1

2

i

i+1

m

1 2 j j+1 n

a13i,j

l13i,j

(i+a13i,j , j+l13

i,j)

I3

C12(a12,l12 ) C13(a13,l13 )

C23(a12,a13, l12,l13 )

Fig. A.1. Pairwise cost functions between three images.

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 327

2009; Mercure et al., 2011; Zakaria et al., 2010; Korukonda andDoyley, 2012; Korukonda et al., 2013).

Accumulating strains significantly outperforms 2D AM in thesimulation and phantom experiments. This improvement, how-ever, mostly diminishes in the patient trials. This is mainly dueto the fact that in the challenging freehand intra-operative settings,it is hard to find three ‘‘good’’ frames for strain computations.Therefore, one of the strain images can be noisy or blurry, and ad-versely affect the accumulated strain. ElastMI, however, does notsuffer from this problem for two main reasons. First, the additionalprior, and second, simultaneous estimation of displacement fieldsfrom three images using robust estimation methods.

We proved that for simple additive Gaussian noise, ElastMI sig-nificantly reduces the estimation noise. For ablation monitoring,however, the nature of the noise changes dramatically both withtime and location because of the gas bubbles. Nevertheless, our re-sults on the patient data shows that ElastMI performs well in thepresence of such complex noise.

In the analysis of Section 3, we assumed quasi-static deforma-tion, so that the dynamics of the continuum can be ignored. Thisassumption is generally valid for freehand palpation elastography.In other methods of measuring tissue elastic properties where theexcitation is dynamic (Parker et al., 2005; Greenleaf et al., 2003),Kalman filters can be used to fuse the noisy displacement esti-mates and tissue dynamics models.

For noise analysis, we assumed additive Gaussian noise, whichallowed us to analytically derive estimates for measurement vari-ance. More accurate techniques for motion estimation have beenproposed based on more realistic models of ultrasound noise(Insana et al., 2000; Maurice et al., 2007). In the future, we willconsider more realistic speckle statistics, such as the models in Riv-az et al., 2007a,b, 2010.

As suggested by Eq. (15), the regularization can be a function ofthe two estimated displacement estimations. For example, u 0or u p indicate that the two estimated displacement fields arein fact similar up to a scale factor, which is what we assume in thiswork. However, u �p=2 indicates that the two displacementsare not similar, meaning that either one of the displacement esti-mates is incorrect or that the tissue is highly nonlinear. Futurework will exploit u in the regularization term (Eq. (15)).

In the future, we will also extend the framework presented inthis paper for calculating the displacement field from three imagesto the more general case where more than three images are uti-lized. Finally, direct estimation of the strain from ultrasoundframes (Brusseau et al., 2008) will also be incorporated intoElastMI.

Acknowledgments

Hassan Rivaz was supported by the DoD Predoctoral Trainee-ship Award and by the Advanced Simulation Fellowship from theLink Foundation. We thank anonymous reviewers for their con-structive feedback, Dr. Stanislav Emelianov for his comments, Dr.Pezhman Foroughi, Ioana Fleming and Mark van Vledder for valu-able discussions, and Shelby Brunke for technical support. We alsothank Siemens for providing us with an Ultrasound machine withresearch interface, and the Hopkins Radiology department forintramural funds.

Appendix A

We now show that the additional constraint of Eq. (8) is criticalin reducing the error in displacement estimation. Consider 3images I1; I2 and I3 from the set of p images (Fig. A.1). Let d12;d23

and d31 be the displacement between I1; I2, between I2; I3 and be-

tween I3; I1 (using the notation of the previous Section, d12 ¼ d2

and d31 ¼ �d3). These three displacements are not independentsince d12 þ d23 þ d31 ¼ 0. The axial component of this equationgives a12 þ a23 þ a31 ¼ 0, which allows us to replace a23 witha12 ¼ a2 and a31 ¼ �a3:

CDða21; . . . ; a2

m; a31; . . . ; a3

mÞ ¼Xm

i¼1

I1ðiÞ � I2ðiþ a2i Þ

� �2

þXm

i¼1

I1ðiÞ � I3ðiþ a3i Þ

� �2

þXm

i¼1

I2ðiþ a2i Þ � I3ðiþ a3

i Þ� �2 ðA:1Þ

328 H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329

where we have modified the data term of the cost function by add-ing the intensity similarity between each two of the three images.Note that this equation is for the displacements of any three imagesand the images need not be consecutive. Using the noise model ofEq. (1) and linearizing I2 and I3 around i we have

CDða21; . . . ;a

2m; a3

1; . . . ;a3mÞ ¼

Xm

i¼1

�ða2i � ~a2

i Þ �~I0aðiÞþn1ðiÞ�n2ðiþa2i Þ

� �2

þXm

i¼1

�ða3i � ~a3

i Þ �~I0aðiÞþn1ðiÞ�n3ðiþa3i Þ

� �2

þXm

i¼1

�ða2i � ~a2

i Þ �~I0aðiÞþ ða3i � ~a3

i Þ �~I0aðiÞþn2ðiþa2i Þ�n3ðiþa3

i Þ� �2

ðA:2Þ

The optimum value of a2i and a3

i will minimize CD. Setting@CD=@a2

i ¼ 0 and @CD=@a3i ¼ 0 will result in a coupled 2-equations-

2-unknowns linear system. Solving the set of equations will give

a2i ¼ ~a2

i � ~I0aðiÞh i�1

n1ðiÞ � n2ðiþ a2i Þ

� �a3

i

¼ ~a3i � ~I0aðiÞ

h i�1n1ðiÞ � n3ðiþ a3

i Þ� �

ðA:3Þ

which are the same as Eq. (5). Interestingly, the solution of the cou-pled linear system shows that a2

i does not depend on n3, and simi-larly a3

i does not depend on n2. Therefore, the implicit constraint ofa12 þ a23 þ a31 ¼ 0 will not reduce the noise in the displacementestimation. In the other words, the third term in the RHS of Eq.(A.1), i.e.

Pmi¼1 I2ðiþ a2

i Þ � I3ðiþ a3i Þ

� �2 will add no information tothe cost function.

We have developed and implemented an algorithm that en-forces the implicit constraint of this Appendix to calculate two mo-tion fields from three images. Our simulation and experimentalresults showed that, compared to AM, this method has negligibleimpact on bias, variance, SNR and CNR of the calculated motionfield and strain image as predicted by our Gaussian noise model.We do not present these results here because of space limitations.

References

Arnal, B., Pernot, M., Tanter, M., 2011. Monitoring of thermal therapy based on shearmodulus changes: shear wave thermometry. IEEE Trans. Ultrason. Ferroelectr.Freq. Control 58 (3), 369–378.

Ayache, N., Lustman, F., 1991. Trinocular stereo vision for robotics. IEEE Trans.Pattern Anal. Mach. Intell. 13 (1), 73–84.

Bharat, S., Varghese, T., Madsen, E., Zagzebski, J., 2008. Radio-frequency ablationelectrode displacement elastography: a phantom study. Med. Phys. 35, 2432–2442.

Boctor, E., Fichtinger, G., Yeung, A., Awad, M., Taylor, R., Choti, M., 2004. Roboticstrain imaging for monitoring thermal ablation of liver, medical imagecomputing & computer assisted interventions, MICCAI, New York, NY, pp. 81–88.

Brown, M., Burschka, D., Hager, G., 2003. Advances in computational stereo. IEEETrans. Pattern Anal. Mach. Intell. 25 (8), 993–1008.

Brusseau, E., Kybic, J., Deprez, J., Basset, O., 2008. 2-D locally regularized tissuestrain estimation from radio-frequency ultrasound images: theoreticaldevelopments and results on experimental data. IEEE Trans. Med. Imag. 27(2), 145–160.

Chaturvedi, P., Insana, M., Hall, T., 1998. 2-D companding for noise reduction instrain imaging. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 45 (1), 179–191.

Chen, L., Housden, R., Treece, G., Gee, A., Prager, R., 2010. A data weighting schemefor quasistatic ultrasound elasticity imaging, Univ. of Cambridge TR651.

Deprez, J., Brusseau, E., Schmitt, C., Cloutier, G., Basset, O., 2009. 3D estimation ofsoft biological tissue deformation from radio-frequency ultrasound volumeacquisitions. Med. Image Anal. 13, 116–127.

DeWall, R., Varghese, T., Brace, C., 2012a. Visualizing ex vivo radiofrequency andmicrowave ablation zones using electrode vibration elastography. Med. Phys.39, 6692–6702.

DeWall, R.J., Bharat, S., Varghese, T., Hanson, M.E., Agni, R.M., Kliewer, M.A., 2012b.Characterizing the compression-dependent viscoelastic properties of humanhepatic pathologies using dynamic compression testing. Phys. Med. Biol. 57 (8),2273.

Doyley, M., Bamber, J., Fuechsel, F., Bush, N., 2001. A freehand elastographic imagingapproach for clinical breast imaging: system development and performanceevaluation. Ultrasound Med. Biol. 27, 1347–1357.

Emelianov, S., Erkamp, R., Lubinski, M., Skovoroda, A., O’Donnell, M., 1998.Nonlinear tissue elasticity: adaptive elasticity imaging for large deformations.In: IEEE Ultrasonics Symposium, vol. 2, pp. 1753–1756.

Erkamp, R., Emelianov, S., Skovoroda, A., O’Donnell, M., 2004a. Measuring thenonlinear elastic properties of tissue-like phantoms. IEEE Trans. Ultrason.Ferroelectr. Freq. Control 51 (4), 410–419.

Erkamp, R., Emelianov, S., Skovoroda, A., O’Donnell, M., 2004b. Nonlinear elasticityimaging: theory and phantom study. IEEE Trans. Ultrason. Ferroelectr. Freq.Control 51 (5), 532–539.

Fahey, B.J., Hsu, S.J., Wolf, P.D., Nelson, R.C., Trahey, G.E., 2006. Liver ablationguidance with acoustic radiation force impulse imaging: challenges andopportunities. Phys. Med. Biol. 51, 3785–3808.

Fellingham, L., Sommer, F., 1983. Ultrasonic characterization of tissue structure inthe in vivo human liver and spleen. IEEE Trans. Sonics Ultrason. 31 (4), 418–428.

Foroughi, P., Rivaz, H., Fleming, I., Hager, G., Boctor, E., 2010. Tracked UltrasoundElastography (TrUE), medical image computing & computer assistedinterventions, Beijing, China, pp. 6–17.

Goenezen, S., Dord, J., Sink, Z., Barbone, P., Jiang, J., Hall, T., Oberai, A., 2012. Linearand nonlinear elastic modulus imaging: an application to breast cancerdiagnosis. IEEE Trans. Med. Imag. 31 (8), 1628–1637.

Goldberg, S., Gazelle, G., Mueller, P., 2000. A unified approach to underlyingprinciples, techniques, and diagnostic imaging guidance. AJR Roentgen 174,323–331.

Greenleaf, J., Fatemi, M., Insana, M., 2003. Selected methods for imaging elasticproperties of biological tissues. Annu. Rev. Biomed. Eng. 5, 57–78.

Hager, G., Belhumeur, P., 1998. Efficient region tracking with parametric models ofgeometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20 (10),1025–1039.

Hall, T., Zhu, Y., Spalding, C., 2003. In vivo real-time freehand palpation imaging.Ultrasound Med. Biol. 29, 427–435.

Hall, T., Oberai, A., Barbone, P., Sommer, A., Gokhale, N., Goenzen, S., Jiang, J., 2009.Elastic nonlinearity imaging. IEEE EMBS, 1967–1970.

Hall, T., Oberai, A., Barbone, P., Sommer, A., Jiang, J., Gokhale, N., Goenzen, S., 2007,Imaging the elastic nonlinearity of tissues. In: IEEE Int. Ultrasonics Symp., pp.452–455.

Hansen, H., Lopata, R., de Korte, C., 2009. Noninvasive carotid strain imaging usingangular compounding at large beam steered angles: validation in vesselphantoms. IEEE Trans. Med. Imag. 28, 872–880.

Hiltawsky, K., Kruger, M., Starke, C., Heuser, L., Ermert, H., Jensen, A., 2001. Freehandultrasound elastography of breast lesions: clinical results. Ultrasound Med. Biol.27, 1461–1469.

Huber, P., 1997. Robust Statistics. Wiley, New York.Insana, M., Cook, L., Bilgen, M., Chaturvedi, P., Zhu, Y., 2000. Maximum-liklihood

approach to strain imaging using ultrasound. IEEE Trans. Ultrason. Ferroelectr.Freq. Control 107 (3), 1421–1434.

Jensen, A., 1996. Field: a program for simulating ultrasound systems. Med. Biol. Eng.Comput. 34, 351–353.

Jiang, J., Hall, T., 2006. A regularized real-time motion tracking algorithm usingdynamic programming for ultrasonic strain imaging. In: IEEE Ultrasonics Symp.,Vancouver, Canada, pp. 606–609.

Jiang, J., Varghese, T., 2009. Youngs modulus reconstruction for RF ablationelectrode-induced displacement fields: a feasibility study. IEEE Trans. Med.Imag. 28, 1325–1334.

Jiang, J., Hall, T., Sommer, A., 2006. A novel performance descriptor for ultrasonicstrain imaging: a preliminary study. IEEE Trans. Ultrason. Ferroelectr. Freq.Control 53 (6), 1088–1102.

Jiang, J., Hall, T., Sommer, A., 2007. A novel image formation method for ultrasonicstrain imaging. Ultrasound Med. Biol. 53 (6), 1088–1102.

Jiang, J., Brace, C., Andreano, A., DeWall, R., Rubert, N., Pavan, T., Fisher, T., Varghese,T., Lee, F., Hall, T., 2010. Ultrasound-based relative elastic modulus imaging forvisualizing thermal ablation zones in a porcine model. Phys. Med. Biol. 55,2281–2306.

Konofagou, E., Ophir, J., 1998. A new elastographic method for estimation andimaging of lateral displacements, lateral strains corrected axial strains, andPoisson’s ratios in tissues. Ultrasound Med. Biol. 24 (8), 1183–1199.

Korukonda, S., Doyley, M., 2012. Visualizing the Radial and Circumferential StrainDistribution Within Vessel Phantoms Using Synthetic-Aperture UltrasoundElastography. IEEE Trans. UFFC 59 (8), 1639–1653.

Korukonda, S., Nayak, R., Carson, N., Schifitto, G., Dogra, V., Doyley, M., 2013.Noninvasive vascular elastography using plane-wave and sparse-array imaging.IEEE Trans. UFFC 60 (2), 332–341.

Krouskop, T., Wheeler, T., Kallel, F., Garra, B., Hall, T., 1998. The elastic moduli ofbreast and prostate tissues under compression. Ultrason. Imag. 20, 260–274.

Le Floc’h, S., Ohayon, J., Tracqui, P., Finet, G., Gharib, A., Maurice, R., Cloutier, G.,Pettigrew, R., 2009. Vulnerable atherosclerotic plaque elasticity reconstructionbased on a segmentation-driven optimization procedure using strainmeasurements: theoretical framework. IEEE Trans. Med. Imag. 28, 1126–1137.

Lubinski, M., Emelianov, S., O’Donnell, M., 1999. Adaptive strain estimation usingretrospective processing. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 46 (1),97–107.

Ma, C., Varghese, T., 2012. Lagrangian displacement tracking using a polar gridbetween endocardial and epicardial contours for cardiac strain imaging. Med.Phys. 39, 1779–1792.

H. Rivaz et al. / Medical Image Analysis 18 (2014) 314–329 329

Maurice, R., Ohayon, J., Fretigny, Y., Bertrand, M., Soulez, G., Cloutier, G., 2004.Noninvasive vascular elastography: theoretical framework. IEEE Trans. Med.Imag. 23 (2), 164–180.

Maurice, R., Fromageau, J., Brusseau, E., Finet, R., Rioufol, G., Cloutier, G., 2007. Onthe potential of the Lagrangian estimator for endovascular ultrasoundelastography: in vivo human coronary artery study. Ultrasound Med. Biol. 33(8), 1199–1205.

McCormick, M., Rubert, N., Varghese, T., 2011. Bayesian regularization applied toultrasound strain imaging. IEEE Trans. Biomed. Eng. 58 (6), 1612–1620.

Mercure, E., Deprez, J., Fromageau, J., Basset, O., Soulez, G., Cloutier, G., Maurice, R.L.,2011. A compensative model for the angle-dependence of motion estimates innoninvasive vascular elastography. Med. Phys. 38 (2), 727–735.

Moon, T., 1996. The expectation-maximization algorithm. IEEE Signal Proc. 13 (6),47–60.

Mulligan, J., Isler, V., Daniilidis, K., 2002. Trinocular stereo: a real-time algorithmand its evaluation. Intern. J. Comput. Vis. 47, 51–61.

Oberai, A., Gokhale, N., Goenezen, S., Barbone, P., Hall, T., Sommer, A., Jiang, J., 2009.Linear and nonlinear elasticity imaging of soft tissue in vivo: demonstration offeasibility. Phys. Med. Biol. 54, 1191–1207.

O’Donnell, M., Skovoroda, A.R., Shapo, B.M., Emelianov, S.Y., 1994. Internaldisplacement and strain imaging using ultrasonic speckle tracking. IEEETrans. Ultrason. Ferroelectr. Freq. Control 41 (3), 314–325.

Ophir, J., Alam, S., Garra, B., Kallel, F., Konofagou, E., Krouskop, T., Varghese, T., 1999.Elastography. Annu. Rev. Biomed. Eng. 213, 203–233.

Parker, K., Taylor, L., Gracewski, S., Rubens, D., 2005. A unified view of imaging theelastic properties of tissue. J. Acoust. Soc. Am. 117, 2705–2712.

Pellot-Barakat, C., Frouin, F., Insana, M., Herment, A., 2004. Ultrasound elastographybased on multiscale estimations of regularized displacement fields. IEEE Trans.Med. Imag. 23 (2), 153–163.

Richards, M., Doyley, M., 2011. Investigating the impact of spatial priors on theperformance of model-based IVUS elastography. Phys. Med. Biol. 56, 7223–7246.

Righetti, R., Kallel, F., Stafford, J., Price, R., Krouskop, T., Hazle, J., Ophir, J., 1999.Elastographic characterization of HIFU-induced lesions in canine livers.Ultrasound Med. Biol. 25 (7), 1099–1113.

Rivaz, H., Boctor, E., Fichtinger, G., 2007a. A robust meshing and calibrationapproach for sensorless freehand 3D ultrasound. In: Medical Imaging,International Society for Optics and Photonics, pp. 651318–651318.

Rivaz, H., Zellars, R., Hager, G., Fichtinger, G., Boctor, E., 2007b. Beam steeringapproach for speckle characterization and out-of-plane motion estimation inreal tissue. In: Ultrasonics Symposium. IEEE, pp. 781–784.

Rivaz, H., Boctor, E., Foroughi, P., Fichtinger, G., Hager, G., 2008a. Ultrasoundelastography: a dynamic programming approach. IEEE Trans. Med. Imag. 27(10), 1373–1377.

Rivaz, H., Fleming, I., Assumpcao, L., Fichtinger, G., Hamper, U., Choti, M., Hager, G.,Boctor, E., 2008b. Ablation monitoring with elastography: 2d in-vivo and 3d ex-vivo studies, medical image computing & computer assisted interventions,MICCAI, New York, NY, pp. 458–466.

Rivaz, H., Foroughi, P., Fleming, I., Zellars, R., Boctor, E., Hager, G., 2009. Trackedregularized ultrasound elastography for targeting breast radiotherapy. In:

Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009,Springer, pp. 507–515.

Rivaz, H., Kang, H.J., Stolka, P.J., Zellars, R., Wacker, F., Hager, G., Boctor, E., 2010.Novel reconstruction and feature exploitation techniques for sensorlessfreehand 3D ultrasound. In: SPIE Medical Imaging, International Society forOptics and Photonics, pp. 76291D–76291D.

Rivaz, H., Boctor, E., Choti, M., Hager, G., 2011a. Real-time regularized ultrasoundelastography. IEEE Trans. Med. Imag. 30 (4), 928–945.

Rivaz, H., Boctor, E., Choti, M., Hager, G., 2011b. Ultrasound elastography using threeimages, medical image computing & computer assisted interventionsconference, MICCAI, London, UK, pp. 371–378.

Shi, H., Varghese, T., 2007. Two-dimensional multi-level strain estimation fordiscontinuous tissue. Phys. Med. Biol. 52, 1325–1334.

Shi, H., Mitchell, C., McCormick, M., Kliewer, M., Dempsey, R., Varghese, T., 2008.Preliminary in-vivo atherosclerotic carotid plaque characterization using theaccumulated axial strain and relative lateral shift based strain indices. Phys.Med. Biol. 53, 6377–6394.

Sumi, C., 2008. Regularization of tissue shear modulus reconstruction using strainvariance. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 55 (2), 297–307.

Sumi, C., Sato, K., 2008. Regularization for ultrasonic measurements of tissuedisplacement vector and strain tensor. IEEE Trans. Ultrason. Ferroelectr. Freq.Control 55 (4), 787–799.

Varghese, T., Ophir, J., Cespedes, I., 1994. Mean-scatterer spacing estimates withspectral correlation. J. Acoustic. Soc. Am. 96 (6), 3504–3515.

Varghese, T., Ophir, J., Cespedes, I., 1996. Noise reduction in elastograms usingtemporal stretching with multicompression averaging. Ultrasound Med. Biol. 22(8), 1043–1052.

Varghese, T., Techavipoo, U., Liu, W., Zagzebski, J., Chen, Q., Frank, G., Lee Jr, F., 2003.Elastographic measurement of the area and volume of thermal lesions resultingfrom radiofrequency ablation: pathologic correlation. Am. J. Roentgenol. 181,701–707.

Varghese, T., Techavipoo, U., James, M., Zagzebski, A., Lee, F., 2004. Impact of gasbubbles generated during interstitial ablation on elastographic depiction ofin vitro thermal lesions. J. Ultrasound Med. 23 (1), 535–544.

Viola, F., Walker, W., 2005. A spline-based algorithm for continuous time-delayestimation using sampled data. IEEE Trans. Ultrason. Ferroelectr. Freq. Control52 (1), 80–93.

Yamakawa, M., Nitta, N., Shiina, T., Matsumura, T., Tamano, S., Mitake, T., Ueno, E.,2003. High-speed freehand tissue elasticity imaging for breast diagnosis. Jpn. J.Appl. Phys. 42 (5B), 3265–3270.

Yeh, W., Li, P., Jeng, Y., Hsu, H., Kuo, P., Li, M., Yang, P., Lee, P., 2002. Elastic modulusmeasurements of human liver and correlation with pathology. Ultrasound Med.Biol. 28 (4), 467–474.

Zahiri, R., Salcudean, S., 2006. Motion estimation in ultrasound images using timedomain cross correlation with prior estimates. IEEE Trans. Biomed. Eng. 53 (10),1990–2000.

Zakaria, T., Qin, Z., Maurice, R.L., 2010. Optical-flow-based b-mode elastography:application in the hypertensive rat carotid. IEEE Trans. Med. Imag. 29 (2), 570–578.