DRAMMS: Deformable Registration via Attribute Matching and...

transcript

DRAMMS: Deformable Registration via Attribute Matching and Mutual-Saliency weighting

Yangming Ou, Christos Davatzikos

Section of Biomedical Image Analysis (SBIA)University of Pennsylvania

Outline

1. Background 2. Motivations3. Framework4. Methods5. Results6. Discussions

Image Registration is the process of finding the optimal transformation that aligns different imaging data into spatial correspondence.[Maintz & Viergever’98, Lester & Arridge’99, Hill’01, Zitova’03, Pluim’03, Crum’04, Holden’08]

Source (Subject) Target (Template)S2T

(overlaid on T)Transformation

(Deformation Field)

1. Background – Definition of Registration

1. Background – Registration Literature

Division of Most Registration Methods:

Category 1

Landmark/feature-based

Category 2

voxel-wise (intensity-based)

DRAMMS

1. Background – Registration Literature (1)

Category 1: Landmark/feature-based methods[Davatzikos’96, Thompson’98, Rohr’01, Johnson’02, Shen’02, Joshi’00, Chui’03, ...]

[Figure from Rohl’03].

Pros: 1) Intuitive; 2) Fast;

Cons:1) Errors in landmark detection & matching;2) Task-specific: different registration tasks

need different landmark detection methods.

Expected: “General-purpose” registration methods!

brain heart breast prostate

brain Abrain B

heart Aheart B

breast Abreast B

prostate Aprostate B

Not suitable for general-purpose

1. Background – Registration Literature (2)

Category 2: (Intensity-based) Voxel-wise methods[Christensen'94, Collins'94, Thirion'98, Rueckert'99, Vercauteren'07, Glocker'08, ...]

Joint Histogram After Reg.Images Under Registration

Pro: General-purpose registration methods (only rely on intensities).Con: 1) 2) => motivations for DRAMMS

Assumption:

Consistentrelationship between intensity distributions

[figure from Rueckert’99]

[figures from Papademetris]

Bblack

blackwhite

Bblack whiteblack

Outline

1. Background2. Motivations3. Framework4. Methods5. Results6. Discussions

2. Motivations (1): Why?

Histological Image (Prostate)

MR Image (Same Prostate)

Challenge 1:No consistent relationship in intensity distributions

intensity-based voxel-wise methods (e.g. MI) fail.

blackblack blackwhite

BlackWhite

BlackMatching Ambiguity

Reason:Matching ambiguity <= characterizing voxels only by intensities.

Not Distinctive!

2. Motivations (1): How?

Proposed Solution to Challenge 1:

- To reduce matching ambiguity, 1-dim image intensity => high-dim attribute vector

Attribute Matching

DRAMMS

Similarity map (by attributes)

High similarity

Low similarity

2. Motivations (2)

Challenge 2: Partial loss of correspondence

Histological Image (Prostate)

MR Image (Prostate)

Inspiration 2:

A continuous weighting mechanism for all voxels:

- Weight high for voxels able to establish reliable correspondence;=> let them drive the registration

- Weight low for voxels not able to establish reliable correspondence. => reduce their negative impact to the registration

Normal Brain Brain w/ lesion

Mutual-Saliency weighting

DRAMMS

Outline

Framework

1. Attribute MatchingTo reduce matching ambiguities

2. Mutual-Saliency weightingTo account for loss of correspondence

Deformable Registration via

DRAMMS

u T(u)

1. Attribute-Matching2. Mutual-Saliency

Outline

1. Background2. Motivations3. Framework4. Methods

4.1. Attribute Extraction and Selection4.2. Mutual-Saliency Weighting4.3. Implementation

5. Results6. Discussions

4.1.1. Attribute Extraction

Ideal Attributes1) Generally Applicable: to diverse registration tasks;2) Discriminative: voxels similar iff true correspondence.

Recent work[Shen and Davatzikos’01, Liu’02, Xue’04, Verma’04, Wu’07, etc]- Intensity attributes- Edge attributes- Tissue membership attributes (based on segmentation)- Geometric moment invariant attributes- Wavelet attributes- Local histogram attributes

4.1.1 Attribute Extraction – Gabor Attributes

A(0)(x)

A(1)(x)

A(2)(x)

A(3)(x)

Gabor filter bank (multi-scale, multi-orientation)

Why Gabor Attributes?

Reason 0: Sometimes, maybe one reason is enough ☺

Dennis Gabor (1900-1979)

Nobel Prize in Physics (1971)

Reason 1/3: General Applicability

Success in Texture segmentation [e.g., Jain’91];Cancer detection [e.g., Zhang’04];Prostate tissue differentiation [e.g., Zhan’06]; Brain registration tasks [e.g., Liu’02, Verma’04, Elbacary’06];…

brain heart breast prostate

Reason 2/3: Multi-scale and Multi-orientation.characterize voxels distinctively

Original Image

orientation

Gabor Attributes

orientation

Reason 3/3: Suitable for Registration

High Freq.Gabor Attributes

Edge maps => relatively independent of intensity distributions

Low Freq.Gabor Attributes

Smoothed (coarse) version => reduce local minimum in reg..

Original Image

Gabor Attributes characterize voxels distinctively

Special Points Ordinary Points

4.1.2. Select Optimal Gabor Attributes

Why? 1) Non-orthogonality among Gabor filters

redundancy;2) Attribute vector A( ) being too long

computational expensive.

How?Step 1: Select training voxel pairs:

Step 2: Select attribute on training voxel pairs:Training voxel pairs

by iterative backward elimination and forward inclusion.

Mutual-Saliency

Role of Gabor Attributes and Optimal Gabor Attributes

distinctiveness

Role of Gabor Attributes and Optimal Gabor Attributes

distinctiveness

Outline

4.1. Attribute Extraction and Selection4.2. Mutual-Saliency Weighting4.3. Implementation

Framework

1. Attribute MatchingTo reduce matching ambiguities

DRAMMS

u T(u)

1. Attribute-Matching2. Mutual-Saliency

4.2. Mutual-Saliency weighting

Recent work [Bond’05, Wu’07, Mahapha’08]

Their approach: Higher weights for more salient regions

Their assumption: Salient regions more likely to establish reliable correspondence.

Saliency in one image=> Matching reliability between two images?

A counter-example

Our work: saliency in one image => mutual-saliency b/w two images

Directly measure matching reliability[Anandan’89, McEache’97]

4.2. Mutual Saliency (MS) weighting

Idea:True correspondence should

Calculation of MS:

similarity

u T(u)

similar to each other;not similar to anything else.

Delta fun.

Reliable matching

High MS value

Role of Mutual-Saliency Map

Account for partial loss of correspondence

Source image Target image

Registrationwithout MS map

Registrationwith MS map

MS map

Outline

4.1. Attribute Extraction and Selection4.2. Mutual-Saliency Weighting 4.3. Implementation

4.3. Implementation

Optimized and regularized by Free Form Deformation (FFD) model [Rueckert’99]

Diffeomorphism FFD [Rueckert’06]

Multi-resolution to reduce local minimaGradient descent optimizationImplemented in CRun on 2.8G CPU, Unix OS

Outline

1. Background2. Motivations3. Framework4. Methods5. Results

5.1. Cross-subject registration;5.2. Multi-modality registration;5.3. Longitudinal registration;5.4. Atlas construction.

6. Discussions

5.1. Cross-subject Registrations

A (Subject) B (Template)

Cardiac

Evaluate registration accuracy by mean sq. diff. (MSD) and corr. coef. (CC)

between registered image and target image.

A2B deformation

deformation

5.1. Cross-subject Registrations

Observations:

1) In images that intensity-based method can register, attribute matching increased registration accuracy considerably;

2) Each of DRAMMS’ components provides additive improvement for registration accuracy.

5.2. Multi-modality Registrations

Human ProstateHistology MR Histology2MR Mutual-saliency

Histological

Joint histogram after registration

5.2. Multi-modality Registrations

Mouse Brain

Histology MR Histology2MR Mutual-saliency

Joint histogram after registration

5.3. Longitudinal Registration

5.4. Atlas Construction

Images from 30 training subjects

template

5.4 Atlas Construction (cont.)

By intensity-based FFD (mutual-information)

By DRAMMS

5.4. Atlas Construction (cont.)

Lesion

Low MS weight

5.4. Atlas Construction (cont.)

Mean Mutual-Saliency Map in 3D

Outline

Discussions

DRAMMS: a general-purpose registration method;DiffeomorphismImproves MI-based methods, especially when 1) no consistent relationship between intensity distributions;2) loss of correspondence

DRAMMS

A bridge between two categories of methods

Category 1

Landmark/feature-based

Category 2

voxel-wise (intensity-based)

DRAMMS

DRAMMS – bridge 1

Category 1

Landmark-based

Category 2

voxel-wise

DRAMMSAttribute Matching

Still using all voxelsEvery voxel will become a landmark to some extent.

DRAMMS – bridge 2

Category 1

Landmark-based

Category 2

voxel-wise

DRAMMSMutual-Saliency weighting

Weight = 1 for all voxelsWeight =

1 for landmarks

0 otherwise

Take-home message

1. (optimal) Attribute MatchingTo reduce matching ambiguities

DRAMMS

Thank you!

Code to be available:

(Lab) https://www.rad.upenn.edu/sbia/

(Personal)https://www.rad.upenn.edu/sbia/Yangming.Ou/

DRAMMS: Deformable Registration via Attribute Matching and...

Documents