3D single and multimodal medical imageregistration using robust voxel similarity measuresand statistically constrained deformable models

Christoforos NIKOU

Laboratoire des Sciences de l’Image de l’Informatiqueet de la Télédétection (LSIIT)

Institut de Physique Biologique (IPB) – Faculté de Médecine

Centre National de la Recherche Scientifique (CNRS)Université Louis Pasteur-Strasbourg I (ULP)

Research framework

• General framework– brain imaging– intra-subject– rigid registration– images

• Single modal (MR/MR,SPECT/SPECT)

• multimodal(MRI/SPECT)

Introduction

MRI SPECT

Registration shortcomings

• Robustness to:– non gaussian noise;– lesion evolution;– non overlaping informations or structures;– incomplete acquisitions.

Introduction

Example 1 : lesion evolution

Reference image Image to beregistered

Introduction

Example 2 : non overlaping structures

Introduction

Example 3 : incomplete acquisition

Introduction

Methods

• Robust similarity functions– reject outliers

• Deformable modeling– Statistical constraints through training

Introduction

Presentation outline

• State of the art• Robust similarity metric-based registration• Statistically constrained deformable model-

based registration• Conclusion

Medical image registration (1)

• Transformation– rigid– affine– projective– deformable

• Image primitives– non image-based methods

• stereostatic frame• markers

– image-based methods• deformable models• contours• surfaces• anatomical landmarks• voxels

State of the art

Medical image registration (2)

• Similarity measure– standard distances;– principal axes;– correlation;– histogram

• variance;• entropy.

• Image modality– single modal;– multimodal;– modality to model

(atlas).

State of the art

Medical image registration (3)

• Validation– precision

• blind evaluation• simulations

– robustness– CPU time– clinical routine application

State of the art

Robustness to outliers

• Single modal images– Median least squares [Alexander-96]– Sign changes [Herbin-89]

• Multimodal images– Mutual information [Collignon-94, Wells-96]

State of the art

Presentation outline

• State of the art• Robust similarity metric-based registration• Statistically constrained deformable model-

based registration• Conclusion

Rigid registration

I x I T x

E F I x I T x

t t t x y z s s sx y z x y zT

1 2

1 2

( ) ( ( ))

( ) ( ( ), ( ( )))

[ , , , � , � , �, , , ]

⇔

=

=

Θ

ΘΘ

Θ

Robust voxel similarity measures

Standard similarity measures (1)

• Quadratic error [Hajnal-95, Alpert-96]• Correlation [Van den Elsen-95]• Sign changes [Venot-84]

• Ratio uniformity [Woods-92]• Inter-image uniformity [Woods-93]• Mutual information [Wells-96]

Single modal images

Multiomodalimages

Robust voxel similarity measures

Standard similarity measures(2)

• Single modal images (quadratic error)

E I x I T xx

( ) [ ( ) ( ( ))]Θ Θ= −� 1 22

Robust voxel similarity measures

Inter-image uniformity (1)

• Fundamental hypothesis:– correspondance between

uniform regions

• Application:– partitioning of the reference

image to its grey levels– Projection of the

partitioning to the floatingimage

– variance minimization in theprojected regions

Robust voxel similarity measures

Inter-image uniformity(2)

• Multimodal images (inter-image uniformity)

E N

I T x

NI T x

g gg

G

gx I x g

g

gg x I x g

( ) ( )

( ) ( ( )) ( )

( ) ( ( ))

( )

( )

Θ Θ

Θ Θ

Θ

Θ

Θ

= ×

= −

=

=

=

=

�

�

�

σ

σ µ

µ

1

2

2

1

1

1

Robust voxel similarity measures

How realistic is the uniformity assumption?

Robust voxel similarity measures

Uniformity validation?

Ideal histogram Real histogram

Robust voxel similarity measures

Robust similarity measures (1)

• Single modal images (robust quadratic error)

� Θ−=Θx

CxTIxIE ))),(()(()( 21ρ

Robust voxel similarity measures

Robust similarity measures(2)

• Multimodal images (robust inter-image uniformity)

E N

I T x C

NI T x C

g gg

G

g gx I x g

gg x I x g

gg

( ) ~ ( )

~ ( ) ( ( ( )) ~ ( ), )

~ ( ) argmin ( ( ( )) , )

( )

( )

Θ Θ

Θ Θ

Θ

Θ

Θ

= ×

= −

= −

=

=

=

�

�

�

σ

σ ρ µ

µ ρ µµ

1

2

2

1

1

1

Robust voxel similarity measures

The Geman-McClure robust estimator

ρ function ψ function

Robust voxel similarity measures

Registration algorithm

• Multiresolusion strategy• Stochastic optimization

– Fast simulated annealing• Deterministic optimisation

– Iterated Conditionnal Modes (ICM)

Robust voxel similarity measures

Experimental results

• MRI phantom• simulation (noise)• blind evaluation (comparative protocol)• Clinical applications cliniques (epilepsy,

MS)

Robust voxel similarity measures

Approach Translation (vox) Rotation (deg)

LS 2,30 ± 1,75 4,71 ± 2,88

RLS 0,03 ± 0,07 0,41 ± 0,21

MRI/MRI : simulation

LS : robust least squaresRLS : robust least squares

Robust voxel similarity measures

MRI/SPECT : simulation

Approach Translation (vox) Rotation (deg)

IU 3,85 ± 5,59 8,33 ± 4,51

MI 1,41 ± 0,74 0,94 ± 1,58

RIU 0,82 ± 0,53 0,21 ± 0,48

IU : inter-image uniformityMI : mutual informationRIU : robust inter-image uniformity

Robust voxel similarity measures

2D MRI/MRI (1)

Reference image Image to beregistered

Robust voxel similarity measures

2D MRI/MRI(2)

LS RLS

Robust voxel similarity measures

3D MRI/MRI (1)

Before registration

Robust voxel similarity measures

3D MRI/MRI (2)

Least squares

Robust voxel similarity measures

3D MRI/MRI(3)

Robust leastsquares

Robust voxel similarity measures

3D MRI/SPECT (1)

Inter-image uniformity

Robust voxel similarity measures

3D MRI/SPECT(2)

Mutual information

Robust voxel similarity measures

3D MRI/SPECT(3)

Robust inter-imageuniformity

Robust voxel similarity measures

Blind evaluation (1)

• Vanderbilt University (Nashville, TN, USA)image database

• CT/MRI et PET/MRI• Registration error computed on VOIs

Robust voxel similarity measures

Blind evaluation (2)

• Several MRI modalities(T1, T2, PD …)

• MRI/PET registrationwithout removing nonbrain structures

Robust voxel similarity measures

Blind evaluation (3)

Erreur médiane (mm) Erreur maximale (mm)

Type RIU Others RIU Others MRI/CT 1,6 - 2,9 3,3 - 4,0 4,3 - 6,2 10,7 - 12,2

MRI/PET 1,9 - 4,3 3,3 - 3,6 5,2 - 9,0 7,4 - 9,7

Robust voxel similarity measures

Blind evaluation (4)

Rang de classement MRI/CT

(15 groups) MRI/PET

(13 groups)

MRI modality Méd Max Méd Max

T1 6 3 12 4 DP 7 4 5 1 T2 3 4 4 1

T1 rect. 7 5 13 11 DP rect. 7 5 1 1 T2 rect. 6 4 5 2

Robust voxel similarity measures

CPU time (3D)CPU time (3D)

HP C360 (image 1283)

LS RLS IU MI RIU

5 mn 7 mn 10 mn 10 mn 20 mn

IU : inter-image uniformityMI : mutual informationRIU : robust inter-image uniformity

LS : least squaresRLS : robust least squares

Robust voxel similarity measures

Clinical application

Jan 1998 – May 1999 : 108 cases treatedictal SPECT

Inter-ictal SPECT

superimpoposition

registration

registration

Robust voxel similarity measures

Partial conclusion

• fully automatic method• redundant information is considered• local minima is not a problem• brain extraction is overcome• blind evaluation• clinical routine• other applications

Robust voxel similarity measures

Visible / IR Registration

Robust voxel similarity measures

Visible / IR Registration

Robust voxel similarity measures

Image uniformity Robust image uniformity

Perspectives

• connected components approach inRIU

• robust estimator parametersadaptation to image modality

• robust mutual information

Perspectives