Fundamental Performance Limits in Image Registration

By Dirk Robinson and Peyman Milanfar

IEEE Transactions on Image Processing Vol. 13, No. 9, 9/2004

CS679: Pattern RecognitionJosh Gleason and Rod Pickens

Topics Example Application of Performance Limits Image registration and errors Parameter estimation and errors Performance limits (bounds) of estimators Cramer-Rao lower bound

Robotic Helicopter toInspect Fukushima Reactors Purpose

Fly through damaged buildings Navigation Approach: SLAM

Install stereo sensors on craft Stereo vision 3D model Fly through 3D model

Critical Algorithm Image registration

Issue: Probability of collision How much bias in position? How much variance in position?

Analyze Accuracy of SLAM Errors in image registration

Decision: Will or will not helicopter successfully perform inspection?

Wiki Commons: Digital Globe

Fukushima Facility Building

Helicopter: http://flickrhivemind.net/Tags/apache,lego

SLAM: Simultaneous Localization and Mapping

Image Registration Errors

Errors Assume only translational

errors Δx and Δy

Higher order errors Not modeled

Asymptotic performance Bias

E(Δx) ≠ 0 and E(Δy) ≠ 0

Variance σ2 = E{(Δx+Δy)2} – E(Δx)E(Δy) >

0

Errors: Δx and Δy > 0

WikiCommons: Jazzjohn, 2012

Estimation: Accuracy and Precision

PDF: WikiCommons: PekajeTargets: www.caroline.com/teacher-resources

Target Practice 1D Error Distribution

(Bias)

(Variance)

LikelihoodFunction

ParameterValue

Truth

Performance Limits How accurate? Bias

Error about true position How precise? Variance

Error about mean of estimator What is best? Optimal

What are performance limits?

Is this best performance?

Targets: www.caroline.com/teacher-resources

Registration Errors Impact Navigation

[𝑥𝑦𝑧 ]𝑝

=[𝑥𝑦𝑧 ]𝑡𝑟𝑢𝑒

+[𝑏𝑥

𝑏𝑦

𝑏𝑧]𝑏𝑖𝑎𝑠

+[𝑛𝑥

𝑛𝑦

𝑛𝑧]𝑛𝑜𝑖𝑠𝑒

Stereo Vision

Navigation

3D Model

Image 1

Image 2[𝑥𝑦𝑧 ]

𝑝

Registration Errors Impact Navigation(Image registration errors cause 3D world model errors)

Small Bias, Small Variance

Large Bias, Small Variance

Small Bias, Large VarianceLarge Bias, Large Variance

Room 2: Fukushima Reactor

Damaged Wall Δ𝑥Δ𝑥 Δ𝑥

Enters Room 2: 3D mapping algorithm is a minimum variance, unbiased estimator (MVUE).

Room 1: Fukushima Reactor

Damaged Wall

Collides with Wall: not MVUE algo

Enters Room 2: MVUE algo

Minimum Variance, Unbiased Estimator: Cramer-Rao Lower Bound (CRLB)

Var(θ)

CRLB is a minimum variance unbiased estimator

Best

CRLBxp

E

2

2 ));(ln(

1)var(

CRLB is Best MVUE

Modeling Registration Errors and CRLB

CRLBxp

E

2

2 ));(ln(

1)var(

J=Fisher Information Matrix (FIM)

Log likelihood function as in Maximum Likelihood (ML) Estimation

)

BiasVariance

MSE = Mean Square Error used as measure of registration error

Registration, ML Estimation, and Objective Function

),(),(),( 11 nmnmfnmz ),(),(),( 2212 nmvnvmfnmz

cvnvmfnmznmfnmzvzPnm

2

,212

212

]),(),([)],(),([2

1));(log(

Log-likelihood function

f(m,n) = truth v = shift ε(m,n) = Gaussian noiseImage courtesy Matlab

Imagery

Objective Function

nm

nmDC vnvmf

vnvmfnmzvQ

, 212

, 212

),(

),(),()(

),(1 nm ),(2 nm

Deriving J(Φ) = FIM (Fisher Information)

𝜕2 ln (𝑃 (𝑧 ;𝑣 ))𝜕𝑣 𝑖

2 =𝜕𝜕𝑣 𝑖

[ 1𝜎2 ∑𝑚 ,𝑛

(𝑧 2−~𝑓 ) 𝜕

~𝑓𝜕𝑣 𝑖

]−𝐸 {𝜕2 ln (𝑃 (𝑧 ;𝑣 ))

𝜕 𝑣12 }= 1

𝜎2 ( 𝜕~𝑓

𝜕𝑣1 )2

Second partials of log likelihood

Expected value of second partials

−𝐸 {𝜕2 ln (𝑃 (𝑧 ;𝑣 ))𝜕 𝑣2

2 }= 1𝜎2 ( 𝜕

~𝑓

𝜕𝑣2 )2

−𝐸 {𝜕2 ln (𝑃 (𝑧 ;𝑣 ))𝜕𝑣1

❑𝑣2❑ }= 1

𝜎2 ( 𝜕~𝑓

𝜕𝑣1 )( 𝜕~𝑓

𝜕𝑣2 )

= =

Equating to x and to y

The Fisher Information Matrix (FIM)

= =

Given

The FIM is

𝐽 (𝑣 )= 1

𝜎2 [𝑎1 𝑎2𝑎3 𝑎4 ]

𝑎1=∑𝑚 ,𝑛

𝑓 𝑥2 (𝑚−𝑣1 ,𝑛−𝑣2) 𝑎3=∑

𝑚 ,𝑛

𝑓 𝑦2 (𝑚−𝑣1 ,𝑛−𝑣2)

𝑎2=∑𝑚 ,𝑛

𝑓 𝑥(𝑚−𝑣1 ,𝑛−𝑣2) 𝑓 𝑦(𝑚−𝑣1 ,𝑛−𝑣2)

Where

𝑟𝑒𝑐𝑎𝑙𝑙 :𝑀𝑆𝐸 (𝑣 )≥ 𝐽 (𝑣 )−1=𝐶𝑅𝐿𝐵(𝑢𝑛𝑏𝑖𝑎𝑠𝑒𝑑)

Results: Registration Error Analysis

CRLB: MSE)ASD: average square distanceDC: maximum direct correlatorPyr: multiscale gradient-basedGB: gradient-based methodProj-GB: project GBPyr-Proj: Project PyrPhase: relative phase

Flat: Estimator biasSloping: Estimator variance

Conclusion Variance & Bias of an estimator Fisher Information Cramer-Rao lower bound (CRLB)

Quantitative measure of estimator performance Application of CRLB to image registration

BACKUP: Registration Algorithms Approximate Minimum Average Square Difference Approximate Maximum Direct Correlator (DC) Linear Gradient-Based Method (GB)

Fit and solve least squares

Multiscale (Pyramid) Gradient-Based Method (Pyr) Perform GB on multiresolution pyramid Iterate from coarsest to finest resolution

Projection Gradiant-Based Method (Proj-GB)

Projection Multiscale Gradient-Based Method (Pyr-Proj) Multiresolution version of Projection Gradiant-Based Method (Proj-GB)

Relative Phase (Phase) Match phases (in Fourier Domain) Solve using weighted least squares

BACKUP: Estimator Variance Example of estimator variance