Date post: | 19-Jan-2016 |
Category: |
Documents |
Upload: | james-carr |
View: | 218 times |
Download: | 0 times |
CSci 6971: Image Registration Lecture 4: First Examples
March 3, 2005
CSci 6971: Image Registration Lecture 4: First Examples
March 3, 2005
Prof. Charlene TsaiProf. Charlene Tsai
Image Registration Lecture 4 2
OutlineOutline
Example Intensity-based registration SSD error function Image mapping Function minimization:
Gradient descent Derivative calculation
Algorithm Results and discussion
Example Intensity-based registration SSD error function Image mapping Function minimization:
Gradient descent Derivative calculation
Algorithm Results and discussion
Image Registration Lecture 4 3
Reading MaterialReading Material
Paper: Hill, Batchelor, Holden and Hawkes,
Medical Image Registration, Physics of Medicine and Biology 46 (2001) R1-R45.
Available electronically from my website Excellent introduction to registration problem,
but heavily slanted toward medical applications using similarity transformations
Paper: Hill, Batchelor, Holden and Hawkes,
Medical Image Registration, Physics of Medicine and Biology 46 (2001) R1-R45.
Available electronically from my website Excellent introduction to registration problem,
but heavily slanted toward medical applications using similarity transformations
Image Registration Lecture 4 4
Running ExampleRunning Example
MRI image registration, similarity transformation (rotated by 10 degrees, with a translation of 17mm and 13mm)
MRI image registration, similarity transformation (rotated by 10 degrees, with a translation of 17mm and 13mm)
Image Registration Lecture 4 5
Intensity-Based RegistrationIntensity-Based Registration
Use the intensities more or less directly
Compare intensities between Mapped (transformed) version
of the moving image Im (based on an estimated transformation) and
Fixed image If
Need: Pixel-by-pixel error measure Mapping technique Minimization technique
Use the intensities more or less directly
Compare intensities between Mapped (transformed) version
of the moving image Im (based on an estimated transformation) and
Fixed image If
Need: Pixel-by-pixel error measure Mapping technique Minimization technique
Image Registration Lecture 4 6
Example Error Measure: SSDExample Error Measure: SSD
Region of intersection between images
Pixel location within region
Image Registration Lecture 4 7
SSD Example: Initial AlignmentSSD Example: Initial Alignment
Image Registration Lecture 4 8
SSD: Sum of Squared Errors SSD: Sum of Squared Errors
Advantages: Simple to compute Differentiable Optimal for Gaussian error distributions
Disadvantages: Doesn’t allow varying “gain” between the
images, which may be caused by different illuminations or different camera settings
Biased by large errors in intensity E.g. caused by contrast agent injection
Advantages: Simple to compute Differentiable Optimal for Gaussian error distributions
Disadvantages: Doesn’t allow varying “gain” between the
images, which may be caused by different illuminations or different camera settings
Biased by large errors in intensity E.g. caused by contrast agent injection
Image Registration Lecture 4 9
Working in the ParametersWorking in the Parameters
Remember:
This means that to evaluate the effect of a transformation estimate what we really want to evaluate is
Remember:
This means that to evaluate the effect of a transformation estimate what we really want to evaluate is
Image Registration Lecture 4 10
Aside: The Role of the RegionAside: The Role of the Region
Observe: the region over which the transformation is evaluated depends on the parameters:
This can cause problems in practice: A transformation resulting
in no overlap leads to 0 error!
Observe: the region over which the transformation is evaluated depends on the parameters:
This can cause problems in practice: A transformation resulting
in no overlap leads to 0 error!
Image Registration Lecture 4 11
Evaluating the Objective FunctionEvaluating the Objective Function
Pixel-by-pixel evaluation within the region Apply the inverse mapping at each pixel Problem: inverse mapping of pixel does not
“land” on a discrete pixel location!
Pixel-by-pixel evaluation within the region Apply the inverse mapping at each pixel Problem: inverse mapping of pixel does not
“land” on a discrete pixel location!
Image Registration Lecture 4 12
Many Interpolation OptionsMany Interpolation Options
Nearest neighbor Bilinear (or trilinear in 3d) Spline Sinc
Nearest neighbor Bilinear (or trilinear in 3d) Spline Sinc
Image Registration Lecture 4 13
Bilinear Interpolation in Moving ImageBilinear Interpolation in Moving Image
Weighted average of 4 surrounding pixels 8 surrounding pixels in
3d Weight proportional to
distance in x and in y
Weighted average of 4 surrounding pixels 8 surrounding pixels in
3d Weight proportional to
distance in x and in y
Image Registration Lecture 4 14
Bilinear: Resulting IntensityBilinear: Resulting Intensity
Image Registration Lecture 4 15
Two Options In PracticeTwo Options In Practice
Create intensity, pixel-by-pixel, but don’t create an explicit image Im’
Create actual image Im’
Create intensity, pixel-by-pixel, but don’t create an explicit image Im’
Create actual image Im’
Image Registration Lecture 4 16
Resetting the StageResetting the Stage
We have: Formulated the SSD objective function Discussed how to evaluate it
Next step is how to minimize it with respect to the transformation parameters
We have: Formulated the SSD objective function Discussed how to evaluate it
Next step is how to minimize it with respect to the transformation parameters
Image Registration Lecture 4 17
Before ProceedingBefore Proceeding
We will estimate the parameters of the backward transformation
Abusing notation, we will minimize the equation
It should be understood (implicitly) that this is the inverse transformation and the parameter values will be different
We will estimate the parameters of the backward transformation
Abusing notation, we will minimize the equation
It should be understood (implicitly) that this is the inverse transformation and the parameter values will be different
Image Registration Lecture 4 18
Thinking Abstractly: Function MinimizationThinking Abstractly: Function Minimization
Function to minimize:
Possibilities Amoeba (simplex) methods - non-
differential Gradient / steepest descent Linearization (leading to least-
squares) Newton’s method Many more …
Function to minimize:
Possibilities Amoeba (simplex) methods - non-
differential Gradient / steepest descent Linearization (leading to least-
squares) Newton’s method Many more …
Image Registration Lecture 4 19
Gradient / Steepest DescentGradient / Steepest Descent
Compute gradient of objective function (with respect to transformation parameters), evaluated at current parameter estimate
Make tentative small change in parameters in the negative gradient direction
is called the “learning rate” Re-evaluate objective function and accept change if it is
reduced (otherwise reduce the learning rate) Continue until no further changes are possible
Compute gradient of objective function (with respect to transformation parameters), evaluated at current parameter estimate
Make tentative small change in parameters in the negative gradient direction
is called the “learning rate” Re-evaluate objective function and accept change if it is
reduced (otherwise reduce the learning rate) Continue until no further changes are possible
Image Registration Lecture 4 20
Computing the DerivativeComputing the Derivative
Issue: Images are discrete Parameters are continuous
Two methods Numerical Continuous (eventually numerical as well)
Abstract definition of parameter vector:
Issue: Images are discrete Parameters are continuous
Two methods Numerical Continuous (eventually numerical as well)
Abstract definition of parameter vector:
Image Registration Lecture 4 21
Numerical DerivativesNumerical Derivatives
Form each partial derivative by taking a small step in each parameter, i = 1,..,k:
Choice of step size can be difficult Requires k+1 function evaluations to compute
the derivative Sometimes this is the only thing you can do!
Form each partial derivative by taking a small step in each parameter, i = 1,..,k:
Choice of step size can be difficult Requires k+1 function evaluations to compute
the derivative Sometimes this is the only thing you can do!
Image Registration Lecture 4 22
Continuous Computation of DerivativeContinuous Computation of Derivative
Apply chain rule: Apply chain rule:
Intensity gradient in moving image
Change in transformation wrt change in parameters
Current error at pixel location
Image Registration Lecture 4 23
Computing Image DerivativesComputing Image Derivatives
Many ways. Simplest is pixel differences.
More sophisticated methods account for image noise
Computed at each pixel
Many ways. Simplest is pixel differences.
More sophisticated methods account for image noise
Computed at each pixel
Image Registration Lecture 4 24
Derivative In Moving ImageDerivative In Moving Image
Equation
In detail Pre-compute derivatives in moving image
Im
During minimization, map pixels back into moving image coordinate system and interpolate
Equation
In detail Pre-compute derivatives in moving image
Im
During minimization, map pixels back into moving image coordinate system and interpolate
Image Registration Lecture 4 25
Image Derivative ExampleImage Derivative Example
Image Registration Lecture 4 26
dT/ddT/d
Similarity transform:
Where
So derivative is 2x4 matrix (Jacobian):
Similarity transform:
Where
So derivative is 2x4 matrix (Jacobian):
Image Registration Lecture 4 27
Putting It All TogetherPutting It All Together
At each pixel in overlap region: Calculate intensity difference (scalar) Multiply by 1x2 intensity gradient vector
computed by mapping pixel location back to moving image
Multiply by 2x4 Jacobian of the transformation, evaluated at pixel location
Result is 1x4 gradient vector at each pixel Sum each component of vector over all pixels
At each pixel in overlap region: Calculate intensity difference (scalar) Multiply by 1x2 intensity gradient vector
computed by mapping pixel location back to moving image
Multiply by 2x4 Jacobian of the transformation, evaluated at pixel location
Result is 1x4 gradient vector at each pixel Sum each component of vector over all pixels
Image Registration Lecture 4 28
Algorithm OutlineAlgorithm Outline
Initialize transformation Repeat
Compute gradient Make step in gradient direction Update mapping equation Remap image
Until convergence
Initialize transformation Repeat
Compute gradient Make step in gradient direction Update mapping equation Remap image
Until convergence
Image Registration Lecture 4 29
InitializationInitialization
Since this is a minimization technique, an initial estimate is required,
There are many ways to generate this estimate: Identity transformation, e.g.
Prior information Different technique
Steepest descent only finds a local minimum of the objective function
We’ll revisit initialization in Lectures 16 and 17
Since this is a minimization technique, an initial estimate is required,
There are many ways to generate this estimate: Identity transformation, e.g.
Prior information Different technique
Steepest descent only finds a local minimum of the objective function
We’ll revisit initialization in Lectures 16 and 17
Image Registration Lecture 4 30
ConvergenceConvergence
Ideal is that gradient is 0. In practice, algorithm is stopped when:
Step size becomes too small Objective function change is sufficiently
small Maximum number of iterations is reached
Ideal is that gradient is 0. In practice, algorithm is stopped when:
Step size becomes too small Objective function change is sufficiently
small Maximum number of iterations is reached
Image Registration Lecture 4 31
ExampleExample
Initial errors Iteration 100 Iteration 200
Iteration 300 Final: 498 iterations
Image Registration Lecture 4 32
DiscussionDiscussion
Steepest descent is simple, but has limitations: Local minima Slow (linear) convergence
Steepest descent is simple, but has limitations: Local minima Slow (linear) convergence
Image Registration Lecture 4 33
SummarySummary
Intensity-based registration is driven by direct functions of image intensity
SSD is a common, though simple example Evaluating the SSD objective function (and
most other intensity-based functions) requires image interpolation
Gradient descent is a simple, commonly-used minimization technique
Derivatives may be calculated using either numerical approximations or differentiation of the objective function.
Intensity-based registration is driven by direct functions of image intensity
SSD is a common, though simple example Evaluating the SSD objective function (and
most other intensity-based functions) requires image interpolation
Gradient descent is a simple, commonly-used minimization technique
Derivatives may be calculated using either numerical approximations or differentiation of the objective function.
Image Registration Lecture 4 34
Looking Ahead to Lecture 5Looking Ahead to Lecture 5
Feature-based registration Topics:
Features Correspondences Least-squares estimation ICP algorithm Comparison to intensity-based registration
Feature-based registration Topics:
Features Correspondences Least-squares estimation ICP algorithm Comparison to intensity-based registration