Post on 05-Jan-2016
description
transcript
CSci 6971: Image Registration Lecture 5: Feature-Base Regisration
January 27, 2004
CSci 6971: Image Registration Lecture 5: Feature-Base Regisration
January 27, 2004
Prof. Chuck Stewart, RPIDr. Luis Ibanez, KitwareProf. Chuck Stewart, RPIDr. Luis Ibanez, Kitware
Image Registration Lecture 5 2
OverviewOverview
What is feature-based (point-based) registration?
Feature points The correspondence problem Solving for the transformation estimate Putting it all together: ICP Discussion and conclusion
What is feature-based (point-based) registration?
Feature points The correspondence problem Solving for the transformation estimate Putting it all together: ICP Discussion and conclusion
Image Registration Lecture 5 3
What is Feature-Based Registration?What is Feature-Based Registration?
Images are described as discrete sets of point locations associated with a geometric measurement Locations may have additional properties
such as intensities and orientations Registration problem involves two parts:
Finding correspondences between features Estimating the transformation parameters
based on these correspondences
Images are described as discrete sets of point locations associated with a geometric measurement Locations may have additional properties
such as intensities and orientations Registration problem involves two parts:
Finding correspondences between features Estimating the transformation parameters
based on these correspondences
Image Registration Lecture 5 4
Feature Examples: Range DataFeature Examples: Range Data
Range image points: (x,y,z) values Triangulated mesh Surface normals are
sometimes computed Notice:
Some information (locations) is determined directly by the sensor (“raw data”)
Some information is inferred from the data
Range image points: (x,y,z) values Triangulated mesh Surface normals are
sometimes computed Notice:
Some information (locations) is determined directly by the sensor (“raw data”)
Some information is inferred from the data
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Image Registration Lecture 5 5
Feature Examples: Vascular LandmarksFeature Examples: Vascular Landmarks
Branching points pulmonary images: Lung vessels Airway branches Retinal image
branches and cross-over points
Typically augmented (at least) with orientations of vessels meeting to form landmarks
Branching points pulmonary images: Lung vessels Airway branches Retinal image
branches and cross-over points
Typically augmented (at least) with orientations of vessels meeting to form landmarks
Image Registration Lecture 5 6
Points Along Centers of Vessels and AirwaysPoints Along Centers of Vessels and Airways
Airways and vessels modeled as tubular structures
Sample points spaced along center of tubes Note that the entire
tube is rarely used as a unit
Augmented descriptions: Orientation Radius
Airways and vessels modeled as tubular structures
Sample points spaced along center of tubes Note that the entire
tube is rarely used as a unit
Augmented descriptions: Orientation Radius
Image Registration Lecture 5 7
“Interest” Points“Interest” Points
Locations of strong intensity variation in all directions
Augmented with summary descriptions (moments) of surrounding intensity structures
Recent work in making these invariant to viewpoint and illumination.
We’ll discuss interest points during Lectures 16 and 17
Locations of strong intensity variation in all directions
Augmented with summary descriptions (moments) of surrounding intensity structures
Recent work in making these invariant to viewpoint and illumination.
We’ll discuss interest points during Lectures 16 and 17
Brown and Lowe, Int. Conf. On Computer Vision, 2003
Image Registration Lecture 5 8
Feature Points: DiscussionFeature Points: Discussion
Many different possible features Problem is reliably extracting features in all
images This is why more sophisticated features are
not used Feature extraction methods do not use all
intensity values Use of features dominates range-image
registration techniques where “features” are provided by the sensor
Many different possible features Problem is reliably extracting features in all
images This is why more sophisticated features are
not used Feature extraction methods do not use all
intensity values Use of features dominates range-image
registration techniques where “features” are provided by the sensor
Image Registration Lecture 5 9
Preamble to Feature-Based Registration: NotationPreamble to Feature-Based Registration: Notation
Set of moving image features
Set of fixed image features
Each feature must include a point location in the coordinate system of its image. It may include more
Set of correspondences
Set of moving image features
Set of fixed image features
Each feature must include a point location in the coordinate system of its image. It may include more
Set of correspondences
Image Registration Lecture 5 10
Error objective function depends on unknown transformation parameters and unknown feature correspondences Each may depend on the other!
Transformation may include mapping of more than just locations
Distance function, D, could be as simple as the Euclidean distance between location vectors.
We are using the forward transformation model.
Error objective function depends on unknown transformation parameters and unknown feature correspondences Each may depend on the other!
Transformation may include mapping of more than just locations
Distance function, D, could be as simple as the Euclidean distance between location vectors.
We are using the forward transformation model.
Mathematical FormulationMathematical Formulation
Image Registration Lecture 5 11
Correspondence ProblemCorrespondence Problem
Determine correspondences before estimating transformation parameters Based on rich description of features Error prone
Determine correspondences at the same time as estimation of parameters “Chicken-and-egg” problem
For the next few minutes we will assume a set of correspondences is given and proceed to the estimation of parameters Then we will return to the correspondence
problem
Determine correspondences before estimating transformation parameters Based on rich description of features Error prone
Determine correspondences at the same time as estimation of parameters “Chicken-and-egg” problem
For the next few minutes we will assume a set of correspondences is given and proceed to the estimation of parameters Then we will return to the correspondence
problem
Image Registration Lecture 5 12
Example: Estimating ParametersExample: Estimating Parameters
2d point locations:
Similarity transformation:
Euclidean distance:
2d point locations:
Similarity transformation:
Euclidean distance:
Image Registration Lecture 5 13
Putting This TogetherPutting This Together
Image Registration Lecture 5 14
What Do We Have?What Do We Have?
Least-squares objective function Quadratic function of each parameter We can
Take the derivative with respect to each parameter
Set the resulting gradient to 0 (vector) Solve for the parameters through matrix
inversion We’ll do this in two forms: component and
matrix/vector
Least-squares objective function Quadratic function of each parameter We can
Take the derivative with respect to each parameter
Set the resulting gradient to 0 (vector) Solve for the parameters through matrix
inversion We’ll do this in two forms: component and
matrix/vector
Image Registration Lecture 5 15
Component Derivative (a)Component Derivative (a)
Image Registration Lecture 5 16
Component Derivative (b)Component Derivative (b)
At this point, we’ve dropped the leading factor of 2. It will be eliminated when this is set to 0.
Image Registration Lecture 5 17
Component Derivatives tx and tyComponent Derivatives tx and ty
Image Registration Lecture 5 18
GatheringGathering
Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have:
Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have:
Image Registration Lecture 5 19
SolvingSolving
This is a simple equation of the form
Provided the 4x4 matrix X is full-rank (evaluate SVD) we easily solve as
This is a simple equation of the form
Provided the 4x4 matrix X is full-rank (evaluate SVD) we easily solve as
Image Registration Lecture 5 20
Matrix VersionMatrix Version
We can do this in a less painful way by rewriting the following intermediate expression in terms of vectors and matrices:
We can do this in a less painful way by rewriting the following intermediate expression in terms of vectors and matrices:
Image Registration Lecture 5 21
Matrix Version (continued)Matrix Version (continued)
This becomes
Manipulating:
This becomes
Manipulating:
Image Registration Lecture 5 22
Matrix Version (continued)Matrix Version (continued)
Taking the derivative of this wrt the transformation parameters (we didn’t cover vector derivatives, but this is fairly straightforward):
Setting this equal to 0 and solving yields:
Taking the derivative of this wrt the transformation parameters (we didn’t cover vector derivatives, but this is fairly straightforward):
Setting this equal to 0 and solving yields:
Image Registration Lecture 5 23
Comparing the Two VersionsComparing the Two Versions
Final equations are identical (if you expand the symbols)
Matrix version is easier (once you have practice) and less error prone
Sometimes efficiency requires hand-calculation and coding of individual terms
Final equations are identical (if you expand the symbols)
Matrix version is easier (once you have practice) and less error prone
Sometimes efficiency requires hand-calculation and coding of individual terms
Image Registration Lecture 5 24
Resetting the StageResetting the Stage
What we have done: Features Error function of transformation parameters
and correspondences Least-squares estimate of transformation
parameters for fixed set of correspondences
Next: ICP: joint estimation of correspondences
and parameters
What we have done: Features Error function of transformation parameters
and correspondences Least-squares estimate of transformation
parameters for fixed set of correspondences
Next: ICP: joint estimation of correspondences
and parameters
Image Registration Lecture 5 25
Iterative Closest Points (ICP) AlgorithmIterative Closest Points (ICP) Algorithm
Given an initial transformation estimate 0
t = 0 Iterate until convergence:
Establish correspondences: For fixed transformation parameter estimate, t,
apply the transformation to each moving image feature and find the closest fixed image feature
Estimate the new transformation parameters, For the resulting correspondences, estimate
t+1
Given an initial transformation estimate 0
t = 0 Iterate until convergence:
Establish correspondences: For fixed transformation parameter estimate, t,
apply the transformation to each moving image feature and find the closest fixed image feature
Estimate the new transformation parameters, For the resulting correspondences, estimate
t+1
ICP algorithm was developed almost simultaneous by at least 5 research groups in the early 1990’s.
Image Registration Lecture 5 26
Finding CorrespondencesFinding Correspondences
Map feature into coordinate system of If
Find closest point
Map feature into coordinate system of If
Find closest point
Image Registration Lecture 5 27
Finding Correspondences (continued)Finding Correspondences (continued)
Enforce unique correspondences Avoid trivial minima of objective function
due to having no correspondences Spatial data structures needed to make
search for correspondences efficient K-d trees Digital distance maps More during lectures 11-15…
Enforce unique correspondences Avoid trivial minima of objective function
due to having no correspondences Spatial data structures needed to make
search for correspondences efficient K-d trees Digital distance maps More during lectures 11-15…
Image Registration Lecture 5 28
Initialization and ConvergenceInitialization and Convergence
Initial estimate of transformation is again crucial because this is a minimization technique
Determining correspondences and estimating the transformation parameters are two separate processes With Euclidean distance metrics you can show
they are working toward the same minimum In general this is not true
Convergence in practice is sometimes problematic and the correspondences oscillate between points.
Initial estimate of transformation is again crucial because this is a minimization technique
Determining correspondences and estimating the transformation parameters are two separate processes With Euclidean distance metrics you can show
they are working toward the same minimum In general this is not true
Convergence in practice is sometimes problematic and the correspondences oscillate between points.
Image Registration Lecture 5 29
2d Retinal Example2d Retinal Example
White = vessel centerline points from one image
Black = vessel centerline points from second image
Yellow line segments drawn between corresponding points
Because of the complexity of the structure, initialization must be fairly accurate
White = vessel centerline points from one image
Black = vessel centerline points from second image
Yellow line segments drawn between corresponding points
Because of the complexity of the structure, initialization must be fairly accurate
Image Registration Lecture 5 30
ComparisonComparison
For a given transformation estimate, we can only find a new, better estimate, not the best estimate, based on the gradient step.
We then need to update the constraints and re-estimate
For a given transformation estimate, we can only find a new, better estimate, not the best estimate, based on the gradient step.
We then need to update the constraints and re-estimate
Intensity-Based Feature-Based
For given set of correspondences, we can directly (least-squares) estimate the best transformation
BUT, the transformation depends on the correspondences, so we generally need to re-establish the correspondences.
For given set of correspondences, we can directly (least-squares) estimate the best transformation
BUT, the transformation depends on the correspondences, so we generally need to re-establish the correspondences.
Image Registration Lecture 5 31
SummarySummary
Feature-based registration Feature types and properties Correspondences Least-squares estimate of parameters based
on correspondences ICP Comparison
Feature-based registration Feature types and properties Correspondences Least-squares estimate of parameters based
on correspondences ICP Comparison
Image Registration Lecture 5 32
Looking Ahead to Lecture 6Looking Ahead to Lecture 6
Introduction to ITK and the ITK registration framework.
Introduction to ITK and the ITK registration framework.