Date post: | 22-Dec-2015 |
Category: |
Documents |
View: | 217 times |
Download: | 1 times |
Miguel Lourenço, João P. Barreto, Abed Malti
Institute for Systems and Robotics,
Faculty of Science and Technology
University of Coimbra, Portugal
Feature Detection and Matching in Images with Radial Distortion
Presentation Outline
SIFT Features – brief overview
RD problems in keypoint detection and matching Theoretical reasoning Experimental validation
Improvement to the SIFT algorithm to enhance it with RD
Real experiments – a comparison study
Motion estimation and 3D reconstruction in endoscopic images
Name / Location / Date Slide 2
Motivation for keypoint detection and matching
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 3
Point correspondence across multiple views Camera calibration Sparse 3D reconstruction Recover camera/robot motion Visual Slam
Representation of image content Image retrieval applications Recognition tasks (e.g. Voc-tree) Image compression
Partioning of the descriptor space
SIFT Features (Lowe, IJCV 2004 – 6725 citations on google scholar)
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 4
SIFT is probably the most broadly used algorithm for keypoint detection and matching
How does SIFT work ? Image salient points detected in a scale space framework
Incr
ease
sca
le
Gaussian pyramid DoG pyramid
(x,y,sigma)
SIFT Features (Lowe, IJCV 2004 – 6725 citations on Scholar google)
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 5
SIFT is probably the most broadly used algorithm for keypoint detection and matching
How does SIFT work ? Image salient points detected in a scale space framework SIFT descriptor is computed based on local image gradient on a scale and rotation normalized
patch
Problem statement (1/2)
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 6
XX
ZZO'O'
Q
What is radial distortion? Bending of the light rays pulling image points towards the center along
radial direction
OO
Cameras with radial lens distortion are often used in computer and robotic vision applications
Mini-lens Fish-eye lensBoroscope
Problem statement (2/2)
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 7
SIFT is invariant to rotation and scale but it is not invariant to RD
Our Contribution: Modifications to the original SIFT for invariance to image RD
Assumptions: RD can be fairly described by the division model (Fitzgibbon, CVPR 2001)
RD is roughly known ( e.g. line stretching ) (Barreto, CVIU 2006)
336 correct matches421 correct matches
Tracking RD effects in SIFT
How does RD affect the SIFT algorithm? Study using images with artificially added distortion
Isolate the RD effect in SIFT detection and matchingReliable ground truth
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 8
Improvement to the SIFT algorithm to handle RD issues
Results on real imagery
RD = 0% RD = 15% RD = 35% RD = 55%
How does RD affect keypoint detection?
Repeatability of keypoint detection decreases with increasing distortion
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 9
Filt
erin
g bo
unds
Regular DoG pyramid ‘Distorted’ DoG pyramid
Small features (fine scale) tend to disappear during the blurring process Coarse features tend to be detected at finer levels of scale Flat regions (e.g. edges) start gain to strong gradient variations
Proposed Solution: Adaptive smoothing
We can avoid the reconstruction artifacts by using an adaptive filter
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 10
Radial distortion must be removed before the Gaussian smoothing
Rectification (~ 1.5 seconds in Matlab)
Standard vs Adaptive Gaussian smoothing
Inherent properties of the standard Gaussian filter Decouple the convolution mask in X and Y directions
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 11
Advantages of the Simplified Adaptive Filter Shape only depends on the radius of the convolution window Isotropic filter that can be decoupled for each image radius
Simplification of the adaptive filter
Detection repeatability (synthetic adding of RD)
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 12
Better Repeatability results for keypoint detection
Repeatability
More robust to calibration errors
Error in calibration
Lower computational time than image rectification
Computational time
How does RD affect matching?
RD modifies the local structures in the image and by consequence the gradients are affected
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 13
Changes in local gradients of the image deteriorates SIFT descriptor performance
Proposed solution: Compute gradients in the distorted image and perform implicit correction using the jacobian matrix of the distortion function
Matching evaluation (synthetic adding of RD)
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 14
Compressive effect adds new contributions to the descriptor that do not occur in undistorted views
The matching performance can be improved by correcting image gradients before building the descriptor
Implicit gradient correction outperforms explicit image rectification for distortion amounts up to 25%.
Implicit gradient correction SIFT in Rectified Images SIFT in RD Images
Experiments with Real Images
Planar scenes for repeatability test and scenes with depth variation for motion estimation
Firewire camera with regular lens (~ 10% ) of distortion
Dragonfly camera with mini lens (~ 25% ) of distortion
Firewire camera with fish-eye lens (~ 45% ) of distortion
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 15
Planar images
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 16
10% distortion
25% distortion
45% distortion
SIFT in RD Images SIFT in Rectified Images Our method
176 matches 294 matches 364 matches
201 matches 310 matches 401matches
112 matches 253 matches 326 matches
Motion recovery / Sparse 3D reconstruction
Scenes with depth variation where wrong matches are discarded using epipolar geometric constraints
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 17
10% distortion
25% distortion
45% distortion
Main Scene
Experimental evaluation
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 18
Main Scene Number of Inliers RMS rotation angle 3D reconstruction Inliers Distribution
Conclusions / Future Work
We proposed a set of modifications to the original SIFT algorithm (RD-SIFT) for achieving invariance to radial distortion. The additional computational overhead is minimum
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 19
RD – SIFT proved to be superior to explicit image correction Better repeatability and retrieval performance Less computational overhead Increased robustness to calibration errors
Future Work Extend the approach to other keypoint detectors (e.g. MSER and SURF) Real-time implementation using GPGPU (to make available to the
community) Get rid of calibration dependence
Name / Location / Date Slide 20
THANKS FOR COMING
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 21
Detection after explicit RD correction
Correct the radial distortion via image rectification
Miguel Lourenço – Anchorage, Alaska – ICRA 2010 Slide 22
Rectification (~ 1.5 seconds in Matlab)
Drawbacks of this approach Signal reconstruction introduces artifacts affecting SIFT performance
Image 1.5x Image (Bilinear) 1.5x Image (Bicubic)