Date post: | 16-Dec-2015 |
Category: |
Documents |
Upload: | coleen-sullivan |
View: | 218 times |
Download: | 0 times |
Feature Extraction and Matching
Feature Tracking
Sudipta N SinhaSep 19, 2006
Outline• Feature Extraction and Matching (for Larger Motion)
– What are features ?– Tasks
• Detection: finding the feature locations • Representation: computing a compact descriptor• Matching: Finding distances in feature space.
– Algorithms: • Harris Corner Detector, SIFT.• More complex (wide-baseline correspondence)
• Tracking (for Small Motion)– Track geometric primitives (points, lines, patches, objects …)
from frame to frame in video.- High temporal coherence.- Typically required in a real-time system.
Matching comes up in all kinds of problems in computer vision
Panoramas, mosaics
Object recognition Structure from Motion ( F, T , … )
More: Detect object in clutter, Motion segmentation,
Image-based retrieval, Video mining .. (Check Papers in References)
The Correspondence Problem and Invariance
Invariance: Features need to be detected repeatedly at the same locations and the computed descriptors must be similar in-spite of the following type of changes observed in two images of the same scene.
Point Features (Interest Points)
Goal:• To detect the same
point in each imageindependently
Challenges:• Need repeatability in presence of Scale, Rotation, Affine
distortions and Illumination change
• Not all pixels are good candidates.– Texture-less regions, edges.
• Effect of noise on feature extraction.• Examples:
– Harris Corner Detector, SIFT
Harris Corner Detector
Idea: • Detect a patch which looks locally unique.• Shifting the patch in any direction will
give a large change in intensity.
Texture-less region: no change in all directions
Edge: no change along one direction.
Corner: large changes in all
direction.
A symmetric matrix represents an ellipse
Matrix is symmetric semi-definite
Harris Corner DetectorEigen-value analysisof the 2x2 matrix M:
Corners: Feature Descriptors and Matching.
• Simple Descriptor: convert a patch of n x n pixels centered at that pixel into a vector.
• Matching: SAD, SSD, ZMNCC• Invariance:
– Translation ? Yes – Rotation ? No. But the image patch could be re-sampled
using eigen-vector pair as the local coordinate frame.– Scale and Affine ? No – Brightness Change ? Yes, normalize image intensity
(ZMNCC)
Feature point in high dim feature space
Point Features: SIFT
First: Scale Invariant Feature Detection,Later: SIFT descriptors (rotational
invariance)
The SIFT Algorithm
Create Scale Space Stack : (Lowe IJCV’04)
• Intensity
• Gradient
• DoG
Images from SIFT Tutorial [Thomas F. El-Maraghi May 2004 ]
The SIFT Algorithm
• Find Local Extrema of DoG
in Scale Space.
Remove
• Low Contrast Point
• Points on Edges.
Images from SIFT Tutorial [Thomas F. El-Maraghi May 2004 ]
The SIFT Algorithm
• Descriptor represents Local Patch Appearance.
• Oriented Histograms built from Weighted Gradients.
Images from Lowe IJCV’04
SIFT: Results
Wide Baseline Matching:
Elliptical and Parallelogram features(Tuteylaar, Van Gool et. al. IJCV 2004)
Anchor point:
Traditional Corners
Wide Baseline Matching:
Elliptical and Parallelogram features(Tuteylaar, Van Gool et. al. IJCV 2004)
Anchor point:
local intensity maxima
Tracking Corners – The KLT algorithm
Main Idea: Assuming brightness constancy, try to find the new positions of some ‘salient’ image points in the second image (where the motion is small)
Steps:1. Detecting Salient Points to track (in current frame)2. Track those features in next frame
Could be done by Searching (Template matching) BUTKLT algorithm does this analytically, hence its faster !
KLT equations:
Assumption – Brightness Constancy
Find a displacement d, such that the error given by the following equation is minimized (over a tracking window )
KLT equations:
Assumption – Brightness Constancy
Find a displacement d, such that the error given by the following equation is minimized (over a tracking window )
KLT equations:
A symmetric form was later proposed by Tomasi, as follows
To estimate d, differentiate w.r.t d,
KLT equations:
Substituting Taylor Series Expansion for J(.) and I(.)
We get,
Setting derivative to zero at the minima, and re-arranging, we get
a linear system of equations for d
KLT equations
Multiscale and Iterative KLT Build Image PyramidCoarse to Fine Tracking Increases Effective Spatial Range within which features can be tracked.
View Dependent Effects : If surface patch is small, then large persective distortions can be approximated by an affine transformation
Brightness change = gain + offset (2 more parameters)
Affine KLT
Invariance to illumination
Acknowledgments
Slides/Figures were taken from –
• SIFT MATLAB tutorial - [Thomas F. El-Maraghi May 2004]
• Lecture Notes by Bill Freeman• Lecture Notes on Tracking: UWA Computer
Science, CITS 4240.• David Lowe’s SIFT papers• Stan Birchfield’s article on Symmetric
Version of KLT equations.
References and Papers• Stan Birchfield. KLT: An Implementation of the Kanade-Lucas-Tomasi Feature Tracker
• [2] Bruce D. Lucas and Takeo Kanade. An Iterative Image Registration Technique with an Application to Stereo Vision. International Joint Conference on Artificial Intelligence, pages 674-679, 1981.
• David Lowe, ‘Distinctive image features from scale-invariant keypoints’, Int. Journal of Computer Vision, 60(2):91–110, 2004.
• J. Matas, O. Chum, U. Martin, and T. Pajdla. Robust wide baseline stereo from maximally stable extremal regions. In Proc. British Machine Vision Conference, volume 1, pages 384–393, Sep 2002.
• • K. Mikolajczyk and C. Schmid. Scale and affine invariant interest point detectors. Int. Journal of
Computer Vision, 1(60):63–86, 2004• • T.Tuytelaars and L. Van Gool. Matching widely separated views based on affine invariant
regions. Int. Journal of Computer Vision, 1(59):61–85, 2004• • K. Mikolajczyk, T. Tuytelaars, C. Schmid, A. Zisserman, J. Matas, F. Schaffalitzky, T. Kadir, and
L. Van Gool. A comparison of affine region detectors. Technical Report, accepted to IJCV, 2005
• KLT src code: http://www.ces.clemson.edu/~stb/klt/
• SIFT Matlab code: see Link at http://robots.stanford.edu/cs223b04/project9.html