Final Exam
• Final exam will be comprehensive.– Midterm Exam material
– SIFT
– Object recognition
– Face recognition using eigenfaces
– Camera parameters
– Camera calibration
– Stereo
SIFT feature computation
• Steps – Scale space extrema detection (how is it different from
Harris-Laplace? different parameters)
– Keypoint localization (need to know main ideas, no equations; two thresholds, which ones?)
– Orientation assignment (how are the histograms built? multiple peaks?)
– Keypoint descriptor (how are the histograms built? partial voting, main parameters, invariance to illumination changes)
SIFT features
• Properties – Scale and rotation invariant– Highly distinctive– Partially invariant to 3D viewpoint and illumination changes– Fast and efficient computation
• Main parameters?• Matching
– How do we match SIFT features?– How do we evaluate the performance of a feature matcher?
• Applications
SIFT variations
• PCA SIFT
• SURF
• GLOH
• Need to know key ideas and steps (no need to remember exact parameter values)
• Similarities/Differences with SIFT
• Strengths/Weakeness
Object Recognition
• Model-based vs category-specific recognition– Preprocessing & Recognition
• Challenges? – Photometric effects, scene clutter, changes in shape (e.g.,
non-rigid objects), viewpoint changes
• Requirements? – Invariance, robustness
• Performance Criteria?– Efficiency (time + memory), accuracy
Object Recognition (cont’d)
• Representation schemes – advantages/disadvantages– Object centered (3D/3D or 3D/2D matching)
– Viewer centered (2D/2D matching)
• Matching schemes – advantages/disadvantages– Geometry-based
– Appearance-based
Object Recognition (cont’d)
• Main steps in matching:– Hypothesis generation
– Hypothesis verification
• Efficient hypothesis generation– Which scene features to choose?
– How to organize and search the model database?
Object Recognition using SIFT
• Main ideas and steps– Perform nearest neighbor search
– Find clusters of features (pose clustering)
– Perform verification
• Practical issues– Approximate nearest neighbors
Bag of Features
• Origins of bag of features method
• Computing Bag of Features– Feature extraction
– Learn “visual vocabulary” (e.g., K-Means clustering)
– Quantize features using “visual vocabulary”.
– Represent images by frequencies of “visual words” (bugs of features)
Bag of Features (cont’d)
• Object categorization using bags of features.– Represent objects using Bag of Features
– Classification (NN, kNN, SVM)
PCA
• Need to know steps and equations.
• What criterion does PCA minimize?
• How is the “best” low-dimensional space determined using PCA?
• What is the geometric interpretation of PCA?
• Practical issues (e.g., choosing K, computing error, standardization)
Using PCA for Face Recognition
• Represent faces using PCA – need to know steps and practical issues (e.g., AAT vs ATA)
• Face recognition using PCA (i.e., eigenfaces)– DIFS
• Face detection using PCA– DFFS
• Limitations
Camera Parameters
• Reference frames – what are they?– World
– Camera
– Image plane
– Pixel plane
• Perspective projection– Should know how to derive equations
– Matrix notation
– Properties of perspective projection
– Vanishing points, vanishing lines.
Camera Parameters
• Orthographic projection – How is related to perspective?
– Study equations
– Matrix notation
– Properties
• Weak perspective projection – How is related to perspective?
– Study equations
– Matrix notation
– Properties
• Extrinsic camera parameters – What are they and what is their meaning?– Study equations
• Intrinsic camera parameters– What are they and what is their meaning?– Study equations
• Projection matrix– What does it represent?
Camera Parameters (cont’d)
Camera Calibration
• What is the goal of camera calibration and how is it performed?
• Camera calibration using the projection matrix (study equations for step 1 only; you should remember how this method works in general)
• Direct parameter calibration (do not memorize equations but remember how they work); how is the orthogonally constraint of the rotation matrix enforced?
Stereo
• What is the goal of stereo vision?
• Triangulation principle.
• Familiarity with terminology (e.g., baseline, epipolar plane, epipolar lines, epipoles, disparity)
• Two main problems of stereo (i.e., correspondence + reconstruction)
• Recover depth from disparity – study proof.
Correspondence Problem
• What is the correspondence problem and why is it difficult?
• Main methods: intensity-based, feature-based– How do intensity-based methods work?
– Main parameters of intensity-based methods. How can we choose them?
– How do feature-based methods work?
– Comparison between intensity-based and feature-based methods
Epipolar Geometry
• Stereo parameters: extrinsic + intrinsic
• What is the epipolar constraint, why is it important?
• How is epipolar geometry represented?– Essential matrix
– Fundamental matrix
Essential Matrix
• What is the essential matrix?
• Properties of essential matrix
• Study equations
• Equation satisfied by corresponding points
Fundamental Matrix
• What is the fundamental matrix?
• Properties of fundamental matrix
• Study equations
• Equation satisfied by corresponding points
Eight-point algorithm
• What is it useful for?
• Study steps
• How is the rank(2) constraint enforced?
• Normalized eight-point algorithm
• Estimate epipoles and epipolar lines using the fundamental matrix?