Computer Vision : CISC 4/689
Going Back a little
• Cameras.ppt
Computer Vision : CISC 4/689
Applications of RANSAC: Solution for affine parameters
• Affine transform of [x,y] to [u,v]:
• Rewrite to solve for transform parameters:
Computer Vision : CISC 4/689
Assignment
• Program-1
• info-Link
• DataNote: You can generate, bring-in, your own images from www, as long as:For n+1 levels, image must be Mr£2n+1 rows by Mc£2n+1 colsMr and Mc are any +ve integers
Sunday 10pm
Computer Vision : CISC 4/689
Another app. : Automatic Homography H Estimation
– Homographies describe image transformation of...• General scene when camera motion is rotation about camera center• Planar surfaces under general camera motion
• How to get correct correspondences without human intervention?from Hartley & Zisserman
Computer Vision : CISC 4/689
Computing a Homography
• 8 degrees of freedom in 3 x 3 matrix H, so at least n = 4 pairs of 2-D points are sufficient to determine it
• Use same basic algorithm for P (aka Direct Linear Transformation, or DLT) to compute H
– Now stacked matrix A is 2n x 9 vs. 2n x 12 for camera matrix P estimation because all points are 2-D
• 3 collinear points in either image is a degenerate configuration preventing a unique solution
Lets Side-track
Computer Vision : CISC 4/689
Estimating H: DLT Algorithm
• x0i = Hxi is an equation involving homogeneous vectors, so Hxi
and x0i need only be in the same direction, not strictly equal
• We can specify “same directionality” by using a cross product formulation:
• See Hartley & Zisserman, Chapter 3.1-3.1.1 (linked on course page) for details
Computer Vision : CISC 4/689
Texture Mapping• Needed for nice display when applying transformations (like a homography H) to
a whole image• Simple approach: Iterate over source image coordinates and apply x0 = H x to
get destination pixel location– Problem: Some destination pixels may not be “hit”, leaving holes
• Easy solution: Iterate over destination image and apply inverse transform x = H-1
x0 – Round off H-1
x0 to address “nearest” source pixel value– This ensures every destination pixel is filled in
Computer Vision : CISC 4/689
Automatic H Estimation: Feature Extraction
• Find features in pair of images using corner detection—e.g., eigenvalue threshold of:
from Hartley & Zisserman
~500 features found
Computer Vision : CISC 4/689
Automatic H Estimation: Finding Feature Matches
• Best match over threshold within square search window (here §300 pixels) using SSD or normalized cross-correlation
from Hartley & Zisserman
Computer Vision : CISC 4/689
Automatic H Estimation: Finding Feature Matches
• Best match over threshold within square search window (here §300 pixels) using SSD or normalized cross-correlation
from Hartley & Zisserman
Computer Vision : CISC 4/689
Automatic H Estimation: Initial Match Hypotheses
268 matched features (over SSD threshold) in left image pointing to locations of corresponding right image features
from Hartley & Zisserman
Computer Vision : CISC 4/689
Automatic H Estimation: Applying RANSAC
• Sampling– Size: Recall that 4 correspondences suffice to define homography, so sample size
s = 4– Choice
• Pick SSD threshold conservatively to minimize bad matches• Disregard degenerate configurations• Ensure points have good spatial distribution over image
• Distance measure– Obvious choice is symmetric transfer error:
Computer Vision : CISC 4/689
Automatic H Estimation: Outliers & Inliers after RANSAC
• 43 samples used with t = 1.25 pixels
117 outliers (² = 0.44) 151 inliersfrom Hartley & Zisserman
Computer Vision : CISC 4/689
A Short Review of Camera Calibration
Computer Vision : CISC 4/689
Pinhole Camera Terminology
Camera center/ pinhole
Principal point/image center
Image pointCamera point
Focal length
Optical axis
Image plane
Computer Vision : CISC 4/689
Calibration
• Slides (calibration.ppt)
Computer Vision : CISC 4/689
Calibration and Pose estimation example
• Recover intrinsic and extrinsic parameters of camera by using calibration board.
• 3D points are given, can find 2D image coordinates for the corresponding 3D points.
• Assume world is located at the folded lower corner, principal point is center of the image, fold is 90 degrees,
• Total length and width of board is 9in by 9in.
Next 8 slides, courtesy UCF.
Computer Vision : CISC 4/689
Matlab code
• Matlab• fx = 1.5031• fy =1.2773
• Rc = -0.0201 -0.2000 -0.9796 0.2198 0.9588 -0.1797 0.9752 -0.2189 0.0247
• Tc = (29.0725, -2.8850, 53.4196)
• camera position:( -51.0289, 19.7118 26.6985)
Computer Vision : CISC 4/689
Multi-View GeometryRelates
• 3D World Points
• Camera Centers
• Camera Orientations
Computer Vision : CISC 4/689
Multi-View GeometryRelates
• 3D World Points
• Camera Centers
• Camera Orientations
• Camera Intrinsic Parameters
• Image Points
Computer Vision : CISC 4/689
Stereo
scene pointscene point
optical centeroptical center
image planeimage plane
Computer Vision : CISC 4/689
Stereo
• Basic Principle: Triangulation– Gives reconstruction as intersection of two rays– Requires
• calibration• point correspondence
Computer Vision : CISC 4/689
Stereo Constraints
p p’?
Given p in left image, where can the corresponding point p’in right image be?
Computer Vision : CISC 4/689
Stereo Constraints
X1
Y1
Z1O1
Image plane
Focal plane
M
p p’Y2
X2
Z2O2
Epipolar Line
Epipole
Computer Vision : CISC 4/689
Stereo
• The geometric information that relates two different viewpoints of the same scene is entirely contained in a mathematical construct known as fundamental matrix.
• The geometry of two different images of the same scene is called the epipolar geometry.
Computer Vision : CISC 4/689
Stereo/Two-View Geometry
• The relationship of two views of a scene taken from different camera positions to one another
• Interpretations– “Stereo vision” generally means
two synchronized cameras or eyes capturing images
– Could also be two sequential views from the same camera in motion
• Assuming a static scene
http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo
Computer Vision : CISC 4/689
3D from two-views
There are two ways of extracting 3D from a pair of images. • Classical method, called Calibrated route, we need to calibrate both
cameras (or viewpoints) w.r.t some world coordinate system. i.e, calculate the so-called epipolar geometry by extracting the essential matrix of the system.
• Second method, called uncalibrated route, a quantity known as fundamental matrix is calculated from image correspondences, and this is then used to determine the 3D.
Either way, principle of binocular vision is triangulation. Given a single image, the 3D location of any visible object point must lie on the straight line that passes through COP and image point (see fig.). Intersection of two such lines from two views is triangulation.
Computer Vision : CISC 4/689
Mapping Points between Images
• What is the relationship between the images x, x’ of the scene point X in two views?
• Intuitively, it depends on:– The rigid transformation between cameras (derivable from the
camera matrices P, P’)– The scene structure (i.e., the depth of X)
• Parallax: Closer points appear to move more
Computer Vision : CISC 4/689
Example: Two-View Geometry
courtesy of F. Dellaert
x1 x’1
x2x’2
x3 x’3
Is there a transformation relating the points xi to x’i ?
Computer Vision : CISC 4/689
Epipolar Geometry
• Baseline: Line joining camera centers C, C’• Epipolar plane ¦: Defined by baseline and scene point X
from Hartley& Zisserman
baseline
Computer Vision : CISC 4/689
Epipolar Lines
• Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes• Epipoles e, e’: Where baseline intersects image planes
– Equivalently, the image in one view of the other camera center.
C C’from Hartley& Zisserman
Computer Vision : CISC 4/689
Epipolar Pencil
• As position of X varies, epipolar planes “rotate” about the baseline (like a book with pages)
– This set of planes is called the epipolar pencil• Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines
from Hartley& Zisserman
Computer Vision : CISC 4/689
Epipolar Constraint• Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other
view (since it’s on the epipolar plane)• 3-D point X on this ray, so image of X in other view x’ must be on l’• In other words, the epipolar geometry defines a mapping x ! l’, of points in one image to
lines in the other
from Hartley& Zisserman
C C’
x’
Computer Vision : CISC 4/689
Example: Epipolar Lines for Converging Cameras
from Hartley & ZissermanLeft view Right view
Intersection of epipolar lines = Epipole ! Indicates direction of other camera
Computer Vision : CISC 4/689
Special Case: Translation Parallel to Image Plane
Note that epipolar lines are parallel and corresponding points lie on correspond-ing epipolar lines (the latter is true for all kinds of camera motions)
Computer Vision : CISC 4/689
From Geometry to Algebra
O O’
P
pp’
Courtesy, UCF
Computer Vision : CISC 4/689
From Geometry to Algebra
O O’
P
pp’
Computer Vision : CISC 4/689
Linear Constraint:Should be able to express as matrix multiplication.