Post on 24-Jun-2015
transcript
Computer vision: models, learning and inference
Chapter 14 The pinhole camera
Please send errata to s.prince@cs.ucl.ac.uk
2
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
2Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Motivation
3Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Sparse stereo reconstruction
Compute the depth at a set of sparse matching points
Pinhole camera
4Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Real camera image is inverted
Instead model impossible but more convenient virtual image
Pinhole camera terminology
5Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Normalized Camera
6Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
By similar triangles:
Focal length parameters
7Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Can model both• the effect of the distance to the focal plane• the density of the receptors
with a single focal length parameter f
In practice, the receptors may not be square:
So use different focal length parameter for x and y dirns
Focal length parameters
8Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Offset parameters
9Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Current model assumes that pixel (0,0) is where the principal ray strikes the image plane (i.e. the center)
• Model offset to center
• Finally, add skew parameter• Accounts for image plane being not exactly perpendicular to
the principal ray
Skew parameter
10Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Position w=(u,v,w)T of point in the world is generally not expressed in the frame of reference of the camera.
• Transform using 3D transformation
or
Position and orientation of camera
11Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Point in frame of reference of camera
Point in frame of reference of world
• Intrinsic parameters (stored as intrinsic matrix)
• Extrinsic parameters
Complete pinhole camera model
12Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
For short:
Add noise – uncertainty in localizing feature in image
Complete pinhole camera model
13Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Radial distortion
14Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
15
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
15Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problem 1: Learning extrinsic parameters (exterior orientation)
16Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
Problem 2 – Learning intrinsic parameters (calibration)
17Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
Calibration
18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Use 3D target with known 3D points
Problem 3 – Inferring 3D points (triangulation / reconstruction)
19Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
Solving the problems
20Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• None of these problems can be solved in closed form
• Can apply non-linear optimization to find best solution but slow and prone to local minima
• Solution – convert to a new representation (homogeoneous coordinates) where we can solve in closed form.
• Caution! We are not solving the true problem – finding global minimum of wrong problem. But can use as starting point for non-linear optimization of true problem
21
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
21Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Homogeneous coordinates
22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Convert 2D coordinate to 3D
To convert back
Geometric interpretation of homogeneous coordinates
23Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Pinhole camera in homogeneous coordinates
24Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Camera model:
In homogeneous coordinates:
(linear!)
Pinhole camera in homogeneous coordinates
25Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Writing out these three equations
Eliminate l to retrieve original equations
Adding in extrinsic parameters
26Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Or for short:
Or even shorter:
27
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
27Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problem 1: Learning extrinsic parameters (exterior orientation)
28Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
29
Exterior orientation
29Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Start with camera equation in homogeneous coordinates
Pre-multiply both sides by inverse of camera calibration matrix
30
Exterior orientation
30Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Start with camera equation in homogeneous coordinates
Pre-multiply both sides by inverse of camera calibration matrix
31
Exterior orientation
31Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
The third equation gives us an expression for l
Substitute back into first two lines
32
Exterior orientation
32Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Linear equation – two equations per point – form system of equations
33
Exterior orientation
33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Minimum direction problem of the form , Find minimum of subject to .
To solve, compute the SVD and then set to the last column of .
34
Exterior orientation
34Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Now we extract the values of and from .
Problem: the scale is arbitrary and the rows and columns of the rotation matrix may not be orthogonal.
Solution: compute SVD and then choose .
Use the ratio between the rotation matrix before and after to rescale
Use these estimates for start of non-linear optimisation.
35
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
35Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problem 2 – Learning intrinsic parameters (calibration)
36Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
Calibration
37Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
One approach (not very efficient) is to alternately
• Optimize extrinsic parameters for fixed intrinsic
• Optimize intrinsic parameters for fixed extrinsic
Then use non-linear optimization.
Intrinsic parameters
38Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Maximum likelihood approach
This is a least squares problem.
Intrinsic parameters
39Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
The function is linear w.r.t. intrinsic parameters. Can be written in form
Now solve least squares problem
40
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
40Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Problem 3 – Inferring 3D points (triangulation / reconstruction)
41Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Use maximum likelihood:
Reconstruction
42Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Write jth pinhole camera in homogeneous coordinates:
Pre-multiply with inverse of intrinsic matrix
Reconstruction
43Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Last equations gives
Substitute back into first two equations
Re-arranging get two linear equations for [u,v,w]
Solve using >1 cameras and then use non-linear optimization
44
Structure
• Pinhole camera model• Three geometric problems• Homogeneous coordinates• Solving the problems– Exterior orientation problem– Camera calibration– 3D reconstruction
• Applications
44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Depth from structured light
45Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Depth from structured light
46Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Depth from structured light
47Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Shape from silhouette
48Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Shape from silhouette
49Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Shape from silhouette
50Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
Conclusion
51Computer vision: models, learning and inference. ©2011 Simon J.D. Prince
• Pinhole camera model is a non-linear function who takes points in 3D world and finds where they map to in image
• Parameterized by intrinsic and extrinsic matrices
• Difficult to estimate intrinsic/extrinsic/depth because non-linear
• Use homogeneous coordinates where we can get closed from solutions (initial solns only)