Date post: | 22-Dec-2015 |
Category: |
Documents |
View: | 217 times |
Download: | 1 times |
ComputerVision
Calibration
Marc PollefeysCOMP 256
Read F&P Chapter 2Some slides/illustrations from Ponce, Hartley & Zisserman
ComputerVision
2
Jan 16/18 - Introduction
Jan 23/25 Cameras Radiometry
Jan 30/Feb1 Sources & Shadows Color
Feb 6/8 Linear filters & edges Texture
Feb 13/15 Multi-View Geometry Stereo
Feb 20/22 Optical flow Project proposals
Feb27/Mar1 Affine SfM Projective SfM
Mar 6/8 Camera Calibration Segmentation
Mar 13/15 Springbreak Springbreak
Mar 20/22 Fitting Prob. Segmentation
Mar 27/29 Silhouettes and Photoconsistency
Linear tracking
Apr 3/5 Project Update Non-linear Tracking
Apr 10/12 Object Recognition Object Recognition
Apr 17/19 Range data Range data
Apr 24/26 Final project Final project
Tentative class schedule
ComputerVision
3
PreviouslyHierarchy of 3D transformations
vTv
tAProjective15dof
Affine12dof
Similarity7dof
Euclidean6dof
Intersection and tangency
Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞
Angles, ratios of length
The absolute conic Ω∞
Volume
10
tAT
10
tRT
s
10
tRT
ComputerVision
4
Camera calibration
Compute relation between pixels and rays in space
?
ComputerVision
5
Pinhole camera
ComputerVision
6
Pinhole camera model
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
linear projection in homogeneous coordinates!
homogeneous coordinates
non-homogeneous coordinates
ComputerVision
7
Pinhole camera model
101
0
0
Z
Y
X
f
f
Z
fY
fX
101
01
01
1Z
Y
X
f
f
Z
fY
fX
PXx
0|I)1,,(diagP ff
ComputerVision
8
Principal point offset
Tyx
T pZfYpZfXZYX )/,/(),,(
principal pointT
yx pp ),(
101
0
0
1
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
Z
Y
X
y
x
x
x
ComputerVision
9
Principal point offset
101
0
0
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
y
x
x
x
camX0|IKx
1y
x
pf
pf
K calibration matrix
ComputerVision
11
Object motion
ComputerVision
12
Camera motion
ComputerVision
13
CCD camera
1yy
xx
pp
K
11y
x
y
x
pfpf
mm
K
ComputerVision
14
General projective camera
1yx
xx
p
ps
K
1yx
xx
p
p
K
t|IKRP
non-singular
11 dof (5+3+3)
t|RKP
intrinsic camera parametersextrinsic camera parameters
ComputerVision
15
Camera matrix decomposition
Finding the camera center
0PC (use SVD to find null-space)
Finding the camera orientation and internal parameters
KR (use RQ decomposition ~QR)
Q R=( )-1= -1 -1QR
(if only QR, invert)
PXλC)P(Xx (for all X and λ C must be camera center)
t|RKP
ComputerVision
16
Affine cameras
ComputerVision
17
Radial distortion
• Due to spherical lenses (cheap)• Model:
R
yxyxKyxKyx ...))()(1(),( 222
2
22
1R
http://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpgstraight lines are not straight anymore
pincushion dist.
barrel dist.
ComputerVision
18
Radial distortion example
ComputerVision
19
Camera model
Relation between pixels and rays in space
?
ComputerVision
20
Projector model
Relation between pixels and rays in space(dual of camera)
(main geometric difference is vertical principal point offset to reduce keystone effect)
?
ComputerVision
21
Meydenbauer camera
vertical lens shiftto allow direct ortho-photographs
ComputerVision
22
Action of projective camera on points and lines
forward projection of line
μbaμPBPAμB)P(AμX
back-projection of line
lPT
PXlX TT PX x0;xlT
PXx projection of point
ComputerVision
23
Action of projective camera on conics and quadricsback-projection to cone
CPPQ Tco 0CPXPXCxx TTT
PXx
projection of quadric
TPPQC ** 0lPPQlQ T*T*T
lPT
ComputerVision
24
ii xX ? P
Resectioning
ComputerVision
25
Direct Linear Transform (DLT)
ii PXx ii PXx
rank-2 matrix
ComputerVision
26
Direct Linear Transform (DLT)
Minimal solution
Over-determined solution
5½ correspondences needed (say 6)
P has 11 dof, 2 independent eq./points
n 6 points
Apminimize subject to constraint
1p
use SVD
ComputerVision
27
Singular Value Decomposition
XXVT XVΣ T XVUΣ T
Homogeneous least-squares
TVUΣA
1X AXmin subject to nVX solution
TVΣU
X
ComputerVision
28
Degenerate configurations
(i) Points lie on plane or single line passing through projection center
(ii) Camera and points on a twisted cubic
ComputerVision
29
Scale data to values of order 1
1. move center of mass to origin2. scale to yield order 1 values
Data normalization
D3
D2
ComputerVision
30
Line correspondences
Extend DLT to lines
ilPT
ii 1TPXl
(back-project line)
ii 2TPXl (2 independent eq.)
ComputerVision
31
Geometric error
ComputerVision
32
Gold Standard algorithm
ObjectiveGiven n≥6 2D to 2D point correspondences {Xi↔xi’}, determine the Maximum Likelyhood Estimation of P
Algorithm
(i) Linear solution:
(a) Normalization:
(b) DLT
(ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error:
(iii) Denormalization:
ii UXX~ ii Txx~
UP~
TP -1
~ ~~
ComputerVision
33
Calibration example
(i) Canny edge detection(ii) Straight line fitting to the detected edges(iii) Intersecting the lines to obtain the images corners
typically precision <1/10
(H&Z rule of thumb: 5n constraints for n unknowns)
ComputerVision
34
Errors in the world
Errors in the image and in the world
ii XPx
iX
Errors in the image
iPXx̂
i
(standard case)
ComputerVision
35
Restricted camera estimation
Minimize geometric error impose constraint through parametrization
Find best fit that satisfies• skew s is zero• pixels are square • principal point is known• complete camera matrix K is known
Minimize algebraic error assume map from param q P=K[R|-RC], i.e. p=g(q)minimize ||Ag(q)||
ComputerVision
36
Restricted camera estimation
Initialization • Use general DLT• Clamp values to desired values, e.g. s=0, x= y
Note: can sometimes cause big jump in error
Alternative initialization• Use general DLT• Impose soft constraints
• gradually increase weights
ComputerVision
37
ComputerVision
38
Image of absolute conic
ComputerVision
39
A simple calibration device
(i) compute H for each square (corners (0,0),(1,0),(0,1),(1,1))
(ii) compute the imaged circular points H(1,±i,0)T
(iii) fit a conic to 6 circular points(iv) compute K from through cholesky factorization
(≈ Zhang’s calibration method)
ComputerVision
40
Some typical calibration algorithmsTsai calibration
Zhangs calibration
http://research.microsoft.com/~zhang/calib/
Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000.
Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision (ICCV'99), Corfu, Greece, pages 666-673, September 1999.
http://www.vision.caltech.edu/bouguetj/calib_doc/
ComputerVision
41
Sequential SfM
• Initialize motion from two images• Initialize structure• For each additional view
– Determine pose of camera– Refine and extend structure
• Refine structure and motion
ComputerVision
42
Initial projective camera motion
• Choose P and P´compatible with F
Reconstruction up to projective ambiguity
(reference plane;arbitrary)
•Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
Same for more views?
different projective basis
ComputerVision
43
Initializing projective structure
• Reconstruct matches in projective frame by minimizing the reprojection error
Non-iterative optimal solution •Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
ComputerVision
44
Projective pose estimation
• Infere 2D-3D matches from 2D-2D matches
• Compute pose from (RANSAC,6pts)
F
X
x
Inliers: inx,X x X DD iii P
•Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
ComputerVision
45
• Refining structure
• Extending structure2-view triangulation
X~
P
1
3
(Iterative linear)
•Initialize motion•Initialize structure•For each additional view
•Determine pose of camera•Refine and extend structure
•Refine structure and motion
Refining and extending structure
ComputerVision
46
Refining structure and motion
• use bundle adjustment
Also model radial distortion to avoid bias!
ComputerVision
47
Metric structure and motion
Note that a fundamental problem of the uncalibrated approach is that it fails if a purely planar scene is observed (in one or more views)
(solution possible based on model selection)
use self-calibration (see next class)
ComputerVision
48
Dealing with dominant planes
ComputerVision
49
PPPgric
HHgric
ComputerVision
50
Farmhouse 3D models
(note: reconstruction much larger than camera field-of-view)
ComputerVision
51
Application: video augmentation
ComputerVision
52
Next class: Segmentation
Reading: Chapter 14