Date post: | 24-Jan-2016 |
Category: |
Documents |
View: | 218 times |
Download: | 0 times |
Camera Modelsclass 8
Multiple View GeometryComp 290-089Marc Pollefeys
Multiple View Geometry course schedule(subject to change)
Jan. 7, 9 Intro & motivation Projective 2D Geometry
Jan. 14, 16
(no class) Projective 2D Geometry
Jan. 21, 23
Projective 3D Geometry (no class)
Jan. 28, 30
Parameter Estimation Parameter Estimation
Feb. 4, 6 Algorithm Evaluation Camera Models
Feb. 11, 13
Camera Calibration Single View Geometry
Feb. 18, 20
Epipolar Geometry 3D reconstruction
Feb. 25, 27
Fund. Matrix Comp. Structure Comp.
Mar. 4, 6 Planes & Homographies Trifocal Tensor
Mar. 18, 20
Three View Reconstruction
Multiple View Geometry
Mar. 25, 27
MultipleView Reconstruction
Bundle adjustment
Apr. 1, 3 Auto-Calibration Papers
Apr. 8, 10
Dynamic SfM Papers
Apr. 15, 17
Cheirality Papers
Apr. 22, 24
Duality Project Demos
N measurements (independent Gaussian noise ) model with d essential parameters(use s=d and s=(N-d))
(i) RMS residual error for ML estimator
(ii) RMS estimation error for ML estimator
2/12/12
/1/XX̂ NdNEeres
2/12/12
//XX̂ NdNEeest
nX
X
X
SM
Error in two images
2/1
2
4
n
neres
2/1
2
4
n
neest
nNnd 4 and 28
Backward propagation of covariance
X f -1P
X -11
xT
P JJ
Over-parameterizationA
JJ 1x
TP
Jf v
Forward propagation of covariance
JJ PT
X
Monte-Carlo estimation of covariance
=1 pixel =0.5cm (Crimisi’97)
Example:
Single view geometry
Camera model
Camera calibration
Single view geom.
Pinhole camera model
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
Pinhole camera model
101
0
0
Z
Y
X
f
f
Z
fY
fX
101
01
01
1Z
Y
X
f
f
Z
fY
fX
PXx
0|I)1,,(diagP ff
Principal point offset
Tyx
T pZfYpZfXZYX )/,/(),,(
principal pointT
yx pp ),(
101
0
0
1
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
Z
Y
X
y
x
x
x
Principal point offset
101
0
0
Z
Y
X
pf
pf
Z
ZpfY
ZpfX
y
x
x
x
camX0|IKx
1y
x
pf
pf
K calibration matrix
Camera rotation and translation
C~
-X~
RX~
cam
X10
RCR
1
10
C~
RRXcam
Z
Y
X
camX0|IKx XC~
|IKRx
t|RKP C~
Rt PXx
CCD camera
1yx
xx
p
p
K
11y
x
x
x
pf
pf
m
m
K
Finite projective camera
1yx
xx
p
ps
K
1yx
xx
p
p
K
C~
|IKRP
non-singular
11 dof (5+3+3)
decompose P in K,R,C?
4p|MP 41pMC
~ MRK, RQ
{finite cameras}={P4x3 | det M≠0}
If rank P=3, but rank M<3, then cam at infinity
Camera anatomy
Camera centerColumn pointsPrincipal planeAxis planePrincipal pointPrincipal ray
Camera center
0PC
null-space camera projection matrix
λ)C(1λAX
λ)PC(1λPAPXx
For all A all points on AC project on image of A,
therefore C is camera center
Image of camera center is (0,0,0)T, i.e. undefined
Finite cameras:
1
pM 41
C
Infinite cameras: 0Md,0
d
C
Column vectors
0
0
1
0
ppppp 43212
Image points corresponding to X,Y,Z directions and origin
Row vectors
1p
p
p
0 3
2
1
Z
Y
X
y
x
T
T
T
1p
p
p0
3
2
1
Z
Y
X
w
yT
T
T
note: p1,p2 dependent on image reparametrization
The principal point
principal point
0,,,p̂ 3332313 ppp
330 Mmp̂Px
The principal axis vector
3m
camcamcam X0|IKXPx T1,0,0mMdetv 3
camcam PP k vv 4k
4p|MC~
|IKRP k
0)Rdet(
vector defining front side of camera
(direction unaffected)
vmMdetv 43 kkk camcam PP k
because
Action of projective camera on point
PXx
MdDp|MPDx 4
Forward projection
Back-projection
xPX 1PPPP
TT IPP
(pseudo-inverse)
0PC
λCxPλX
1
p-μxM
1
pM-
0
xMμλX 4
-14
-1-1
xMd -1
CD
Depth of points
C~
X~
mCXPXPT3T3T3 w
(dot product)(PC=0)
1m;0det 3 MIf , then m3 unit vector in positive direction
3m
)sign(detMPX;depth
T
w
TX X,Y,Z,T
Camera matrix decomposition
Finding the camera center
0PC (use SVD to find null-space)
432 p,p,pdetX 431 p,p,pdetY
421 p,p,pdetZ 321 p,p,pdetT
Finding the camera orientation and internal parameters
KRM (use RQ decomposition ~QR)
Q R=( )-1= -1 -1QR
(if only QR, invert)
When is skew non-zero?
1yx
xx
p
ps
K
1
arctan(1/s)
for CCD/CMOS, always s=0
Image from image, s≠0 possible(non coinciding principal axis)
HPresulting camera:
Euclidean vs. projective
homography 44
0100
0010
0001
homography 33P
general projective interpretation
Meaningfull decomposition in K,R,t requires Euclidean image and space
Camera center is still valid in projective space
Principal plane requires affine image and space
Principal ray requires affine image and Euclidean space
Cameras at infinity
00
dP
Camera center at infinity
0Mdet
Affine and non-affine cameras
Definition: affine camera has P3T=(0,0,0,1)
Affine cameras
Affine cameras
C~
rr
C~
rr
C~
rr
KC~
|IKRP3T3T
2T2T
1T1T
0
C~
r3T0 d
t
t
dt
t
t
3T
2T2T
1T1T
33T3T
32T2T
31T1T
r
C~
rr
C~
rr
K
r-C~
rr
r-C~
rr
r-C~
rr
KP
modifying p34 corresponds to moving along principal ray
Affine cameras
003T
2T2T
1T1T
0
3T
2T2T
1T1T
0
0
/r
C~
rr
C~
rr
K
r
C~
rr
C~
rr
1
/
/
KP
dddd
d
d
dd
dd
t
t
t
t
t
t
now adjust zoom to compensate
0
2T2T
1T1T
0
C~
rr
C~
rr
KPlimP
dt
t
Error in employing affine cameras
1
βrαrX
21
1
rβrαrX
321
XPXPXP t0
point on plane parallel with principal plane and through origin, then
general points
Δ
~
~
KXPx
0
0proj
d
y
x
0
affine~
~
KXPx
d
y
x
projxaffinex
0x
Affine imaging conditions
0proj0
projaffine x-xx-xd
Approximation should only cause small error
1. much smaller than d0
2. Points close to principal point (i.e. small field of view)
Decomposition of P∞
0
02x2
0
t~R~
10
x~KP
d
10
t~R~
10
x~K 02x2-1
0d
absorb d0 in K2x2
10
0R~
10
x~t~KK
10
x~Kt~R~
10
0K
10
x~t~KR~
K
02x22x2
0-12x22x202x22x2
10
0R~
10
x~K
10
t~R~
10
0KP 02x22x2
alternatives, because 8dof (3+3+2), not more
Summary parallel projection
1000
0010
0001
P canonical representation
10
0KK 22 calibration matrix
principal point is not defined
A hierarchy of affine cameras
Orthographic projection
Scaled orthographic projection
1000
0010
0001
P
10
tRH
10rr
P 21T
11T
tt
ktt
/10rr
P 21T
11T
(5dof)
(6dof)
A hierarchy of affine cameras
Weak perspective projection
ktt
y
x
/10rr
1α
αP 2
1T1
1T
(7dof)
1. Affine camera=camera with principal plane coinciding with ∞
2. Affine camera maps parallel lines to parallel lines
3. No center of projection, but direction of projection PAD=0(point on ∞)
A hierarchy of affine camerasAffine camera
ktts
y
x
A
/10rr
1α
αP 2
1T1
1T
(8dof)
1000P 2232221
1131211
tmmmtmmm
A
affine 44100000100001
affine 33P
A
Pushbroom cameras
T)(X X,Y,X,T T),,(PX wyx T)/,( wyx
Straight lines are not mapped to straight lines!(otherwise it would be a projective camera)
(11dof)
Line cameras
ZYX
pppppp
yx
232221
131211(5dof)
Null-space PC=0 yields camera center
Also decomposition c~|IRKP 22222232
Next class: Camera calibration