Agenda
11/26/2012 [email protected] 2
Homogeneous Coordinates
Projective Transformation
Pinhole Camera Model
Camera Calibrations
Zhengyou Zhang
http://www.cs.unc.edu/~marc/
Homogeneous coordinates
11/26/2012 [email protected] 4
0 cbyax Ta,b,c
0,0)()( kkcykbxka TTa,b,cka,b,c ~
Homogeneous representation of lines
equivalence class of vectors, any vector is representative
Set of all equivalence classes in R3(0,0,0)T forms P2
Homogeneous representation of points
0 cbyax Ta,b,cl x , ,1x yTon if and only if
0l 11 x,y,a,b,cx,y,T 0,1,,~1,, kyxkyx
TT
The point x lies on the line l if and only if xTl=lTx=0
Homogeneous coordinates
Inhomogeneous coordinates Tyx,
, ,x y zT
but only 2DOF
Points from lines and vice-versa
11/26/2012 [email protected] 5
l'lx
Intersections of lines
The intersection of two lines and is l l'
Line joining two points
The line through two points and is x'xl x x'
Example
1x
1y
Ideal points and the line at infinity
11/26/2012 [email protected] 6
T0,,l'l ab
Intersections of parallel lines
TTand ',,l' ,,l cbacba
Example
1x 2x
Ideal points T0,, 21 xx
Line at infinity T1,0,0l
l22RP
tangent vector
normal direction
ab ,
ba,
Note that in P2 there is no distinction
between ideal points and others
A model for the projective plane
11/26/2012 [email protected] 7
exactly one line through two points
exactly one point at intersection of two lines
Duality
11/26/2012 [email protected] 8
x l
0xl T0lx T
l'lx x'xl
Duality principle:
To any theorem of 2-dimensional projective geometry
there corresponds a dual theorem, which may be
derived by interchanging the role of points and lines in
the original theorem
Projective transformations
11/26/2012 [email protected] 11
A projectivity is an invertible mapping h from P2 to itself
such that three points x1,x2,x3 lie on the same line if and
only if h(x1),h(x2),h(x3) do.
Definition:
A mapping h:P2P2 is a projectivity if and only if there
exist a non-singular 3x3 matrix H such that for any point
in P2 reprented by a vector x it is true that h(x)=Hx
Theorem:
Definition: Projective transformation
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
xx' Hor
8DOF
projectivity=collineation=projective transformation=homography
Mapping between planes
11/26/2012 [email protected] 12
central projection may be expressed by x’=Hx
(application of theorem)
Removing projective distortion
11/26/2012 [email protected] 13
333231
131211
3
1
'
''
hyhxh
hyhxh
x
xx
333231
232221
3
2
'
''
hyhxh
hyhxh
x
xy
131211333231' hyhxhhyhxhx
232221333231' hyhxhhyhxhy
select four points in a plane with know coordinates
(linear in hij)
(2 constraints/point, 8DOF 4 points needed)
11/26/2012 [email protected] 14
11/26/2012 [email protected] 15
http://www.youtube.com/watch?v=NOU1lXR4JKo
http://www.youtube.com/watch?v=-tuS3GbHtV8
11/26/2012 [email protected] 16
http://www.youtube.com/watch?v=i7woG0pqFjs
http://www.youtube.com/watch?v=zFMOtzpkgQo
Transformation for lines
11/26/2012 [email protected] 18
Transformation for lines
ll' -TH
xx' HFor a point transformation
Isometries
11/26/2012 [email protected] 19
1100
cossin
sincos
1
'
'
y
x
t
t
y
x
y
x
1
11
orientation preserving:
orientation reversing:
x0
xx'
1
tT
RHE IRR T
special cases: pure rotation, pure translation
3DOF (1 rotation, 2 translation)
Invariants: length, angle, area
Similarities
11/26/2012 [email protected] 20
1100
cossin
sincos
1
'
'
y
x
tss
tss
y
x
y
x
x0
xx'
1
tT
RH
sS IRR T
also know as equi-form (shape preserving)
metric structure = structure up to similarity (in literature)
4DOF (1 scale, 1 rotation, 2 translation)
Invariants: ratios of length, angle, ratios of areas, parallel lines
Affine transformations
11/26/2012 [email protected] 21
11001
'
'
2221
1211
y
x
taa
taa
y
x
y
x
x0
xx'
1
tT
AH A
non-isotropic scaling! (2DOF: scale ratio and orientation)
6DOF (2 scale, 2 rotation, 2 translation)
Invariants: parallel lines, ratios of parallel lengths, ratios of areas
DRRRA
2
1
0
0
D
Projective transformations
11/26/2012 [email protected] 22
xv
xx'
vP T
tAH
Action non-homogeneous over the plane
8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)
Invariants: cross-ratio of four points on a line (ratio of ratio)
T21,v vv
vv
sPAS TTTT v
t
v
0
10
0
10
t AIKRHHHH
Ttv RKA s
K 1det Kupper-triangular, decomposition unique (if chosen s>0)
Overview Transformations
11/26/2012 [email protected] 23
100
2221
1211
y
x
taa
taa
100
2221
1211
y
x
tsrsr
tsrsr
333231
232221
131211
hhh
hhh
hhh
100
2221
1211
y
x
trr
trr
Projective
8dof
Affine
6dof
Similarity
4dof
Euclidean
3dof
Concurrency, collinearity,
order of contact (intersection,
tangency, inflection, etc.),
cross ratio
Parallellism, ratio of areas,
ratio of lengths on parallel
lines (e.g midpoints), linear
combinations of vectors
(centroids).
The line at infinity l∞
Ratios of lengths, angles.
The circular points I,J
lengths, areas.
Line at infinity
11/26/2012 [email protected] 24
2211
2
1
2
1
0v
xvxv
x
x
x
x
v
AAT
t
000 2
1
2
1
x
x
x
x
v
AAT
t
Projective Transformation : Line at infinity becomes finite,
allows to observe vanishing points, horizon,
Affine Transformation : Line at infinity stays at infinity
The line at infinity
11/26/2012 [email protected] 25
v1 v2
l1
l2 l4
l3
l∞
21 vvl
211 llv
432 llv
The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity
l
1
0
0
1t
0ll
A
AH
TT
A
Recovery of affine properties from images
11/26/2012 [email protected] 26
Projective transformation Affine rectification
Affine transformation
Projective 3D geometry
11/26/2012 [email protected] 28
vTv
tAProjective
15dof
Affine
12dof
Similarity
7dof
Euclidean
6dof
Intersection and tangency
Parallellism of planes,
Volume ratios, centroids,
The plane at infinity π∞
The absolute conic Ω∞
Volume
10
tAT
10
tRT
s
10
tRT
Pinhole Camera Model
11/26/2012 [email protected] 30
TT ZfYZfXZYX )/,/(),,(
101
0
0
1
Z
Y
X
f
f
Z
fY
fX
Z
Y
X
101
01
01
1Z
Y
X
f
f
Z
fY
fX
Pinhole Camera Model
11/26/2012 [email protected] 31
camX0|IKx
1
y
x
pf
pf
K
Principal point offset Calibration Matrix
Camera rotation and translation
11/26/2012 [email protected] 33
C~
-X~
RX~
cam
X10
RCR
1
10
C~
RRXcam
Z
Y
X camX0|IKx
XC~
|IKRx
t|RKP C~
Rt
PXx
11/26/2012 [email protected] 37
Correcting Radial Distortion of Camera with
Rotation Angle
11/26/2012 [email protected] 39
2 4 6
1 2 3
2 4 6
1 2 3
( )(1 )cos
( )(1 )sin
u x d x d d d
d y d d d
x c x c k r k r k r
y c k r k r k r
2 4 6
1 2 3
2 4 6
1 2 3
( )(1 )sin
( )(1 )cos
u y d x d d d
d y d d d
y c x c k r k r k r
y c k r k r k r
0 1 2
6 7 1
u up
u u
m x m y mx
m x m y
3 4 5
6 7 1
u up
u u
m x m y my
m x m y
11/26/2012 [email protected] 40
Camera Models
11/26/2012 [email protected] 42
“An empirical evaluation of factors influencing camera calibration accuracy using three publicly
available techniques”,Wei Sun and Jeremy R. Cooperstock
Machine Vision and Applications Volume 17, Number 1 (2006), 51-67,
Roger Y. Tsai
11/26/2012 [email protected] 43
"An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision", Roger Y. Tsai,
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL,
1986, pages 364-374,
Zhengyou Zhang
11/26/2012 [email protected] 44
Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision, Corfu, Greece, pages 666-673, September 1999.
Camera Calibration Toolbox
11/26/2012 [email protected] 58
http://www.vision.caltech.edu/bouguetj/calib_doc/
Robust Multi-camera Calibration
11/26/2012 [email protected] 61
http://graphics.stanford.edu/~vaibhav/projects/calib-
cs205/cs205.html
ARToolKit Camera Calibration
11/26/2012 [email protected] 64
http://www.hitl.washington.edu/artoolkit/
OpenCV
11/26/2012 [email protected] 65
http://opencv.willowgarage.com/documentation/
camera_calibration_and_3d_reconstruction.html
OpenCV 2.1 Camera Calibration
11/26/2012 [email protected] 68
http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html
OpenCV 2.1 Camera Calibration
11/26/2012 [email protected] 69
http://mmlab.disi.unitn.it/wiki/index.php/Camera_Calibration_Tool_in_OpenCV
OpenCV 2.1 Camera Calibration
11/26/2012 [email protected] 70
xcorrected = x + [2P1xy + P2(r2 + 2x
2)]
ycorrected = y + [P1(r2 + 2y
2) + 2P2xy]
Using the Polynomial Distortion Model to Correct Tangential Distortion The polynomial distortion model uses two parameters, P1 and P2, to characterize tangential distortion.
The distortion model for tangential distortion can be represented as:
http://zone.ni.com/reference/en-XX/help/372916L-01/nivisionconcepts/spatial_calibration_indepth/
xcorrected = x(1+ K1r2+ K2r
4 + K3r
6 + Knr
(n × 2))
ycorrected = y(1+ K1r2+ K2r
4 + K3r
6 + Knr
(n × 2))
Brown, D. C. Decentric distortion of lenses.
Journal of Photogrammetric Engineering and Remote Sensing, 32(3), 444-462, 1966.
OpenCV 2.1 Camera Calibration
11/26/2012 [email protected] 71
http://www.nowpublishers.com/product.aspx?product=CGV&doi=0600000001§ion=x1-10r1
11/26/2012 [email protected] 80
http://dprg.geomatics.ucalgary.ca/system/files/AKAM_ENGO667_CH_1Handouts.pdf
11/26/2012 [email protected] 81
11/26/2012 [email protected] 82
11/26/2012 [email protected] 83
11/26/2012 [email protected] 85
Capture chessboard image
11/26/2012 [email protected] 86
Chessboard no of row and col
10 : no of calibration images
Auto : automatically add images after 3*150 milliseconds
Start : start calibration
Calibration parameters
11/26/2012 [email protected] 87
Focal length f_x, f_y
Center c_x, c_y
Radial : k1, k2, k3(check box)
Tangential : p1, p2
Und : Undistorted live image
Filenames
11/26/2012 [email protected] 89
Capture072309511500
month day hour min sec index
Corner072309511500
Image0723095115.xml
Object0723095115.xml
Distortion0723095115.xml
Intrinsics0723095115.xml
Image & Object xml
11/26/2012 [email protected] 90
<?xml version="1.0"?>
<opencv_storage>
<Image0723095127 type_id="opencv-matrix">
<rows>480</rows>
<cols>2</cols>
<dt>f</dt>
<data>
2.25150803e+002 1.92720139e+002 2.42047302e+002 1.88396301e+002
2.59520660e+002 1.83857040e+002 2.77741913e+002 1.79775314e+002
2.96598541e+002 1.75898483e+002 3.15514801e+002 1.72283661e+002
<?xml version="1.0"?>
<opencv_storage>
<Object0723095127 type_id="opencv-matrix">
<rows>480</rows>
<cols>3</cols>
<dt>f</dt>
<data>
0. 1.37500000e+001 0. 2.75000000e+000 1.37500000e+001 0.
5.50000000e+000 1.37500000e+001 0. 8.25000000e+000 1.37500000e+001
0. 11. 1.37500000e+001 0. 1.37500000e+001 1.37500000e+001 0.
Distortion & Intrinsics xml
11/26/2012 [email protected] 91
<?xml version="1.0"?>
<opencv_storage>
<Distortion0723095127 type_id="opencv-matrix">
<rows>4</rows>
<cols>1</cols>
<dt>f</dt>
<data>
-4.46110874e-001 2.08227739e-001 9.08007473e-003 -3.17067979e-003</data></Distortion0723095127>
</opencv_storage>
<?xml version="1.0"?>
<opencv_storage>
<Intrinsics0723095127 type_id="opencv-matrix">
<rows>3</rows>
<cols>3</cols>
<dt>f</dt>
<data>
2.66292694e+002 0. 3.24350769e+002 0. 2.58133820e+002
2.33235382e+002 0. 0. 1.</data></Intrinsics0723095127>
</opencv_storage>