Post on 09-Aug-2020
transcript
Silvio Savarese
• Recapofcameramodels• Cameracalibrationproblem• Cameracalibrationwithradialdistortion• Example
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Reading: [FP] Chapter 1 “Geometric Camera Calibration”[HZ] Chapter 7 “Computation of Camera Matrix P”
Lecture3CameraModels2&CameraCalibration
P '3×1 =M Pw = K3×3 R T"#
$% 3×4
Pw4×1
=
m1
m2
m3
!
"
####
$
%
&&&&
PW =m1PWm2PWm3PW
!
"
####
$
%
&&&&
→ P 'E = (m1Pwm3Pw
, m2Pwm3Pw
)E
f
Projective camera
Ow
kw
jwR,T
úúú
û
ù
êêê
ë
é=
3
2
1
mmm
M
Exercise!
Ow
iw
kw
jwR,T
M = K R T!"
#$
P
P’
f
Suppose we have no rotation or translationZero skew, square pixels, no distortion, no off-set
= (m1Pwm3Pw
, m2Pwm3Pw
)P 'E Pw =
xwywzw1
!
"
#####
$
%
&&&&&
à
Exercise!
Ow
iw
kw
jwR,T
M = K R T!"
#$
= (m1Pwm3Pw
, m2Pwm3Pw
)
P
P’
f
P 'E = ( f xwzw, f yw
zw)
=
f 0 0 00 f 0 00 0 1 0
!
"
###
$
%
&&&
Pw =
xwywzw1
!
"
#####
$
%
&&&&&
à
= K I 0!"
#$
Canonical Projective Transformation
P ' =xyz
!
"
####
$
%
&&&&
=1 0 0 00 1 0 00 0 1 0
!
"
###
$
%
&&&
xyz1
!
"
####
$
%
&&&&
xzyz
!
"
####
$
%
&&&&
Pi ' =
P ' =M P
M3
H4 ®Â
Projective camera
p’q’r’ O
f’R
Q
P
Weak perspective projection
R
Q
P
O
f’
When the relative scene depth is small compared to its distance from the camera
zo
p’q’r’ Q_
P_
R_
π
Weak perspective projection
O
f’ zo
p’q’r’ Q_
P_
R_
x ' = f 'zx
y ' = f 'zy
!
"
###
$
###
→
x ' = f 'z0x
y ' = f 'z0y
!
"
##
$
##
Magnification m
R
Q
Pπ
úû
ùêë
é=
10bA
M[ ]TRKM = úû
ùêë
é=
1vbA
O
f’ zo
p’q’r’ Q_
P_
R_R
Q
P
π
à
Projective (perspective) Weak perspective
Weak perspective projection
úúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é=
10002
1
3
2
1
mm
mmm
=
m1
m2
m3
!
"
####
$
%
&&&&
PW
)P,P( 21 ww mm®E
wPMP =¢úû
ùêë
é=
10bA
M
magnification
úúú
û
ù
êêê
ë
é=
3
2
1
mmm
wPMP =¢ úû
ùêë
é=
1vbA
Múúú
û
ù
êêê
ë
é=
úúú
û
ù
êêê
ë
é=
W3
W2
W1
W
3
2
1
PPP
Pmmm
mmm
)PP,
PP(
3
2
3
1
w
w
w
w
mm
mm
®E
Perspective
Weakperspective
=
m1 PWm2 PW1
!
"
####
$
%
&&&&
Orthographic (affine) projectionDistance from center of projection to image plane is infinite
x ' = f 'zx
y ' = f 'zy
!
"
###
$
###
→x ' = xy ' = y
!"#
$#
ProsandConsofTheseModels
• Weakperspectiveresultsinmuchsimplermath.– Accuratewhenobjectissmallanddistant.– Mostusefulforrecognition.
• Pinholeperspectiveismuchmoreaccurateformodelingthe3D-to-2Dmapping.– UsedinstructurefrommotionorSLAM.
Lecture 3 -Silvio Savarese 18-Jan-18
• Recapofcameramodels• Cameracalibrationproblem• Cameracalibrationwithradialdistortion• Example
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Lecture3CameraCalibration
Reading: [FP] Chapter 1 “Geometric Camera Calibration”[HZ] Chapter 7 “Computation of Camera Matrix P”
Whyisthisimportant?Estimatecameraparameterssuchposeorfocallengthfromimages!
?
Projective camera
úúú
û
ù
êêê
ë
é -
=
100v0
ucot
K o
o
sinqb
qaa
úúú
û
ù
êêê
ë
é
=T3
T2
T1
Rrrr
úúú
û
ù
êêê
ë
é
=
z
y
x
ttt
T
43´
wPMP =¢ [ ] wPTRK=Internal parameters
External parameters
Goal of calibration
Estimate intrinsic and extrinsic parameters from 1 or multiple images
Change notation:P = Pw
p = P’
wPMP =¢ [ ] wPTRK=Internal parameters
External parameters
Calibration Problem
•P1… Pn with known positions in [Ow,iw,jw,kw]
jC
Calibration rig
Calibration Problem
•P1… Pn with known positions in [Ow,iw,jw,kw]•p1, … pn known positions in the image
jC
Calibration rig
pi
Goal: compute intrinsic and extrinsic parameters
pi
image
image
Calibration Problem
jC
Calibration rig
How many correspondences do we need?• M has 11 unknowns • We need 11 equations • 6 correspondences would do it
pi
jCpi
pi
image
Calibration Problem
jC
Calibration rig
In practice, using more than 6 correspondences enables more robust results
pi
jCpi
pi
image
Calibration Problem
jC
Calibration rig
pi
jCpi
pi
image
=M Pipi =uivi
!
"##
$
%&&
úúúú
û
ù
êêêê
ë
é
=
i3
i2
i3
i1
PPPP
mmmm
in pixels[Eq. 1]
úúú
û
ù
êêê
ë
é=
3
2
1
Mmmm
Calibration Problem
i3
i1i P
Pumm
=
i2i3i P)P(v mm =®
i1i3i P)P(u mm =®
i3
i2i P
Pvmm
=
úû
ùêë
é
i
i
vu
úúúú
û
ù
êêêê
ë
é
=
i3
i2
i3
i1
PPPP
mmmm
0P)P(v 23 =-® iii mm
→ ui (m3 Pi )−m1 Pi = 0
[Eq. 1]
[Eqs. 2]
Calibration Problem
……
0)( 12131 =- PPv mm
0)( 11131 =- PPu mm
0)( 23 =- iii PPv mm
0)( 13 =- iii PPu mm
0)( 23 =- nnn PPv mm
0)( 13 =- nnn PPu mm
[Eqs. 3]
Block Matrix Multiplication
úû
ùêë
é=ú
û
ùêë
é=
2221
1211
2221
1211
BBBB
BAAAA
A
What is AB ?
úû
ùêë
é++++
=2222122121221121
2212121121121111
BABABABABABABABA
AB
÷÷÷
ø
ö
ççç
è
æ
=T3
T2
T1def
mmm
mP = def
P1T
0T0T
P1T
−u1P1T
−v1P1T
!PnT
0T0T
PnT
−unPnT
−vnPnT
"
#
$$$$$$$$
%
&
''''''''
Calibration Problem
2n x 1212x1
1x44x1
…
Homogenous linear system
knownunknown
0)( 12131 =+- PPv mm
0)( 11131 =+- PPu mm
0)( 23 =+- nnn PPv mm
0)( 13 =+- nnn PPu mm
0=P m [Eq. 4]
Homogeneous M x N Linear Systems
P m 0=
Rectangular system (M>N)
• 0 is always a solution
Minimize |P m|2
under the constraint |m|2 =1
M=number of equations = 2nN=number of unknown = 11
• To find non-zero solution
N
M
• How do we solve this homogenous linear system?
Calibration Problem
• Via SVD decomposition!
0=P m
Calibration Problem
1212T
121212n2 VDU ´´´
Last column of V gives m
M
SVD decomposition of P
Why?Seepag 592ofHZ
÷÷÷
ø
ö
ççç
è
æ
=T3
T2
T1def
mmm
m
0=P m
Extracting camera parameters
M = ρ
Extracting camera parameters
A
úúú
û
ù
êêê
ë
é
=T3
T2
T1
Aaaa
[ ]TRK=
3
1a±
=r
úúú
û
ù
êêê
ë
é
=
3
2
1
bbb
b
Estimated values
)( 312 aa ×= rou
)(v 322
o aa ×= r
( ) ( )3231
3231cosaaaaaaaa
´×´´×´
=q
Intrinsic
b úúú
û
ù
êêê
ë
é -
=
100v0
ucot
K o
o
sinqb
qaarM =
Box 1
See [FP], Sec. 1.3.1
Extracting camera parameters
úúú
û
ù
êêê
ë
é
=T3
T2
T1
Aaaa
[ ]TRK=
úúú
û
ù
êêê
ë
é
=
3
2
1
bbb
b
Estimated values
Intrinsic
qra sin312 aa ´=
qrb sin322 aa ´=
A brM =
úúú
û
ù
êêê
ë
é -
=
100v0
ucot
K o
o
sinqb
qaa
Box 1
See [FP], Sec. 1.3.1
Theorem (Faugeras, 1993)
Extracting camera parameters
úúú
û
ù
êêê
ë
é
=T3
T2
T1
Aaaa
[ ]TRK=
úúú
û
ù
êêê
ë
é
=
3
2
1
bbb
b
Estimated values
A brM =
úúú
û
ù
êêê
ë
é -
=
100v0
ucot
K o
o
sinqb
qaa
Box 1
See [FP], Sec. 1.3.1
Extrinsic( )
32
321 aa
aar´´
=3
33 a
ar ±=
132 rrr ´= b1KT -= r
Degenerate cases
•Pi’s cannot lie on the same plane!• Points cannot lie on the intersection curve of two
quadric surfaces [FP] section 1.3
Lecture 3 -Silvio Savarese 18-Jan-18
• Recapofprojectivecameras• Cameracalibrationproblem• Cameracalibrationwithradialdistortion• Example
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Lecture3CameraCalibration
Reading: [FP] Chapter 1 “Geometric Camera Calibration”[HZ] Chapter 7 “Computation of Camera Matrix P”
Lecture 3 -Silvio Savarese 18-Jan-18
Some slides in this lecture are courtesy to Profs. J. Ponce, F-F Li
Lecture3CameraCalibration• Recapofprojectivecameras• Cameracalibrationproblem• Cameracalibrationwithradialdistortion• ExampleReading: [FP] Chapter 1 “Geometric Camera Calibration”
[HZ] Chapter 7 “Computation of Camera Matrix P”
Camera Calibration video from Matlab
Calibration ProcedureCamera Calibration Toolbox for MatlabJ. Bouguet – [1998-2000]
http://www.vision.caltech.edu/bouguetj/calib_doc/index.html#examples
Calibration Procedure
Calibration Procedure
Calibration Procedure
Calibration Procedure
Calibration Procedure
Calibration Procedure
Calibration Procedure
Next lecture
•Single view reconstruction
12
1
1
.--
úúúú
û
ù
êêêê
ë
é
=L= SSSSA
Nl
ll
Eigenvectors of A are columns of S
Eigenvalues and Eigenvectors
[ ]NS vv1=
Eigendecomposition
Singular Value decomposition
úúúú
û
ù
êêêê
ë
é
=S
Ns
ss
.2
1
1-S= VUA
ii ls = s = singular valuel = eigenvalue of At A
U, V = orthogonal matrix