Date post: | 01-Jan-2016 |
Category: |
Documents |
Upload: | cody-kelly |
View: | 234 times |
Download: | 2 times |
Recap of Lecture 7
• Inverse perspective transformation and its issues
•Many to one mapping
• Generalized perspective transformation
• Fundamentals of camera calibration
Outline of Lecture 8
• Fundamentals of stereo imaging
• Calculation of disparity
• Search space for point correspondence
• Correlation based correspondence
Camera calibration
𝑝11 𝑋+𝑝12𝑌+𝑝13𝑍+𝑝14−𝑥𝑋 𝑝41−𝑥𝑌 𝑝42−𝑥𝑍𝑝43−𝑥 𝑝44=0
𝑝21𝑋+𝑝22𝑌 +𝑝23 𝑍+𝑝24− 𝑦 𝑋 𝑝41− 𝑦 𝑌 𝑝42− 𝑦 𝑍𝑝43−𝑦 𝑝44=0
….. (1)
….. (2)6 pairs of points are required
and
and
and
and
and
and
Solving for unknowns
𝐶𝑃=0
[𝑋1 𝑌 1 𝑍1 1 0 0 0 0 −𝑥1 𝑋1 −𝑥1𝑌 1 −𝑥1𝑍 1 −𝑥1𝑋 2 𝑌 2 𝑍2 1 0 0 0 0 −𝑥2 𝑋 2 −𝑥2𝑌 2 −𝑥2𝑍 2 −𝑥2⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮𝑋 6 𝑌 6 𝑍6 1 0 0 0 0 −𝑥6 𝑋 6 −𝑥6𝑌 6 − 𝑥6𝑍 6 − 𝑥60 0 0 0 𝑋1 𝑌 1 𝑍1 1 − 𝑦1𝑋 1 − 𝑦1𝑌1 − 𝑦1𝑍1 − 𝑦10 0 0 0 𝑋 2 𝑌 2 𝑍2 1 − 𝑦2𝑋 2 − 𝑦2𝑌2 − 𝑦2𝑍2 − 𝑦2⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮0 0 0 0 𝑋 6 𝑌 6 𝑍6 1 − 𝑦 6𝑋 6 −𝑦 6𝑌6 − 𝑦6𝑍6 − 𝑦6
] [𝑎11𝑎12𝑎13𝑎14𝑎21𝑎22𝑎23𝑎24𝑎41𝑎42𝑎43𝑎44
]=[0⋮000⋮0]
2𝑛×1212×1
12×1
Perspective transformation
P
PI
𝑍 , 𝑧
𝑌 , 𝑦
𝑋 ,𝑥
World co-ordinate
Image plane
𝑥=𝜆𝑋𝜆−𝑍 𝑦=
𝜆𝑌𝜆−𝑍
𝑋=𝑥𝜆
(𝜆−𝑍 ) 𝑌=𝑦𝜆
(𝜆−𝑍 )Two equations, three unknowns
Introducing a second imaging plane
𝑃 :(𝑋 ,𝑌 ,𝑍 )
𝑧
𝑃 𝐼 ′𝑦
𝑥
𝑧 ′
𝑦 ′
𝑥 ′
Focal length of C1
Coordinate system for C1Image point w.r.to C1
Coordinate system for C2Image point w.r.to C2
Focal length of C2
Relationship between coordinate systems
[𝑥 ′𝑦 ′𝑧 ′ ]=[𝑟11 𝑟 12 𝑟13𝑟 14 𝑟 15 𝑟16𝑟 17 𝑟 18 𝑟19 ] [
𝑥𝑦𝑧 ]+[𝑡𝑥𝑡𝑦𝑡 𝑧 ]
Coordinates of Camera #2
Rotation matrix
Translation matrix
Coordinates of Camera #1
Assumptions
•World coordinate w.r.to camera #1:
•World coordinate w.r.to camera #2:
• Two cameras are having identical focal length:
• Coordinate of the point w.r.to x-y-z coordinate system
• Coordinate of the point w.r.to x’-y’-z’ coordinate system
Mathematical relationship between points
• For camera #1
• For camera #2
𝑥0𝑥 𝑖
=𝑦 0𝑦 𝑖
=𝜆− 𝑧0𝜆
𝑥0 ′𝑥 𝑖 ′
=𝑦0 ′𝑦 𝑖′
=𝜆−𝑧 0 ′𝜆
Coordinate transformation is required
Rectified camera configuration
• Assume pure translation, without any rotation
[𝑥 ′𝑦 ′𝑧 ′ ]=[1 0 00 1 00 0 1 ][
𝑥𝑦𝑧 ]+[𝛿𝑥00 ]
[𝑥 ′𝑦 ′𝑧 ′ ]=[1 0 00 1 00 0 1 ][
𝑥𝑦𝑧 ]+[ 0𝛿 𝑦0 ]
[𝑥 ′𝑦 ′𝑧 ′ ]=[1 0 00 1 00 0 1 ][
𝑥𝑦𝑧 ]+[ 00𝛿𝑧 ]
Lateral stereo geometry
Axial stereo geometry
Modified camera configuration after lateral shift along x-axis
LEFT
𝑧
𝑥
𝑦
𝝀𝑂 𝐿
𝐶𝐿
𝑧 ′
𝑥 ′
𝑦 ′
𝝀𝑂𝑅
𝐶𝑅
RIGHT
𝛿𝑥
𝑃 (𝑥0 , 𝑦0 ,𝑧 0)
𝑃 𝐿(𝑥𝐿 , 𝑦𝐿) 𝑃 𝑅(𝑥𝑅 , 𝑦𝑅)
Assumption
• : w.r.to x-y-z coordinate system
• : w.r.to x-y-z coordinate system
• : Origin of the left camera coordinate system
• : Origin of the right camera coordinate system
•World coordinate w.r.to left camera is
• : Lateral shift between to cameras
Mathematical relationship
• For camera #1
• For camera #2
𝑥0𝑥 𝐿
=𝑦0𝑦 𝐿
=𝜆− 𝑧0𝜆
𝑥0𝑥𝑅
=𝑦0𝑦𝑅
=𝜆−𝑧 0𝜆
𝑥0+𝛿𝑥𝑥𝑅+𝛿𝑥
=𝑦 0𝑦𝑅
=𝜆−𝑧 0𝜆
Incorrect
Solve for unknowns
𝑥0𝑥 𝐿
=𝜆−𝑧 0𝜆
…….. (1)
𝑦0𝑦 𝐿
=𝜆−𝑧 0𝜆
…….. (2)
𝑥0+𝛿𝑥𝑥𝑅+𝛿𝑥
=𝜆−𝑧0𝜆
…….. (3)
𝑦 0𝑦𝑅
=𝜆−𝑧 0𝜆
…….. (4)
Coordinate of the 3D world point
𝑧 0=𝜆+𝛿𝑥 .𝜆
𝑥𝐿−(𝑥𝑅+𝛿𝑥)
𝑥0=𝛿𝑥 .𝜆 . 𝑥𝐿
𝑥𝐿−(𝑥𝑅+𝛿𝑥)
𝑦 0=𝛿 𝑥 .𝜆 . 𝑦 𝐿
𝑥𝐿−(𝑥𝑅+𝛿𝑥 )
Depth
Disparity
• Denominator term is significant
• Translating the point to the left camera plane
• Relative displacement: disparity
• Object at infinity
• Depth is inversely related to the disparity
Token Based Stereo
• Detect token• Corners, interest point, edges
• Find correspondences
• Interpolate surface
Correlation Based Stereo Methods
• Depth is computed only at tokens and interpolated/ extrapolated to remaining pixel
• Disparity map is constructed based on a correlation measure
|| 1 tt IIAD
tt II 1CC
tt
tt
II
II
.
.1NC
2
1 tt IISSD
Correlation Based Stereo Methods
• Once disparity is available compute depth using
𝑍=𝜆𝐵𝑑 Separation between the cameras disparity
Error
Index of points