Date post: | 11-Jan-2016 |
Category: |
Documents |
Upload: | curtis-singleton |
View: | 215 times |
Download: | 1 times |
Lecture 11Lecture 11 Stereo Reconstruction IStereo Reconstruction I
Mata kuliah : T0283 - Computer VisionTahun : 2010
January 20, 2010 T0283 - Computer Vision 3
Learning ObjectivesLearning Objectives
After carefullyAfter carefully listening this lecture, students will listening this lecture, students will be able to do the following :be able to do the following :
demonstrate 3D Stereo Computation by solving demonstrate 3D Stereo Computation by solving point-point- correspondence problems and fundamental correspondence problems and fundamental matrixmatrix
Calculate object-depth information using Calculate object-depth information using disparity and disparity and triangulation techniques.triangulation techniques.
January 20, 2010 T0283 - Computer Vision 4
Stereo Stereo Reconstruction Reconstruction
knownknowncameracamera
viewpointsviewpoints
Shape (3D) from two (or more) images
January 20, 2010 T0283 - Computer Vision 5
ExamplesExamples
images
shape
surface reflectance
January 20, 2010 T0283 - Computer Vision 6
ScenariosScenarios
The two images can arise fromThe two images can arise from A stereo rig consisting of two camerasA stereo rig consisting of two cameras
the two images are acquired the two images are acquired simultaneouslysimultaneously
OROR
A single moving camera (static scene)A single moving camera (static scene)
the two images are acquired the two images are acquired sequentiallysequentially
The two scenarios are geometrically equivalentThe two scenarios are geometrically equivalent
January 20, 2010 T0283 - Computer Vision 7
Stereo head
Camera on a mobile vehicle
January 20, 2010 T0283 - Computer Vision 8
Imaging geometryImaging geometry
central projectioncentral projection
camera centre, image point camera centre, image point and scene point are collinearand scene point are collinear
an image point back projects an image point back projects to a ray in 3-spaceto a ray in 3-space
depth of the scene point is depth of the scene point is unknownunknown
cameracentre image plane
imagepoint
scenepoint
?
January 20, 2010 T0283 - Computer Vision 9
The objective The objective
Given two images of a scene acquired by known cameras, compute the 3D position of the scene (structure recovery)
Basic principle: triangulate from corresponding image points
• Determine 3D point at intersection of two back-projected rays
January 20, 2010 T0283 - Computer Vision 10
Corresponding points are images of the same scene point
Triangulation
C C /
The back-projected points generate rays which intersect at the 3D scene point
January 20, 2010 T0283 - Computer Vision 11
An algorithm for stereo An algorithm for stereo reconstructionreconstruction
1. For each point in the first image determine the corresponding point in the second image
(this is a search problem)
2. For each pair of matched points determine the 3D point by triangulation
(this is an estimation problem)
January 20, 2010 T0283 - Computer Vision 12
The correspondence problemThe correspondence problem
Given a point x in one image, find the corresponding point in the other image
This appears to be a 2D search problem, but it is reduced to a 1D search by the epipolar constraint
January 20, 2010 T0283 - Computer Vision 13
NotationNotation
x x /
X
C C /
The two cameras are P and P/, and a 3D point X is imaged as
for equations involving homogeneous quantities ‘=’ means ‘equal up to scale’
P P/
Warning
January 20, 2010 T0283 - Computer Vision 14
Epipolar geometryEpipolar geometry
January 20, 2010 T0283 - Computer Vision 15
Epipolar geometryEpipolar geometry
Given an image point in one view, where is Given an image point in one view, where is the corresponding point in the other view?the corresponding point in the other view?
• A point in one view “generates” an epipolar line in the other view
• The corresponding point lies on this line
epipolar line
?
baselineepipole C
/C
January 20, 2010 T0283 - Computer Vision 16
Epipolar lineEpipolar line
Epipolar constraint• Reduces correspondence problem to 1D search
along an epipolar line
January 20, 2010 T0283 - Computer Vision 17
Epipolar geometry continuedEpipolar geometry continued
Epipolar geometry is a consequence of the Epipolar geometry is a consequence of the coplanarity coplanarity of the camera centers and scene pointof the camera centers and scene point
x x /
X
C C /
The camera centers, corresponding points and scene point lie in a single plane, known as the epipolar plane
January 20, 2010 T0283 - Computer Vision 18
NomenclatureNomenclature
• The epipolar line l/ is the image of the ray through x• The epipole e is the point of intersection of the line joining the camera centres with the image plane
this line is the baseline for a stereo rig, and the translation vector for a moving camera
• The epipole is the image of the centre of the other camera: e = PC/ , e/ = P/C
xx /
X
C C /
e
left epipolar line
right epipolar line
e /
l/
January 20, 2010 T0283 - Computer Vision 19
The epipolar pencilThe epipolar pencil
e e /
baseline
X
As the position of the 3D point X varies, the epipolar planes “rotate” about the baseline. This family of planes is known as an epipolar pencil. All epipolar lines intersect at the epipole.(a pencil is a one parameter family)
January 20, 2010 T0283 - Computer Vision 20
The epipolar pencilThe epipolar pencil
e e /
baseline
X
As the position of the 3D point X varies, the epipolar planes “rotate” about the baseline. This family of planes is known as an epipolar pencil. All epipolar lines intersect at the epipole.(a pencil is a one parameter family)
January 20, 2010 T0283 - Computer Vision 21
Epipolar geometry ex. 1: parallel Epipolar geometry ex. 1: parallel camerascameras
Epipolar geometry depends only on the relative pose (position and orientation) and internal parameters of the two cameras, i.e. the position of the camera centres and image planes. It does not depend on the scene structure (3D points external to the camera).
January 20, 2010 T0283 - Computer Vision 22
Epipolar geometry ex. 2: converging Epipolar geometry ex. 2: converging camerascameras
Note, epipolar lines are in general not parallel
e e /
January 20, 2010 T0283 - Computer Vision 23
Homogeneous notation for linesHomogeneous notation for lines
January 20, 2010 T0283 - Computer Vision 24
• The line l through the two points p and q is l = p x q
Example: compute the point of intersection of the two lines l and m in the figure below
Proof
y
x
1
2
• The intersection of two lines l and m is the point x = l x m
l
m
which is the point (2,1)
January 20, 2010 T0283 - Computer Vision 25
Matrix representation of the vector cross Matrix representation of the vector cross productproduct
January 20, 2010 T0283 - Computer Vision 26
Example: compute the cross product of l and m
January 20, 2010 T0283 - Computer Vision 27
Algebraic representation of epipolar Algebraic representation of epipolar geometrygeometryWe know that the epipolar geometry defines a We know that the epipolar geometry defines a mappingmapping
x l/
point in first image
epipolar line in second
image
January 20, 2010 T0283 - Computer Vision 28
Derivation of the algebraic expressionDerivation of the algebraic expression
Outline
Step 1: for a point x in the first image back project a ray with camera P
Step 2: choose two points on the ray and project into the second image with camera P/
Step 3: compute the line through the two image points using the relation l/ = p x q
P
P/
January 20, 2010 T0283 - Computer Vision 29
• choose camera matrices
internal calibration
rotation translation
from world to camera coordinate
frame
• first camera
world coordinate frame aligned with first camera
• second camera
January 20, 2010 T0283 - Computer Vision 30
Step 1: for a point x in the first image back project a ray with camera
P
A point x back projects to a ray
where Z is the point’s depth, since
satisfies
January 20, 2010 T0283 - Computer Vision 31
Step 2: choose two points on the ray and project into the second image with camera P/
P/
Consider two points on the ray
• Z = 0 is the camera centre
• Z = is the point at infinity
Project these two points into the second view
January 20, 2010 T0283 - Computer Vision 32
Using the identity
Compute the line through the points
F
F is the fundamental matrix
Step 3: compute the line through the two image points using the relation l/ = p x q
January 20, 2010 T0283 - Computer Vision 33
Example 1: compute the fundamental matrix for a parallel camera
• reduces to y = y/ , i.e. raster correspondence (horizontal scan-lines)
f
f
X Y
Z
January 20, 2010 T0283 - Computer Vision 34
f
f
X Y
Z
January 20, 2010 T0283 - Computer Vision 35
Example 2: compute F for a forward translating camera
f
f
X YZ
January 20, 2010 T0283 - Computer Vision 36
first image second image
f
f
X YZ
January 20, 2010 T0283 - Computer Vision 37
January 20, 2010 T0283 - Computer Vision 38
January 20, 2010 T0283 - Computer Vision 39
Summary: Properties of the Fundamental Summary: Properties of the Fundamental matrixmatrix