Spring2016ENGN2502---3DPhotography
Lecture7
GabrielTaubinBrownUniversity
Desktop3DPhotography
• http://www.vision.caltech.edu/bouguetj/ICCV98/.index.html
3Dtriangulation:ray-planeIntersection
ray
plane
intersectionpoint
projector/
coordinate
systemsworld
coordinate system
3DTriangulationbyRay-Planeintersection
object being
scanned
pq n
projected light plane
p
illuminated
point on object
camera ray
v
intersection
of light plane
with object
world coordinate
system
image plane
center of projection
ray
direction
vector
Ifcameraandprojectorarecalibrated
object being
scanned
pq n
projected light plane
p
illuminated
point on object
camera ray
vintersection
of light plane
with object
common
world
coordinate
systemcenter of projection
ray direction
vector from
detected pixel
Lq
3DTriangulationbyRay-RayIntersection
object being
scanned
camera ray2v
projected
light ray
p
lines may not intersect !
Implicitequationoftheplane
pq n
}0)(:{ =−= pt qpnpP
v
world coordinate
system Plane normal vector
(unit length)
A fixed point
on the plane
p
Parametricequationoftheray
pq n
L = {p = qL +λv :λ > 0}
camera ray
Lq
v
world coordinate
system
ray direction
vector
(unit length)
p
v
TriangulationbyLine-PlaneIntersection
world coordinate
system
}{ vqpL L λ+==
camera ray
Lqv
Compose implicit and
parametric equations
p
pq n
}0)(:{ =−= pt qpnpP
projected light plane
TriangulationbyLine-PlaneIntersection
world coordinate
system
}{ vqpL L λ+==
camera ray
Lqv
Replace computed λ
in parametric equation
p
pq n
}0)(:{ =−= pt qpnpP
projected light plane
TriangulationbyLine-PlaneIntersection
world coordinate
system
}{ vqpL L λ+==
camera ray
Lqv
Eliminate parameter λ
p
pq n
}0)(:{ =−= pt qpnpP
projected light plane
TriangulationbyLine-LineIntersection
object being
scanned
}{ 2222 vqpL λ+==camera ray
2q
2v
projected light ray
1q1v
}{ 1111 vqpL λ+==
p
lines may not intersect !
TriangulationbyLine-LineIntersection
L2 = {p2 = q2 +λ2v2}
2q2v1q
1v
L1 = {p1 = q1 +λ1v1}
p1 = q1 +λ1v1
p2 = q2 +λ2v2
p1 − p2
v1t (p1 − p2 ) = 0v2t (p2 − p1) = 0
E(λ1,λ2 ) = dist(p2 − p1)2
Minimize
Necessary conditions
p = (p1 + p2 ) / 2
p
Approximate Line-Line Intersection
1q1v
2q
2v
1111 vqp λ+=
2222 vqp λ+=
),( 2112 λλp
Midpoint of segment joining
arbitrary points in the two lines
1q1v
2q
2v
),( 2112 λλp
Least-squares approach
Find parameters which minimize
Approximate Line-Line Intersection
1111 vqp λ+=
2222 vqp λ+=
3DLaserScanner
Whatistheequationoftheplane
inthecameracoordinatesystem?
Cameraandlaserareattached:
usecameracoordinatesystem
Plane defined by image line and center of projection
center of projection
qn
image plane
}0)(:{ =−= qpnpP t
}0:{ == uluL tImplicit equation of line in image coordinates
Triangulation by Laser Striping
M. J. Leotta, A. Vandergon, and G. Taubin. 3D Slit Scanning With Planar Constraints. Computer Graphics Forum, 2008
§ Manuallyormechanicallytranslatedlaserstripe
§ Per-pixeldepthbyray-planetriangulation
§ Requiresaccuratecameraandlaserplanecalibration
§ PopularsolutionforcommercialandDIY3Dscanners
3D Photography on Your Desk: Bouguet and Perona [ICCV 1998]
J.-Y. Bouguet and P. Perona. 3D photography on your desk. Intl. Conf. Comp. Vision, 1998
§ DIYscannerusingonlyacamera,ahalogenlamp,andastick
§ Per-pixeldepthbyray-planetriangulation
§ Requiresaccuratecameraandshadowplanecalibration
3D Photography on Your Desk: Bouguet and Perona [ICCV 1998]
J.-Y. Bouguet and P. Perona. 3D photography on your desk. Intl. Conf. Comp. Vision, 1998
Assembling Your Own Scanner
§ Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]
§ Step1:Buildthecalibrationboards(includefiducialsandchessboard)
§ Step2:Buildthepointlightsource(removereflectorandplaceinscene)
§ Step3:Arrangethecamera,lightsource,andcalibrationboards
Assembling Your Own Scanner
§ Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]
§ Step1:Buildthecalibrationboards(includefiducialsandchessboard)
§ Step2:Buildthepointlightsource(removereflectorandplaceinscene)
§ Step3:Arrangethecamera,lightsource,andcalibrationboards
Assembling Your Own Scanner
§ Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]
§ Step1:Buildthecalibrationboards(includefiducialsandchessboard)
§ Step2:Buildthepointlightsource(removereflectorandplaceinscene)
§ Step3:Arrangethecamera,lightsource,andcalibrationboards
Assembling Your Own Scanner
§ Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]
§ Step1:Buildthecalibrationboards(includefiducialsandchessboard)
§ Step2:Buildthepointlightsource(removereflectorandplaceinscene)
§ Step3:Arrangethecamera,lightsource,andcalibrationboards
Swept-Plane Reconstruction Geometry
CΧ1
2
3
)(tlΠ
),( yxCΛ
)(),( tyxP lC Π∩Λ=
P
p
)(tlΠ),( yxCΛ
Demo: Data Capture
Imax (x, y) = maxtI(x, y, t)
§ ConvertfromRGBtograyscale(forluminance-domainprocessing)
§ Determineper-pixelminimumandmaximumvalueoversequence
Video Processing: Assigning Per-Pixel Shadow Thresholds
Imin(x, y) = mintI(x, y, t)
Ishadow(x, y) =I m a x ( x ;y ) + I m i n ( x ;y )
2
§ ConvertfromRGBtograyscale(forluminance-domainprocessing)
§ Determineper-pixelminimumandmaximumvalueoversequence
§ Evaluateper-pixel�shadowthreshold�asaverageofmin.andmax.
Video Processing: Assigning Per-Pixel Shadow Thresholds
400 600 800
-60
-40
-20
0
20
40
60
column index
I 60(x
,y) -
I sha
dow(x
,y)
Video Processing: Spatial Shadow Edge Localization
§ Selectregionofinterestoneachcalibrationplane(occlusion-free)
§ Estimatezero-crossingstofindleadingandtrailingshadowboundaries
§ Fitalinetothesetofpointsalongeachshadowboundary
èResult:Best-fit2Dlinesforeachshadowedge(inimagecoordinates)
540 550 560 570 580
-60
-40
-20
0
20
40
column index
I 60(x
,y) -
I sha
dow(x
,y)
Video Processing: Temporal Shadow Edge Localization
80 100 120 140-40
-20
0
20
40
frame index
I(x,y
,t) -
I sha
dow(x
,y)
§ Tabulateper-pixeltemporalsequence(minusshadowthreshold)
§ Estimatezero-crossingstofindshadow-crossingtimes
èResult:Useshadow-crossingtimetolookupcorresponding3Dplane
crossing frame index for leading shadow crossing frame index for trailing shadow
fram
e in
dex
80
100
120
140
Intrinsic Camera Calibration
1 12
23
3
4
Estimated Camera Lens Distortion Map Camera Calibration Input
§ Howtoestimateintrinsicparametersanddistortionmodel?(unknowns:
focallength,skew,scale,principalpoint,anddistortioncoeffs.)
§ Popularsolution:Observeaknowncalibrationobject(Zhang[2000])
§ Each2Dchessboardcorneryieldstwoconstraintsonthe6-11unknowns
§ But,mustalsofind6extrinsicparametersperimage(rotation/translation)
§ èResult:Twoormoreimagesofachessboardaresufficient
camera coordinate system
CΧ
12 3 u
world coordinate system
WΧ
1 2
3p
TRXWC +=Χ
intrinsic parameters extrinsic parameters
Demo: Camera Calibration in Matlab
J.-Y. Bouguet. Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/
Extrinsic Camera Calibration
CΧ1
2
3
WΧ 12
3
?},{ TRTRXWC +=Χ
hΠ
vΠvΧ 1
2
3
vvvC TXR +=Χ
hΧ1
23
hhhC TXR +=Χ
w h
Demo: Mapping Pixels to Optical Rays
CΧ 12
3
),( yxCΛ
p
)( TRPKp +=λ
P
n
§ Howtomapanimagepixeltoanopticalray?
§ Solution:Invertthecalibratedcameraprojectionmodel
§ But,alsorequiresinversionofdistortionmodel(whichisnon-linear)
§ MappingimplementedinCameraCalibrationToolboxwithnormalize.m
èResult:Aftercalibration,pixelscanbeconvertedtoopticalrays
Shadow Plane Calibration
CΧ1
2
3
)(tlΠ
hΠ
vΠ
)(tvΛ
)(thλ
)(tvλ
)(tn
)()()( tttn vh Λ⊗Λ=)(thλ
)(tvλ
)(thΛ
)(tlΠ
)()()( tttP vh Λ∩Λ=
)(tP
Alternatives for Shadow Plane Calibration
J.-Y. Bouguet and P. Perona. 3D photography on your desk. Intl. Conf. Comp. Vision, 1998
CΧ1
2
3b st
B
h
sT
TL