Post on 11-Jul-2020
transcript
Perspective projection
Albrecht Dürer, Mechanical creation of a perspective image, 1525
Slides Credit: Prof. David Forsyth, Prof. Svetlana Lazebnik
Overview of next two lectures
• The pinhole projection model• Qualitative properties
• Perspective projection matrix
• Cameras with lenses• Depth of focus
• Field of view
• Lens aberrations
• Digital cameras• Sensors
• Color
• Artifacts
Let’s design a camera
Idea 1: put a piece of film in front of an object
Do we get a reasonable image?
Slide by Steve Seitz
Pinhole camera
Add a barrier to block off most of the rays
Slide by Steve Seitz
Pinhole camera
• Captures pencil of rays – all rays through a single
point: aperture, center of projection, optical center,
focal point, camera center
• The image is formed on the image plane
Slide by Steve Seitz
Camera obscura
• Latin. camera: chamber
or room,
obscura: darkened
• Basic principle known to
Mozi (470-390 BCE),
Aristotle (384-322 BCE)
• Drawing aid for artists:
described by Leonardo
da Vinci (1452-1519)
Gemma Frisius, 1558
Source: A. Efros
Turning a room into a camera obscura
Abelardo Morell, Camera Obscura Image of Manhattan View Looking South in Large
Room, 1996
http://www.abelardomorell.net/camera_obscura1.html
From Grand Images Through a Tiny Opening, Photo
District News, February 2005
Turning a room into a camera obscura
A. Torralba and W. Freeman, Accidental Pinhole and Pinspeck Cameras, CVPR 2012
Pinhole cameras everywhere
Tree shadow during a solar eclipsephoto credit: Nils van der Burg
http://www.physicstogo.org/index.cfm
Slide by Steve Seitz
Point of observation
Figures © Stephen E. Palmer, 2002
Dimensionality reduction: from 3D to 2D
3D world 2D image
What properties of the world are preserved?• Straight lines, incidence
What properties are not preserved?• Angles, lengths Slide by A. Efros
Single-view Geometry
How tall is this woman?
Which ball is closer?
How high is the camera?
What is the camera
rotation?
What is the focal length of
the camera?
Modeling projection
• To compute the projection P’ of a scene point P, form
the visual ray connecting P to the camera center O and
find where it intersects the image plane
• All scene points that lie on this visual ray have the same
projection in the image
• Are there scene points for which this projection is undefined?
P
O
Modeling projection
f z
P
?O
P’
The coordinate system• The optical center (O) is at the origin
• The image plane is parallel to xy-plane or perpendicular to the z-axis,
which is the optical axis
Projection equations• Derived using similar triangles ),(),,(
z
yf
z
xfzyx
y
Point of observation
Fronto-parallel planes
• What happens to the projection of a pattern
on a plane parallel to the image plane?• All points on that plane are at a fixed depth z
• The pattern gets scaled by a factor of f / z, but angles and
ratios of lengths/areas are preserved
),(),,(z
yf
z
xfzyx
Fronto-parallel planes
• What happens to the projection of a pattern
on a plane parallel to the image plane?• All points on that plane are at a fixed depth z
• The pattern gets scaled by a factor of f / z, but angles and
ratios of lengths/areas are preserved
Jan Vermeer, The Music Lesson, 1662-1665Piero della Francesca, Flagellation of Christ, 1455-1460
What about non-fronto-parallel planes?
Jan Vermeer, The Music Lesson, 1662-1665Piero della Francesca, Flagellation of Christ, 1455-1460
Projection can be tricky…Slide source: Seitz
Projection can be tricky…Slide source: Seitz
Making of 3D sidewalk art:
http://www.youtube.com/watch?v=3SNYtd0Ayt0
Vanishing points
• All parallel lines converge to a vanishing point
• Each direction in space is associated with its own vanishing point
• Exception: directions parallel to the image plane
Constructing the vanishing point of a line
image plane
cameracenter
line in the scene
vanishing point
Slide by Steve Seitz
Perspective cues
Slide by Steve Seitz
Perspective cues
Slide by Steve Seitz
Perspective cues
Slide by Steve Seitz
Vanishing points and lines
Vanishingpoint
Vanishingline
Vanishingpoint
Vertical vanishingpoint
(at infinity)
Slide from Efros, Photo from Criminisi
Vanishing points and lines
Photo from online Tate collection
Note on estimating vanishing points
Use multiple lines for better accuracy
… but lines will not intersect at exactly the same point in practice
One solution: take mean of intersecting pairs
… bad idea!
Instead, minimize angular differences
Vanishing objects
Vanishing lines of planes
Image source: S. Seitz
How do we construct the vanishing line of a plane?
Vanishing lines of planes
Slide by Steve Seitz
cameracenter
plane in the scene
• Horizon: vanishing line of the ground plane
– All points at the same height as the camera project to the
horizon
– Points higher (resp. lower) than the camera project
above (resp. below) the horizon
– Provides way of comparing height of objects
Comparing heights
Vanishing
Point
Slide by Steve Seitz
Measuring height
1
2
3
4
55.4
2.8
3.3
Camera height
What is the height of the camera?
Slide by Steve Seitz
Perspective distortion
• Are the widths of the projected columns equal?• The exterior columns are wider
• This is not an optical illusion, and is not due to lens flaws
• Phenomenon pointed out by Da Vinci
Source: F. Durand
Perspective distortion
• What is the shape of the projection of a sphere?
Image source: F. Durand
Perspective distortion
• What is the shape of the projection of a sphere?
Perspective distortion: People
Some Applications
• Various applications of perspective geometry
& vanishing points
• Credits: David Forsyth & Derek Hoiem
Inverse Perspective Transformation
Object Recognition (CVPR 2006)
Reconstruction
Single-view reconstruction (SIGGRAPH 2005)
Secene Geometry Estimation
Getting spatial layout in indoor scenes (ICCV
2009)
3D Scene Reconstruction
Creating detailed and complete 3D scene models from a
single view (ongoing)
Lane Detection
B. Benligiray, C. Topal, C. Akinlar, Video-based lane detection using a fast vanishing point estimation
method, IEEE Int. Symposium Multimedia (ISM), 2012.
Modeling projection
Projection equation:
Source: J. Ponce, S. Seitz
),(),,(z
yf
z
xfzyx
x
y
z
f
Homogeneous coordinates
Is this a linear transformation?
Trick: add one more coordinate:
homogeneous image
coordinates
homogeneous scene
coordinates
Converting from homogeneous coordinates
• no—division by z is nonlinear
Slide by Steve Seitz
),(),,(z
yf
z
xfzyx
Projection: world coordinates image coordinates
Camera
Center
(tx, ty, tz)
Z
Y
X
P.
.
. f Z
Y
v
up
.
Optical
Center
(u0, v0)
v
u
X
Homogeneous coordinates
Invariant to scaling
Point in Cartesian is ray in Homogeneous
w
y
wx
kw
ky
kwkx
kw
ky
kx
w
y
x
k
Homogeneous Coordinates
Cartesian Coordinates
Basic geometry in homogeneous coordinates
Line equation: ax + by + c = 0
Append 1 to pixel coordinate to get homogeneous
coordinate
Line given by cross product of two points
Intersection of two lines given by cross product of the lines
1
i
i
i v
u
p
jiij ppline
jiij linelineq
i
i
i
i
c
b
a
line
Another problem solved by homogeneous coordinates
Cartesian: (Inf, Inf)
Homogeneous: (1, 1, 0)
Intersection of parallel lines
Cartesian: (Inf, Inf)
Homogeneous: (1, 2, 0)
divide by the third
coordinate
Perspective Projection Matrix
Projection is a matrix multiplication using homogeneous
coordinates
z
yf
xf
z
y
x
f
f
10100
000
000
),(z
yf
z
xf
In practice: lots of coordinate transformations…
World to
camera coord.
trans. matrix
(4x4)
Perspective
projection matrix
(3x4)
Camera to
pixel coord.
trans. matrix
(3x3)
=2D
point
(3x1)
3D
point
(4x1)
X0IKx
10100
000
000
1z
y
x
f
f
v
u
w
K
Slide Credit: Saverese
Projection matrix
Intrinsic Assumptions
• Unit aspect ratio
• Optical center at (0,0)
• No skew
Extrinsic Assumptions• No rotation
• Camera at (0,0,0)
Remove assumption: known optical center
X0IKx
10100
00
00
1
0
0
z
y
x
vf
uf
v
u
w
Intrinsic Assumptions
• Unit aspect ratio
• No skew
Extrinsic Assumptions• No rotation
• Camera at (0,0,0)
Remove assumption: square pixels
X0IKx
10100
00
00
1
0
0
z
y
x
v
u
v
u
w
Intrinsic Assumptions• No skew
Extrinsic Assumptions• No rotation
• Camera at (0,0,0)
Remove assumption: non-skewed pixels
X0IKx
10100
00
0
1
0
0
z
y
x
v
us
v
u
w
Intrinsic Assumptions Extrinsic Assumptions• No rotation
• Camera at (0,0,0)
Note: different books use different notation for parameters
Degrees of freedom
XtRKx
1100
0
1 333231
232221
131211
0
0
z
y
x
trrr
trrr
trrr
v
us
v
u
w
z
y
x
5 6
Vanishing Point = Projection from Infinity
R
R
R
z
y
x
z
y
x
z
y
x
KpKRptRKp
0
R
R
R
z
y
x
vf
uf
v
u
w
100
0
0
1
0
0 0uz
fxu
R
R
0vz
fyv
R
R
Orthographic Projection
Special case of perspective projection• Distance from center of projection to image plane is infinite
• Also called “parallel projection”
Image World
Slide by Steve Seitz
Orthographic Projection
Special case of perspective projection• Distance from center of projection to image plane is infinite
• Also called “parallel projection”
Orthographic Projection
Special case of perspective projection• Distance from center of projection to image plane is infinite
• Also called “parallel projection”
• What’s the projection matrix?
Image World
Slide by Steve Seitz