EECS490: Digital Image Processing
Cameras
• First photographs due to Niepce• First experiment 1822; first photograph 1826
• Basic abstraction is the pinhole camera• lenses required to ensure image is not too
dark• various other abstractions can be applied
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
EECS490: Digital Image Processing
Pinhole cameras
Abstract camera model -box with a small hole in it
Pinhole cameras actually workin practice!
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Each point on the image plane sees light from only one direction which passes through the pinhole
EECS490: Digital Image Processing
Distant objects are smaller
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Image plane
Common toput virtualimage here
{Approximatelythe focal length
Various objectsin the 3-D world
Due to camerascaling imagesB’ and C’ arethe sameheight althoughthe 3-D B andC are verydifferent.
EECS490: Digital Image Processing
Distant objects are smaller
real image
pinhole
object
virtual image
pinhole
object
EECS490: Digital Image Processing
Parallel lines meet
Common to draw filmplane in front of thepinhole.Moving the film planemerely scales the image.
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
virtual image plane
Parallel lines in realworld
Vanishing point
plane containing pinhole
Parallel lines converge to aline H in the virtual imageplane called the horizon
EECS490: Digital Image Processing
Vanishing points
• Each set of parallel lines(same direction) meetsat a different point inthe image plane– Called the vanishing
point for thisdirection
• Sets of parallel lines onthe same plane lead tocollinear vanishingpoints.– The line is called the
horizon for that plane
• Good ways to spotfaked images– scale and
perspective don’twork
– vanishing pointsbehave badly
– supermarkettabloids are a greatsource of fakedimages.
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
EECS490: Digital Image Processing
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Vanishing points
EECS490: Digital Image Processing
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Vanishing points
The blocks wereobviously cut andpasted. Only onecorrectly points tothe vanishing pointon the horizon.
EECS490: Digital Image Processing
Recent Controversial Image
Chinese Moon Photo Not Fake, But Not Pristine!
A new feature on the Moon?A 4-kilometer-diameter crater in a Clementine imageof the Moon from 1994 seems to have been struckby another crater in a Chang'e image of the Moonfrom 2007. Credit: NASA / DOD / CAST
Not a new feature on the MoonWith a repair to an image seam in a Chang'e 1image of the Moon, an apparently new featuredisappears. Credit: CAST / Emily Lakdawalla
How did that happen? the Chang’e 1 image is actually a mosaic of 19 image stripstaken on different orbits. It is incredibly difficult to get every tiny feature in alarge mosaic to line up properly; you have to have very, very precise knowledge ofthe shape of the body being photographed, and we actually don't have very goodtopographic models of the Moon (Kaguya will change that). So it's no surprise thatthere are seams in this image. I commented last week when I first posted this imageon the beauty of the mosaic, and the fact that there were no obvious seams. It'sevident from this little snafu that the seams are there, just blended well to make itdifficult to see them.
Clementine Chang’e
FixedChang’e
EECS490: Digital Image Processing
The equation of projection
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Since OP and OP’ are collinearwe can use similar triangles
x’= xy’= xf’= z
=x '
x=y '
y=f '
z
Use f’ instead of z’
pinhole
x ' = f 'x
z
y ' = f 'y
z
EECS490: Digital Image Processing
The equation of projection
• Cartesian coordinates:– We have, by similar triangles, that
– Ignore the third coordinate, and get
(x, y, z) ( fx
z, fy
z)
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
(x, y, z) ( fx
z, fy
z, f )
EECS490: Digital Image Processing
Homogenous coordinates
• Add an extracoordinate and use anequivalence relation
• for 2D– equivalence relation
(X/K,Y/K,1) is thesame as(X,Y,K)
• for 3D– equivalence relation
(X/K,Y/K,Z/K,1) isthe same as(X,Y,Z,K)
Basic notion• Possible to represent
points “at infinity”– Where parallel lines
intersect– Where parallel
planes intersect• Possible to write the
action of a perspectivecamera as a matrix
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
EECS490: Digital Image Processing
The camera matrix
Re-write previousexpression for cameratransformation inhomogeneous coordinates(HC’s)
• HC’s for 3D point are(X,Y,Z,T)
• HC’s for point in imageare (U,V,W)
U
V
W
=
1 0 0 0
0 1 0 0
0 0 1f 0
X
Y
Z
T
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
2-D imagecoordinates 3-D world
coordinates
EECS490: Digital Image Processing
The camera matrix
U
V
W
=
1 0 0 0
0 1 0 0
0 0 1f 0
X
Y
Z
T
2-D imagecoordinates
3-D worldcoordinates
U = X
V = Y
W =Z
f
Multiplying
U =XZ
f
= fX
Z
V =YZ
f
= fY
Z
Scale factor
EECS490: Digital Image Processing
Weak perspective
Issue• perspective effects,
but not over the scaleof individual objects
• collect points into agroup at about the samedepth, then divide eachpoint by the depth ofits group
• Adv: easy• Disadv: wrong
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
If depth of field of objects in agroup is similar, assume magnificationof group is constant.
U = fX
ZmX
V = fY
ZmY
EECS490: Digital Image Processing
Orthographic projection
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Extreme case of weak perspective. Fromprevious drawing we normalize imagecoordinates to have magnification m=-1which looks wrong but is correct.
EECS490: Digital Image Processing
Projection matrix for orthographicprojection
U
V
W
=
1 0 0 0
0 1 0 0
0 0 0 1
X
Y
Z
T
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
This is the mathematicalsimplification of theprevious projection matrixin the limit that f->
EECS490: Digital Image Processing
Pinhole too big -many directions areaveraged, blurring theimage
Pinhole too small-diffraction effectsblur the image
Generally, pinholecameras are dark,because a very small setof rays from aparticular point hits thescreen.
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Pinhole imaging
EECS490: Digital Image Processing
The reason for lenses
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
A lens can gather(focus) many light raysto a single image pointmaking a brighterimage.
Each point on theimage plane sees lightfrom only the onedirection whichpasses through thepinhole
EECS490: Digital Image Processing
The thin lens
1
z'-1
z=1
fComputer Vision - A Modern Approach: Cameras
Slides by D.A. Forsyth
z’ image distancez object distancef focal length
object plane
image plane
Lens of focallength f
EECS490: Digital Image Processing
The lens law
1
z'-1
z=1
f
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
ASSUMPTIONS AND OBSERVATIONS• From Snell’s Law n1sin 1=n2sin 2 where n1
is the index of refraction of air and n2 isthe index of refraction of the lens
• Assumes y and y’ are close to the z-axis(paraxial ray assumption) so that sin 1 1
• Ray PP’ is not refracted since 1= 2 but allother rays are refracted.
• f is the point at which entering raysparallel to the z-axis are focused, i.e.,cross the z-axis
EECS490: Digital Image Processing
Spherical aberration
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
Focal plane
Circle of leastconfusion
Because the paraxial ray assumptions sin 1 1 and sin
2 2 is only an approximation the focal point fordifferent rays is not at the same z-coordinate for allrays.
EECS490: Digital Image Processing
Lens systems
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
• Complex lens systems are used to minimizeaberrations, etc.
• It is common for modern lenses (exceptthe front and rear lenses) to be plastic.– Light-weight– Easy to manufacture, i.e., low-cost– aspheric
EECS490: Digital Image Processing
Vignetting
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
• Light follows a path which prevents itfrom reaching the image plane, i.e., itreaches the limit of the lens or runs into astop. This can be accidental or deliberate.
EECS490: Digital Image Processing
Other (possibly annoying)phenomena
Chromatic aberration• Light at different wavelengths follows different
paths; hence, some wavelengths are defocussed• Machines: coat the lens• Humans: live with it
Scattering at the lens surface• Some light entering the lens system is reflected off
each surface it encounters (Fresnel’s law givesdetails)
• Machines: coat the lens, interior• Humans: live with it (various scattering phenomena
are visible in the human eye)Geometric phenomena (Barrel distortion, etc.)
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
EECS490: Digital Image Processing
Camera parameters
Issue• camera may not be at the origin, looking down the z-
axis• extrinsic parameters
• one unit in camera coordinates may not be the sameas one unit in world coordinates• intrinsic parameters - focal length, principal point,
aspect ratio, angle between axes, etc.
U
V
W
=
Transformation
representing
intrinsic parameters
1 0 0 0
0 1 0 0
0 0 1 0
Transformation
representing
extrinsic parameters
X
Y
Z
T
Computer Vision - A Modern Approach: CamerasSlides by D.A. Forsyth
EECS490: Digital Image Processing
Camera calibration
Issues:• what are intrinsic
parameters of thecamera?
• what is the cameramatrix?
(intrinsic+extrinsic)General strategy:• view calibration object• identify image points• obtain camera matrix by
minimizing error• obtain intrinsic
parameters from cameramatrix
Error minimization:• Linear least squares
– easy problemnumerically
– solution can be ratherbad
• Minimize image distance*– more difficult numerical
problem– solution usually rather
good,– start with linear least
squares• Numerical scaling is an
issueComputer Vision - A Modern Approach: Cameras
Slides by D.A. Forsyth
EECS490: Digital Image Processing
Camera calibration
Radial alignment constraint (due to Tsai)• The idea is that spherical distortion changes the radial
scale but not the direction.• Use least squared error (LSE) to find all camera
parameters– Tz — z-translation of camera– Magnification– Distortion parameters (usually we only use the first two
terms) where d is the distance between the image pointand the image center.
= 1+ 1d2+ 2d
4+ 3d
6
EECS490: Digital Image Processing
Simple Camera CalibrationExample
• Set up basic transformation from 3-D object(homogeneous) coordinates to 2-D image (homogeneous)coordinates.
• Let all matrix parameters be unknown.• Keep wi as an unknown in the image plane but explicitly
write out scaled coordinates wixi and wiyi.
Pi =
wixiwiyiwi
= APo =
a11 a12 a13 a14a21 a22 a23 a24a31 a32 a33 a34
xoyozo1
EECS490: Digital Image Processing
Simple Camera CalibrationExample
Multiply out
wixi = a11xo + a12yo + a13zo + a14wiyi = a21xo + a22yo + a23zo + a24wi = a31xo + a32yo + a33zo + a34
Eliminate wi
xi =a11xo + a12yo + a13zo + a14a31xo + a32yo + a33zo + a34
yi =a21xo + a22yo + a23zo + a24a31xo + a32yo + a33zo + a34
EECS490: Digital Image Processing
Simple Camera CalibrationExample
Cross-multiply and rearrange with aij as unknowns
a11xo + a12yo + a13zo + a14 = a31xoxi + a32yoxi + a33zoxi + a34xia21xo + a22yo + a23zo + a24 = a31xoyi + a32yoyi + a33zoyi + a34yi
xoa11 + yoa12 + zoa13 + a14 xoxia31 xiyoa32 xizoa33 = a34xixoa21 + yoa22 + zoa23 + a24 yixoa31 yiyoa32 yizoa33 = a34yi
We can set a34=1 since it simply scales the equations
xoa11 + yoa12 + zoa13 + a14 xoxia31 xiyoa32 xizoa33 = xixoa21 + yoa22 + zoa23 + a24 yixoa31 yiyoa32 yizoa33 = yi
EECS490: Digital Image Processing
Simple Camera CalibrationExample
Write in matrix form
xo yo zo 1 0 0 0 0 xoxi xiyo xizo0 0 0 0 xo yo zo 1 yixo yiyo yizo
a11a12a13a14a21a22a23a24a31a32a33
=xiyi
This is two equations in 11 unknowns. We will need at leastsix calibration pairs (xo,yo,zo)-(xi,yi) to solve this equation.Write the set of equations as UA=X. Since you will havemore equations than unknowns use MATLAB pseudo-inverseto calculate A=U\X