Date post: | 13-Dec-2015 |
Category: |
Documents |
Upload: | lizbeth-cole |
View: | 220 times |
Download: | 5 times |
3D IMAGING
Midterm Review
Pinhole cameras
Abstract camera model - box with a small hole in it
Pinhole cameras work in practice
The equation of projection
Cartesian coordinates: We have, by similar triangles, that (x, y, z) -> (f x/z, f
y/z, -f) Ignore the third coordinate, and get
(x,y, z) ( fxz
, fyz)
Prove at home
The camera matrix
Homogenous coordinates for 3D point are (X,Y,Z,T) Homogenous coordinates for point in image are
(U,V,W)
U
V
W
1 0 0 0
0 1 0 0
0 0 1f 0
X
Y
Z
T
Properties of “thin” lens (i.e., ideal lens)
Light rays passing through the center are not deviated.
Light rays passing through a point far away from the center are deviated more.
focal point
f
Thin lens equation (cont’d)
fuv
1 1 1u v f+ =
image
Combining the equations(do at home):
Thin lens equation (cont’d)
The thin lens equation implies that only points at distance u from the lens are “in focus” (i.e., focal point lies on image plane).
Other points project to a “blur circle” or “circle of confusion”
in the image (i.e., blurring occurs).
“circle of confusion”
Depth of field
Changing the aperture size or focal length affects depth of field
f / 5.6
f / 32
Basic models of reflection
Specular: light bounces off at the incident angle E.g., mirror
Diffuse: light scatters in all directions E.g., brick, cloth, rough wood
incoming lightspecular reflection
ΘΘ
incoming light
diffuse reflection
Bidirectional Reflectance Distribution Function (BDRF)
);,,,( eeii
surface normal
( , )
( , )e e e
i i i
L
E
• Model of local reflection that tells how bright a surface appears when viewed from one direction when light falls on it from another
The Retina
Cross-section of eye
Ganglion cell layer
Bipolar cell layer
Receptor layer
Pigmentedepithelium
Ganglion axons
Cross section of retina
© Stephen E. Palmer, 2002
Cones cone-shaped less sensitive operate in high light color vision
Two types of light-sensitive receptors
cone
rod
Rods rod-shaped highly sensitive operate at night gray-scale vision
Color Vision.
400 450 500 550 600 650
RE
LAT
IVE
AB
SO
RB
AN
CE
(%
)
WAVELENGTH (nm.)
100
50
440
S
530 560 nm.
M L
The raster image (pixel matrix)
0.92 0.93 0.94 0.97 0.62 0.37 0.85 0.97 0.93 0.92 0.990.95 0.89 0.82 0.89 0.56 0.31 0.75 0.92 0.81 0.95 0.910.89 0.72 0.51 0.55 0.51 0.42 0.57 0.41 0.49 0.91 0.920.96 0.95 0.88 0.94 0.56 0.46 0.91 0.87 0.90 0.97 0.950.71 0.81 0.81 0.87 0.57 0.37 0.80 0.88 0.89 0.79 0.850.49 0.62 0.60 0.58 0.50 0.60 0.58 0.50 0.61 0.45 0.330.86 0.84 0.74 0.58 0.51 0.39 0.73 0.92 0.91 0.49 0.740.96 0.67 0.54 0.85 0.48 0.37 0.88 0.90 0.94 0.82 0.930.69 0.49 0.56 0.66 0.43 0.42 0.77 0.73 0.71 0.90 0.990.79 0.73 0.90 0.67 0.33 0.61 0.69 0.79 0.73 0.93 0.970.91 0.94 0.89 0.49 0.41 0.78 0.78 0.77 0.89 0.99 0.93
Color Images: Multi-chip
wavelengthdependent
Color Images: Bayer Grid
Estimate RGBat ‘G’ cells from neighboring values
Slide by Steve Seitz
HSV Color Space
Moravec corner detectorChange of intensity for the shift [u,v]:
2,
( , ) ( , ) ( , ) ( , )x y
E u v w x y I x u y v I x y
IntensityShifted intensity
Window function
Four shifts: (u,v) = (1,0), (1,1), (0,1), (-1, 1)Look for local maxima in min{E}
Harris Corner Detector
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmflkgnmhlk
[.,.]h[.,.]f
Image filtering
111
111
111
],[g
Practice with linear filters
Original
111
111
111
000
020
000
-
Sharpening filter- Accentuates differences with
local average
Source: D. Lowe
Other filters
-101
-202
-101
Vertical Edge(absolute value)
Sobel
Spatially-weighted average
0.003 0.013 0.022 0.013 0.0030.013 0.059 0.097 0.059 0.0130.022 0.097 0.159 0.097 0.0220.013 0.059 0.097 0.059 0.0130.003 0.013 0.022 0.013 0.003
5 x 5, = 1
Important filter: Gaussian
Smoothing with Gaussian filter
Smoothing with box filter
A sum of sines
Our building block:
Add enough of them to get any signal f(x) you want!
xAsin(
Fourier analysis in images
Intensity Image
Fourier Image
cos( )A ux vy
Gaussian
Box Filter
Source: S. Marschner
1D example (sinewave):
Aliasing problem
Aliasing in video
Aliasing in graphics
Subsampling without pre-filtering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Template matching
Goal: find in image
Main challenge: What is a good similarity or distance measure between two patches? Correlation Zero-mean correlation Sum Square Difference Normalized Cross
Correlation
Normalized Cross Correlation
mean image patchmean template
5.0
,
2,
,
2
,,
)],[()],[(
)],[)(],[(
],[
lknm
lk
nmlk
flnkmfglkg
flnkmfglkg
nmh
Matching with filters (Normalized Cross Correlation)
Input Normalized X-Correlation Thresholded Image
True detections
Reducing salt-and-pepper noise by Gaussian smoothing
3x3 5x5 7x7
Alternative idea: Median filtering• A median filter operates over a
window by selecting the median intensity in the window
• Is median filtering linear?
Median filterSalt-and-pepper noise Median filtered
Effects of noise• Consider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is the edge?
Solution: smooth first
• To find edges, look for peaks in )( gfdx
d
f
g
f * g
)( gfdx
d
Source: S. Seitz
• Differentiation is convolution, and convolution is associative:
• :
gdx
dfgf
dx
d )(
Derivative theorem of convolution
gdx
df
f
gdx
d
Derivative of Gaussian filter
Final Canny Edges
Alper Yilmaz, CAP5415, Fall 2004
47
Estimating Camera Parameters
11, yx 111 ,, ZYX
222 ,, ZYX
333 ,, ZYX
NNN ZYX ,,
22 , yx
33 , yx
NN yx ,
Ames Room
Julesz: had huge impact because it showed that recognition not needed for stereo.
Epipolar Constraint
Basic Stereo Derivations
Disparity: 21 xxd
d
fBZ
We can always achieve this geometry with image rectification
Image Reprojection reproject image planes onto
common plane parallel to line between optical centers
(Seitz)
Using these constraints we can use matching for stereo
For each epipolar lineFor each pixel in the left image
• compare with every pixel on same epipolar line in right image
• pick pixel with minimum match cost• This will never work, so:
Improvement: match windows
Stereo matching as energy minimizationI1 I2 D
• Energy functions of this form can be minimized using graph cuts
Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy Minimization via Graph Cuts, PAMI 2001
W1(i ) W2(i+D(i )) D(i )
)(),;( smooth21data DEIIDEE 2
,neighborssmooth )()(
ji
jDiDE 221data ))(()( i
iDiWiWE
Active stereo with structured light
L. Zhang, B. Curless, and S. M. Seitz. Rapid Shape Acquisition Using Color Structured Light and Multi-pass Dynamic Programming. 3DPVT 2002
Random Dot
Time to Collision
f
L
v
L
Do
l(t)
t=0t
D(t)
And time to collision:
Can be directly measured from image
Can be found, without knowing L or Do or v !!
2D Motion Field
2D Optical Flow
Apparent motion of image brightness pattern
2D Motion Field and 2D Optical Flow Motion field: projection of 3D motion vectors on image plane
• Optical flow field: apparent motion of brightness patterns
• We equate motion field with optical flow field
0 0
00
Object point has velocity , induces in imagei
ii
P
d d
dt dt
v v
r rv v
Brightness Constancy Equation
Taking derivative wrt time:
Normal Motion/Aperture Problem
Barber Pole Illusion
Full 3D Rotation
cossin0
sincos0
001
cos0sin
010
sin0cos
100
0cossin
0sincos
R
• Any rotation can be expressed as combination of three rotations about three axes.
100
010
001TRR
• Rows (and columns) of R are orthonormal vectors.
• R has determinant 1 (not -1).
Velocity Model in 2D
Perspective projection
321
213
132
VXYZ
VZXY
VYZX
Z
Yfy
Z
Xfx
yv
xu
212331
2
2213
32
1
)(
)(
yf
xyf
yZ
Vx
Z
Vfv
xf
xyf
yxZ
V
Z
Vfu