Date post: | 02-Apr-2018 |
Category: |
Documents |
Upload: | naachiz-faqeer |
View: | 220 times |
Download: | 0 times |
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 1/169
Optical Flow
http://www2.ece.ohio-state.edu/~gotardop/
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 2/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 3/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 4/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 5/169
Where did each pixel in image 1 go to in image 2
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 6/169
General Scenario
Possibilities:
• camera moving, stationary scene
• camera stationary, moving objects
• both camera and scene moving
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 7/169
Scene Interpretation
• How is the camera moving?
• How many moving objects are there?
• Which directions are they moving in?
• How fast are they moving?
• Can we recognize their type of motion (e.g.
walking, running, etc.)?
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 8/169
Motion Field & Optical Flow Field
• Motion Field = Real world 3D motion
• Optical Flow Field = Projection of the
motion field onto the 2d image3D motion vector
2D optical flow
vector
vu,u
CCD
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 9/169
Optical Flow
• Goal:
Find for each pixel a velocity vector
which says: – How quickly is the pixel moving across the image
– In which direction it is moving
vu,u
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 10/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 11/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 12/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 13/169
Estimating Optical Flow
• Assume the image intensity is constant
t y x I ,, dt t dy ydx x I ,,
I Time = t Time = t+dt
dy ydx x ,
y x,
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 14/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 15/169
Sobel Example
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 16/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 17/169
Assume constant (u,v) in small neighborhood
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 18/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 19/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 20/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 21/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 22/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 23/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 24/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 25/169
Time t+dt
Aperture problem
Time t
?
Time t+dt
• Where did the blue point move to?
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 26/169
Aperture problem
Sometimes enlarging the aperture can help
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 27/169
Aperture problem in Real Life
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 28/169
Local Patch Analysis
• How certain are the motion estimates?
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 29/169
Edge
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 30/169
Low texture region
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 31/169
High textured region
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 32/169
When does it break?
The screen is
stationary yet
displays motion
Homogeneous
objects
generate zero
optical flow.
Fixed sphere.
Changing light
source.
Non-rigid
texture motion
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 33/169
33
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 34/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 35/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 36/169
Verbal Assignment
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 37/169
Examples of Motion fields
Forward
motion
Rotation Horizontal
translation
Closer objects
appear to move
faster!!
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 38/169
Optical Flow for an Overhead View of a
Dragonfly
• Travelling forward
• Optical flow is faster on the
left & right than in the front
or back
• Rotating to the right
• Optical flow is to the left in all
directions
• If flying a curved path?
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 39/169
What is it useful for? - Flying
Hovering: keep the optic flow zero
everywhere
Avoiding obstacles: the focus of expansion
(FOE) is inside the rock, and the optic flow is
expanding rapidly.
Speed control through dense environments:
by reducing optical flow
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 40/169
Optical Flow as Seen from an Aircraft
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 41/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 42/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 43/169
Block-based motion estimation
• Break image up into square blocks
• Estimate translation for each block
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 44/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 45/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 46/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 47/169
Pyramids
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 48/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 49/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 50/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 51/169
???
….
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 52/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 53/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 54/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 55/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 56/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 57/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 58/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 59/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 60/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 61/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 62/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 63/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 64/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 65/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 66/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 67/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 68/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 69/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 70/169
Low-Res.
High-Res.
Interpolation
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 71/169
1-D Interpolation
Given any two points, )(,,)(, 1100 x f x x f x
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 72/169
1-D Interpolation
The line that interpolates the two points is:
Example :
Find a polynomial that interpolates (1,2) and (2,4).
001
0101
)()()()( x x
x x
x f x f x f x f
x x x f 2112
242)(1
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 73/169
Low-Res.
High-Res.
2-D Interpolation
l
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 74/169
1
1.5
2
1
1.5
2
0
5
10
15
2-D Interpolation
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 75/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 76/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 77/169
Assignment
• At least 10 Problems around us/in our daily
life you see the Optical Flow can solve?
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 78/169
3D Rigid Motion
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 79/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 80/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 81/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 82/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 83/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 84/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 85/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 86/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 87/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 88/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 89/169
I M i M d l
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 90/169
Image Motion Models
• Displacement Model
• Instantaneous Velocity Model
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 91/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 92/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 93/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 94/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 95/169
Perspective Effects:
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 96/169
Perspective EffectsFunny things happen…
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 97/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 98/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 99/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 100/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 101/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 102/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 103/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 104/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 105/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 106/169
3D Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 107/169
3D Transformations• Right-handed / left-handed systems
3D Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 108/169
3D Transformations …
• Positive rotation angles for right-handed
systems: (counter-clockwise rotations)
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 109/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 110/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 111/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 112/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 113/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 114/169
Perspective Effects:
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 115/169
Funny things happen…
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 116/169
Mapping between planes
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 117/169
Mapping between planes
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 118/169
Ax + By + Cz + D = 0
Assignment:
Given an equation of the line. How would you determine a point is on the line or not?
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 119/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 120/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 121/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 122/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 123/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 124/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 125/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 126/169
Geometric Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 127/169
Geometric Transformations
Coordinate Mapping
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 128/169
Coordinate Mapping
• Transformation: ( x’, y’ ) = T {( x, y)}
f(x, y) g(x’, y’ )
T(x, y)
x'
y'
x
y
Rotation
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 129/169
Rotation
• Rotates points by an angle θ about origin
(θ >0: counterclockwise rotation)
• From ABP triangle:
• From ACP ’ triangle: A
BC
Rotation …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 130/169
Rotation …
y
x
y
x*
cossin
sincos
y x, y x ,
x
y
0
Scaling
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 131/169
Scaling
y s
x s
y
x
s
s
y
x
y
x
y
x*
0
0
x
y
0
Scaling …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 132/169
Scaling …
• Uniform vs non-uniform scaling
• Effect of scale factors:
Shear
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 133/169
Shear
• along X-axis
y
dy x
y
xd
y
x
10
1
x
y
0
Shear …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 134/169
Shear …
•along Y-axis
dx y
x
y
x
d y
x
1
01
x
y
0
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 135/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 136/169
Summary
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 137/169
ummary
1
*
100
00
00
1
y
x
s
s
y
x
y
x
1
*
100
0cossin0sincos
1
y x
y x
1
*
100
01
01
1
y x
sh sh
y x
y
x
1
*
100
10
01
1
y
x
t
t
y
x
y
x
Translation Scaling
ShearingRotation
Affine Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 138/169
ff f m
• Perceptually, Affine Transformations are linear combinations of
Affine Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 139/169
ff f
11001
''
2221
1211
y x
t aat aa
y x
y
x
• How to put together
– Translation
– Rotation
– Scaling
– Shearing
Affine Transformations …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 140/169
ff f
• Approximates viewpoint changes for roughly
planar objects and roughly orthographic cameras
Affine Transformations …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 141/169
center at
infinity
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 142/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 143/169
Vanishing
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 144/169
Vanishing
Point
Vanishing
Point Horizon
Line
Assignment
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 145/169
g
• How to put together successive:
– Translation
– Rotation – Scaling
– Shearing
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 146/169
Coordinate Mapping
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 147/169
pp gT(x, y)
x
y
x'
y'
f(x, y) g(x’, y’ )
Transformation: ( x’, y’ ) = T {( x,
Geometric Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 148/169
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 149/169
Rotation …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 150/169
y
x
y
x*
cossin
sincos
y x, y x ,
x
y
0
Scaling
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 151/169
y s
x s
y
x
s
s
y
x
y
x
y
x*
0
0
x
y
0
Scaling …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 152/169
g
• Uniform vs non-uniform scaling
• Effect of scale factors:
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 153/169
Shear
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 154/169
• along X-axis
y
dy x
y
xd
y
x
10
1
x
y
0
Shear …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 155/169
•
along Y-axis
dx y
x
y
x
d y
x
1
01
x
y
0
Translation
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 156/169
• Moves a point to a new location by adding
translation amounts to the coordinates of the
point.
or
or
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 157/169
Summary
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 158/169
1
*
100
00
00
1
y
x
s
s
y
x
y
x
1
*
100
0cossin
0sincos
1
y
x
y
x
1
*
100
01
01
1
y
x
sh
sh
y
x
y
x
1
*
100
10
01
1
y
x
t
t
y
x
y
x
Translation Scaling
ShearingRotation
Affine Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 159/169
• Perceptually, Affine Transformations are linear combinations of
Affine Transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 160/169
11001
''
2221
1211
y x
t aat aa
y x
y
x
• How to put together
– Translation
– Rotation
– Scaling
– Shearing
Affine Transformations …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 161/169
center at
infinity
Perspective Effects:Funny things happen
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 162/169
Funny things happen…
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 163/169
Translation …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 164/169
• e.g., stereo pairs, video sequence assumingsmall camera motion
Affine Transformations …
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 165/169
• Approximates viewpoint changes for roughly
planar objects and roughly orthographic cameras
Overview transformations
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 166/169
1002221
1211
y
x
t aa
t aa
100
2221
1211
y
x
t sr sr
t sr sr
333231
232221
131211
hhh
hhhhhh
100
2221
1211
y
x
t r r
t r r
Projective
8dof
Affine
6dof
Similarity
4dof
Euclidean
3dof
Concurrency, collinearity,order of contact (intersection,
tangency, inflection, etc.),
cross ratio
Parallellism, ratio of areas,
ratio of lengths on parallel
lines (e.g midpoints), linear combinations of vectors
(centroids).
Ratios of lengths, angles.
Angles, lengths, areas.
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 167/169
Assignment
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 168/169
• How to put together successive:
– Translation
– Rotation
– Scaling
– Shearing
Think of This
7/27/2019 Optical Flow, Motion Field, Horn Schunck, Lucas Kanade, Aperture Problem, Optical Flow Failure, Pyramids, Interpolation, 3d Rigid Motion, Displacement, Instantaneous Velocity, Affine Transformation
http://slidepdf.com/reader/full/optical-flow-motion-field-horn-schunck-lucas-kanade-aperture-problem-optical 169/169