Freeman Ran Sac 2010

8/6/2019 Freeman Ran Sac 2010

1/165

Homographies and RANSAC

Computer vision 6.869

Bill Freeman and Antonio Torralba

March 30, 2011

Wednesday, March 30, 2011


2/165

Homographies and RANSAC

Homographies

RANSAC

Building panoramas

Phototourism

2



3/165

Depth-based ambiguity of position

Camera A Camera B



4/165

Under what conditions can you know where

to translate each point of image A to where

it would appear in camera B (with calibrated

cameras), knowing nothing about image

depths?

4

Camera A Camera B



5/165

(a) camera rotation

5



6/165

and (b) imaging a planar surface

6



7/165

Geometry of perspective projection

7

pinhole

sensor plane



8/165


7

pinhole

inverted copy

of sensor plane



9/165


7

pinhole

inverted copy

of sensor plane

Lets look at this scene from above...



10/165

Two cameras with same center of projection

common pinhole

position of the

cameras



11/165


camera A

common pinhole

position of the

cameras



12/165


camera A camera B

common pinhole

position of the

cameras



13/165


camera A camera B

Can generate any synthetic camera view

as long as it has the same center of projection!

common pinhole

position of the

cameras



14/165

camera A camera B

camera A center

Two cameras with offset centers of projection



15/165

camera A camera B

camera A center

camera B center




16/165

camera A camera B

camera A center

camera B center




17/165

camera A camera B

camera A center

camera B center




18/165

Recap

When we only rotate the camera (around nodal

point) depth does not matter It only performs a 2D warp

one-to-one mapping of the 2D plane

plus of course reveals stuff that was outside the fieldof view

Now we just need to figure out this mapping

A B



19/165

Aligning images: translation?


Ali i i l i ?


20/165


left on top right on top


Ali i i t l ti ?


21/165


Translations are not enough to align the images

left on top right on top


h h


22/165

homography


homograph


23/165

homography


homography


24/165

homography


homography


25/165

homography

How many pairs of points does it take to specify M_10?


Homography


26/165

Homography

PP2

PP1

S

eeSzeliskiSect2

.1.5,

Mappingfro

mone

c

ameratoanother.


Homography


27/165

Homography Projective mapping between any two projection

planes with the same center of projection called Homography

represented as 3x3 matrix in homogenous

coordinates

PP2

PP1H pp

S

eeSzeliskiSect2

.1.5,

Mappingfromone

c

ameratoanother.


Homography


28/165

Homography Projective mapping between any two projection

planes with the same center of projection called Homography

represented as 3x3 matrix in homogenous

coordinates

PP2

PP1H pp

To apply a homography H Compute p = Hp (regular matrix multiply)

Convert p from homogeneous to image

coordinates (divide by w)S

eeSzeliskiSect2.1.5,

Mappingfromone

c

ameratoanother.



29/165

Images of planar

objects, taken bygenerically offsetcameras, are alsorelated by ahomography.

FromS

zeliskibo

ok

camera A

camera B


Measurements on planes


30/165

CSE 576, Spring 2008 Projective Geometry 6





31/165






32/165



Approach: unwarp then measure




33/165







34/165


1 2 3 4

1

2

3

4






35/165


1 2 3 4

1

2

3

4






36/165


1 2 3 4

1

2

3

4






37/165


1 2 3 4

1

2

3

4



How to unwarp?


Image rectification


38/165


Image rectification

To unwarp (rectify) an image solve for homography H given p and p

solve equations of the form: wp = Hp

linear in unknowns: w and coefficients ofH

H is defined up to an arbitrary scale factor

how many points are necessary to solve forH?

p p


Solving for homographies


39/165


g g p




40/165


g g p




41/165


g g p




42/165


g g p




43/165


g g

A h 0




44/165


A h 02n 9 9 2n




45/165


A h 0

Defines a least squares problem:

2n 9 9 2n




46/165


A h 0


2n 9 9 2n

Since h is only defined up to scale, solve for unit vector




47/165


A h 0


2n 9 9 2n

Since h is only defined up to scale, solve for unit vector

Solution: = eigenvector ofATA with smallest eigenvalue Works with 4 or more points


Image warping with homographies


48/165

image plane below

homography so

that image isparallel to floor

homography sothat image is

parallel to rightwall




49/165

image plane below

black areawhere no pixelmaps to

homography so







50/165

image plane below

black areawhere no pixelmaps to

homography so





automatic image mosaicing


51/165


automatic image mosaicing


52/165

Basic Procedure

Take a sequence of images from the same position. Rotate the camera about its optical center (entrance pupil).

Robustly compute the homography transformation

between second image and first.

Transform (warp) the second image to overlap with first.

Blend the two together to create a mosaic.

If there are more images, repeat.


Robust feature matching through


53/165

RANSAC

15-463: Computational Photography

Alexei Efros, CMU, Fall 2005

with a lot of slides stolen from

Steve Seitz and Rick Szeliski

Krister ParmstrandNikon D70. Stitched Panorama. The sky has been retouched. No other image manipulation.


Feature matching


54/165

?

descriptors for left image feature points descriptors for right image feature points


Strategies to match images robustly


55/165

(a) Working with individual features: For each feature

point, find most similar point in other image (SIFTdistance)

Reject ambiguous matches where there are too many similar points

(b) Working with all the features: Given some good featurematches, look for possible homographies relating the twoimages

Reject homographies that dont have many feature matches.

25


(a) Feature-space outlier rejection


56/165

Lets not match all features, but only these that

have similar enough matches?

How can we do it?

SSD(patch1,patch2) < threshold

How to set threshold?Not so easy.


Feature-space outlier rejection


57/165

A better way [Lowe, 1999]:

1-NN: SSD of the closest match

2-NN: SSD of the second-closest match

Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN

That is, is our best match so much better than the rest?




58/165

A better way [Lowe, 1999]:

1-NN: SSD of the closest match

2-NN: SSD of the second-closest match

Look at how much better 1-NN is than 2-NN, e.g. 1-NN/2-NN

That is, is our best match so much better than the rest?




59/165




60/165

Can we now compute H from the bluepoints?




61/165

Can we now compute H from the bluepoints?

No! Still too many outliers

What can we do?


(b) Matching many features--looking fora good homography


62/165

What do we do about the bad matches?

Note: at this point we dont know which ones are good/bad

Simplified illustration with translation instead of homography


RA d SA l C


63/165

RAndom SAmple Consensus

Select one match, count inliers




64/165



0 inliers




65/165



4 inliers




66/165


Select one match, count inliersSelect one match, count inliers

Keep match with largest set of inliers


At the end: Least squares fit


67/165


Find average translation vector,but with only inliers




68/165


Find average translation vector,but with only inliers


Reference


69/165

M. A. Fischler, R. C.Bolles. Random SampleConsensus: A Paradigm forModel Fitting with

Applications to ImageAnalysis and AutomatedCartography. Comm. of theACM, Vol 24, pp 381-395,1981.

http://portal.acm.org/citation.cfm?id=358692

35


RANSAC for estimating homography
http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692http://portal.acm.org/citation.cfm?id=358692


70/165





71/165


RANSAC loop:




72/165


RANSAC loop:

Select four feature pairs (at random)




73/165


RANSAC loop:


Compute homography H (exact)




74/165


RANSAC loop:



Compute inliers where ||pi, Hpi|| <




75/165


RANSAC loop:







76/165


RANSAC loop:




Keep largest set of inliers




77/165


RANSAC loop:




Keep largest set of inliers

Re-compute least-squares H estimate using all ofthe inliers


Simple example: fit a line


78/165

p p

Rather than homography H (8 numbers)

fit y=ax+b (2 numbers a, b) to 2D pairs

37




79/165

p p

38




80/165

p p

Pick 2 points

38




81/165

p p

Pick 2 points

Fit line

38




82/165

p p

Pick 2 points

Fit line

Count inliers

38

3 inlier




83/165

p p

39




84/165

Pick 2 points

39



85/165



86/165

Pick 2 points

Fit line

Count inliers

39

4 inlier




87/165

40




88/165

Pick 2 points

40




89/165

Pick 2 points

Fit line

40




90/165

Pick 2 points

Fit line

Count inliers

40

9 inlier




91/165

41




92/165

Pick 2 points

41




93/165

Pick 2 points

Fit line

41




94/165

Pick 2 points

Fit line

Count inliers

41

8 inlier




95/165

42




96/165

Use biggest set of inliers

42




97/165

Use biggest set of inliers

Do least-square fit

42


RANSAC


98/165

red:rejected by 2nd nearestneighbor criterion

blue:Ransac outliers

yellow:inliers


Robustness


99/165

44


Robustness


100/165

Proportion of inliers in our pairs is G (for

good)

Our model needs P pairs

P=4 for homography

44


Robustness


101/165


good)


P=4 for homography

Probability that we pick P inliers?

44


Robustness


102/165


good)


P=4 for homography


GP

44


Robustness


103/165


good)


P=4 for homography


GP

Probability that after N RANSAC iterations

we have not picked a set of inliers?44



104/165

Robustness: example Matlab: p=4; x=0.5; n=1000; (1-x^p)^n


105/165

Proportion of inliers G=0.5

Probability that we pick P=4 inliers?

0.54=0.0625 (6% chance)

Probability that we have not picked a set of

inliers?

N=100 iterations:

(1-0.54)100=0.00157 (1 chance in 600)

N=1000 iterations:

1 chance in 1e28 45

p ; ; ; ( p)


Robustness: example


106/165



0.34=0.0081 (0.8% chance)


inliers?

N=100 iterations:

(1-0.34)100=0.44 (1 chance in 2)

N=1000 iterations:

1 chance in 3400 46


Robustness: example


107/165



0.14=0.0001 (0.01% chances, 1 in 10,000)


inliers?

N=100 iterations: (1-0.14)100=0.99

N=1000 iterations: 90%

N=10,000: 36%

N=100,000: 1 in 22,00047


Robustness: conclusions


108/165

48




109/165

Effect of number of parameters of model/

number of necessary pairs

48




110/165



Bad exponential

48




111/165



Bad exponential

Effect of percentage of inliers

48




112/165



Bad exponential


Base of the exponential

48




113/165



Bad exponential



Effect of number of iterations

48




114/165



Bad exponential



Effect of number of iterations

Good exponential

48


RANSAC recap


115/165

For fitting a model with low number P of

parameters (8 for homographies)

Loop

Select P random data points

Fit model

Count inliers

(other data points well fit by this model) Keep model with largest number of inliers

49


Example: Recognising


116/165

Panoramas

M. Brown and D. Lowe,University of British Columbia

* M. Brown and D. Lowe. Automatic Panoramic Image Stitching using Invariant

Features. International Journal of Computer Vision, 74(1), pages 59-73, 2007 (pdf

3.5Mb | bib)

* M. Brown and D. G. Lowe. Recognising Panoramas. In Proceedings of the 9th

International Conference on Computer Vision (ICCV2003), pages 1218-1225, Nice,

France, 2003 (pdf 820kb | ppt | bib)


Recognising Panoramas?


117/165


RANSAC for Homography


118/165


RANSAC for Homography


119/165



120/165

Probabilistic model for verification


121/165



122/165

Finding the panoramas


123/165




124/165




125/165


Results


126/165



127/165

61


Benefits of Laplacian image compositing


128/165

62M. Brown and D. Lowe. Automatic Panoramic Image Stitching using Invariant

Features. International Journal of Computer Vision, 74(1), pages 59-73, 2007


Photo Tourism:


129/165

2006 Noah Snavely

Photo Tourism:Exploring Photo Collections in 3D

Noah Snavely

Steven M. SeitzUniversity of Washington

Richard SzeliskiMicrosoft Research

2006 Noah Snavely



130/165

2006 Noah Snavely



131/165

2006 Noah Snavely



132/165

2006 Noah Snavely

15,464



133/165

2006 Noah Snavely

15,464

37,383



134/165

2006 Noah Snavely

15,464

76,389

37,383


Movie


135/165

2006 Noah Snavely


Photo Tourism overview


136/165

2006 Noah Snavely




137/165

2006 Noah Snavely

Input photographs




138/165

2006 Noah Snavely

Scene

reconstruction

Input photographs




139/165

2006 Noah Snavely

Scene

reconstruction

Input photographsRelative camerapositions and orientations

Point cloud

Sparse correspondence




140/165

2006 Noah Snavely

Scene

reconstruction PhotoExplorer

Input photographsRelative camerapositions and orientations

Point cloud

Sparse correspondence




141/165

2006 Noah Snavely

Scene

reconstructionPhoto

ExplorerInput photographs




142/165

2006 Noah Snavely

Scene

reconstructionPhoto

ExplorerInput photographs


Scene reconstruction


143/165

2006 Noah Snavely



Automatically estimate

position, orientation, and focal length of cameras

3D positions of feature points


144/165

2006 Noah Snavely







145/165

2006 Noah Snavely

Feature detection







146/165

2006 Noah Snavely

Feature detection

Pairwisefeature matching







147/165

2006 Noah Snavely

Feature detection


Correspondenceestimation







148/165

2006 Noah Snavely

Feature detection


Incrementalstructure

from motion

Correspondenceestimation


Feature detection

Detect features using SIFT [Lowe, IJCV 2004]


149/165

2006 Noah Snavely


Feature detection



150/165

2006 Noah Snavely

Wednesday March 30 2011

Feature detection



151/165

2006 Noah Snavely


Feature detection



152/165

2006 Noah Snavely


Feature detection



153/165

2006 Noah Snavely


Feature matching

Match features between each pair of images


154/165

2006 Noah Snavely


Feature matching

Match features between each pair of images


155/165

2006 Noah Snavely


Feature matchingRefine matching using RANSAC [Fischler & Bolles 1987]

to estimate fundamental matrices between pairs(See 6.801/6.866 for fundamental matrix, or Hartley and Zisserman, Multi-View

Geometry.

See also the fundamental matrix song: http://danielwedge.com/fmatrix/ )
http://danielwedge.com/fmatrix/http://danielwedge.com/fmatrix/


156/165

2006 Noah Snavely


Feature matchingRefine matching using RANSAC [Fischler & Bolles 1987]

to estimate fundamental matrices between pairs(See 6.801/6.866 for fundamental matrix, or Hartley and Zisserman, Multi-View

Geometry.

See also the fundamental matrix song: http://danielwedge.com/fmatrix/ )
http://danielwedge.com/fmatrix/http://danielwedge.com/fmatrix/http://danielwedge.com/fmatrix/


157/165

2006 Noah Snavely

http://danielwedge.com/fmatrix/


158/165

Structure from motion


159/165

2006 Noah Snavely

Camera 1

Camera 2

Camera 3




160/165

2006 Noah Snavely

Camera 1

Camera 2

Camera 3



p1

p4

p3p2


161/165

2006 Noah Snavely

Camera 1

Camera 2

Camera 3

p5

p6

p7



p1

p4

p3p2


162/165

2006 Noah Snavely

Camera 1

Camera 2

Camera 3

R1,t1R2,t2

R3,t3

p5

p6

p7



p1

p4

p3p2


163/165

2006 Noah Snavely

Camera 1

Camera 2

Camera 3

R1,t1R2,t2

R3,t3

p5

p6

p7



p1

p4

p3p2

minimizef(R,T,P)


164/165

2006 Noah Snavely

Camera 1

Camera 2

Camera 3

R1,t1R2,t2

R3,t3

p5

p6

p7

( , , )


Links

Code available: http://phototour.cs.washington.edu/bundler/

http://phototour.cs.washington.edu/ http://livelabs.com/photosynth/
http://livelabs.com/photosynth/http://livelabs.com/photosynth/http://phototour.cs.washington.edu/http://phototour.cs.washington.edu/http://phototour.cs.washington.edu/bundler/http://phototour.cs.washington.edu/bundler/


165/165

2006 Noah Snavely

http://www.cs.cornell.edu/~snavely/

http://www.cs.cornell.edu/~snavely/http://www.cs.cornell.edu/~snavely/http://livelabs.com/photosynth/

Date post:	07-Apr-2018
Category:	Documents
Upload:	chaohong-wu
View:	220 times
Download:	0 times

Freeman Ran Sac 2010

Documents