1

Comparative Survey on Fundamental Matrix Estimation

Computer Vision and Robotics Group Institute of Informatics and Applications

University of Girona. Girona (Spain)

{armangue, jpages, qsalvi, jbatlle}@eia.udg.es

X. Armangué, J. Pagès, J. Salvi and J. Batlle

2

Contents:

1.- Stereo Vision

2.- The Epipolar Geometry

3.- Computing the Fundamental Matrix

3.1.- Linear methods

3.2.- Iterative methods

3.3.- Robust methods

4.- Experimental Results

3

Y

Z

X

I

M

f

mI’

m’

X’

Y’

Z’

f’

OO’

C C’

Zw

Xw

Yw

Captured Image Captured Image

• The principle of Triangulation

3D point

Stereo Vision

4

T21 = [R1(:,1:3) T1];invT21=inv(T21);P2Dw1=invT21*[Xu1; Yu1; f1; 1];Ocw1=invT21(:,4);

T22 = [R2(:,1:3) T2];invT22=inv(T22);P2Dw2=invT22*[Xu2; Yu2; f2; 1] Ocw2=invT22(:,4);WOc2

{W}

WOc1

WP2D2

WP2D1u

v

pq

pq=Ocw2(1:3)-Ocw1(1:3);

u=P2Dw1(1:3,i)-Ocw1(1:3); v=P2Dw2(1:3,i)-Ocw2(1:3); alpha=(pq'*v-(pq'*u)*norm(v)^2/(u'*v))/((u'*v)-norm(u)^2*norm(v)^2/(u'*v));beta=(-pq'*u+alpha*norm(u)^2)/(u'*v); r=Ocw1(1:3)+alpha.*u; s=Ocw2(1:3)+beta.*v; P3Dstereo = (r+s)./2;disterror = norm(r-s);

WP3D

• Getting the 3D pointStereo Vision

5

Camera Pose

Stereo Vision

I I C WC Ws m A K M

' ' '' '' ' ' 'I I C W

C Ws m A K M

3D Reconstruction:

'';I I

C CA A

'';C C

W WK K

Optics and Internal Geometry

Constraints:

• The Correspondence Problem.• Active Systems: Non static Camera Position nor Orientation

Epipolar Geometry

I’I

OC’OC

mm’

MOI

OI’

OW

6

The Epipolar Geometry

OW coincides with OC’

I’I

OC’OC

mm’

e e’

MOI

OI’

lm’l’m

OW

CKC’

'';I I

C CA A

'''''CCC

CCCCCC OKORtO

Intrinsic

Extrinsic

0'' mT lm

ml m '' F0' m

T lm

'' mlm F 1][' ARAF xTT t

1][ ARAF xT t

0'' mm T F0'mmT F

TFF

7

Epipole

Epipole

Epipolar lines

Epipolar lines

Area 1

Area 2

CorrespondencepointsZoom

Area 1

Zoom Area 2

Epipolar geometry of Camera 1 Epipolar geometry of Camera 2

The Epipolar Geometry

8

Computing the Fundamental Matrix: The Survey

• Linear Methods.

• Iterative Methods.

• Robust Methods.

Driving INRIA Aerial Underwater

9

Linear Iterative Robust Optimisation Rank-2

Seven points (7p) X — yes

Eight points (8p) X LS or Eig. no

Rank-2 constraint X LS yes

Iterative Newton-Raphson

X LS no

Linear iterative X LS no

Non-linear minimization in parameter space

X Eig. yes

Gradient technique X LS or Eig. no

M-Estimator X LS or Eig. no / yes

LMedS X 7p / LS or Eig. no

RANSAC X 7p / Eig no

LS: Least-SquaresEig: Eigen Analysis

Computing the Fundamental Matrix: Survey

10

Computing the Fundamental Matrix: Linear Methods

• Seven points It depends extremely on the seven points used.

• Eight points• Least-squares minimization• Eigen analysis Better results increasing the number of points. Eigen minimization is more realistic.

• Analytical method with Rank-2 constraint. Forces a unique epipole, but results do not improve.

Least-squares Eigen Analysis

11

Linear Methods

Seven Points

8-points Least Squares 8-points Eigen Analysis

ComputingNoiseOutliers

X X

12

Computing the Fundamental Matrix: Iterative Methods

• Iterative Newton-Raphson

Good results. Depends on the initial guess.

• Linear iterative method.

• i is based on the F of the previous step.

• F is computed by using Least-squares in each iteration.

Improves linear least-squares considerably.

• Non-linear minimization in parameter space

Forces a Rank-2 F but the discrepancy is high.

• Gradient technique

• Least-squares or Eigen Analysis.

Better results with eigen analysis.

2 2

Fmin ,F ' ' , Fi i i i

i

d m m d m m

2 2 2 2 2 21 2 1 2

Fmin F ' ' 'T

i i i ii

m m g g l l l l

1/ 2

22

2 2 2 2F1 2 1 2

1 1min F '

' 'T

i i i ii

m ml l l l

13

Iterative Methods

Newton-Raphson Linear minimization

Forcing Rank-2

ComputingNoiseOutliers

X X

14

Computing the Fundamental Matrix: Robust Methods

• Robust Methods:• M-Estimator

Reduces effect of outliers weighting the residual of each point.

Lots of methods just defining a new weight-function.

• Leasts-squares, eigen analysis, Torr, etc...

Good results in the presence gaussian noise in point localization.

Bad results in the presence of outliers.

• LMedS & RANSAC

Points used to compute F randomly selected.

LMedS uses the median of distances.

RANSAC maximises de number of inliers.

LMedS is more restrictive than RANSAC (removes more points).

Once the outliers are removed, F is recalculated.

15

Robust Methods

Initial Matching

16

Robust Methods: M-estimator Torr

17

Robust Methods: RANSAC

18

Robust Methods: LMedS Eigen

ComputingNoiseOutliers

X X

19

Methods Implemented with mean and std. of error:1.- seven points; 2.- least-squares (LS); 3.- orthogonal LS; 4.- rank-2 constraint; 5.- iterative lineal using LS;6.- iterative Newton-Raphson using LS; 7.- minimization in parameter space using eigen; 8.- gradient using LS;9.- gradient using eigen; 10.- M-Estimator using LS; 11.- M-Estimator using eigen;12.- M-Estimator proposed by Torr; 13.- LMedS using LS; 14.- LMedS using eigen; 15.- RANSAC using eigen.

Linear Iterative Robust

Computing the Fundamental Matrix: Results

20

Computing the Fundamental Matrix: Time

Linear Iterative Robust

21

Conclusions

• Survey of fifteen methods of computing F.

• Conditions: Gaussian Noise, Outliers and Real Images.

Linear methods: Good results if the points are well located and the correspondence problem previously solved (without outliers).

Iterative methods: Can cope with noise but inefficient in the presence of outliers.

Robust methods: Cope with both noise and outliers.

• Eigen Analysis is better than least-squares.

• Rank-2 matrices are preferred if a good geometry is required.

• Better results if data is previously normalized.• traslated so that they centroid is placed at the origin.• Scaled so that the mean of distances to the origin is

• [-1, 1] 2

Code available: http://eia.udg.es/~armangue/research.html