Post on 04-Jan-2016
transcript
Epipolar geometry: basic equation0Fxx'T
separate known from unknown
0'''''' 333231232221131211 fyfxffyyfyxfyfxyfxxfx
0,,,,,,,,1,,,',',',',',' T333231232221131211 fffffffffyxyyyxyxyxxx
(data) (unknowns)(linear)
0Af
0f1''''''
1'''''' 111111111111
nnnnnnnnnnnn yxyyyxyxyxxx
yxyyyxyxyxxx
If A has rank 8 soln unique up to a scale factorFor noisy data, rank of A might be larger than 8 least squares solnMinimize || Af || subject to ||f|| = 1 singular vector for smallest sing. value of AThis is called “8 point” algorithm
the singularity constraint: 0Fe'T 0Fe 0detF 2Frank
T333
T222
T111
T
3
2
1
VσUVσUVσUVσ
σσ
UF
SVD from linearly computed F matrix (rank 3)
Replace F with F‘ that minimizes Forbenius norm || F – F‘ ||F subject to det(F‘) = 0
T222
T111
T2
1
VσUVσUV0
σσ
UF'
FF'-FminCompute closest rank-2 approximation
Two steps: (a) linear soln F obtained from sing. Vect corresp. To smallest singular value (b) replace F by F’ the closest singular matrix to F under Forbenius norm
|| A||F = square root of sum of squares of ai,j
the minimum case – 7 point correspondences
0f1''''''
1''''''
777777777777
111111111111
yxyyyxyxyxxx
yxyyyxyxyxxx
T9x9717x7 V0,0,σ,...,σdiagUA
7x298 0]VA[V T8
T ] 000000010[Ve.g.V
Assume matrix A has rank 7 7 correspondences
A is 7x9 ;
If A rank 7 soln to Af =0 is 2D space of the form α F1 + ( 1 – α ) F2 whereF1 and F2 are matrices corresponding to generators f1 and f2 of the right null space of ADet (F) = 0 det (α F1 + ( 1 – α ) F2 ) = 0 cubic polynomial in α either one or 3 real solns either one or 3 FThis is called the “7 point “ algorithm.
0
1´´´´´´
1´´´´´´
1´´´´´´
33
32
31
23
22
21
13
12
11
222222222222
111111111111
f
f
f
f
f
f
f
f
f
yxyyyyxxxyxx
yxyyyyxxxyxx
yxyyyyxxxyxx
nnnnnnnnnnnn
~10000 ~10000 ~10000 ~10000~100 ~100 1~100 ~100
!Orders of magnitude differenceBetween column of data matrix least-squares yields poor results
the NOT normalized 8-point algorithm
Transform image to ~[-1,1]x[-1,1]
(0,0)
(700,500)
(700,0)
(0,500)
(1,-1)
(0,0)
(1,1)(-1,1)
(-1,-1)
1
1500
2
10700
2
Least squares yields good results (Hartley, PAMI´97)
the normalized 8-point algorithm
Translate and scale each image so that the centroid of the reference points is at the origin of the coordinates and the RMS distance of the points from the origin is √2
algebraic minimization
Find F‘ with rank 2 directly, i.e. Find singular matrix F‘ that minimizes || A F‘|| subject to ||F‘|| = 1
• Arbitrary singular matrix F can be written as F = M [e]x where M is nonsingular and [e]x is any skew symmetrix matrix
• For now, assume e is known
• F = M [e]x f = Em where f and m are vectors corresponding to F and M matrices and
• Minimize || A E m || subject to || Em || = 1
• Apply Algorithm A.5.6 from Hartley and Zisserman
Gold standard
Maximum Likelihood Estimation
i
iiii dd 22 'x̂,x'x̂,x
(= least-squares for Gaussian noise)
0x̂F'x̂ subject to T
iXt],|[MP'0],|[IP Parameterize:
Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi
iiii XP''x̂,PXx̂
Minimize cost using Levenberg-Marquardt(preferably sparse LM, see book)
(overparametrized)
xi and x’I measured correspondence; and true correspondence
Find F to minimize
i'x̂ix̂
Symmetric epipolar error
2
221
2
2T2
1T
T
FxFx
1
Fx'Fx'
1Fxx'
i
iiii dFd2T2 x'F,xx,x'
Minimize distance of a point from its projected epipolar line
Algorithm comparison
• Normalized 8 point (alg. 11.1)• Min. Algebraic error while imposing singularlity (Alg.
11.2)• Gold standard algorithm (Alg. 11.3)• N = total number of correspondences• Choose n out of N points randomly 100 times• Find average residual error vs. n
• Squared distance between a point’s epipolar line and the other image averaged over all N
• This is not directly minimized by any of the 3 algorithms
NdFdi
iiii /)x'F,xx,x'(2T2
Recommendations:
1. Do not use unnormalized algorithms
2. Quick and easy to implement: 8-point normalized (alg. 11.1)
3. Better: enforce rank-2 constraint during minimization (alg. 11.2)
4. Best: Maximum Likelihood Estimation (alg. 11.3) (minimal parameterization, sparse implementation)
Automatic computation of F
(i) Interest points(ii) Putative correspondences(iii) RANSAC (iv) Non-linear re-estimation of F(v) Guided matching(repeat (iv) and (v) until stable)
• Extract feature points to relate images
• Required properties:• Well-defined
(i.e. neigboring points should all be different)
• Stable across views(i.e. same 3D point should be extracted as feature for neighboring viewpoints)
Feature points
homogeneous
edge
corner
dxdyyxwyI
xI
W yIxI
,
M
If M has two small eigen value homogeneous; if one small and one large eigenvalue edge; if two large eigenvalues corner
(e.g.Harris&Stephens´88; Shi&Tomasi´94)
MTSSD
Find points that differ as much as possible from all neighboring points
Feature = local maxima (subpixel) of F(1, 2)
Feature points
),( yxT
Evaluate NCC for all features withsimilar coordinates
Keep mutual best matchesKeep mutual best matches
Still many wrong matches!Still many wrong matches!
10101010 ,,´´, e.g. hhww yyxxyx
?
Feature matching
0.96 -0.40 -0.16 -0.39 0.19
-0.05 0.75 -0.47 0.51 0.72
-0.18 -0.39 0.73 0.15 -0.75
-0.27 0.49 0.16 0.79 0.21
0.08 0.50 -0.45 0.28 0.99
1 5
24
3
1 5
24
3
Gives satisfying results for small image motions
Feature example
• Requirement to cope with larger variations between images• Translation, rotation, scaling• Foreshortening• Non-diffuse reflections• Illumination
geometric transformations
photometric changes
Wide-baseline matching…
Wide baseline matching for two different region types
(Tuytelaars and Van Gool BMVC 2000)
Wide-baseline matching…
restrict search range to neighborhood of epipolar line (1.5 pixels)
relax disparity restriction (along epipolar line)
Finding more matches
Step 1. Extract featuresStep 2. Compute a set of potential matchesStep 3. do
Step 3.1 select minimal sample (i.e. 7 matches)
Step 3.2 compute solution(s) for F
Step 3.3 determine inliers
until (#inliers,#samples)<95%
samples#7)1(1
matches#inliers#
#inliers 90%
80%
70% 60%
50%
#samples
5 13 35 106 382
Step 4. Compute F based on all inliersStep 5. Look for additional matchesStep 6. Refine F based on all correct matches
(generate hypothesis)
(verify hypothesis)
RANSAC
How many samples?
Choose N so that, with probability p, at least one random sample is free from outliers. e.g. p=0.99
sepN 11log/1log
peNs 111
proportion of outliers es 5% 10% 20% 25% 30% 40% 50%2 2 3 5 6 7 11 173 3 4 7 9 11 19 354 3 5 9 13 17 34 725 4 6 12 17 26 57 1466 4 7 16 24 37 97 2937 4 8 20 33 54 163 5888 5 9 26 44 78 272 117
7
• Absence of sufficient features (no texture)• Repeated structure ambiguity
(Schaffalitzky and Zisserman, BMVC‘98)
• Robust matcher also finds Robust matcher also finds support for wrong hypothesissupport for wrong hypothesis• solution: detect repetition solution: detect repetition
More problems:
geometric relations between two views is fully
described by recovered 3x3 matrix F
two-view geometry