58
1 Range Maps Alignment
1
Range Maps Alignment
2
The problem
Many range maps for a single object Each one in its own 3D reference frame Need to put them in a common frame
Example: Scanning the Minerva
Last scan: Minolta laser scanner (03/2002)
No. range scans: 297
Sampling resolution: ~0.3 mm
3
Rotations
2D rotation of a point p around the origin: movement of p such that its distance from the origin is constant
5
Rotations
3D rotation of a point p around a principal axis
6
Rotation group in 3D: SO(3)
Set: of all the rotations about the origin = all orthonormal matrices (det = 1)
Operation: composition Properties:
7
Matrices frames↔
every 3x3 orthonormal matrix is a orthogonal frame whose axes are the matrix's columns
every orthonormal matrix is a rotation → every frame is a rotation
8
Rotations about an arbitrary axis
Can be obtained by composing:– The rotation between the red frame and the identity– The rotation about the Z axis➔ The rotation between the identity and the red frame
9
Other representations...
Rotations can be represented in other ways:
– Euler angles– Axis-angle (without matrices)– Quaternions– ...
10
Rototranslations
A rototranslation is a rotation followed by a translation
In (handy) matrix form:
11
Manual alignment
Specifies few correspondences between two range maps and solve for the rototranslation:
12
Correspondence problem 1
Given two set of n points, find the roto-translation the maps the first on the second
– At least 3 points necessary (d in d-dimensions)– In the practical case the more the better– The “more accurate” the better
13
Correspondence problem 1
Given two set of n points, find the roto-translation the maps the first on the second
– At least 3 points necessary (d in d-dimensions)– In the practical case the more the better– The “more accurate” the better
14
Correspondence problem 2
Solution:1. solve the least square problem
2. find the rotation part of R:
15
Correspondence problem 3
Better solutions exists...➔ With quaternions [BeslMcKay92]
– With matrices [Horn88, Faugeras86]
..a lot of linear algebra calculations we are not interested in right now..
16
Finding Correspondences
Finding correspondences manually is ok for 2 range maps, boring for 3, unbearable for 400...
Quest for automatic solutions: finding “features” points on two (or more) surfaces and their correspondence
17
Alignment pipeline
Picking point of the surface that are notable w.r.t. to some criterion
Not just 4 points but a lot of them
Understand the correspondence between notable points
A lot of correspondence. Some correct, some uncorrect: need to tell them apart
All the range maps are aligned on the basis of the correspondences.
Finalize the alignment considering the whole surfaces
18
Alignment pipeline
Picking point of the surface that are notable w.r.t. to some criterion
Not just 4 points but a lot of them
Understand the correspondence between notable points
A lot of correspondence. Some correct, some uncorrect: need to tell them apart
All the range maps are aligned on the basis of the correspondences.
Finalize the alignment considering the whole surfaces
19
Feature Desc. Example [Pingi05]
Yellow: low varianceBlue: medium var.Red: high variance
A set of points are chosen in the region of medium variance
The distance between two points on two range maps is computed as:
20
Feature Descriptors
21
Feature Descriptors
22
Quality of a feature descriptor
A good descriptor finds notable pointsIt is resilient to:
– Rotation (i.e. point of view)– Noise– Scale
There is a good metric to compare two feature points
– Avoid false positives– Avoid false negatives
23
Feature desc.:Curvature Maps [Gatzke05]
Descriptor of a vertex v as a function of the curvature in the neighborhood of v
(approximated) geodesic distance from v
v Plot of a function of the distance from v
24
Feature desc.:Curvature Maps
1D descriptor: curvature values along the rings
26
Feature desc.:Curvature Maps
2D descriptor: curvature values inside the fans. Similar to 1D but:
– the spokes are sampled
– F is a function of the spoke index...
...the function depends on the starting spoke
– Try all the rotations when comparing and pick the best
– It gives a hint on the rotation between meshes
27
Curvature Maps
28
Curvature Maps
Most unique vertices are found by sorting by the sum of similarity of each vertex with all the others.
29
Feature desc.:Spin Images [Johnson97]
Estimate the normal per-vertex
Place a rectangle along the normal Accumulate projection of all the vertices on the
image while it rotates
30
Feature desc.:Spin Images
Metric: image comparison pixel-by-pixel Descriptor depends on:
– size of the image
– Resolution of the image
– Resolution of the mesh
– Angle step of the rotation
Many false positives & false negatives: it suffers from symmetries!
31
Multi-scale features [Li05]
➔ Range maps normal, curvature maps, spin images all have a crucial parameter: the scale
➔ Range maps-> size of kernel in pixels
➔ Curvature maps-> size of the ring
➔ Spin images-> radius of the ring
➔ In an uncontrolled setting it is not obvious that they work (i.e. probably they don't)
➔ Multi-scale approaches consider multiple scales at the same time....
32
Multi-scale features [Li05]
➔ Smoothing a mesh: move its points so that the surface is smoother (i.e. lower curvatures)
➔ Larger-scale features “survive” to smoothing
The nose is still a feature
The hair disapperared..
34
Multi-scale features [Li05]
➔ Let➔ Build a sequence of smoothed version
(called levels) j=1..n for h=F^j (i.e. increasing)
➔ Define the distance between levels as:
b(q , R)={i :∥pi−q∥<R }
d j (i):=nij ( pi
j− pij−1)
35
Multi-scale features [Li05]
➔ Def: A point i on level j is a neighborhood maximum if:
➔ Def: neighborhood minimum. The same with “<” instead of “>”
d j (i)>d j−1(i ' )∀ i '∈b( pi , h j−1)d j (i)>d j (i ' )∀ i '∈b( pi , h j)d j (i)>d j+1(i ' )∀ i '∈b( pi , h j+1)
36
Multi-scale features [Li05]
➔ Mapping of
Level 0
Level 8
d j(i)
37
Multi-scale features [Li05]
➔ Circled points are extrema (i.e. neighborhood max or min)Level 0
Level 8
38
Alignment pipeline
Picking point of the surface that are notable w.r.t. to some criterion
Not just 4 points but a lot of them
Understand the correspondence between notable points
A lot of correspondences. Some correct, some uncorrect: need to tell them apart
All the range maps are aligned on the basis of the correspondences.
Finalize the alignment considering the whole surfaces
39
Random Sample Consensus [Fischler81]
Problem data– Input: a set of N correspondences– 4 enough to provide a solution – Output: the solution that satisfies as
many as possibleAlgorithm:
– For a predefined number of trials T:
● Pick 4 correspondences● Find the candidate rototranslation Rt● S* = set of satisfied correspondences
– Return the Rt corresponding to the largest set
40
RANSAC: parameters
How many trials for success?
Rule of thumb: E[k]+2*SD[k]
k=number of trials
w= probof picking a correct correspondenceb=wn probof picking acorrect set
E [k ]=average number of trial=∑i=0
∞
i P (i)
E [k ]=b+2(1−b)b+3(1−b)2b+...+i(1−b)i−1b+.......
E [k ]=1/b
41
RANSAC: parameters
How many trials for succeeding with given prob?
How do we know about b ?
ps= probability of successs(1− ps)= probability of failure
(1−b)k<(1− p s)
k>log(1− ps)
log(1−b)
42
4 Point Congruent Sets [Aiger08]
Forget notable points, find an efficient way of exploring the search space of all the transformations
– Find a way to recognize similar points quicklyAlgorithm:
– Iteration:● Pick a base: four points in the first mesh (RM)● Find four points in the second that are affine (i.e.
there is a rototranslation..)– Use Ransac to find the best match
43
4 Point Congruent Sets
How many possible corresponding quadruples?
– O(n^4), n number of verticesPrune the search space...
P Q
44
4 Point Congruent Sets
Choosing the base on mesh P:– 4 coplanar points (a,b,c,d)– Characterize the quadruple as the barycentric coordinates
of the intersection point of the segments (a,b) and (c,d)
45
4 Point Congruent Sets
Finding the corresponding quadruple on Q:– Take every pairs of points at distance ||a-b|| or ||c-d|| [ O (n) ]– For each pair (=segment) generate the intermediate points at barycentric
coordinates r1,r2,1-r1,1-r2➔ Put these 3D points in a search data structure [ O (n^2 log n) ]– Every pair of coincident points corresponds to a possibly affine quadruple
46
4 Point Congruent Sets
Robust to noiseDoes not use feature points
Amount of overlap between P and Q:– If known, the basis are chosen as wide
as the overlap– If not known, the algorithm is launched
for 1, 0.9,0.8 ...
47
4 Point Congruent Sets
If overlap is known:
α= fraction of overlapb=α
4
N=number of points
k>log (1− ps)
log (1−α4)
48
Alignment pipeline
Picking point of the surface that are notable w.r.t. to some criterion
Not just 4 points but a lot of them
Understand the correspondence between notable points
A lot of correspondences. Some correct, some uncorrect: need to tell them apart
All the range maps are aligned on the basis of the correspondences.
Finalize the alignment considering the whole surfaces
49
Fine Alignment
INPUTrough alignment
OutputPrecise alignment withinterror tolerance
53
ICP: Iterative Closest Point [BeslMcKay92]
Q
P
54
ICP: convergence proof
55
Improvements over ICP
Trying to avoid matching wrong correspondences. Discard matches if:
– Their normals are more than than 45° apart [Godin94]
– If the closest point is on the boundary [Turk94]
Converge faster [Pulli99]
– Keep only the p% closest points (p increasing along the iterations)
– Discard pairs more distant than a fixed threshold (decreasing along the iterations)
56
Improvements over ICP
Point-to-plane minimization– Minimize the distance from the point
and the plane tangent to the closest point
– Faster convergence
57
Global Registration
A set of range map can be registered pairwise, but pairwise alignment leads to accumulation of errors when walking across the surface of an object
An optimal solution should minimize distances between all range scans simultaneously
– i.e. to solve the global registration
58
Global Registration
Formalization: a connected graph– Nodes are RM– Arcs are pairwise alignment– Arcs are double labeled:
● Alignment value (rototranslation)● alignment error
– Goal: minimize the sum of arcs' error
59
Global Registration
Many different global registration approaches
Global optimization– Simulated annealing [Blais95]– Springs [Stoddart, Hilton 96]– Hierarchical linearized least squares [Neugebauer
97]Incremental
– Iterate moving one scan, keep others fixed [Bergevin et al 96]
– Speed up using hardware [Benjemaa, Schmitt 97]– Preserve pairwise matches [Pulli 99]
60
Global Registration [Pulli99]
Idea: incrementally add the most connected RM and distribute the error
61
Global Registration [Pulli99]
The virtual pair– Principle: trust the pairwise matching
The error is only the one due to the change from the pairwise matching
62
[BeslMcKay92] Besl, Paul J.; N.D. McKay (1992). "A Method for Registration of 3-D Shapes".
IEEE Trans. on Pattern Analysis and Machine Intelligence (Los Alamitos, CA, USA: IEEE Computer Society) 14 (2): 239-256
[Blais95]
Blais, G. and M.D. Levine, Registering Multiview Range Data to Create 3D Computer Objects. IEEE
Transactions on PAMI, 1995. 17(8): p. 820-4.
[Stoddart96] Stoddart, A.J., Hilton, A., 1996. Registration of multiple point sets. In: Proc. 13th Int. Conf. on Pattern Recognition
[Neugebauer 97] Reconstruction of real-world objects via simultaneous registration and robust
combination of multiple range images. International Journal of Shape Modeling,
3(1,2):71-90, 1997
[Bergevin96]
Bergevin, R., Soucy, M., Gagnon, H., and Laurendeau,
D. "Towards a General Multi-View Registration Technique," Trans.
PAMI, Vol. 18, No. 5, 1996
63
[Benjemaa97]
Benjemaa, R. and Schmitt, F. "Fast Global Registration of 3D Sampled Surfaces Using a Multi-Z-Buffer Technique," Proc.
3DIM, 1997.
[Pulli 99]
Pulli, K. "Multiview Registration for Large Data Sets," Proc. 3DIM, 1999.
[Horn88]
Horn, B., Hilden, H., and Negahdaripour, S. "Closed-Form Solution of Absolute Orientation Using Orthonormal Matrices," JOSA
A, Vol. 5, No. 7, 1988.
[Faugeras86] Faugeras, O. and Hebert, M. "The Representation, Recognition, and Locating of 3-D Objects,"
Int. J. Robotic Res., Vol. 5, No. 3, 1986
[Pingi05]
Exploiting the scanning sequence for automatic registration of large sets of range maps
Paolo Pingi, Andrea Fasano, Paolo Cignoni, Claudio Montani, Roberto Scopigno
Computer Graphics Forum, Volume 24, Number 3, page 517-526 - 2005
64
[Gatzke05]
Curvature Maps for Local Shape Comparison, Timothy Gatzke , Cindy Grimm , Michael Garland , Steve Zelinka
In Shape Modeling International, SMI 05
[Johnson97]
Johnson, A. and Hebert, M. "Surface Registration by Matching Oriented Points," Proc. 3DIM, 1997.
[Li05]
Xinju Li, Igor Guskov: Multiscale Features for Approximate Alignment of Point-based Surfaces. Symposium on Geometry Processing 2005: 217-226
[Fischler81]
Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography
Martin A. Fischler,Robert C. Bolles Communications of the ACM Volume 24 Issue 6, June 1981 Pages 381 - 395
[Turk94]
Turk, G. and Levoy, M. "Zippered Polygon Meshes from
Range Images," Proc. SIGGRAPH, 1994.
[GOdin94]
Godin, G., Rioux, M., and Baribeau, R. "Three-dimensional
Registration Using Range and Intensity Information," Proc. SPIE:
Videometrics III, Vol. 2350, 1994.
