Registration of Colored 3D Point Clouds with a Kernel-based Extension to the
Normal Distributions Transform
Benjamin Huhle1, Martin Magnusson2, Wolfgang Straßer1, Achim J. Lilienthal2
1 WSI/GRIS, University of Tübingen, Germany2 AASS, Dept. of Technology, Örebro University,
Sweden
05/23/2008 ICRA '08, Pasadena
Benjamin Huhle - WSI/GRIS, Tübingen
Motivation
2
• point cloud registration for Localization & Mapping
• Problems– geometric features
(structure) required– small field-of-view– noise
• additional color data available!
Outline of the talk
3
• Related Work– Normal Distributions Transform (NDT)– Vision-aided registration (SIFT-Features)– Combined approach (SIFT-Features+NDT)
• Color-NDT– straightforward approach fails– Kernel-based Color-NDT
• Experiments– mobile robot with time-of-flight camera
Normal Distributions Transform (NDT)
4
• Biber & Straßer, 2003– cell grid– approximate point
distributions– multiple overlapping
grids– optimization using
analytical (2nd order) derivatives from: Biber, 2003
3D Normal Distributions Transform
5
• in 3D
• comparison with ICP: Magnusson et al., 2007
Vision-Aided RegistrationAndreasson & Lilienthal, 2007
6
• robust registration with image features (SIFT)– feature detection in images– lookup of 3D coordinates
• challenges:– noise– dynamic environments from: Andreasson et al., 2007
Combined Energy ApproachHuhle, Jenke & Straßer, 2008
7
• Sum of NDT score and feature distances
• must favor features (small )
Ad-hoc approach to using color with NDT
8
• colored point cloud: [x,y,z,r,g,b]• 6D color–space distribution
toy example:• 2D position• 1 color-
dimension (hue)
• model is 3D normal distribution
Single-Mode Color–Space Distribution
9
• conditional distributions of 2 test-points
Single-Mode Color–Space Distribution
10
• ... for a test-point with different color
Kernel-Based Color-NDT
11
• Gaussian mixture-model in color-space (EM-Algorithm)
• components are weighting kernels for point distributions
• use 3 kernels
• for each kernel: compute spatial Normal Distribution using– weighted mean– weighted covariance
Toy Example revisited
12
• perfect match of model and data
Toy Example revisited
13
• ... test point with different color
Registration with Color-NDT
14
• optimize score (mixture model of normal distributions)
• Newton's method• translation + rotation vector (6D parameters)
Experiments
15
• Time-of-flight camera – PMDTec 19k– 160x120 pixels– 30° fov– significant noise level
• additional color camera
Results
16
• 21 incrementally registered frames• odometry as initial poses
Results using combined approach
17
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
Results using combined approach
18
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
Results using combined approach
19
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
Results using combined approach
20
SIFT only
• 11 frames
combinedSIFT+NDT
combinedSIFT+Color-NDT
Conclusion
21
• Color-NDT:– more robust/stable– more weight on Color-NDT score in combined
approach– can fix inaccuracies of SIFT registration
• towards registration of low-end sensor data with integrated use of color and depth data
Thank you for your attention!
22
• Peter Biber and Wolfgang Straßer. The Normal Distributions Transform: A New Approach to Laser Scan Matching. In Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2743–2748, 2003
• Martin Magnusson, Achim Lilienthal and Tom Duckett, Scan Registration for Autonomous Mining Vehicles Using 3D-NDT. Journal of Field Robotics, 24:10, 2007, pp. 803-827
• Henrik Andreasson and Achim J. Lilienthal, Vision Aided 3D Laser Based Registration. Proceedings of the 3rd European Conference on Mobile Robots (ECMR), 2007
• Benjamin Huhle, Philipp Jenke and Wolfgang Straßer. On-the-Fly Scene Acquisition with a Handy Multi-Sensor System. Int. J. of Intelligent Systems Technologies and Applications, to appear, 2008