Point Cloud Registration Using A Viewpoint...

transcript

Local-to-Global Point Cloud Registration Using a Viewpoint

Dictionary

David Avidar

MSc Research under the supervision of

Prof. David Malah and Dr. Meir Barzohar

02/07/18

Omek Consortium

Signal and Image

Processing Lab

SIPL Annual Event 2018

What is a point cloud?

• Point cloud: a set of points given in a common coordinate system 𝑥

What is a point cloud?

• Point cloud: a set of points given in a common coordinate system

• Typically represented as a list of coordinates:

𝑥1 𝑦1 𝑧1⋮ ⋮ ⋮𝑥𝑁 𝑦𝑁 𝑧𝑁

• Acquisition methods: Structure-from-Motion, stereo

cameras Laser sensors (e.g., LiDAR) IR cameras, mostly indoors (e.g.,

Kinect, RealSense)

𝑥𝑦

Problem definition

• Alignment between:- a local cloud (e.g., terrestrial LiDAR)

- and a global cloud (e.g., airborne LiDAR)

Problem definition

• Alignment between:- a local cloud (e.g., terrestrial LiDAR)

- and a global cloud (e.g., airborne LiDAR)

6DoF:Translation

+Rotation

Motivation

• High-resolution urban modeling Airborne LiDAR –

coarse overall structure Terrestrial LiDAR –

fine details

• Accurate localization in large scale urban environment Better reliability than GPS No dependency on

inertial sensors

Motivation

fine details

• Accurate localization in large scale urban environment Better reliability than GPS No dependency on

inertial sensors

Motivation

fine details

• Accurate localization in large scale urban environments Better reliability than GPS No dependency on

inertial sensors

Motivation

fine details

• Accurate localization in large scale urban environments Better reliability than GPS No dependency on inertial

sensors for position and orientation (Yaw,Pitch,Roll)

Challenges

Figure: Williams et al., 2013

• Different point density distributions: Airborne: high density on

horizontal surfaces Terrestrial: high density on

vertical surfaces

• Different types of occlusion Airborne: mostly vertical

occlusion Terrestrial: mostly horizontal

occlusion

• Different typical noise levels Airborne: 𝜎 ≈ 15𝑐𝑚 @ 1𝑘𝑚 Terrestrial: 𝜎 ≈ 0.5𝑐𝑚 @ 100𝑚

Challenges

vertical surfaces

occlusion

Challenges

vertical surfaces

occlusion

Outline

• Problem definition and motivation

• Prior art - keypoint-based registration

• Registration using a viewpoint dictionary

• Summary

Keypoint-based registration - 1

1. Keypoint detection Intrinsic Shape Signature (ISS)

(Zhong, 2009)

3D Harris (Sipiran et al., 2011)

3D SIFT (Rusu et al., 2011)

(Zhong, 2009)

2. 3D descriptor computation Spin-Images (Johnson, 1997)

Fast Point Feature Histogram (FPFH) (Rusu et al., 2009)

Rotational Projection Statistics (RoPS) (Guo et al., 2013)

(Zhong, 2009)

2. 3D descriptor computation Spin-Images (Johnson, 1997)

Fast Point Feature Histogram (FPFH) (Rusu et al., 2009)

Rotational Projection Statistics (RoPS) (Guo et al., 2013)

3. Descriptor matching k-d tree (Bentley, 1975), FLANN (Muja and Lowe, 2009)

4. Coarse registration RANdom SAmple Consensus

(Fischler and Bolles, 1981) or variants

5. Registration refinement Iterative Closest Point

(Besl and McKay, 1992) (Chen and Medioni, 1991)

Many other variations…

5. Registration refinement Iterative Closest Point

(Besl and McKay, 1992) (Chen and Medioni, 1991)

Many other variations…

Find nearest neighbors (NN)

From 𝑃 to 𝑄

Find 𝑹, 𝒕 thatminimize MSE

𝑀𝑆𝐸 =1

𝑖=1

𝑹𝒑𝑖 + 𝒕 − 𝒒𝑁𝑁 𝒑𝑖

• Limitations: Captures only local information (sensitive to large

differences in point density)

Lack of salient features in mostly flat scenes

Ambiguity in repetitive scenes

Found to be unreliable for terrestrial Vs. airborneregistration

Outline

• Summary

Registration using a viewpoint dictionary

• Main concepts: Large-scale global cloud

Characterize each viewpoint using the geometric information of its environment

Local-to-global registration

Viewpoint-based dictionary

Viewpoint dictionary search

(scene matching)

Block diagram

“Local-to-global point cloud registration using a dictionary of viewpoint descriptors”, ICCV 2017, Venice, Italy

Block diagram

• Define a viewpoint grid over the global cloud

• Create a descriptor for each viewpoint - panoramic range-image

Viewpoint dictionary creation

Viewpoint grid

Dictionary range-image

Local viewpoint descriptor creation

Viewpoint grid, Local sensor

• Create a panoramic range-image of the local cloud

Local range-image

Dictionary range-image

Block diagram

• If GPS location is available, define dictionary search area around it

• Otherwise, search over entire grid

Candidate viewpoint selection - 1

Viewpoint grid, Search area, GPS

• Match local and dictionary range-images using phase correlation Select a small subset of candidate grid viewpoints (e.g.,

10 with largest phase correlation peaks)

Phase correlation

• Match local and dictionary range-images using phase correlation

Phase correlation

• Main steps:• Compute 2D DFT of both images𝑮𝑎 𝑢, 𝑣 = ℱ 𝑔𝑎 𝑥, 𝑦 𝑮𝑏 𝑢, 𝑣 = ℱ 𝑔𝑏 𝑥, 𝑦

• Compute normalized cross-power spectrum

𝑅 𝑢, 𝑣 =𝑮𝑎∗𝑮𝑏

𝑮𝑎∗𝑮𝑏

• Find the phase correlation using inverse Fourier transform𝑟 𝑥, 𝑦 = ℱ−1 𝑅

• Recover image shift - find peak in phase correlation Δ𝑥, Δ𝑦 = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥,𝑦 𝑟 𝑥, 𝑦

ℱ 𝑔𝑎 ⋆ 𝑔𝑏⋆ = cyclic cross-

correlation

Phase correlation

𝑔𝑎 𝑥, 𝑦 𝑔𝑏 𝑥, 𝑦

Cyclic cross-correlation𝑔𝑎 ⋆ 𝑔𝑏

Phase correlation 𝑟 𝑥, 𝑦

Images: Foroosh et al., 2002

Phase correlation - example

• Match local and dictionary range-images using phase correlation Select a small subset of candidate grid viewpoints (e.g.,

10 with largest phase correlation peaks)

• Grid viewpoints

• Ground truth

• 10 candidates with largest phase correlation peaks

Block diagram

RMSE verification

• Select a smaller subset of candidate viewpoints with lowest RMSE between the local and global clouds (e.g., lowest 3)

• Grid viewpoints

• Ground truth

• 10 candidates with largest phase correlation peaks

• 3 candidates with lowest RMSE

Block diagram

Registration refinement

• Refine using ICP and select registration with lowest RMSE

• Ground truth sensor location

Test dataset (Vancouver)

• Global cloud area: ~1𝑘𝑚2

• # Local clouds: 104

Registration results - 1

RegistrationRefinement

Method

Localization error [m]

Mean STD Max

Using GPS (30m search radius)

0.40 0.20 1.13

No GPS (search entire grid)

0.44 0.27 1.94

RRE [deg]

Mean STD Max

0.67 0.30 1.73

0.78 0.39 1.96

Runtime [sec]*

Mean STD Max

2.0 0.4 3.7

14.5 1.2 19.4

* Run on PC (i7-5820K CPU @ 3.30 GHz), MATLAB implementation

(Registration time per local cloud)

RRE: Relative Rotation Error

𝑹𝑹𝑬 = 𝒀𝒂𝒘 𝒆𝒓𝒓𝒐𝒓 +𝑷𝒊𝒕𝒄𝒉 𝒆𝒓𝒓𝒐𝒓 +𝑹𝒐𝒍𝒍 𝒆𝒓𝒓𝒐𝒓

Our registration

Ground Truth

Comparison to keypoint-based registration methods

Registration method # Tested local clouds

# Local clouds with localization

error < 3m

Avg. runtime per local cloud [sec]

Mean STD

FPFH (Rusu et al., 2009) 24 6 (25%) 53.7 23.1

RoPS (Guo et al., 2015) 104 8 (7.7%) 88.7 23.7

Viewpoint dictionary (proposed)

104 104 (100%) 2.0 0.4

• Using the proposed method, the maximal localization error over all 104 clouds was 1.13m The mean error was 0.40m (STD=0.20m)

Outline

• Summary

Summary• Proposed a viewpoint-dictionary-based point

cloud registration method

Achieved significant improvement in registration performance over state-of-the-art keypoint-based methods (FPFH, RoPS)

Viewpoint grid, Search area, GPS

• Take-home message: When matching two very

different types of 3D data (e.g., terrestrial vs. airborne LiDAR), better to use viewpoint descriptors than local features

02/07/18

Omek Consortium

Signal and Image

Processing Lab

SIPL Annual Event 2018

Thank You!Questions?

Point Cloud Registration Using A Viewpoint...

Documents