Point Cloud Registration Using A Viewpoint...

Local-to-Global Point Cloud Registration Using a Viewpoint

Dictionary

David Avidar

MSc Research under the supervision of

Prof. David Malah and Dr. Meir Barzohar

02/07/18

Omek Consortium

Signal and Image

Processing Lab

SIPL Annual Event 2018

What is a point cloud?

• Point cloud: a set of points given in a common coordinate system 𝑥

𝑦

𝑧

2

What is a point cloud?

• Point cloud: a set of points given in a common coordinate system

• Typically represented as a list of coordinates:

𝑥1 𝑦1 𝑧1⋮ ⋮ ⋮𝑥𝑁 𝑦𝑁 𝑧𝑁

• Acquisition methods: Structure-from-Motion, stereo

cameras Laser sensors (e.g., LiDAR) IR cameras, mostly indoors (e.g.,

Kinect, RealSense)

𝑥𝑦

𝑧

3

Problem definition

• Alignment between:- a local cloud (e.g., terrestrial LiDAR)

- and a global cloud (e.g., airborne LiDAR)

5

Problem definition

• Alignment between:- a local cloud (e.g., terrestrial LiDAR)

- and a global cloud (e.g., airborne LiDAR)

6DoF:Translation

+Rotation

6

Motivation

• High-resolution urban modeling Airborne LiDAR –

coarse overall structure Terrestrial LiDAR –

fine details

• Accurate localization in large scale urban environment Better reliability than GPS No dependency on

inertial sensors

7

Motivation



fine details

• Accurate localization in large scale urban environment Better reliability than GPS No dependency on

inertial sensors

8

Motivation



fine details

• Accurate localization in large scale urban environments Better reliability than GPS No dependency on

inertial sensors

9

Motivation



fine details

• Accurate localization in large scale urban environments Better reliability than GPS No dependency on inertial

sensors for position and orientation (Yaw,Pitch,Roll)

10

Challenges

Figure: Williams et al., 2013

• Different point density distributions: Airborne: high density on

horizontal surfaces Terrestrial: high density on

vertical surfaces

• Different types of occlusion Airborne: mostly vertical

occlusion Terrestrial: mostly horizontal

occlusion

• Different typical noise levels Airborne: 𝜎 ≈ 15𝑐𝑚 @ 1𝑘𝑚 Terrestrial: 𝜎 ≈ 0.5𝑐𝑚 @ 100𝑚

11

Challenges




vertical surfaces



occlusion


12

Challenges




vertical surfaces



occlusion


13

Outline

• Problem definition and motivation

• Prior art - keypoint-based registration

• Registration using a viewpoint dictionary

• Summary

14

Keypoint-based registration - 1

1. Keypoint detection Intrinsic Shape Signature (ISS)

(Zhong, 2009)

3D Harris (Sipiran et al., 2011)

3D SIFT (Rusu et al., 2011)

15



(Zhong, 2009)



2. 3D descriptor computation Spin-Images (Johnson, 1997)

Fast Point Feature Histogram (FPFH) (Rusu et al., 2009)

Rotational Projection Statistics (RoPS) (Guo et al., 2013)

16



(Zhong, 2009)



2. 3D descriptor computation Spin-Images (Johnson, 1997)

Fast Point Feature Histogram (FPFH) (Rusu et al., 2009)

Rotational Projection Statistics (RoPS) (Guo et al., 2013)

17

3. Descriptor matching k-d tree (Bentley, 1975), FLANN (Muja and Lowe, 2009)


4. Coarse registration RANdom SAmple Consensus

(Fischler and Bolles, 1981) or variants

18




19




5. Registration refinement Iterative Closest Point

(Besl and McKay, 1992) (Chen and Medioni, 1991)

Many other variations…

20




5. Registration refinement Iterative Closest Point

(Besl and McKay, 1992) (Chen and Medioni, 1991)

Many other variations…

Find nearest neighbors (NN)

From 𝑃 to 𝑄

Find 𝑹, 𝒕 thatminimize MSE

𝑀𝑆𝐸 =1

𝑁

𝑖=1

𝑁

𝑹𝒑𝑖 + 𝒕 − 𝒒𝑁𝑁 𝒑𝑖

2

21


• Limitations: Captures only local information (sensitive to large

differences in point density)

Lack of salient features in mostly flat scenes

Ambiguity in repetitive scenes

Found to be unreliable for terrestrial Vs. airborneregistration

22

Outline




• Summary

23

Registration using a viewpoint dictionary

• Main concepts: Large-scale global cloud

Characterize each viewpoint using the geometric information of its environment

Local-to-global registration

Viewpoint-based dictionary

24

Viewpoint dictionary search

(scene matching)

Block diagram

“Local-to-global point cloud registration using a dictionary of viewpoint descriptors”, ICCV 2017, Venice, Italy

25

Block diagram

26

Block diagram

27

• Define a viewpoint grid over the global cloud

• Create a descriptor for each viewpoint - panoramic range-image

Viewpoint dictionary creation

Viewpoint grid

Dictionary range-image


28

Local viewpoint descriptor creation

Viewpoint grid, Local sensor

• Create a panoramic range-image of the local cloud

Local range-image


29

Block diagram

30

• If GPS location is available, define dictionary search area around it

• Otherwise, search over entire grid

Candidate viewpoint selection - 1

Viewpoint grid, Search area, GPS

31

• Match local and dictionary range-images using phase correlation Select a small subset of candidate grid viewpoints (e.g.,

10 with largest phase correlation peaks)


Phase correlation

32


• Match local and dictionary range-images using phase correlation

Phase correlation

33

• Main steps:• Compute 2D DFT of both images𝑮𝑎 𝑢, 𝑣 = ℱ 𝑔𝑎 𝑥, 𝑦 𝑮𝑏 𝑢, 𝑣 = ℱ 𝑔𝑏 𝑥, 𝑦

• Compute normalized cross-power spectrum

𝑅 𝑢, 𝑣 =𝑮𝑎∗𝑮𝑏

𝑮𝑎∗𝑮𝑏

• Find the phase correlation using inverse Fourier transform𝑟 𝑥, 𝑦 = ℱ−1 𝑅

• Recover image shift - find peak in phase correlation Δ𝑥, Δ𝑦 = 𝑎𝑟𝑔𝑚𝑎𝑥𝑥,𝑦 𝑟 𝑥, 𝑦

ℱ 𝑔𝑎 ⋆ 𝑔𝑏⋆ = cyclic cross-

correlation

Phase correlation

34

𝑔𝑎 𝑥, 𝑦 𝑔𝑏 𝑥, 𝑦

Cyclic cross-correlation𝑔𝑎 ⋆ 𝑔𝑏

Phase correlation 𝑟 𝑥, 𝑦

Images: Foroosh et al., 2002

Phase correlation - example

35


• Match local and dictionary range-images using phase correlation Select a small subset of candidate grid viewpoints (e.g.,

10 with largest phase correlation peaks)

• Grid viewpoints

• Ground truth

• 10 candidates with largest phase correlation peaks

36

Block diagram

37

Block diagram

38

RMSE verification

• Select a smaller subset of candidate viewpoints with lowest RMSE between the local and global clouds (e.g., lowest 3)

• Grid viewpoints

• Ground truth

• 10 candidates with largest phase correlation peaks

• 3 candidates with lowest RMSE

39

Block diagram

40

Registration refinement

• Refine using ICP and select registration with lowest RMSE

• Ground truth sensor location

41




42




43

Test dataset (Vancouver)

• Global cloud area: ~1𝑘𝑚2

• # Local clouds: 104

44

Registration results - 1

RegistrationRefinement

Method

Localization error [m]

Mean STD Max

Using GPS (30m search radius)

0.40 0.20 1.13

No GPS (search entire grid)

0.44 0.27 1.94

RRE [deg]

Mean STD Max

0.67 0.30 1.73

0.78 0.39 1.96

Runtime [sec]*

Mean STD Max

2.0 0.4 3.7

14.5 1.2 19.4

* Run on PC (i7-5820K CPU @ 3.30 GHz), MATLAB implementation

(Registration time per local cloud)

RRE: Relative Rotation Error

𝑹𝑹𝑬 = 𝒀𝒂𝒘 𝒆𝒓𝒓𝒐𝒓 +𝑷𝒊𝒕𝒄𝒉 𝒆𝒓𝒓𝒐𝒓 +𝑹𝒐𝒍𝒍 𝒆𝒓𝒓𝒐𝒓

45


Our registration

46


Ground Truth

47

Comparison to keypoint-based registration methods

48

Registration method # Tested local clouds

# Local clouds with localization

error < 3m

Avg. runtime per local cloud [sec]

Mean STD

FPFH (Rusu et al., 2009) 24 6 (25%) 53.7 23.1

RoPS (Guo et al., 2015) 104 8 (7.7%) 88.7 23.7

Viewpoint dictionary (proposed)

104 104 (100%) 2.0 0.4

• Using the proposed method, the maximal localization error over all 104 clouds was 1.13m The mean error was 0.40m (STD=0.20m)

Outline




• Summary

49

Summary• Proposed a viewpoint-dictionary-based point

cloud registration method

50

Achieved significant improvement in registration performance over state-of-the-art keypoint-based methods (FPFH, RoPS)

Viewpoint grid, Search area, GPS

• Take-home message: When matching two very

different types of 3D data (e.g., terrestrial vs. airborne LiDAR), better to use viewpoint descriptors than local features

02/07/18

Omek Consortium

Signal and Image

Processing Lab

SIPL Annual Event 2018

Thank You!Questions?

Date post:	17-Aug-2020
Category:	Documents
Upload:	others
View:	2 times
Download:	0 times

Point Cloud Registration Using A Viewpoint...

Documents