Home >Documents >Precise Point Set Registration with Color Assisted and...

Precise Point Set Registration with Color Assisted and...

Date post:07-Mar-2021
Category:
View:1 times
Download:0 times
Share this document with a friend
Transcript:
  • Abstract—Iterative closest point (ICP) algorithm, as its accuracy and efficiency, is widely used in rigid registration. However, ICP algorithm is easily failed when point sets lack of structure variety, such as semicircles. To solve this problem, a precise point set registration method for RGB-D data is proposed. Firstly, the color information provides a new information for registration, and the correntropy is introduced to deal with the noises and outliers. With color assisted and correntropy, a more robust objective function is built. Secondly, a variant ICP algorithm is used to deal with optimization problem via multiple iterations. Finally, as shown in the experimental results and scene reconstruction, our method obtains more precise results than other ICP algorithms.

    Keywords—iterative closest point; RGB-D point set registration; color assisted; maximum correntropy criterion

    I. INTRODUCTION

    The point set registration is a classic issue of computer vision which applies to three-dimension reconstruction, medical image processing, simultaneous localization and mapping (SLAM) and so on. For rigid body registration, ICP algorithm is most commonly used [1]. In recent decades, various ICP algorithms are designed to enhance the speed, accuracy and robustness of registration.

    First, to improve the speed, Benjemaa et al. [2] presented a fast registration method with multi-z-buffer technique for 3D sampled surfaces. The Levenberg-Marquardt algorithm was proposed by Fitzgibbon et al. [3] which is both faster and more robust than the ICP algorithm. Jost et al. [4] presented the fast ICP algorithm for shape registration with a tree search method. Rusinkiewicz et al. [5] presented an efficient variant of ICP algorithm which could match two range images fast. Logarithmic data point search and hierarchical model point selection are proposed by Kim et al. [6] to speed up the matching process.

    Second, to enhance the veracity and robustness, a hybrid genetic algorithm was proposed by Silva et al. [7] to obtain precise registration with a robust surface interpenetration measure. Chetverikov et al. [8] proposed the trimmed ICP. Ridene et al. [9] proposed the RANSAC method with adaptive dynamic threshold. Du et al. [10,11] presented a probability ICP algorithm with a Gaussian model which obtain more accuracy and faster registration. Xu et al. [12] proposed a robust registration algorithm by using the maximum correntropy criterion to dispose the outliers and noises

    simultaneously. Du et al. [13] presented a variant ICP algorithm with corner point constraint which realizes precise registration.

    Third, for the past few years, the studies of RGB-D point set registration are presented. Hao et al. [14] presented the 4D ICP algorithm with hue assist in HSV color space. Korn et al. [15] proposed the generalized-ICP algorithm with color assisted in LAB color space. Both of them can register point sets fast but are easily affected by noises and outliers. Danelljan et al. [16] proposed a precise point set registration with a probabilistic framework for color assisted methods but it have high computational complexity and takes more time. Moreover, the above algorithms cannot obtain accurate result in the weak structural point sets. To solve this problem, a precise registration algorithm is presented which could register point sets fast even if they lack sufficient structure changes like curved surface.

    The structure of this paper is as follows. In section 2, the ICP algorithm is introduced briefly. Section 3 introduce the color analysis and correntropy amply, and the new algorithm is presented. In section 4, the experimental results of simulation datasets and real data are given. In section 5, a conclusion is obtained finally.

    II. THE ICP ALGORITHM

    The ICP algorithm was presented for point set registration by McKay and Besl [1]. It aligns two m-dimension point sets in

    m , the data point set 1{ } ( )xN

    i i xX x N= ∈ and the target point set 1{ } ( )y

    Nj j yY y N= ∈ are registered via calculating the

    rigid transformation iteratively. The objective function of registration can be expressed as follows:

    { }2

    ( ) 2R, , ( ) 1,2, , 1

    T

    min || (R ) ||

    . . R R I ,det(R) 1

    x

    y

    N

    i c it c i N i

    m

    x t y

    s t

    ∈ =

    + −

    = = (1)

    where R m m×∈ is a rotation matrix and mt ∈ is a translation vector. The objective function (1) is solved by the ICP algorithm which includes two basic steps as follows:

    First, establish the correspondences between two point sets:

    { }

    21 1 ( ) 2

    ( ) 1,2, ,( ) arg min || (R ) || , 1,2, ,

    y

    k k i k c i xc i N

    c i x t y i N− −∈

    = + − = (2)

    Precise Point Set Registration with Color Assisted and Correntropy for 3D Reconstruction

    Teng Wan1, Shaoyi Du1*, Yiting Xu1, Guanglin Xu1, Yang Yang2*, Yue Gao3, Badong Chen1 1Institute of Artificial Intelligence and Robotics, School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi'an,

    Shaanxi Province, 710049, P.R.China 2Xi’an Jiaotong University Shenzhen Research School, Shenzhen 518057, P.R.China

    3School of Software, Tsinghua University, Beijing, 100084, P.R.China *Corresponding author. E-mail: [email protected], [email protected]

    3970

    2018 IEEE International Conference on Systems, Man, and Cybernetics

    2577-1655/18/$31.00 ©2018 IEEEDOI 10.1109/SMC.2018.00673

  • Second, compute the rigid transformation according to the correspondences:

    T

    21 1 ( ) 2

    R R I ,det(R) 1, 1(R , ) arg min || R(R ) ||

    x

    km

    N

    k k k i k c it i

    t x t t y− −= = =

    = + + − (3)

    The ICP algorithm will repeat the above two steps until convergence.

    III. OUR ALGORITHM

    For weak structural point sets registration, we establish a new objective function with color assist and correntropy based. Then, the ICP algorithm is presented to improve the accuracy and robustness of point set registration.

    A. Problem Statement The ICP algorithm can fast register rigid point sets

    accurately but needs good initial value and sufficient geometry information. However, when point sets are lacking in obvious structure change, the ICP algorithm will easily converge to a local minimum and fail to registration as shown in Figure 1. To deal with this problem, the color information and correntropy are introduced to realize accurate registration.

    (a) Expected registration result (b) ICP registration result

    Figure 1. The registration result of ICP algorithm.

    First, to enhance the veracity of color information, the illumination impact should be removed. Therefore, the HSV (hue, saturation, value) color space is used to replace the RGB color space. After color transformation, the saturation and value show the intensity and brightness of color should be deleted and the hue values remain the position to camera invariant. In this paper, the hue values of the data point set

    1{ } ( )xN

    i i xX x N= ∈ and the target point set 1{ } ( )yN

    j j yY y N= ∈

    are denoted as xih and yjh respectively.

    Second, to deal with the noises and outliers, the maximum correntropy criterion (MCC) [12] is introduced, where MCC can be expressed as follows:

    2 221

    ( , ) exp( || || /(2 ))xN

    i

    g x y x y σ=

    = − − (4)

    where σ is a variance. Therefore, the objective function can be expressed as

    follow:

    2 2

    ( ) 2 ( )1

    2 ,( ) {1,2, , }

    (|| (R ) || ( )max exp( )

    2

    . . R R=I ,det(R) 1.

    x

    y

    Nx y

    i c i i c ii

    R tc i N

    Tm

    x t y w h h

    s t

    σ=

    + − + −−

    =

    (5)

    where ( )c is the correspondence index of the target point, and w represents the weight of hue values xh and yh , which is set to be 20 in this paper.

    B. The Proposed ICP Algorithm To solve the objective function (5), the proposed algorithm

    based on the ICP algorithm can be expressed by two steps.

    Step 1. Set up the ( -1)thk correspondences between two point sets:

    2 21 1 ( ) 2 ( )( ) {1,2, }

    ( ) argmin (|| (R ) || ( ) )y

    x yk k i k c i i c i

    c i Nc i x t y w h h− −

    ∈= + − + − (6)

    To establish a correct correspondence, the HSV (hue, saturation, value) color space is used to remove the illumination impact. As the saturation and value show the intensity and brightness of color do not need to be used and the hue values remain the position to camera invariant, the hue value of each point is used to single out the appropriate points for precise registration. First, the points are divided into 8 classes by their hue values and the classification range as shown in Table 1.

    TABLE I. COLOR CLASSIFICATION ACCORDING TO HUE VALUSES

    Red Orange Yellow Green Cyan Blue Purple Magenta

    minh 0 11 26 35 78 100 125 156

    maxh 10 25 34 77 99 124 155 180

    We calculate color proportion of point set by counting the number of points in each color. If the color proportion of points is less than a threshold a which is set to be 5%, we regard those points as noises. Similarly, if the proportion is more than a threshold b which is set to be 30%, we treat those points as the background points with high possibility. Both noises and background should be rejected for precise registration.

    Step 2. We compute the rotation matrix R and translation vector t based on the correspondences of the ( 1) thk − step:

    2 2

    ( ) 2 ( )1

    2R R I ,det(R) 1,

    (|| (R ) || ( ) )(R , ) arg max exp( )

    2

    x

    k k

    Tm

    Nx y

    i c i i c ii

    k kt

    x t y w h ht

    σ=

    = =

    + − + −= − (7)

    Then, we repeat the above two steps until the registration error is small enough or k reaches the maximum number of iterations.

    C. The Computation of The Rigid Transformation

    In the following, we will derivate the rotation matrix R and translation vector t in detail as follows.

    We calculate the translation vector kt by derivating (7) with respect to translation vector t , we can obtain:

    2 2

    ( ) 2 ( )2

    ||R || ( )( ) 2

    21

    R( ) ( )

    yxi c i i c ik kx

    k

    x t y w h hNi c i

    i

    x t yD t e σ

    σ

    + − + −−

    =

    + −= − (8)

    3971

  • To get the approximate result of kt , we turn the rotation matrix R to 1Rk− and translate the translation vector t to 1kt − . For convenience to expression, let:

    2 21 1 ( ) 2 ( )

    2

    ||R || ( )

    2( )

    yxk i k c i i c ik k

    x t y w h h

    kv i e σ− −+ − + −

    −= (9)

    As ( ) 0D t = , we can get:

    ( )1 1( R ) ( ) / ( )

    x x

    k

    N N

    k c i i k ki i

    t y x v i v i= =

    = − (10)

    Substituting (10) to (6), we can calculate the rotation matrix R k according to the following function:

    2 22 ( )

    2

    ||R || ( )

    2

    R R=I ,det(R) 1 1R arg max

    yxi i i c ikx

    Tm

    m n w h hN

    ki

    e σ− + −

    = =

    = (11)

    where 1 1

    ( ( ) / ( ))x xN N

    i i k i ki i

    m x v i x v i= =

    = − and ( )ki c in y= −

    ( )1 1

    ( ( ) / ( ))x x

    k

    N N

    k c i ki i

    v i y v i= =

    . Consider M and N as xN n× matrices

    where each row represents a point im and in . Therefore, R k can be rewrite it as:

    R R=I ,det(R) 1

    R arg max 1 (R)xT

    m

    Tk N G

    == (12)

    where (R)G g= is a column vectors of length xN and each

    row 2 22 ( )

    2

    || R || ( )exp( )

    2k

    x yi i i c i

    i

    m n w h hG

    σ− + −

    = − . Assume

    ( ) ( )J g diag g= , the singular value decomposition (SVD) method is applied to obtain following equation:

    2(g) USV

    TTN J M

    σ− = (13)

    here, S is a diagonal matrix and U and V are orthogonal matrices. Therefore, the rotation matrix can be denoted as follow:

    1R J Tk U V−= (14)

    where J= ( )idiag d and 1id = or 1− .

    IV. EXPERIMENTAL RESULTS

    In this section, we test our algorithm in simulation and real data to prove the precision and robustness. Then, our algorithm is compared with the ICP, the hue-assist ICP (HICP) and correntropy based ICP (CICP) algorithms. The simulation public RGB-D object dataset is provided by University of Washington, and real data is collected by Kinect.

    A. Simulation Experiment In simulation experiment, the data of rugby is selected for

    model point set. To show our algorithm can obtain precise registration result, the original point set is rotated 60 degrees along the z axis of spatial coordinates to obtain the target point sets. After that, the initial position of data point set and target point set in each data as shown in Figure 2.

    Figure 2. The initail position of data and target point sets.

    Moreover, for quantitative comparison, R 2|| R R ||tε = − and

    2|| ||tt t tε = − are used to measure the registration results. Finally, the proposed algorithm is compared with other algorithms, and the registration results are shown in Table 2 and Figure 3.

    TABLE II. COMPARISON OF SIMULATION RESULTS

    Dataset Error Algorithms

    ICP HICP CICP Ours

    Rugby Rε 7.8 7.71 0.22 2.77e-05

    tε 1.78 1.8 0.03 4.8e-06

    Figure 3. The registration results of constrast experiment.

    As shown in Table 2 and Figure 3, the ICP and CICP algorithm fail to registration because of they only use the coordinate information which is not enough to solve the weak structural problem, such as hemishere. On the contrary, the HICP algorithm is based on color assisted but still fail to registration because the impact of noises and outliers in the point sets.

    In the second part of simulation experiment, we add random color points into the data and target point sets. Therefore, the data sets are displayed in Figure 4. Same as before, our algorithm is compared with other contrast algorithms and the experimental results are shown in Figure 5 and Table 3.

    3972

  • Figure 4. The initail position of data and target point sets.

    TABLE III. COMPARISON OF SIMULATION RESULTS WITH NOISE

    Dataset Error Algorithms

    ICP HICP CICP Ours

    Football Rε 7.1.96 8 1.9 0.04

    tε 1.79 1.87 0.45 3.96e-03

    Figure 5. The registration results with color noise points.

    B. Real Data Experiment In this part, our algorithm is used for real data registration.

    Firstly, we analyze the color distribution of data and target point sets is shown in Figure 8.

    (a) Color distribution of data set (b) Color distribution of target set

    Figure 6. Color distribution of real data.

    Second, according to the rules for color selected, we reject the noise and background points. Only the points with green

    are participated in registration. The registration results of our algorithm for real point sets are shown in Figure 9.

    (a) The initail position of point sets

    (b) ICP (c) HICP

    (d) CICP (e) Ours

    Figure 7. The registration results with real data.

    C. 3D Reconstruction of object In this section, the 3D indoor model is reconstructed by our

    algorithm. First, we obtain the RGB-D information of the room from different visual angles. Secondly, we calculate the rigid transformation of adjacent point sets via our algorithm and the results of registration are shown in Figure 8. Finally, each frame of point set is integrated with the obtained rigid transformation and the results are shown in Figure 9.

    Figure 8. The registration results of each two frames of room.

    3973

  • Figure 9. The reconstruction of room.

    V. CONCLUSION In this paper, to solve the weak structural point sets

    registration problem, we proposed a precise ICP registration algorithm based on color assisted and correntropy. Our algorithm obtains accurate and robust registration results in simulation and real experiments. The contributions of our works are: 1) The color information is used to delete the noise and background point before registration and the hue values are used to further improve the registration accuracy. 2) The maximum correntropy criterion is applied to deal with the noises and outliers and get robust results. In the future works, we will further applied the method for 3D reconstruction of large scale scene.

    ACKNOWLEDGMENT This work was supported by the National Natural Science

    Foundation of China under Grant Nos. 61627811 and 61573274, the Fundamental Research Funds for the Central Universities under Grant Nos. xjj2017005 and xjj2017036, and the project of Shenzhen Technology Plan under Grant No. JCYJ20170816100724089.

    REFERENCES [1] P. J. Besl and N. D. Mckay, "A method for registration of 3-D shapes, "

    IEEE Trans.Pattern Anal. Mach. Intell, vol. 14, pp. 239-256, 1992. [2] R. Benjemaa and F. Schmitt, "Fast global registration of 3D sampled

    surfaces using a multi-z-buffer technique," in Proc. 3-D Digital Imaging and Modeling, 1997. Proceedings, International Conference on Recent Advances in, 1997, pp. 113-120.

    [3] A. W. Fitzgibbon, "Robust registration of 2D and 3D point sets," Image & Vision Computing, vol. 21, no. 13, pp. 1145-1153, 2001.

    [4] T. Jost and H. Hügli, "Fast ICP Algorithms for Shape Registration," in Proc. Dagm Symposium on Pattern Recognition, 2002, pp. 91-99.

    [5] S. Rusinkiewicz and M. Levoy, "Efficient Variants of the ICP Algorithm," in Proc. International Conference on 3-D Digital Imaging and Modeling, 2001. Proceedings, 2002, pp. 145-152.

    [6] D. Kim and D. Kim, "A Fast ICP Algorithm for 3-D Human Body Motion Tracking," IEEE Signal Processing Letters, vol. 17, no. 4, pp. 402-405, 2010.

    [7] L. Silva, O. R. P. Bellon and K. L. Boyer, "Precision range image registration using a robust surface interpenetration measure and

    enhanced genetic algorithms.," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 27, no. 5, pp. 762-776, 2005.

    [8] D. Chetverikov, D. Stepanov and P. Krsek, "Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm," Image & Vision Computing, vol. 23, no. 3, pp. 299-309, 2005.

    [9] T. Ridene, F. Goulette and Ois, "Registration of fixed-and-mobile- based terrestrial Laser data sets with DSM," in Proc. IEEE International Symposium on Computational Intelligence in Robotics and Automation, pp. 375-380,2009.

    [10] S. Du, J. Liu, C. Zhang, J. Zhu, and K. Li, "Probability iterative closest point algorithm for m-D point set registration with noise," Neurocomputing, vol. 157, pp. 187-198, 2015.

    [11] S. Du, J. Liu, B. Bi, J. Zhu, and J. Xue, "New iterative closest point algorithm for isotropic scaling registration of point sets with noise," Journal of Visual Communication & Image Representation, vol. 38, pp. 207-216, 2016.

    [12] G. Xu, S. Du and J. Xue, "Precise 2D point set registration using iterative closest algorithm and correntropy," in International Joint Conference on Neural Networks, 2016, pp. 4627-4631.

    [13] S. Du, W. Cui, L. Wu, S. Zhang, X. Zhang, G. Xu, and M. Xu, "Precise iterative closest point algorithm with corner point constraint for isotropic scaling registration," Multimedia Systems, pp. 1-8, 2017.

    [14] M. Hao, B. Gebre and K. Pochiraju, "Color point cloud registration with 4D ICP algorithm," in Proc. IEEE International Conference on Robotics and Automation, 2011, pp. 1511-1516.

    [15] M. Korn, M. Holzkothen and J. Pauli, "Color supported generalized-ICP," in International Conference on Computer Vision Theory and Applications, 2015, pp. 592-599.

    [16] M. Danelljan, G. Meneghetti, F. S. Khan, and M. Felsberg, "A Probabilistic Framework for Color-Based Point Set Registration," in Proc. Computer Vision and Pattern Recognition, 2016, pp. 1818-1826.

    3974

of 5/5
Abstract—Iterative closest point (ICP) algorithm, as its accuracy and efficiency, is widely used in rigid registration. However, ICP algorithm is easily failed when point sets lack of structure variety, such as semicircles. To solve this problem, a precise point set registration method for RGB-D data is proposed. Firstly, the color information provides a new information for registration, and the correntropy is introduced to deal with the noises and outliers. With color assisted and correntropy, a more robust objective function is built. Secondly, a variant ICP algorithm is used to deal with optimization problem via multiple iterations. Finally, as shown in the experimental results and scene reconstruction, our method obtains more precise results than other ICP algorithms. Keywords—iterative closest point; RGB-D point set registration; color assisted; maximum correntropy criterion I. INTRODUCTION The point set registration is a classic issue of computer vision which applies to three-dimension reconstruction, medical image processing, simultaneous localization and mapping (SLAM) and so on. For rigid body registration, ICP algorithm is most commonly used [1]. In recent decades, various ICP algorithms are designed to enhance the speed, accuracy and robustness of registration. First, to improve the speed, Benjemaa et al. [2] presented a fast registration method with multi-z-buffer technique for 3D sampled surfaces. The Levenberg-Marquardt algorithm was proposed by Fitzgibbon et al. [3] which is both faster and more robust than the ICP algorithm. Jost et al. [4] presented the fast ICP algorithm for shape registration with a tree search method. Rusinkiewicz et al. [5] presented an efficient variant of ICP algorithm which could match two range images fast. Logarithmic data point search and hierarchical model point selection are proposed by Kim et al. [6] to speed up the matching process. Second, to enhance the veracity and robustness, a hybrid genetic algorithm was proposed by Silva et al. [7] to obtain precise registration with a robust surface interpenetration measure. Chetverikov et al. [8] proposed the trimmed ICP. Ridene et al. [9] proposed the RANSAC method with adaptive dynamic threshold. Du et al. [10,11] presented a probability ICP algorithm with a Gaussian model which obtain more accuracy and faster registration. Xu et al. [12] proposed a robust registration algorithm by using the maximum correntropy criterion to dispose the outliers and noises simultaneously. Du et al. [13] presented a variant ICP algorithm with corner point constraint which realizes precise registration. Third, for the past few years, the studies of RGB-D point set registration are presented. Hao et al. [14] presented the 4D ICP algorithm with hue assist in HSV color space. Korn et al. [15] proposed the generalized-ICP algorithm with color assisted in LAB color space. Both of them can register point sets fast but are easily affected by noises and outliers. Danelljan et al. [16] proposed a precise point set registration with a probabilistic framework for color assisted methods but it have high computational complexity and takes more time. Moreover, the above algorithms cannot obtain accurate result in the weak structural point sets. To solve this problem, a precise registration algorithm is presented which could register point sets fast even if they lack sufficient structure changes like curved surface. The structure of this paper is as follows. In section 2, the ICP algorithm is introduced briefly. Section 3 introduce the color analysis and correntropy amply, and the new algorithm is presented. In section 4, the experimental results of simulation datasets and real data are given. In section 5, a conclusion is obtained finally. II. THE ICP ALGORITHM The ICP algorithm was presented for point set registration by McKay and Besl [1]. It aligns two m-dimension point sets in m \ , the data point set 1 {} ( ) x N i i x X x N = G ` and the target point set 1 { } ( ) y N j j y Y y N = G ` are registered via calculating the rigid transformation iteratively. The objective function of registration can be expressed as follows: { } 2 () 2 R, , ( ) 1,2, , 1 T min || (R ) || .. RR I ,det(R) 1 x y N i ci tci N i m x t y st = + = = G " G G G (1) where R m m × \ is a rotation matrix and m t G \ is a translation vector. The objective function (1) is solved by the ICP algorithm which includes two basic steps as follows: First, establish the correspondences between two point sets: { } 2 1 1 () 2 () 1,2, , () arg min || (R ) || , 1, 2, , y k k i k ci x ci N c i x t y i N = + = " G G G " (2) Precise Point Set Registration with Color Assisted and Correntropy for 3D Reconstruction Teng Wan 1 , Shaoyi Du 1 *, Yiting Xu 1 , Guanglin Xu 1 , Yang Yang 2 *, Yue Gao 3 , Badong Chen 1 1 Institute of Artificial Intelligence and Robotics, School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi'an, Shaanxi Province, 710049, P.R.China 2 Xi’an Jiaotong University Shenzhen Research School, Shenzhen 518057, P.R.China 3 School of Software, Tsinghua University, Beijing, 100084, P.R.China *Corresponding author. E-mail: [email protected], [email protected] 3970 2018 IEEE International Conference on Systems, Man, and Cybernetics 2577-1655/18/$31.00 ©2018 IEEE DOI 10.1109/SMC.2018.00673
Embed Size (px)
Recommended