+ All Categories
Home > Documents > 3D registration of the point cloud data using ICP...

3D registration of the point cloud data using ICP...

Date post: 13-Oct-2020
Category:
Upload: others
View: 2 times
Download: 0 times
Share this document with a friend
of 6 /6
3D Registration of the Point Cloud Data Using ICP Algorithm in Medical Image Analysis Martin Sinko, Patrik Kamencay, Robert Hudec, Miroslav Benco Department of multimedia and information-communication technologies University of Zilina Zilina, Slovakia [email protected], [email protected], [email protected], [email protected] Abstract—In this article, we are combining an advanced implementation of the popular ICP algorithm using the transformation of 3D invariant properties based on 3D scale- invariant feature transform to register 3D free-form closed surfaces (3D model of the human skull). Unlike point and surface registers, our method based on the ICP algorithm better captures the bulk nature of data such as bone thickness. The proposed ICP algorithm is divided into three main steps: the 3D function extraction, comparison of the Euclidean metric distance and gross alignment and ICP enhancement. The input to our system are biomedical data (CT, MRI). Our proposed method first performs image segmentation and then 3D reconstruction of the human skull. In addition, 3D models for the point clouds (PCs) are created. Finally, the differences between point clouds of 3D models are visualized. We compare the performance of three different algorithms with their accuracy and robustness. The proposed solution is fast and accurate compared to the state of art in the field of reconstruction and registration of medical imagery. We apply our algorithm to study skull deformities. Experiments on skull models demonstrate the efficiency and robustness of our algorithm. Keywords—3D registration; CT; MRI; ICP; point cloud I. INTRODUCTION Registration is a critical issue for various problems in computer vision and computer graphics. The overall aim is to find the best alignment between two objects or between several instances of the same object, in order to bring the shape data into the same reference system. To solve 3D registration problem (pose estimation, alignment and motion estimation), we decided to use a well-known algorithm for registering two point sets (in 2D or 3D) called the iterative closest point algorithm (ICP) proposed by Besl and Mckay [1]. It has been used in numerous real-world applications in computer vision such as 3D object scanning, 3D mapping or 3D localization. ICP is one of the most popular algorithms for registration because of its conceptual simplicity, as well as its superior performance in practice. Independently, Chen and Medioni [2] published a similar iterative scheme using a different pairing procedure based on surface normal vector. Few new variants of ICP were also provided by Rusinkiewicz and Levoy [3] where they proposed new classified variants of registration process. These modifications of ICP seek to improve robustness, speed and precision. For solving the local minima problem there were proposed several novel approaches in a past few years. For robust Euclidian alignment of 3D point sets D. Chetverikov and D. Stepanov [4] proposed the trimmed ICP algorithm which is based on the consistent use of the Least Trimmed Squares in all phases of the operation. The outline of this paper is organized as follows. In the first part (Section II), the state of the art is discussed. In next part (Section III), the 3D feature detection is described. The multimodal registration is described in the Section IV. The obtained experimental results are listed in Section V. Finally, the Section VI concludes and suggests the future work. II. STATE OF THE ART Our work aims to register two 3D models of human skull obtained from CT/MRI images. We have done this by programming graphical user interface (GUI) inside Matlab environment. We were focused on the whole workflow from image segmentation and visualization; 3D reconstruction and visualization; to 3D registration. However main function of our program is registration of 3D models and estimating the metric error. For this purpose, a review of 3D/2D registration methods can be found here [5] and for 3D modelling there is a comparative study of state-of-the-art evolutionary image registration methods aimed on evolutionary algorithms [6]. Another approach was done by J. Yang and H. Li [7] in the paper called solving 3D registration efficiently and globally optimal, they combined ICP with branch-and-bound (BnB) scheme which searches the 3D motion space efficiently. For volumetric registration of 3D solid shapes Y. Sahilliog and L. Kavan [8] achieved higher efficiency by leveraging a template tetrahedral mesh. In contrast to point- and surface-based registration techniques, their method better captures volumetric nature of the data, such as bone thickness. And lastly B. and D. Lee [9] presented an online learning approach to 3D object registration that vastly improves the performance of ICP methods. They approach achieves better robustness and stable convergence by learning generalized distance functions directly from a stream of object depth data. III. 3D FEATURE DETECTION The SIFT algorithm is robust against local distortions and errors in feature detection. It is considered to be one of the most effective descriptors currently available [9]. For each feature identified, a characteristic local SIFT function marker is created. The SIFT 3D descriptor is characterized by the use of the 16 × 16 × 16 gradient orientation distribution that surrounds the function position and the cube is divided into 4 × 4 × 4 sub areas. The example of the 3D SIFT structure is shown in Fig. 1. This publication is the result of the project implementation: Centre of Translation Medicine, ITMS 26220220021 supported by the Research & Development Operational Programme funded by the ERDF and by the project "Competence Centre for research and development in the field of diagnostic and therapy of oncological diseases", ITMS: 26220220153, co-funded from EU sources and the European Regional Development Fund. 978-1-5386-4759-2/18/$31.00 ©2018 IEEE
Transcript
Page 1: 3D registration of the point cloud data using ICP ...static.tongtianta.site/paper_pdf/75a31750-6b10-11e... · The search algorithm for the closest point is iterative, the point is

3D Registration of the Point Cloud Data Using ICP Algorithm in Medical Image Analysis

Martin Sinko, Patrik Kamencay, Robert Hudec, Miroslav Benco Department of multimedia and information-communication technologies

University of Zilina Zilina, Slovakia

[email protected], [email protected], [email protected], [email protected]

Abstract—In this article, we are combining an advanced

implementation of the popular ICP algorithm using the transformation of 3D invariant properties based on 3D scale-invariant feature transform to register 3D free-form closed surfaces (3D model of the human skull). Unlike point and surface registers, our method based on the ICP algorithm better captures the bulk nature of data such as bone thickness. The proposed ICP algorithm is divided into three main steps: the 3D function extraction, comparison of the Euclidean metric distance and gross alignment and ICP enhancement. The input to our system are biomedical data (CT, MRI). Our proposed method first performs image segmentation and then 3D reconstruction of the human skull. In addition, 3D models for the point clouds (PCs) are created. Finally, the differences between point clouds of 3D models are visualized. We compare the performance of three different algorithms with their accuracy and robustness. The proposed solution is fast and accurate compared to the state of art in the field of reconstruction and registration of medical imagery. We apply our algorithm to study skull deformities. Experiments on skull models demonstrate the efficiency and robustness of our algorithm.

Keywords—3D registration; CT; MRI; ICP; point cloud

I. INTRODUCTION

Registration is a critical issue for various problems in computer vision and computer graphics. The overall aim is to find the best alignment between two objects or between several instances of the same object, in order to bring the shape data into the same reference system. To solve 3D registration problem (pose estimation, alignment and motion estimation), we decided to use a well-known algorithm for registering two point sets (in 2D or 3D) called the iterative closest point algorithm (ICP) proposed by Besl and Mckay [1]. It has been used in numerous real-world applications in computer vision such as 3D object scanning, 3D mapping or 3D localization. ICP is one of the most popular algorithms for registration because of its conceptual simplicity, as well as its superior performance in practice. Independently, Chen and Medioni [2] published a similar iterative scheme using a different pairing procedure based on surface normal vector. Few new variants of ICP were also provided by Rusinkiewicz and Levoy [3] where they proposed new classified variants of registration process. These modifications of ICP seek to improve robustness, speed and precision. For solving the local minima problem there were proposed several novel approaches in a past few years. For robust Euclidian alignment of 3D point sets D. Chetverikov and D. Stepanov [4] proposed the trimmed ICP algorithm

which is based on the consistent use of the Least Trimmed Squares in all phases of the operation.

The outline of this paper is organized as follows. In the first part (Section II), the state of the art is discussed. In next part (Section III), the 3D feature detection is described. The multimodal registration is described in the Section IV. The obtained experimental results are listed in Section V. Finally, the Section VI concludes and suggests the future work.

II. STATE OF THE ART

Our work aims to register two 3D models of human skull obtained from CT/MRI images. We have done this by programming graphical user interface (GUI) inside Matlab environment. We were focused on the whole workflow from image segmentation and visualization; 3D reconstruction and visualization; to 3D registration. However main function of our program is registration of 3D models and estimating the metric error. For this purpose, a review of 3D/2D registration methods can be found here [5] and for 3D modelling there is a comparative study of state-of-the-art evolutionary image registration methods aimed on evolutionary algorithms [6].

Another approach was done by J. Yang and H. Li [7] in the paper called solving 3D registration efficiently and globally optimal, they combined ICP with branch-and-bound (BnB) scheme which searches the 3D motion space efficiently. For volumetric registration of 3D solid shapes Y. Sahilliog and L. Kavan [8] achieved higher efficiency by leveraging a template tetrahedral mesh. In contrast to point- and surface-based registration techniques, their method better captures volumetric nature of the data, such as bone thickness. And lastly B. and D. Lee [9] presented an online learning approach to 3D object registration that vastly improves the performance of ICP methods. They approach achieves better robustness and stable convergence by learning generalized distance functions directly from a stream of object depth data.

III. 3D FEATURE DETECTION

The SIFT algorithm is robust against local distortions and errors in feature detection. It is considered to be one of the most effective descriptors currently available [9]. For each feature identified, a characteristic local SIFT function marker is created. The SIFT 3D descriptor is characterized by the use of the 16 × 16 × 16 gradient orientation distribution that surrounds the function position and the cube is divided into 4 × 4 × 4 sub areas. The example of the 3D SIFT structure is shown in Fig. 1.

This publication is the result of the project implementation: Centre of Translation Medicine, ITMS 26220220021 supported by the Research & Development Operational Programme funded by the ERDF and by the project "Competence Centre for research and development in the field of diagnostic and therapy of oncological diseases", ITMS: 26220220153, co-funded from EU sources and the European Regional Development Fund.

978-1-5386-4759-2/18/$31.00 ©2018 IEEE

Page 2: 3D registration of the point cloud data using ICP ...static.tongtianta.site/paper_pdf/75a31750-6b10-11e... · The search algorithm for the closest point is iterative, the point is

Fig. 1. Construction of the 3D SIFT descriptor [9].

The 3D SIFT descriptor covers the voxel area 16 × 16 × 16 (this area is divided into 4 × 4 × 4 sub-regions). A histogram of 8 × 4 trays is calculated for each sub-region, summarizing the 4 × 4 × 4 voxel orientation in sub-area [9].

IV. MULTIMODAL REGISTRATION

The goal in multimodal registration is to align multiple views of the same object that arise in different types of acquisition systems. In our experiments, the iterated closest point’s (ICP) algorithm using 3D SIFT will be used to registration of input biomedical data.

A. ICP Registration The ICP is a method of aligning 3D geometric models that

are widely used to register 3D scanner outputs. The basic idea is to improve this transformation and minimize the error by rewording the following steps [10]:

• Reducing the number of points on each cloud of points (increase the processing speed).

• Determine the pair of corresponding points (calculate the closest point).

• Minimizing distance between the correspondences.

• Calculate the mean square error between point’s sets.

Pre-processing

Pre-processing

Feature Localization

Feature Localization

ICP

Rotation and

Translation

Feature Extraction

Registration

Target Model

Test Model

DifferenceVector

Fig. 2. The block diagram of the model registration using ICP algorithm.

Firstly, the ICP algorithm is based on looking for pairs of closest points between two sets. Secondly, an estimate of optimal rigid transformation that aligns two sets of data is created. Finally, a solid transformation is applied to points of scenic data. The procedure is repeated until convergence is achieved [10].

ICP Input: Two n-dimensional points sets

• M = {mi | mi ∈ Rn, i = 1, . . . , Nm} - model set

• D = {dj | dj ∈ Rn, j = 1, . . . Nd } - data set

ICP Output:

• Rotation R and translation ∆t which maps D to M.

• Solved as an optimization problem ⇒ Minimizing of error in mapping E:

( ) ( )= =

Δ+−=Δm dN

i

N

jjiji tRdmwtRE

1 1

2

,, , (1)

where weighting coefficients wi,j encode points correspondence, wi,j = 1 for (mi, di) correspondence.

In the ICP process, the search radius affects the calculation time. All retries use a constant search radius. The optimal search radius depends on the cloud point density and the initial position estimate. Nearest neighbour’s comparison is defined in terms of Euclidean distance [10]:

2

minarg ji qpd −= , (2)

where qj is the nearest neighbouring point of interest, pi is the point in the cloud of the source point, i ∈ 0,1,...,N, and N is the number of points in the cloud of the target point.

Although ICP has been successfully used in many registration issues, there are several critical issues to be addressed. Specifically, ICP works well if the following assumptions are met:

• Both views must be close to each other. If not, the ICP is likely to be trapped at the local minimum.

• Both views must completely overlap, or the display of D data must be a subset of model view M.

The speed and accuracy of the ICP calculation are highly dependent on the association process. Two other important questions are the calculation speed and accuracy of the ICP algorithm. When the ICP algorithm is used to register two files, the Delaunay tessellation method is used to speed up the point search (speed and improve registration efficiency) that splits the corresponding point points into the four-point simplex group [10].

B. ICP Registration k-D trees The k-D tree (k-dimensional tree) is a binary tree where

each node represents one point of the cloud of the entry point. This data structure divides the space to two half of the hyperplane space perpendicular to one of the dimensional axes in each non-linear node. The points found in these two halves are two nodes sub-units. The generated k-D tree can then be used to efficiently find the closest neighbourhood of P [10], [11]:

Page 3: 3D registration of the point cloud data using ICP ...static.tongtianta.site/paper_pdf/75a31750-6b10-11e... · The search algorithm for the closest point is iterative, the point is

• Firstly, the initial estimate is made by recursive tree transition from its root in logarithmic time (selecting a subtree that surrounds the half space where P is located until it reaches the node of the leaves).

• Secondly, the tree transition uses an estimate obtained in the previous phase (contain a point with a distance P lower than the first estimate are checked).

The search algorithm for the closest point is iterative, the point is compared to the separating plane, and this determines the search routing decision [10], [11].

V. EXPERIMENTS AND RESULTS

In this section we will evaluate the performance of our proposed algorithm. The proposed registration algorithm is based on the iterative closest point (ICP) and 3D SIFT (ICP + 3D SIFT) algorithm. All tested methods were implemented in the MATLAB programming language. The input of the ICP algorithm consists of the cloud of the source point and the cloud of the target point (see Fig. 3).

Correspondences3D SIFT

SVD

Iteration

Transform OutputSource Model

Target Model

Fig. 3. The registration algorithm based on ICP.

The point correspondence between these points clouds is determined by the 3D SIFT descriptor. This algorithm directly addresses the smallest square problem. After signing up, the whole process is repeated by removing outbound values and re-aligning the point correspondence.

Fig. 4. GUI for registration of biomedical data.

In our work, we aimed to designing graphical user interface (GUI) in Matlab environment for registration two created 3D models of human skull from input CT/MRI data (see Fig. 4). We’ve made a fully functional application capable of these functions:

• visualization and parsing of the input biomedical data,

• making modifications to DICOM images, such as changing contrast or exposure,

• 3D reconstruction and visualization of 3D models,

• converting 3D models to point clouds (PC),

• making modifications to PCs, such us removing outliers and down-sampling,

• registration of two PCs from database by iterative closest point algorithm (ICP) with 2 different methods,

• estimating metric error between PCs by RMSE error.

For the ICP registration we need to obtain two point clouds (PCs) from input CT/MRI data. The workflow of this process is shown in Fig. 5. DICOM data format is known for storing images in the datasets. This means that the images could be imported in a wrong order. We programmed a parser function to reorder the images into right order. The number of actual slice is obtained by function dicomread. With this function, we also receive all necessary information about images from DICOM data such as series, acquisition and instance number, bit depth, image resolution and so on. We then display this information in slice info panel.

Fig. 5. The block diagram for creating point clouds (PCs).

For visualization DICOM images we use graphical element in GUI called axes. At the same time, we send information about slice number to the main panel. Next step is 3D reconstruction of DICOM images, but to do that we need to

Page 4: 3D registration of the point cloud data using ICP ...static.tongtianta.site/paper_pdf/75a31750-6b10-11e... · The search algorithm for the closest point is iterative, the point is

separate bones from other tissue such as muscles or skin. This process is called image segmentation. We have implemented thresholding method for segmentation, which is widely used segmentation technique in medical image processing.

Displaying a 3D model consists of several steps (see Fig. 6). First, we used the patch function to create one or more filled polygons using X and Y elements as coordinates for each vertex. In 3D visualization, we have used several other functions. These serve to fine-tune the displaying model, such as determining the model view angle, changing the aspect ratio, adjusting the light (angle, color, way of reflecting light, etc.) The patch function associates the vertices in the order given by the variables surface.vertices and surface.faces. Subsequently, we entered the colour of the faces through 'FaceColor'.

Fig. 6. The example of the 3D visualization.

Next, we converted the 3D models to point clouds (PCs) (see Fig. 7). We created the PCs using the pointCloud function, the output of which is an object with the coordinates specified by the x, y, z points. In our case, the input coordinates [x, y, z] were obtained by taking the vertices from the already reconstructed 3D model after we applied the isosurface function that created both the vertices and faces of the 3D model. When this is done we visualize differences between PCs by function pcshowpair in visualization window.

a) b)

Fig. 7. The visualization of a) 3D model and b) point clouds (PCs).

When all these steps are done we have got two separate PCs. One is Fixed PC and another is moving PC. In the next steps, we need to prepare PCs for registration. This process is visualized in PCs registration workflow in Fig. 8. First step is to remove outliers and noise in both PCs with function pcdenoise. The next step of preparation is down sample. This is

a critical step because success of registration depends on the number of points of registered PCs. We perform down sampling using the pcdownsample function. This feature uses three different down sample methods. The first, randomly tagged, randomly divides the point cloud without replacing the points. With this method, only the percentage ratio of the input and output number of desired points is set. The second method, called 'gridAverage', uses a grid-like filter labelled 'gridStep' to down sample it. We use this method to reduce points in our point clouds and we have determined by experiment that the ideal size of the grid filter for our purposes is 5.

Fig. 8. The proposed registration of 3D models using ICP method.

After this process, the PCs are ready for rigid registration using the ICP algorithm, which, being iterative, is performed in a cycle consisting of the following steps:

• match points between fixed and moving PCs,

• remove incorrect matches by filtering outliers,

• recover the shift and rotation (minimizing the error).

The proposed registration of 3D models based on ICP method is described in Fig. 8. Once we have performed rigid registration, we can take three optional steps. The first is to

Page 5: 3D registration of the point cloud data using ICP ...static.tongtianta.site/paper_pdf/75a31750-6b10-11e... · The search algorithm for the closest point is iterative, the point is

check the point cloud alignment with the pctransform function, as the moving cloud point transformation is detected. The second step is to join two point clouds into one object via the pcmerge function. This step is mostly used to reconstruct 3D scenes or to scan the scene. And last one is to apply the transformation matrix to moving point cloud and again show the differences between fixed and aligned PC (see Fig. 9).

Fig. 9. The example of the model registration using ICP method.

An incompatible part of the ICP algorithm is also the determination of the error of registration because one of the steps of the algorithm is to minimize it. We obtain this error by function pcregrigid which returns the root mean squared error (RMSE) of the Euclidean distance between the aligned point clouds. In order to evaluate the convergence of the ICP algorithm based on point cloud features, we used MATLAB to generated 5 sets of positions for “skull”.

TABLE I. THE POINT CLOUD REGISTRATION ERROR

No. of Iterations

Algorithms

ICP ICP k-D tree Proposed ICP +

3D SIFT

Registration error

[m] Registration error

[m] Registration error

[m]

10 8.917x10-6 8.989x10-6 8.473x10-6

15 8.739x10-6 8.803x10-6 8.251x10-6

20 8.385x10-6 8.527x10-6 7.989x10-6

25 8.242x10-6 8.375x10-6 7.952x10-6

30 8.057x10-6 8.159x10-6 7.908x10-6

Table I lists ICP cycle times and ICP algorithm base for k-D strome, cloud point registration error, and the number of iterations required by the ICP algorithm. From the Table I is clear that for ICP algorithm, increasing the initial cloud point position will lead to increase in the number of the iterations. In case that the initial position reached a certain value, the ICP will reach the local extreme.

TABLE II. THE RUN TIME PERFORMANCE COMPARISON

No. of Iterations

Algorithms

ICP ICP k-D tree Proposed ICP +

3D SIFT

Time [s] Time [s] Time [s]

10 120 15 49

15 155 27 55

20 179 32 58

25 187 45 61

30 225 51 65

The proposed ICP + 3D SIFT algorithm is slower than the k-D tree based ICP algorithm (see Table II). On the other hand, the proposed ICP + 3D SIFT algorithm is faster than the known iteration algorithm of the closest point (ICP).

VI. CONCLUSION

In this paper we presented an experimental evaluation of the performance of the proposed algorithm. To solve the 3D registration problem (alignment and motion estimation) the iterative closest point algorithm was used. This algorithm is based on the very popular ICP algorithm using the transformation of 3D invariant properties and 3D scale-invariant feature transform to register 3D free-form closed surfaces (3D models of human skull). The performance of three different algorithms with their accuracy and robustness were compared (ICP, ICP k-D tree and proposed method). The proposed solution (ICP + 3D SIFT) is fast and accurate compared to the state of art in the field of reconstruction and registration of medical imagery.

The proposed method (ICP + 3D SIFT) provides high accuracy of registration. The ICP algorithm is left unmodified to preserve the speed and simplicity. The proposed method deals only with iterative transformation calculations using 3D SIFT. This method based on combination of ICP + 3D SIFT improve overall accuracy. These modifications improve the performance and speed of the classical ICP algorithm. We apply our algorithm to study skull deformities. The experiments on the reference skull models demonstrate the efficiency and robustness of our algorithm.

New challenging scenarios can be addressed through the use of machine learning and computer vision technologies already successfully used for the 2D domain, as well as new advances inspired by the latest computer animation techniques. The timeliness of the proposed ICP + 3D SIFT can be significantly increased.

Page 6: 3D registration of the point cloud data using ICP ...static.tongtianta.site/paper_pdf/75a31750-6b10-11e... · The search algorithm for the closest point is iterative, the point is

ACKNOWLEDGMENT

This publication is the result of the project implementation: Centre of Translation Medicine, ITMS 26220220021 supported by the Research & Development Operational Programme funded by the ERDF and by the project "Competence Centre for research and development in the field of diagnostic and therapy of oncological diseases", ITMS: 26220220153, co-funded from EU sources and the European Regional Development Fund.

REFERENCES [1] P. J. Besl and N.D. Mckay, “A method for registration of 3-D shapes,”

IEEE Transactions on Pattern Analysis and Machine Intelligence [online]. 1992, 14(2), 239-256. DOI: 10.1109/34.121791. ISSN 0162-8828.

[2] Y. Chen and G. Medioni, “Object modelling by registration of multiple range images,” Image and vision computing, 1992, 10.3: 145-155.

[3] S. Rusinkiewicz and M. Levoy, “Efficient variants of the ICP algorithm,” In: 3-D Digital Imaging and Modeling, 2001. Proceedings. Third International Conference on. IEEE, 2001. p. 145-152.

[4] D. Chetverikov, D. Stepanov and P. Krsek, “Robust Euclidean alignment of 3D point sets: the trimmed iterative closest point algorithm,” Image and Vision Computing, 2005, 23.3: 299-309.

[5] P. Markelj, “A review of 3D/2D registration methods for image-guided interventions,” Medical image analysis, 2012, 16.3: 642-661.

[6] J. Santamaría, O. Cordón and S. Damas, “A comparative study of state-of-the-art evolutionary image registration methods for 3D modeling,” Computer Vision and Image Understanding, 2011, 115.9: 1340-1354.

[7] J. Yang, H. Li and Y. Jia, “Go-ICP: Solving 3d registration efficiently and globally optimally,” In: Proceedings of the IEEE International Conference on Computer Vision. 2013. p. 1457-1464.

[8] Y. Sahillioglu and L. Kavan, “Skuller: A volumetric shape registration algorithm for modeling skull deformities,” Medical image analysis, 2015, 23.1: 15-27.

[9] P. Scovanner, S. Ali, and M. Shah, “A 3-dimensional sift descriptor and its application to action recognition,” In Proceedings of the 15th International Conference on Multimedia (ACM ’07), pp. 357–360, New York, NY, USA, September 2007.

[10] B. Lee and D.D. Lee, “Learning anisotropic ICP (LA-ICP) for robust and efficient 3D registration,” In: Robotics and Automation (ICRA), 2016 IEEE International Conference on. IEEE, 2016, p. 5040-5045.

[11] A. Nuchter, K. Lingemann and J. Hertzberg, “Cached k-d tree search for ICP algorithms,” Sixth International Conference on 3-D Digital Imaging and Modeling, 2007, ISBN: 0-7695-2939-9.


Recommended