Fan-Meshes: A Geometric Primitive for Point-based Description of 3D Models and Scenes

Xiaotian Yan, Fang Meng, Hongbin Zha

National Laboratory on Machine Perception Peking University, Beijing, P. R. China {yanxt, mengfang, zha}@cis.pku.edu.cn

Abstract

We propose a new data structure, called Fan-Meshes (FM), for reconstructing 3D models and scenes represented by dense scanning point clouds. It is a local piecewise linear approximation to the data geometry, and can serve as primitives in reconstruction with a good balance between computational loads and reconstruction quality. In our algorithm, local remeshing is performed in preprocessing to obtain regular FMs, and a three-level-point data structure called Triangle Selection Record (TSR) is then used to reduce redundancies in the raw data and overlapping in the original FMs. Furthermore, to apply the method to raw 3D scanning data, we use a smoothing operator to the point cloud in order to eliminate some sensor noises. Experimental results demonstrate that our scheme is effective even for large-scale scenes with real data.

1. Introduction

Reconstruction of 3D models and scenes is an important process in robot vision, digital heritage and virtual reality [1]. With advances in 3D sensing methods, more and more reconstruction systems are based on dense 3D point clouds obtained by using such technologies [2, 3]. In traditional geometry-based reconstruction pipelines, first we have to perform global triangulation on the original point clouds for the subsequent processes, such as feature extraction, scene description and rendering. However, the global triangulation itself is very time-consuming, and in some cases where a huge dataset has to be handled, it may become completely unacceptable for current PC hardware. Moreover, since the raw data usually contains many noises and holes due to occlusion, the resulting triangular meshes may have large topological errors, which will make the final reconstructed result unreliable.

Using 3D points directly as primary primitives for reconstruction is an effective solution to the above problems [4]. In fact, some new algorithms on rendering and point-based model transmission [5, 6] have been proposed recently in the fields of computer graphics and

3D modeling. However, the methods have drawbacks in dealing with problems such as:

How to simplify data clouds directly by local shape-adaptive decimation controlled by some approximation error bounds.

How to explicitly represent the adjacency relationship between points without global meshing.

In order to solve the problems, we propose in the paper a novel geometric primitive, called Fan-Meshes (FM), for locally approximating surfaces in 3D point-sampled models with piecewise linear approximation functions. It has much more flexibility in describing surfaces of complex shapes than other simple primitives such as planes or spheres [4, 6]. At the same time, by using a data arrangement process prior to the on-line reconstruction, we can reduce the computational costs that are usually high for complex primitives [7]. We can also decrease the data redundancies by utilizing a local remeshing technique to make each FM to have largest coverage on the surfaces. To eliminate some sensor noises in a raw data set, we apply a local smoothing operator to the point cloud as a basic pre-processing step. Moreover, since the point-based reconstruction scheme relies only on local adjacency relationships between points, we can partition a large point cloud into pieces and then merge them together easily after processing each piece respectively. These advantages together will lead to a good trade-off among the computational costs, space storage and reconstruction effects.

Experimental results will show the applicability of our primitives to the reconstruction of 3D models and large natural scenes with real scanning point clouds.

2. Conceptual overview

Our system takes as input unorganized point-clouds without any connectivity information. The system is composed of two main parts: data organization and point-based description. The first part deals with the input data and creates FMs based on local piecewise linear approximation to the data geometry. At first, a discrete Laplacian smoothing operator [11,12] is applied to point clouds for eliminating the measuring errors occurred while scanning. Local remeshing of triangles and FMs is

used to reorganize the local meshes in order to get regular distribution over the model surfaces. The second part uses a new data structure, called Triangle Selection Record (TSR), to decimate redundant triangles and decrease overlaps on model surfaces.

3. FM’s creation

At first, the definition of FM is given in this section.

Then we explain in detail the FM’s generation process which consists of three steps: selecting center points, searching its K-nearest neighbors and organizing the neighbors.

3.1. Definition of FM

We use the term FM to define a new kind of geometric primitives in our description scheme. It consists of three components: a center point, its neighbor points and the connection relationship among these points.

In the process of data organization, for a given point P0, we compute its K-nearest neighbors P1, P2,…, PK under the condition that all the neighbors should be distributed around P0. P0 will be referred to as the center point in the following sections. Then we order this neighbor points to make sure that the polygon P1P2…PK is convex. We connect P0 with P1, P2,…, PK respectively and form a local mesh that we called a FM. It can describe local geometric properties around P0 with K triangles incident to the point.

3.2. Selection of center points

There exist many crossovers among FMs if we build

the FM for every input point [7]. Considering that the neighborhoods of many points are already covered by other FMs, we need only treat parts of input data as center points which are used in building FMs.

Weather a point can be regarded as a center point depends on the number of edges that connect to it. In the computation of K-nearest neighbors for a point P0, it is desirable to find K neighbors surrounding P0. Based on analysis of the rendering process for fan-meshes, it is obvious that if a point Pi appears as a center point, K edges will be added to it. If it appears as a neighbor point, the number of added edges will vary between one and three. Therefore we stipulate that a point should not be used as a center point if it has been used (K-2) times as a neighbor point. According to this rule, some points are left out in this step, which can reduce both the computational time and storage space in our scheme.

In our experiments, only less than 70% of the input points need to be used in the computation of K-nearest neighbors. Further analysis on connection information

among fan-meshes and geometric properties over the model surfaces will decrease this percentage further to a lower level. 3.3. Searching the K-nearest neighbors

A new algorithm for searching the K-nearest neighbors

is developed with O(nlogn). The basic idea is that if a point is in the neighborhood of a center point P0, its coordinates of XYZ axes are also in the neighborhood of P0’s coordinates respectively.

At first, we create three index tables to store respectively 3D coordinates of input points in increasing order, and hence the searching of neighbors for a point can be limited to local areas. Since points within the point-cloud models are dense with even distribution, we can form the searching subset quickly by selecting a certain number of points whose coordinates are adjacent to the corresponding coordinates of P0 in the tables. Then we compute the distances from every point in the subset to P0 and array them in increasing order. The first K points are the neighbors we need.

3.4. Neighbors’ organization

After the K-nearest neighbors’ computation for a given center point, we must organize the neighbor points to ensure that they can construct a convex polygon.

First we compute the best fitting plane F of a center point and its neighbors. As has been demonstrated in earlier work [8], the normal of F is the eigenvector corresponding to the minimal eigenvalue of the points’ covariance matrix C. Since the covariance matrix C is a 3× 3 matrix, we can use the Cardano’s equation [13] to compute the eigenvector quickly.

After getting the best fitting plane F we project all points into it and denote their projections as Q0, Q1…QK. We permute the indices of P1, P2…PK such that their projections form increasing angles Ai = ∠ Q1Q0Qi. In this order the points Pi form a convex polygon.

4. Local remeshing

We define a regular fan-mesh as follows: the center point P0 lies in the center of the fan-mesh and triangles within the fan-mesh are approaching regular triangles, i.e. equilateral triangles. In our system, a triangle is a regular one only if each internal angle is between 30 and 135.

Regular triangles and meshes are preferred in 3D modeling because they can describe the model surfaces smoothly and represent the object with better visual quality. However, we may get some irregular FMs because of the incomplete search of the neighbors or wrong selection of the best fitting plane in the processes

above. In order to improve the effect of the reconstruction, we perform the local remeshing process which can be divided into two parts: one is to deal with the triangle that has an internal angle less than 30 degrees by remeshing within a FM, and the other is to handle the triangle with an internal angle beyond 135 degrees by adjusting neighbors for a FM.

Since our goal is to apply our method directly to raw scanning data, we also have to use some techniques to eliminate sensor noises before these processes. 4.1. Smoothing

With the help of current 3D scanners, we can obtain models with high resolutions, which can record the objects accurately. However, there are some measuring errors which can lead to distortion. To eliminate such errors, we apply a local smoothing operator to the point clouds.

A well-known smoothing operator for triangle meshes is the discrete Laplacian smoothing operator which moves a single point P0 to the centroid / ( 1, 2... )iP K i K=∑ of its

1-ring formed by the points P1, P2,…, PK. We replace the 1-rings by the neighborhoods and then apply it to our point clouds.

Tthis Lapplicoeffthe c

also on the geometry. To get good adaptability to detailed changes in raw data, we integrate their methods together and the effects of smoothing is shown in Figure 1(b), which is the smoothed result of Figure 1(a).

4.2. Remeshing within a FM

As the input point-cloud is dense with even distribution, the distances from the neighbors to the center point seem to be equal. It is obvious that the triangles in FMs are isosceles triangles approximately, and hence we need only to consider about the vertex angle, i.e. the angle with the vertex of the center point.

Figure 2. Remeshing for getting regular triangles

Figure 2 illustrates the process of remeshing for transforming irregular triangles into regular ones. Since ∠ P2P0P3 is less than 30 degrees in Figure 2(a), we delete P3 from the neighbor sequence and permute the indices of P1, P2,…,P6 which is illustrated in Figure 2(b).

4.3. Adjusting neighbors for a FM

Sometimes, the neighbor points can not enclose the center point P0. At the same time, we notice that visual effects will be deteriorated if the vertex angle of a triangle is too big. Therefore it is required that the vertex angle should be less than 135 degrees. The local remeshing for the FMs is given as follows:

As shown in Figure 3(a), when P∠ 4P0P5 is larger

than 135 degrees, the most possible reason is that the point density varies sharply around P0. The neighbors have to be adjusted with an insert-criterion. We select the 6th neighbor P6 and see whether ∠ P4P0P6 and ∠ P6P0P5

Figure 3. Adjusting the neighborhood for a regular FM

Figure 1. Effects of the smoothing operator

aubin [11] made further investigation and found that aplacian smoothing operator causes shrinkage if it is

ed to every vertex of a mesh. Guskov [12] added a icient to each point Pi when they are summed, and oefficients not only depend on the connectivity but

are regular. If P6 meets the criterion we insert it and permute the neighbor sequence which is illustrated in Figure 3(b), or we consider the further next neighbor till the insert-criterion is met or a certain threshold number of neighbors has been reached.

When the insert-criterion can not be satisfied in the

step above, we assume that P0 lies on a sharp edge of the surface. In this situation we will get the wrong best fitting plane and incorrect connection order of the neighbors. As shown in Figure 4, if we follow the projections’ order which is computed in the initial best fitting plane F1 we can not get a regular FM. However, we can get a regular fan-mesh when using the plane F2 to compute the connection order. Here the normal of F2 is the eigenvector corresponding to the maximal eigenvalue of the covariance matrix C. By doing so, we rebuild the best fitting plane and permute the neighbor sequence.

If the rebuilding step fails, we assume that P0 lies on the boundary of the surface and use the neighbor sequence we compute originally.

We apply our scheme on a hematite model with 58824 points. Figure 5(b) and Figure 5(c) illustrate the results of the edges before and after local meshing respectively, contained in the box in Figure 5(a). We can observe that the edge of the surface becomes smooth and the visual effects are improved.

Figure 4. Rebuilding the best fitting plane

5. Point-based description

In the process of data organization, we reduce the computational loads by selecting parts of input points in building fan-meshes. However, there still exist much redundancies and overlaps among FMs. Therefore, to further simplify the model description, we propose a data structure, called Triangle Selection Record (TSR). With the help of TSRs, we can eliminate the redundancies and judge whether or not a triangle should be removed for decreasing topological errors. This process is called optimization analysis.

TSR is a three-level-point data structure to record triangles that have been used in the creation of FMs. Nodes in TSRs represent vertices of triangles with its serial number and are arranged into three levels in our scheme. Node in first level records the first vertex of a triangle, and it has only one pointer to its child, the second vertex of the triangle; nodes in second level record the second vertices of triangles, and it has two pointers, one to its child, the other to the second vertices of other triangles; nodes in third level represent the third vertices of triangles, and it has only one pointer to its neighbors, the third vertices of other triangles.

Figure 5. Effects of local remeshing

(a) (b) Figure 6. Creation of a TSR

Redundancies and overlaps always appear among

adjacent FMs. Figure 6(b) demonstrates a TSR that records three triangles: {P0, P1, P2}, {P0, P3, P4} and {P0, P1, P3} among adjacent FMs in Figure 5(a). It works as follows: for any triangle {Px, Py ,Pz}, we first array its vertices by its serial number in increasing order, for example: PxPyPz, Then, we search the first vertex of the triangle in TSRs. If there has no matching record, we built a new TSR for this triangle; if there has a TSR with root node being Px, we compare the vertex Py with its nodes in the second level. A new node is inserted into this level if there has no node being Py; otherwise we begin the next comparison for the third vertex Pz. Since every

point has a unique serial number while reading from input data, we can eliminate redundancies naturally in the process of the TSRs’ creation.

Moreover, based on the fact that overlapping triangles, for example {P0, P1, P2} and {P0, P1, P3} showed in Figure 6(a), share common nodes in the first-level and second-level in the TSR, we can skip the process for triangle {P0, P1, P3} if point P2 and P3 locate in the same way with respect to the edge P0P1. Since it would cost considerable computational time and storage space to finish this work, and in fact, overlaps in our rendering scheme do not affect the visual quality seriously, we did not complete the elimination process for all overlaps in current scheme.

We use a horse model with 48485 points to illustrate

the effects of optimization analysis. Figure 7(a) illustrates the whole model and the area that is magnified in Figure 7(b) and (c). We can observe that the burrs in Figure 7(b) disappear in Figure 7(c) after the optimization analysis. About 15% of the triangles are reduced in this process.

6. Results of experiments

We apply our scheme on some models and two demos are shown in this section. Figure 8 shows a piece of a nose model with 300 points. As compared with the original model in Figure 8(a), our fan-meshes do not result in a continuous surface. However, as shown in Figure 8(b), we can achieve almost the same visual effects.

The time-complexity of our scheme is O(nlogn). Table

1 shows the running times compared with the triangulation of point-clouds using Delaunay tetrahedrizations [10]. Since input data are dense scanning point clouds, Delaunay method is hard to perform the reconstruction and visualization for whole models. On the other hand, since our method just use localized process for some selected points, even huge point sets can be reconstructed easily.

Table 1. Running time

Method Computer Points Time 300 0.03 s

48485 65.92 s

Our scheme

PC with AMD

1.8GHz 58824 84.52 s 1248 45 s 2600 1 min

Delaunay

SGI Indigo2 Extreme

48921 133 min

Figure 8. The property of fan-meshes

Figure 7. Effects of optimization analysis

To illustrate the efficiency of our scheme in particular for huge models, we test our algorithm on point-cloud data from scanning the Yungang Grottos in China directly by a 3D laser scanner.

The dataset contains 2,981,168 points after registration. We cut it into seven pieces, each of them has 100 thousands to 600 thousands points and the computational time varies from 3 minutes to 20 minutes. Figures 9(a), 9(b), 9(c) and 9(d) are four pieces of the model with the number of points and the running time given below. After eliminating the duplicates and overlaps, the number of triangles is between 2.5 and 4 times of the points.

A significant feature of the point-based method is its easiness in merging different data pieces together. For a large model, we can partition it into several pieces and do the local meshing with each of them. Then, the whole model is represented just by aligning them together without any special processing for the piece stitching, which is usually very difficult for mesh models. The whole model merged by those pieces is illustrated in Figure 10(a) and 10(b).

Our scheme may fail when the density of point clouds is too low or it varies too sharply. However, it is suitable for real data acquired by 3D laser scanners, where the point clouds are dense.

7. Conclusions

In this paper, we proposed a data structure FM as primitives for point-based reconstruction of 3D models and scenes. Though the concept of FM is very simple, it is a good idea to realize localized analysis for huge point sets. It is designed based on local piecewise linear surface approximation, which can balance the computational loads and description performance. Compared with the traditional modeling, the method can achieve similar visual effects without the global triangulation process. Moreover, there is no necessity for recording connectivity information among primitives, which makes it more appropriate for huge point-cloud models.

In the future, we will investigate deeply into the geometric properties over the model surfaces in order to optimally distribute FMs and reduce artifacts among those primitives. In addition, several other techniques, such as view-dependent level-of-detail description [9], will be incorporated into our scheme to improve the reconstruction quality.

Acknowledgement We would like to acknowledge the helpful discussions with Dr. Hong Liu. The work was supported in part by the NSFC grant (No. 60333010).

References [1] J. Hu, S. You, and U. Neumann, “Approaches to Large-Scale Urban Modeling”, IEEE Computer Graphics and Applications, Volume 23, Issue 6, 2003, pp. 62-69. [2] P. Allen, I. Stamos, A. Troccoli, etc., “3D Modeling of Historic Sites Using Range and Image Data”, IEEE Int. Conf. on Robotics & Automation, 2003, pp. 145-150. [3] C. Früh and A. Zakhor, "Automated Reconstruction of Building Facades for Virtual Walk-thrus", SIGGRAPH, Sketches and Applications, 2003. [4] M. Gross, H. Pfister and M. Zwicker, “Point-Based Computer Graphics,” Eurographics Tutorial T6, 2002. [5] F. Meng and H. Zha, “Streaming Transmission of Point-Sampled Geometry Based on View-Dependent Level-of-Detail”, Proc. Fourth Int. Conf. on 3-D Digital Imaging and Modeling, 2003, pp.466-473. [6] S. Rusinkiewicz and M. Levoy，“Streaming QSplat: A Viewer for Networked Visualization of Large, Dense Models,” Proc. Symposium on Interactive 3D Graphics, 2001, pp.63-69. [7] L. Linsen, “Point-cloud Representation”, Technical Report, Faculty of Computer Science, University of Karlsruhe, 2001. [8] M. Pauly, M. Gross and L. P. Kobbelt, “Efficient Simplification of Point-Sampled Surface,” Proc. IEEE Visualization, 2002, pp. 163-170. [9] D. Luebke, D. Cohen, R. Heubner, etc., Advanced Issues In Level Of Detail, SIGGRAPH Course Notes, 2002. [10] R. Mencl and H. Müller, “Graph-Based Surface Reconstruction Using Structures in Scattered Point Sets,” Proc. of Computer Graphics International’98, 1998, pp. 298-311. [11] G. Taubin, “A Signal Processing Approach to Fair Surface Design,” Computer Graphics Proceedings, Annual Conf. Series, 1995, pp. 351-358. [12] I. Guskov, W. Sweldens, P. Schroder, “Multiresolution Signal Processing for Meshes,” SIGGRAPH 99, 1999, pp. 325-334. [13] M. Kline, Mathematical Thought from Ancient to Modern Times, Oxford University, Oxford, 1990.

(a) (b)

(c) (d)

Figure 9. Scene description for Yungang Grottos: (a) middle part of the tower: 124,994 points / processing time: 3

minutes; (b) a wall of the grotto: 572,050 points / processing time: 15 minutes; (c) bottom part of the tower: 169,995 points / processing time: 4 minutes; (d) frontispiece of the grotto: 327,379 points / processing time: 9 minutes.

(a)

(b)

Figure 10. (a) The whole model from viewpoint one; (b) The whole model from another viewpoint.