Eurographics Italian Chapter Conference 2006R. De Amicis and G. Conti (Editors)

3D Objects Face Clustering using Unsupervised Mean Shift

M. Farenzena M. Cristani and U. Castellani

Dipartimento di Informatica, University of VeronaStrada Le Grazie 15, 37134 Verona, Italy

AbstractIn this paper, a novel approach to face clustering is proposed. The aim is the extraction of planes of a meshacquired from a 3D reconstruction process. In this context, as 3D coordinates points are inevitably affected byerror, robustness is the main focus. The method is based on mean shift clustering paradigm, devoted to separatethe modes of a multimodal density by using a kernel-based technique. A critic parameter, the kernel bandwidth,is automatically detected. Experimental results on synthetic and real data validate the approach and prove itsrobustness.

Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computational Geometry and Object Model-ing]:

1. Introduction

Polygonal meshes remain a preferred representation for sur-face data because of their ability to approximate complexshapes, algorithmic simplicity, and visualization efficiency.In particular, triangular meshes are widely used in many en-gineering, medical, and entertainment applications. Anyway,in order to process a three-dimentional (3D) object, in theform of a triangular mesh for further and more sophisticatedanalysis, extracting and opportunely organizing higher or-der features represents a fundamental step. In this study, wefocus on face clustering: this operation is useful, for exam-ple, in a Computer Graphic context, for shape simplification[DCSD04, MGH01, KT96], shape modelling and retrieval[TF04, DCSD04], or to accelerate the face culling process.

In the context of Computer Vision, instead, face clusteringstrategies could be useful for image-based modelling appli-cations. Here, an important operation is the automatic ex-traction and division into consistent sets of informative por-tions of a mesh object acquired from images. In particular,planes can be extracted and organized into different entities,depending on their orientation and position. This operationshould be considered as first step for a further refinement ofthe 3D structure as in [MFD06], and in general for a higherlevel analysis and processing of the mesh object. However,this context requires to see the problem from a perspectiveslightly different from a pure Computer Graphic problem:if a mesh derives from a 3D reconstruction process, in fact,

the location of the 3D points is inevitably affected by error,and so robustness should become the main focus of a faceclustering algorithm on these data.

In this paper we propose a robust fully authomatic methodfor face clustering. This approach considers as leadingframework the mean shift (MS) clustering paradigm, a pow-erful general purpose procedure for non-parametric scattereddata, proposed in [CM02]. The main underlying idea of suchapproach is that the data space is regarded as an empiricalprobability density function to estimate. In short, the MSprocedure operates by shifting a fixed size estimation win-dow, the kernel, from each data point towards a local mode,denoted as a high concentration of points. The points con-verging to the same mode are included in the same region.

MS has shown to be a powerful technique for several re-search fields such as image and video segmentation, track-ing, clustering and data mining [CM02, Col03, GSM03]. Inthe context of face clustering, instead, MS clustering hasbeen applied to surface normals only as a pre-processing stepto mesh segmentation, as in [HY05]. Other state-of-the-artapproaches to face clustering [SWG∗03, CSAD04] mainlyuse region growing methods, and the focus is not on robust-ness but on the visual quality of the result. Moreover, mostmethods rely on tuning of several parameters.

This work, instead, builts on a paper by Cristani et al[MCV06], where a method for automatic selection of kernel

c© The Eurographics Association 2006.

M.Farenzena, M. Cristani & U. Castellani / 3D Objects Face Clustering using Unsupervised Mean Shift

parameters in MS algorithm is developed. [MCV06] facesthe problem of 3D segmentation on unorganized 3D points;here, the method is applied to extract planes from noisy 3Dmesh objects. In brief, our method firstly clusterize triangles’normals, organizing together triangles with similar normals.Then, for each cluster, a further clustering operation basedon the distance of the triangles from the origin is carriedout, thus permitting to separate triangles belonging to par-allel planes.

In literature, approaches for automatic estimation of MSparameters are present: a recent and important theoreticalframework has been proposed by Comaniciu in [Com03],but it is based on the assumption that data are locallydistributed with a Gaussian distribution, and corrupted byGaussian noise. This assumption does not hold in generalfor data characterizing rigid geometrical data: for example,punctual information which characterize corners and spikes,such as normals and spatial positions, are far from beingcharacterized by a local Gaussian configuration. Therefore,we do not want to impose Gaussian assumptions: we acceptevery data configuration, only assuming that the clutter af-fecting the data is bounded, with a uniform distribution hold-ing inside the bound. In order to sensibly validate our as-sumptions, an extensive performance comparison betweenour approach and [Com03] has been performed, showingbetter performances of our method in this context.

The rest of the paper is organized as follows. Sec. 2 de-scribes an overview of the mean shift procedure is described,while Sec. 3 illustrates the automatic estimation of the band-width parameter. Sec. 4 depicts the proposed method andSec. 5 shows the experimental results, on synthetic and realdata. Finally, the conclusions in Sec. 6.

2. Mean Shift

The mean shift procedure is a dated non-parametric densityestimation technique [Fuk90, CM02]. The theoretical frame-work of the mean shift arises from the Parzen Windows tech-nique, that, in particular hypotheses of regularity of the inputspace (such as independency among dimensions [CM02]),estimates the density at point x as:

f̂h,k(x) =ck,m

nhm

n

∑i=1

k(∣∣∣∣∣∣x−xi

h

∣∣∣∣∣∣2) (1)

where ck,m is a normalizing constant, n is the number ofpoints available, and k(·) is the kernel profile, that modelshow strongly points are taken into account for the estima-tion, in dependence with their distance h (bandwidth) to x.

Mean shift extends this “static” expression, differentiating(1) and obtaining the gradient of the density as:∇̂ fh,k(x) =

2ck,m

nhm

[n

∑i=1

g(∣∣∣∣∣∣xi−x

h

∣∣∣∣∣∣2)]∑

ni=1 xig

(∣∣∣∣ xi−xh

∣∣∣∣2)∑

ni=1 g

(∣∣∣∣ xi−xh

∣∣∣∣2) −x

(2)

where g(x) = k′(x). In the above equation, the first term insquare brackets is proportional to the normalized densitygradient, and the second term is the mean shift vector, thatis guaranteed to point towards the direction of maximumincrease in the density [CM02]. Therefore, starting from apoint xi in the feature space, the mean shift produces iter-atively a trajectory that converges in a stationary point yi,representing a mode of the whole feature space.

3. Bandwidth automatic estimation

The bandwidth parameter h defines the level of detail of theanalysis. Large bandwidths lead to global but course separa-tion, whereas small bandwidths better identify local modes,but at the risk of over-partitioning. Good segmentation re-sults could be obtained after an accurate parameters tuning.In line with the concept of stable segmentation [Fuk90] weexploit the same strategy developed in [MCV06]. If we sin-gle out extreme values hmin and hmax for h and we uniformlydivide the range [hmin,hmax], for each h-value so identi-fied we perform mean shift clustering. After these trials wechoose as best bandwidth the centre of the largest plateauover which the same number of clusters are obtained.

4. Proposed method

The proposed technique is composed by two steps. Firstly,the normals of every mesh triangle is clusterized. Then, eachcluster must be clusterized again in order to separate parallelplanes. In fact, a plane is expressed by the equation

ax+by+ cz+d = 0 (3)

The parameters a, b and c are the coordinates of the plane’snormal, while d represents the plane’s distance from the ori-gin. Two triangles belonging to different parallel planes dif-fers from d only, so after the first clusterization these trian-gles are in the same cluster. A second clusterization basedon d parameter permits to separate them.

Specifically, for the former clustering data are the pointsxi = ni, where ni is the 3D normal of the ith mesh triangle.

The adopted kernel is [CM02]:

Khn(x) =Ch3

nk(

(∣∣∣∣∣∣∣∣ xhn

∣∣∣∣∣∣∣∣2)

) (4)

where C is the normalization constant and hn the kernelbandwidth. Assuming a bounded error on 3D points, and auniform distribution inside the bound, the appropriate k em-ployed is the Epanechnikov kernel [CM02]:

k(x) ={

1− x if 0≤ x≤ 10 otherwise

(5)

c© The Eurographics Association 2006.

M.Farenzena, M. Cristani & U. Castellani / 3D Objects Face Clustering using Unsupervised Mean Shift

k, differentiated, leads to the uniform kernel g(·), i.e. am-dimensional unit sphere.

The latter clusterization is on data yi = dic, where dic isthe plane’s distance from the origin (3) of the ith triangle inc cluster. The kernel used is the same of the previous clus-terization, with bandwidth hd .

Finally, in order to automatically select the parameters hnand hd as described in Sec. 3 we need to single out theirrange of variability. We fixed hn ∈ [0.05,0.2], while for hdwe consider as maximum value the 10% of the median of themaximum distances between every pair of 3D mesh points,and as minimum value min(di).

5. Experimental Results

The proposed method has been tested on both synthetic andreal data.

For the synthetic experiments, we used the four mesh ob-jects in Fig. 1. As previously said, we assume that the 3Dmesh points are affected by a uniform bounded error. So, the3D points has been perturbed varying the bound’s width anduniformly generating points inside the bound. For each ob-ject, the bound’s widths k are calculated as 0.5-3.0% of themedian among the maximum distances between every pairof mesh points. For each k we performed 50 independent tri-als. The mean percentage of mismatches, wrt the number ofmesh triangles, is detailed in Table 1. Examples of the de-tected planes are depicted in Figs. 2 and 3.

Figure 1: The four objects used for the synthetic exper-iments, here referred as (from the left) test, boxwhole,cutcube and foursix

2 mismatches 0 mismatches

Figure 2: Examples of detected planes for k = 2.0, with thenumber of mismatches. For the test object on the right, themismatches are the two highlighted triangles.

As you can notice, the algorithm works remarkably well,with a very little percentage of mismatches.

0 mismatches 0 mismatches

Figure 3: Examples of detected planes for k = 2.0, with thenumber of mismatches.

0.5 1.0 1.5 2.0 2.5 3.0

test 0 0.12 1.35 3.97 6.88 9.67

boxwhole 0 0 0 0.09 0.36 3.09

cutcube 0 0 0 0.02 0.23 0.78

foursix 0 0 0 0 0.02 0.08

Table 1: Synthetic experiments results: mean percentage ofmismatches wrt the number of mesh triangles vs the bound’swidth k.

As experimental comparative evaluation, we apply thesame hierarchical clustering process, employing instead thebandwidth selection theorem proposed by [Com03]. Thetheorem implies a different MS formulation, giving to eachdata point a particular bandwidth value, instead of choos-ing a fixed bandwidth value for all the data space. Suchbandwidth value is the one that maximize the module of thenormalized MS vector, that from each location in the dataspace points towards the nearest mode. As previously said,the bandwidth selection theorem works well when the datais locally distributed as a Gaussian distribution, corrupted byGaussian noise. The results are reported in Table 2. As ex-pected, our algorithm shows up better performances.

0.5 1.0 1.5 2.0 2.5 3.0

test 1.53 2.63 4.60 6.07 6.97 8.33

boxwhole 0.06 0.44 1.19 2.19 2.88 3.00

cutcube 0.03 0.43 0.60 1.30 1.40 2.27

foursix 5.19 6.14 7.99 9.09 10.03 10.59

Table 2: Synthetic experiments results obtained with the au-tomatic bandwidth selection developed in [Com03]: meanpercentage of mismatches wrt the number of mesh trianglesvs the bound’s width k.

c© The Eurographics Association 2006.

M.Farenzena, M. Cristani & U. Castellani / 3D Objects Face Clustering using Unsupervised Mean Shift

We tested the algorithm even on real cases, using modelsobtained from an image-based reconstruction process. Thechurch model is composed by 43 3D points and 93 triangles.The planes extracted by our algorithm are 30, with 8 trian-gles misclusterized, two of which derives from a wrong nor-mals classification, the others from a wrong parallel planesseparation (see Fig. 4).

Figure 4: Two views of church model, with the planes ex-tracted by our algorithm.

The tribuna model is composed by 272 points and 364triangles. The planes extracted are 52, with about 30 trian-gles misclusterized. In this case, most of misclusterizationsderive from wrong parallel planes separation.

Figure 5: Two views of tribuna model, with the planes ex-tracted by our algorithm.

6. Conclusions

In this paper, we propose a robust and fully automatic ap-proach to face clustering based on the mean shift algorithm.The aim is the extraction of planes of a mesh acquired froma 3D reconstruction process. Robustness is proved by bothsynthetic and real experimental results. Related to the auto-matic choice of the bandwidth value, in this paper we provethat, in case of rigid geometrical structures described bypunctual measurements such as face normals, methods rely-ing on Gaussian assumptions perform poorly. Instead, meth-ods of bandwidth selection based on more general principlessuch as the stability of the partition produced, better behave.

References

[CSAD04] COHEN-STEINER D., ALLIEZ P., DESBRUN

M.: Variational shape approximation. ACM Transactionson Graphics 23, 3 (2004), 905–914. 1

[HY05] H. YAMAUCHI S. LEE Y. L. E. A.: Feature sensi-tive mesh segmentation with mean shift. Shape ModelingInternational (2005), 236–243. 1

[KT96] KALVIN A., TAYLOR R.: Surfaces: Polygonalmesh semplification with bounded error. IEEE ComputerGraphics and Applications 16 (1996), 64–77. 1

[MCV06] M. CRISTANI U. C., V.MURINO: Adaptivefeature integration for segmentation of 3d data by unsu-pervised density estimation. In International Conferenceon Pattern Recognition ICPR 2006 (August 2006), vol. 4,pp. 21–24. 1, 2

[MFD06] M. FARENZENA A. F., DOVIER A.: Recon-struction with interval constraints propagation. In Com-puter Vision and Pattern Recognition (2006), pp. 1185–1190. 1

[SWG∗03] SANDER P. V., WOOD Z. J., GORTLER S. J.,SNYDER J., HOPPE H.: Multi-chart geometry images.In SGP ’03: Proceedings of the 2003 Eurographics/ACMSIGGRAPH symposium on Geometry processing (Aire-la-Ville, Switzerland, Switzerland, 2003), Eurographics As-sociation, pp. 146–155. 1

[TF04] T. FUNKHOUSER M. KAZHDAN P. S. E. A.: Mod-eling by example. ACM Transactions on Graphics 23(2004), 652–663. 1

[DCSD04] D. COHEN-STEINER P. A., DESBREUN M.:Variation shape approximation. ACM Transactions onGraphics 23 (August 2004), 905–914. 1

[MGH01] M. GARLAND A. W., HECKBERT P.: Hier-archcal face clustering on polygonal surfaces. In Pro-ceedings Symposium on Interactive 3D Graphics (2001),pp. 49–58. 1

[Col03] R. COLLINS: Mean-shift blob tracking throughscale space. In CVPR (2), pages 234–240, 2003. 1

[Com03] D. COMANICIU: An algorithm for data-drivenbandwidth selection. IEEE Transactions on Pattern Anal-ysis and Machine Intelligence, 25(2):281–288, 2003. 2, 3,4

[CM02] D. COMANICIU AND P. MEER: Mean shift: A ro-bust approach toward feature space analysis. IEEE Trans-actions on Pattern Analysis and Machine Intelligence,24(5):603–619, 2002. 1, 2

[Fuk90] K. FUKUNAGA: Statistical Pattern Recognition.Academic Press, second edition, 1990. 2

[GSM03] B. GEORGESCU, I. SHIMSHONI, AND

P. MEER: Mean shift based clustering in high dimen-sions: A texture classification example. In ICCV, pages456–463, 2003. 1

c© The Eurographics Association 2006.