PARTITIONAL CLUSTERING

Deniz ÜSTÜN

CONTENT

WHAT IS CLUSTERING ?

WHAT IS PARTITIONAL CLUSTERING ?

THE USED ALGORITHMS IN PARTITIONAL CLUSTERING

What is Clustering ? A process of clustering is classification of the objects which are

similar among them, and organizing of data into groups.

The techniques for Clustering are among the unsupervised methods.

What is Partitional Clustering ? The Partitional Clustering Algorithms separate the similar

objects to the Clusters.

The Partitional Clustering Algorithms are succesful to determine center based Cluster.

The Partitional Clustering Algorithms divide n objects to k cluster by using k parameter.

The techniques of the Partitional Clustering start with a randomly chosen clustering and then optimize the clustering according to some accuracy measurement.

The Used Algorithms in Partitional Clustering

K-MEANS ALGORITHM

K-MEDOIDS ALGORITHM

FUZZY C-MEANS ALGORITHM

K-MEANS ALGORITHM

K-MEANS algorithm is introduced as one of the simplest unsupervised learning algorithms that resolve the clustering problems by J.B. MacQueen in 1967 (MacQueen, 1967).

K-MEANS algorithm allows that one of the data belong to only a cluster.

Therefore, this algorithm is a definite clustering algorithm.

Given the N-sample of the clusters in the N-dimensional space.

K-MEANS ALGORITHM

This space is separated, {C1,C2,…,Ck} the K clusters. The vector mean (Mk) of the Ck cluster is given (Kantardzic, 2003) :

kn

iik

kk Xn

M1

1

where the value of Xk is i.sample belong to Ck.

The square-error formula for the Ck is given :

2

1

2

kn

ikiki MXe

K-MEANS ALGORITHM The square-error formula for the Ck is called the changing in

cluster. The square-error for all the clusters is the sum of the changing

in clusters.

K

kkk eE

1

22

The aim of the square-error method is to find the K clusters that minimize the value of the Ek

2 according to the value of the given K

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

K-MEANS ALGORITHMEXAMPLE

Gözlemler Değişken1 Değişken2 Küme ÜyeliğiX1 3 2 C1X2 2 3 C2X3 7 8 C1

5,52

82,2

731

M

3,213,

12

2

M

K-MEANS ALGORITHMEXAMPLE

2158575253 222221 e

2102122

21

2 eeE

03332 2222 e

K-MEANS ALGORITHMEXAMPLE

41,12332, 2212 XMd

82,22535, 2211 XMd

60,33525, 2221 XMd

03322, 2222 XMd

60,38575, 2231 XMd

07,78372, 2232 XMd

Gözlemler d(M1) d(M2) Küme Üyeliği

X1 2,82 1,41 C2X2 3,60 0 C2X3 3,60 7,07 C1

1112 ,, XMdXMd

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

K-MEANS ALGORITHMEXAMPLE

Gözlemler Değişken1 Değişken2 Küme ÜyeliğiX1 3 2 C2X2 2 3 C2X3 7 8 C1

5.2,5.22

32,2

232

M

8,718,

17

1

M

K-MEANS ALGORITHMEXAMPLE

15.235.225.225.23 222222 e

11022

21

2 eeE

08877 2221 e

K-MEANS ALGORITHMEXAMPLE-1

21,72837, 2211 XMd

7,025,235,2, 2212 XMd

07,73827, 2221 XMd

7,035,225,2, 2222 XMd

08877, 2231 XMd

10,785.275.2, 2232 XMd

Gözlemler d(M1) d(M2) Küme Üyeliği

X1 7,21 0,7 C2X2 7,07 0,7 C2X3 0 7,10 C1

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

9

10

C2

C1

K-MEANS ALGORITHMEXAMPLE-2

Dataset The Number of Attributes

The Number of Features

The Number of Class

Synthetic 1200 2 4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

K-MEANS ALGORITHMEXAMPLE-2

K=2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

K-MEANS ALGORITHMEXAMPLE-2

K=3

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

K-MEANS ALGORITHMEXAMPLE-2

K=4

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

K-MEDOIDS ALGORITHM The aim of the K-MEDOIDS algorithm is to find the K

representative objects (Kaufman and Rousseeuw, 1987). Each cluster in K-MEDOIDS algorithm is represented by

the object in cluster. K-MEANS algorithm determine the clusters by the mean

process. However, K-MEDOIDS algorithm find the cluster by using mid-point.

2

1

2

kn

ikiki OXe

K-MEDOIDS ALGORITHMEXAMPLE-1

K-MEDOIDS ALGORITHMEXAMPLE-1

Select the Randomly K-MedoidsSelect the Randomly K-Medoids

K-MEDOIDS ALGORITHMEXAMPLE-1

Allocate to Each Point to Closest MedoidAllocate to Each Point to Closest Medoid

K-MEDOIDS ALGORITHMEXAMPLE-1

Allocate to Each Point to Closest MedoidAllocate to Each Point to Closest Medoid

K-MEDOIDS ALGORITHMEXAMPLE-1

Allocate to Each Point to Closest MedoidAllocate to Each Point to Closest Medoid

K-MEDOIDS ALGORITHMEXAMPLE-1

Determine New Medoid for Each ClusterDetermine New Medoid for Each Cluster

K-MEDOIDS ALGORITHMEXAMPLE-1

Determine New Medoid for Each ClusterDetermine New Medoid for Each Cluster

K-MEDOIDS ALGORITHMEXAMPLE-1

Allocate to Each Point to Closest MedoidAllocate to Each Point to Closest Medoid

K-MEDOIDS ALGORITHMEXAMPLE-1

Stop ProcessStop Process

K-MEDOIDS ALGORITHMEXAMPLE-2

Dataset The Number of Attributes

The Number of Features

The Number of Class

Synthetic 2000 2 3

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

K-MEDOIDS ALGORITHMEXAMPLE-2

K=2

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

K-MEDOIDS ALGORITHMEXAMPLE-2

K=3

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

FUZZY C-MEANS ALGORITHM Fuzzy C-MEANS algorithm is the best known and widely

used a method. Fuzzy C-MEANS algorithm is introduced by DUNN in 1973

and improved by BEZDEK in 1981 [Höppner vd, 2000]. Fuzzy C-MEANS lets that objects are belonging to two and

more cluster. The total value of the membership of a data for all the

classes is equal to one. However, the value of the memebership of the cluster that

contain this object is high than other clusters. This Algorithm is used the least squares method [Höppner

vd, 2000].

FUZZY C-MEANS ALGORITHM

N

i

C

jii

mij mcxuJm

1 1

2 1 ,

The algorithm start by using randomly membership matrix (U) and then the center vector calculate [Höppner vd, 2000].

N

i

mij

N

ii

mij

j

u

xuc

1

1

FUZZY C-MEANS ALGORITHMAccording to the calculated center vector, the membership matrix (u) is computed by using the given as:

C

k

m

ki

ii

ij

cxcx

u

1

12

1

The new membership matrix (unew) is compared with the old membership matrix (uold) and the the process continues until the difference is smaller than the value of the ε

FUZZY C-MEANS ALGORITHMEXAMPLE

Dataset The Number of Attributes

The Number of Features

The Number of Class

Synthetic 2000 2 3

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

FUZZY C-MEANS ALGORITHMEXAMPLE

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

C=3m=5ε=1e-6

Results

K-MEDOIDS is the best algorithm according to K-MEANS and FUZZY C-MEANS.

However, K-MEDOIDS algorithm is suitable for small datasets.

K-MEANS algorithm is the best appropriate in terms of time.

In FUZZY C-MEANS algorithm, a object can belong to one or more cluster.

However, a object can belong to only a cluster in the other two algorithms.

References [MacQueen, 1967] J.B., MacQueen, “Some Methods for Classification and Analysis of

Multivariate Observations”, Proc. Symp. Math. Statist.and Probability (5th), 281-297,(1967). [Kantardzic, 2003] M., Kantardzic, “Data Mining: Concepts, Methods and Algorithms”, Wiley,

(2003). [Kaufman and Rousseeuw, 1987] L., Kaufman, P. J., Rousseeuw, “Clustering by Means of

Medoids,” Statistical Data Analysis Based on The L1–Norm and Related Methods, edited by Y. Dodge, North-Holland, 405–416, (1987).

[Kaufman and Rousseeuw, 1990] L., Kaufman, P. J., Rousseeuw, “Finding Groups in Data: An Introduction to Cluster Analysis”, John Wiley and Sons., (1990).

[Höppner vd, 2000] F., Höppner, F., Klawonn, R., Kruse, T., Runkler, “Fuzzy Cluster Analysis”, John Wiley&Sons, Chichester, (2000).

[Işık and Çamurcu, 2007] M., Işık, A.Y., Çamurcu, “K-MEANS, K-MEDOIDS ve Bulanık C-MEANS Algoritmalarının Uygulamalı olarak Performanslarının Tespiti”, İstanbul Ticaret Üniversitesi Fen Bilimleri Dergisi, Sayı :11, 31-45, (2007).