28
Clustering By: Avshalom Katz
Clustering
By: Avshalom Katz
We will be talking about…
• What is Clustering?• Different Kinds of Clustering• What is DBSCAN?• Pseudocode• Example of Clustering• Definitions of parameters• Complexity
What is Clustering?
• clustering is the assignment of a set of observations into subsets (called clusters) so that observations in the same cluster are similar in some sense.
Different types of Clustering
• Biology• Information retrieval • Climate• Business • Clustering for utility• Summarization
Example
DIFFERENT KINDS OF CLUSTERS
Well Separated
Prototype based
Graph based
Density based
Share property (conceptual clusters)
DBSCAN-IntroductionDensity-Based Spatial Clustering of Applications with Noise
• Since society has started using databases, the amount of information that we are using is increasing exponentially. Due to that, automatic algorithms are entered to every subject.
Database Example
Density-Based Spatial Clustering of Applications with Noise
• 1. Minimum point in the density (MINEPS)
• 2. The distance of the point to check the density (EPS).
There are four main steps in the algorithm, and the algorithm gets two parameters:
Definition 1
• To find all adjacent points. The so called “adjacent” points are called so only of the distance between them is smaller than EPS from what we refer to as P- “point”. All the adjacent points are later entered into Neps (P).
Definition 2• Is to define the
core group by checking if the point p is in the core with point q by checking if p includes in Neps (q) and the size of the group Neps (p) is grater then MINPTS.
Definition 3
• Density-reachable the point p is density reachable from point q if there is a sequence of points that the first is p and the last is q, then every couple in the sequence is a directly density reachable
Definition 4
• Density connected point refers to a single point that can reach two different points, also in different direction. For example in the diagram below we can see that P and Q are density-reachable from O. Therefore, P and Q are are density connected.
Definition 5
• Cluster C, wrt.erps and MINPTS are non-empty subset of the database, together these two terms below are created:
1. If P is a member of class C and q is density reachable from P and NEPS(P)> MINTPS then q is also a member of C.
2. If p and q are both members of C, then both p and q are density connected to eachother.
Definition 6
• There are groups of clusters, each point that does not belong to any group is called “noise”.
= noise
EB
FA
N
P
Q T
S
R
V
U
JC
H
G
I
DOL
KMε
DBSCAN ( Eps = ε , MinPts = 3 )number of adjacent : 5stack : B,C,D,E,Fcurrent ClusterId : green
number of adjacent : 8stack : C,D,E,F,G,H,I,current ClusterId : green
number of adjacent : 8stack : D,E,F,G,H,I,current ClusterId : green
number of adjacent : 9stack : F,G,H,I,Jcurrent ClusterId : green
number of adjacent : 7stack : E,F,G,H,Icurrent ClusterId : green
number of adjacent : 9stack : G,H,I,Jcurrent ClusterId : green
number of adjacent : 6stack : H,I,Jcurrent ClusterId : green
number of adjacent : 7stack : I,Jcurrent ClusterId : green
number of adjacent : 7stack : Jcurrent ClusterId : green
number of adjacent : 5stack : current ClusterId : green
number of adjacent : stack : current ClusterId : purple
number of adjacent : 0stack : current ClusterId : purple
X
number of adjacent : 3 stack : O,P,Qcurrent ClusterId : purple
number of adjacent : 2stack : P,Qcurrent ClusterId : purple
number of adjacent : 5stack : Q,R,S,Tcurrent ClusterId : purple
number of adjacent : 1stack : current ClusterId : purple
Pseudocode of the algorithm DBSCAN (Eps, MinPts) // SetOfPoints is UNCLASSIFIEDClusterId := nextId(NOISE);FOR i FROM 1 TO SetOfPoints.size DOPoint := SetOfPoints.get(i);IF Point.ClId = UNCLASSIFIED THENIF ExpandCluster(SetOfPoints, Point,ClusterId, Eps, MinPts) THEN ClusterId := nextId(ClusterId)END IFEND IFEND FOREND; // DBSCAN
ExpandCluster(SetOfPoints, Point, ClId, Eps,MinPts) : Boolean;seeds:=SetOfPoints.regionQuery(Point,Eps);IF seeds.size<MinPts THEN // no core pointSetOfPoint.changeClId(Point,NOISE);RETURN False;ELSE // all points in seeds are density- // reachable from PointSetOfPoints.changeClIds(seeds,ClId);seeds.delete(Point);WHILE seeds <> Empty DOcurrentP := seeds.first();result := SetOfPoints.regionQuery(currentP,Eps);IF result.size >= MinPts THENFOR i FROM 1 TO result.size DOresultP := result.get(i);IF resultP.ClId IN {UNCLASSIFIED, NOISE} THENIF resultP.ClId = UNCLASSIFIED THENseeds.append(resultP);
• END IF;• SetOfPoints.changeClId(resultP,ClId);• END IF; // UNCLASSIFIED or NOISE• END FOR;• END IF; // result.size >= MinPts• seeds.delete(currentP);• END WHILE; // seeds <> Empty• RETURN True;• END IF• END; // ExpandCluster
Example
Define the value of parameter EPS bay MINPTS:
The complexityThe complexity of ExpandCluster() is o(logN) in the worst case on a data base in size N and there is n iterations of this function ,so it is on * log (n) )
Bibliography • Ankerst, M., Breunig, M. M., Kriegel, H.-P., and Sander, J. (1999). Optics:
ordering points to identify the clustering structure. SIGMOD Rec., 28(2):49-60
• Clustering. (2010, April 19). In Wikipedia, The Free Encyclopedia. Retrieved 14:14, April 19, 2010
from http://en.wikipedia.org/w/index.php?title=Clustering&oldid=357078594
• Ester, M., Kriegel, H.-p., Jörg, S., and Xu, X. (1996). A density-based algorithm for discovering clusters in large spatial databases with noise.
• Ester, M ., Kriegel, H,. Jörg, S., and Xu, X (1995).A DatabaseIn terface forClustering in Large Spatial Databases, Proc. 1st Int. Conf. onKnowledge Discovery and Data Mining, Montreal, Canada, 1995, AAAI Press, 1995.
• Schikuta E., Erhart M.: “The bang-clustering system:Grid-based data
analysis”. Proc. Sec. Int. Symp. IDA-97,Vol. 1280 LNCS, London, UK, Springer-Verlag, 1997.
