+ All Categories
Home > Documents > K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by...

K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by...

Date post: 14-Jul-2020
Category:

Documents
Upload: others
View: 1 times
Download: 0 times
Download Report this document
Share this document with a friend
14
K-Means Clustering 3/3/17
Transcript
Page 1: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

K-MeansClustering3/3/17

Page 2: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

UnsupervisedLearning• Wehaveacollectionofunlabeled datapoints.• Wewanttofindunderlyingstructureinthedata.

Examples:• Identifygroupsofsimilardatapoints.• Clustering

• Findabetterbasistorepresentthedata.• Principalcomponentanalysis

• Compressthedatatoashorterrepresentation.• Auto-encoders

Page 3: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

UnsupervisedLearning• Wehaveacollectionofunlabeled datapoints.• Wewanttofindunderlyingstructureinthedata.

Applications:• GeneratingtheinputrepresentationforanotherAIorMLalgorithm.• ClusterscouldleadtostatesinastatespacesearchorMDPmodel.• Anewbasiscouldbetheinputtoaclassificationorregressionalgorithm.

• Makingdataeasiertounderstand,byidentifyingwhat’simportantand/ordiscardingwhatisn’t.

Page 4: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

TheGoalofClusteringGivenabunchofdata,wewanttocomeupwitharepresentationthatwillsimplifyfuturereasoning.

Keyidea:groupsimilarpointsintoclusters.

Examples:• Identifyingobjectsinsensordata• Detectingcommunitiesinsocialnetworks• Constructingphylogenetictreesofspecies• Makingrecommendationsfromsimilarusers

Page 5: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

EMAlgorithmEstep:“expectation”…terriblename• Classifythedatausingthecurrentmodel.

Mstep:“maximization”…slightlylessterriblename• Generatethebestmodelusingthecurrentclassificationofthedata.

Initializethemodel,thenalternateEandMstepsuntilconvergence.

Note:TheEMalgorithmhasmanyvariations,includingsomethathavenothingtodowithclustering.

Page 6: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

K-MeansAlgorithmModel:kclusterseachrepresentedbyacentroid.

Estep:• Assigneachpointtotheclosestcentroid.

Mstep:• Moveeachcentroidtothemeanofthepointsassignedtoit.

Convergence:werananEstepwherenopointshadtheirassignmentchanged.

Page 7: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

K-MeansExample

Page 8: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

InitializingK-MeansReasonableoptions:

1. StartwitharandomEstep.• Randomlyassigneachpointtoaclusterin{1,2,…,k}.

2. StartwitharandomMstep.a) Pickrandomcentroidswithinthemaximumrangeof

thedata.b) Pickrandomdatapointstouseasinitialcentroids.

Page 9: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

K-MeansinAction

https://www.youtube.com/watch?v=BVFG7fd1H30

Page 10: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

AnotherEMExample:GMMs

GMM:Gaussianmixturemodel• AGaussiandistributionisamultivariategeneralizationofanormaldistribution(theclassicbellcurve).• AGaussianmixtureisadistributioncomprisedofseveralindependentGaussians.• IfwemodelourdataasaGaussianmixture,we’resayingthateachdatapointwasarandomdrawfromoneofseveralGaussiandistributions(butwemaynotknowwhich).

Page 11: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

EMforGaussianMixtureModelsModel:datadrawnfromamixtureofkGaussians

Estep:• Computethe(log)likelihoodofthedata• Eachpoint’sprobabilityofbeingdrawnfromeachGaussian.

Mstep:• UpdatethemeanandcovarianceofeachGaussian.• WeightedbyhowresponsiblethatGaussianwasforeachdatapoint.

Page 12: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

HowdowepickK?There’snohardrule.

• Sometimestheapplicationforwhichtheclusterswillbeuseddictatesk.• Ifkcanbeflexible,thenweneedtoconsiderthetradeoffs:• Higherkwillalwaysdecreasetheerror(increasethelikelihood).• Lowerkwillalwaysproduceasimplermodel.

Page 13: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

HierarchicalClustering• Organizesdatapointsintoahierarchy.• Everylevelofthebinarytreesplitsthepointsintotwosubsets.• Pointsinasubsetshouldbemoresimilarthanpointsindifferentsubsets.• Theresultingclusteringcanberepresentedbyadendrogram.

Page 14: K-Means Clustering - Swarthmore College · K-Means Algorithm Model: k clusters each represented by a centroid. E step: •Assign each point to the closest centroid. M step: •Move

DirectionofClusteringAgglomerative(bottom-up)• Eachpointstartsinitsowncluster.• Repeatedlymergethetwomost-similarclustersuntilonlyoneremains.

Divisive(top-down)• Allpointsstartinasinglecluster.• Repeatedlysplitthedataintothetwomostself-similarsubsets.

Eitherversioncanstopearlyifaspecificnumberofclustersisdesired.


Recommended
The Centroid

The Centroid

Documents

K-means Clustering

K-means Clustering

Documents

Clustering: K-means and Kernel K-means · Clustering: K-means and Kernel K-means Piyush Rai Machine Learning (CS771A) Aug 31, 2016 Machine Learning (CS771A) Clustering: K-means and

Clustering: K-means and Kernel K-means · Clustering: K-means and Kernel K-means Piyush Rai Machine Learning (CS771A) Aug 31, 2016 Machine Learning (CS771A) Clustering: K-means and

Documents

Applying subclustering and Lp distance in Weighted K-Means ...Applying subclustering and L p distance in Weighted K-Means with distributed centroid Renato Cordeiro de Amorima,, Vladimir

Applying subclustering and Lp distance in Weighted K-Means ...Applying subclustering and L p distance in Weighted K-Means with distributed centroid Renato Cordeiro de Amorima,, Vladimir

Documents

CLUSTERING - University of Belgradeai.fon.bg.ac.rs/wp-content/uploads/2015/04/ML-Clustering-2013_eng.… · K-MEANS: THE ALGORITHMSteps: 1) Initial, random selection of a centroid

CLUSTERING - University of Belgradeai.fon.bg.ac.rs/wp-content/uploads/2015/04/ML-Clustering-2013_eng.… · K-MEANS: THE ALGORITHMSteps: 1) Initial, random selection of a centroid

Documents

Machine Learning for Data-Driven Smart Grid Applications · K-means algorithm 1: Define K centroids randomly. 2: Associate every observation according to the nearest centroid. 3:

Machine Learning for Data-Driven Smart Grid Applications · K-means algorithm 1: Define K centroids randomly. 2: Associate every observation according to the nearest centroid. 3:

Documents

Manual Centroid

Manual Centroid

Documents

Centroid Brochure

Centroid Brochure

Design

Generalized Mean Distance-based k Nearest Centroid ...

Generalized Mean Distance-based k Nearest Centroid ...

Documents

Detecting Anomalies in Irregular Data Using K-means ...Detecting Anomalies in Irregular Data Using K-means Clustered Signal Dictionary 247 centroid c p. x m belongs to C p. Step 5:

Detecting Anomalies in Irregular Data Using K-means ...Detecting Anomalies in Irregular Data Using K-means Clustered Signal Dictionary 247 centroid c p. x m belongs to C p. Step 5:

Documents

K-Means Clustering - totoharyanto.staff.ipb.ac.idtotoharyanto.staff.ipb.ac.id/files/2013/09/LKP-10-K_Means.pdf · Lakukan iterasi step 3-5 sampai tak ada lagi perubahan centroid ...

K-Means Clustering - totoharyanto.staff.ipb.ac.idtotoharyanto.staff.ipb.ac.id/files/2013/09/LKP-10-K_Means.pdf · Lakukan iterasi step 3-5 sampai tak ada lagi perubahan centroid ...

Documents

Faster Algorithms for the Constrained k-means Problemrjaiswal/Files/list-k-means-slides.pdf · k-means Clustering Problem Problem (k-means) Given n points X ˆRd, and an integer k,

Faster Algorithms for the Constrained k-means Problemrjaiswal/Files/list-k-means-slides.pdf · k-means Clustering Problem Problem (k-means) Given n points X ˆRd, and an integer k,

Documents

CENTROID · CENTROID RT-150 Rotary Table Operator’s Manual Serial # _____ Rev. 082704 ©2004 Centroid Corp. Howard, PA 16841

CENTROID · CENTROID RT-150 Rotary Table Operator’s Manual Serial # _____ Rev. 082704 ©2004 Centroid Corp. Howard, PA 16841

Documents

K means clustering

K means clustering

Data & Analytics

Scalable K-Means++

Scalable K-Means++

Documents

Centroid Calculator

Centroid Calculator

Documents

CENTROID (AĞIRLIK MERKEZİ ) - DEUkisi.deu.edu.tr/binnur.goren/Statics2017/13_CENTROID_MOMENT_OF_INERTIA... · CENTROID (AĞIRLIK MERKEZİ ) A centroid is a geometrical concept arising

CENTROID (AĞIRLIK MERKEZİ ) - DEUkisi.deu.edu.tr/binnur.goren/Statics2017/13_CENTROID_MOMENT_OF_INERTIA... · CENTROID (AĞIRLIK MERKEZİ ) A centroid is a geometrical concept arising

Documents

Monte Carlo K-Means Clustering · Centroid . Find the distance from one cluster center point to another cluster center point ...

Monte Carlo K-Means Clustering · Centroid . Find the distance from one cluster center point to another cluster center point ...

Documents