Outline
Two models for One Class ClassificationFrom One Class to Binary ClassFrom Binary Class to Multi ClassFrom Multi Class to ClusteringConclustion
Two models for One Class Classification
One Class SVM Find the optimal hyperplane to separate the tar
get class from the origin with maximum margin
Support Vector Data Description Use the minimum hyperspere to enclose the tar
get class
From SVDD to Binary SVDD with negative data: B_SVDD_Neg
Objective function:
21 2
2 2
2 2
( , , )
|| ||. . , 0, 0,
|| ||
target class, class
i pi p
i ii p
p p
R a R C C
x a Rs t
x a R
i p negative
Drawback: without considering the margin between classes.
The other Version of B_SVDD_Neg
Dong, X., W. Zhaohui, et al. (2001). A new multi-class support vector machines, Systems, Man, and Cybernetics, 2001 IEEE International Conference on.
Does it work really?
1
1
1
0
to KKT, if >0 =0
then 1
n
ii
n
ii
here
according
No support vector of
Negative data.Can’t calculate R
1. Modify the coefficient of R: Biased support vector machine
Chan, C.-H., K. Huang, and M.R.L.a.I. King. Biased support vector machine for relevance feedback in image retrieval. in International Joint Conference on Neural Networks 2004. Budapest, Hungary.
In order to avoid the above problem, b need to less than 1, that is 1b
Equivalent style: Minimum Enclosing and Maximum Excluding Machine
Liu, Y. and Y.F. Zheng. Minimum Enclosing and Maximum Excluding Machine for Pattern Description and Discrimination Pattern Recognition. in Proc of the 18th Int Conf on ICPR 2006.Loa Alamitos: IEEE Computer Society
2. Modify the coefficient of margin
Here,
Wang, J., N. P, et al. (2005). Pattern classification via single spheres, Lecture notes in artificial intelligence.( briefly PCSS)
, 1here K
3. Modify the coefficients of margin and R
张新峰 ; 刘垚巍 : 广义超球面 SVM研究 , 计算机研究与发展 2008.1
1
Generalized HyperSphere SVM(GHSSVM)
Extend to Ellipsoid
Wei2007:Minimum Mahalanobis Enclosing Ellipsoid Machine for Pattern Classification:ICIC 2007,CCIS2, pp. 1176-1185
SVDD with negative data for Multi-Class:M_SVDD_Neg
Drawback: without considering the margin either .
Embedding margin for SVDD_Mulit: MSM_SVM
Pei-Yi Hao, Jung Hsien Chiang, Yen Hsiu lin:A new maximal-margin spherical-structured multi-class support vector machine, Appl Intell, 2009,30,P98-111
OCSVM with negative: Binary OCSVM_Neg
Motivation: using the mean of the other class instead of the optimal point. Doesn’t considering margin either.
1
1 1min
2
1. . ( ) , 0, 1,...,
n
ii
i n i i
n
s t x z i nt
Tw,ξ,
T
w w
w
From OCSVM to Asymmetric SVM: margin embededLike the SVDD Multi with margin, here also describe the target class by core hyperplane, then push the negative class by maximized the margin.
S. H. Wu, K. P. Lin, C. M. Chen, M. S. Chen, Asymmetric support vector machines: low false positive learning under the user tolerance, Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, 749-757, 2008.
Summarize
Model Hyperspere Ellipsoid Hyperplane
One-Class SVDD MVEE ,MVCE, MELM
OCSVM
Binary-Class Without margin
B_SVDD_Neg B_OCSVM_Neg
Embedding margin
BSVM, MEMEMPCSS, GHSSVM
Binary MELM
ASVM
Multi-Class Without margin
Multi SVDD_Neg
One against others Or One-to One
?Embedding margin
MSM SVM ?
One Class Classification for Clustering
Support Vector Clustering(JMLR2002)Iterative strategy integrating two-stage
one-class SVMKernel Growth (PAMI 2005)Soft Clustering for Kernel Growth
Support Vector Clustering
Clustering boundary: same as SVDD, found the support vector to get the boundary.
Clustering number: based on the adjacency matrix which components decided according to:
Ben-Hur, H. A., D., et al. (2002). "Support vector clustering
" Journal of Machine Learning Research 2 125-137.
Iterative strategy integrating two-stage one-class SVM
Yeh, C.-Y. and S.-J. Lee (2007). A Kernel-Based Two-Stage One-Class Support Vector Machines Algorithm. Advances in Neural Networks – ISNN 2007.
Different from SVC, need to know the clustering number in advance, attribute to partition-based clustering algorithm.
First stage: using OCSVM for each cluster to find the non-support vectors ;
Second stage: retrain the OCSVM using those non-support vector for representing each clustering accurately by the optimal hyperplane.