Date post: | 20-Dec-2015 |
Category: |
Documents |
View: | 213 times |
Download: | 0 times |
Direct Convex Relaxations of Sparse SVM
Antoni B. Chan, Nuno Vasconcelos, and Gert R. G. LanckrietThe 24th International Conference on Machine Learning (ICML 2007)
Presented by Shuiwang Ji
Outline
• Introduction;
• Quadratically Constrained Quadratic Programming (QCQP) formulation;
• Semidefinite Programming (SDP) formulation;
• Experiments;
Sparsity of SVM
x1, …, xd
SVM is sparse w.r.t. data points,
but not sparse w.r.t. features.
R
kkkk yα
12
1xw
( ) sgn( )Tf x w x b
Motivations & Related Work
• Features may be noisy, redundant;
• Sparsity enhance interpretability;
• Sparse PCA (Zou et al. & d'Aspremont et al.);
• Sparse Eigen Methods by D.C. Programming (ICML07);
Vector Norm
1 1
2 22 1
0
|| || | | ... | |
|| || ...
|| || ( )
n
Tn
x x x
x x x x x
x Card x
Number of nonzero entries in x
Interpretations of QCQP-SSVM
• Problem 6 and 7 are equivalent;
• QCQP-SSVM is a combination of C-SVM and LP-SVM, 1-norm encourages sparsity and 2-norm encourages large margin;
QCQP-SSVM
• QCQP-SSVM automatically learns an adaptive soft-threshold on the original SVM hyperplane.
SDP-SSVM Dual
• The optimal weighting matrix increases the influence of the relevant features while demoting the less relevant features;
• SDP-SSVM learns a weighting on the inner product such that the hyperplane in the feature space is sparse.