arXiv:1411.6232v2 [cs.LG] 11 Jan 2015 1 Semi-supervised Feature Analysis by Mining Correlations among Multiple Tasks Xiaojun Chang and Yi Yang ✦ Abstract—In this paper, we propose a novel semi-supervised fea- ture selection framework by mining correlations among multiple tasks and apply it to different multimedia applications. Instead of independently computing the importance of features for each task, our algorithm leverages shared knowledge from multiple related tasks, thus, improving the performance of feature selection. Note that we build our algorithm on assumption that different tasks share common structures. The proposed algorithm selects features in a batch mode, by which the correlations between different features are taken into consideration. Besides, considering the fact that labeling a large amount of training data in real world is both time-consuming and tedious, we adopt manifold learning which exploits both labeled and unlabeled training data for feature space analysis. Since the objective function is non-smooth and difficult to solve, we propose an iterative algorithm with fast convergence. Extensive experiments on different applications demonstrate that our algorithm outperforms other state-of-the-art feature selection algorithms. Index Terms—Multi-task feature selection, semi-supervised learn- ing, image annotation, 3D motion data annotation 1 I NTRODUCTION I N many computer vision and pattern recognition applications, dimension of data representation is normally very high. Recent studies have claimed that not all features in the high-dimensional feature space are discriminative and informative, since many fea- tures are often noisy or correlated to each other, which will deteriorate the performances of subsequent data analysing tasks [1], [2], [3]. Consequently, feature se- lection is utilized to select a subset of features from the original high dimensional feature space [4], [5], [6], [7], [8]. It has twofold functions in enhancing performances of learning tasks. First, feature selection eliminates noisy and redundant information to get a better representation, thus facilitating classification and clustering tasks. Second, dimension of selected feature space becomes much lower, which makes the subsequent computation more efficient. Inspired by the motivations, much progress has been made to feature selection during last few years. According to availability of class labels of train- ing data, feature selection algorithms fall into two Xiaojun Chang and Yi Yang are with School of Information Technology and Electric Engineering, The University of Queensland, Australia.(e- mail: [email protected], [email protected]). groups, i.e. supervised feature selection and unsuper- vised feature selection. Supervised feature selection algorithms, for example, Fisher Score [9], only use labeled training data for feature selection. With suf- ficient labeled training samples, supervised feature selection is reliable to train appropriate feature selec- tion functions because of utilization of class labels. However, labeling a large amount of training sam- ples manually is unrealistic in real-world applications. Recent works on semi-supervised learning have indi- cated that it is beneficial to leverage both labeled and unlabeled training data for data analysis. Motivated by the progress of semi-supervised learning, much research attention has been paid to semi-supervised feature selection. For example, Zhao et al. propose a semi-supervised feature selection algorithm based on spectral analysis. A common limitation of the existing supervised and semi-supervised feature selection al- gorithms is that they evaluate the importance of each feature individually, ignoring correlations between different features. To address this problem, some state- of-the-art algorithms are proposed to take feature correlations into consideration for feature selection. For example, [10] and [3] implement their methods in a supervised way and Ma et al. design their approach in a semi-supervise way in [5]. Another limitation of current feature selection al- gorithms is that they select features for each task individually, which fails to mine correlations among multiple related tasks. Recent researches have indi- cated that it is beneficial to learn multiple related tasks jointly [11], [12], [13], [14]. Motivated by this fact, multi-task learning has been introduced to the field of multimedia. For instance, Yang et al. present a novel feature selection algorithm which leverages shared information from related tasks in [6]. Nevertheless, they design their algorithm in a supervised way. The semi-supervised algorithm proposed in this paper combines the strengths of semi-supervised fea- ture selection and multi-task learning. Both labeled and unlabeled training data are utilized for feature selection. Meanwhile, correlations between different features are taken into consideration to improve the performance of feature selection.
Transcript
arX
iv:1
411.
6232
v2 [
cs.L
G]
11
Jan
2015
1
Semi-supervised Feature Analysis by MiningCorrelations among Multiple Tasks
Xiaojun Chang and Yi Yang
Abstract —In this paper, we propose a novel semi-supervised fea-ture selection framework by mining correlations among multipletasks and apply it to different multimedia applications. Instead ofindependently computing the importance of features for each task,our algorithm leverages shared knowledge from multiple relatedtasks, thus, improving the performance of feature selection. Notethat we build our algorithm on assumption that different tasks sharecommon structures. The proposed algorithm selects features in abatch mode, by which the correlations between different features aretaken into consideration. Besides, considering the fact that labelinga large amount of training data in real world is both time-consumingand tedious, we adopt manifold learning which exploits both labeledand unlabeled training data for feature space analysis. Since theobjective function is non-smooth and difficult to solve, we proposean iterative algorithm with fast convergence. Extensive experimentson different applications demonstrate that our algorithm outperformsother state-of-the-art feature selection algorithms.
Index Terms —Multi-task feature selection, semi-supervised learn-ing, image annotation, 3D motion data annotation
1 INTRODUCTION
IN many computer vision and pattern recognitionapplications, dimension of data representation is
normally very high. Recent studies have claimed thatnot all features in the high-dimensional feature spaceare discriminative and informative, since many fea-tures are often noisy or correlated to each other, whichwill deteriorate the performances of subsequent dataanalysing tasks [1], [2], [3]. Consequently, feature se-lection is utilized to select a subset of features fromthe original high dimensional feature space [4], [5],[6], [7], [8]. It has twofold functions in enhancingperformances of learning tasks. First, feature selectioneliminates noisy and redundant information to geta better representation, thus facilitating classificationand clustering tasks. Second, dimension of selectedfeature space becomes much lower, which makes thesubsequent computation more efficient. Inspired bythe motivations, much progress has been made tofeature selection during last few years.
According to availability of class labels of train-ing data, feature selection algorithms fall into two
Xiaojun Chang and Yi Yang are with School of Information Technologyand Electric Engineering, The University of Queensland, Australia.(e-mail: [email protected], [email protected]).
groups, i.e. supervised feature selection and unsuper-vised feature selection. Supervised feature selectionalgorithms, for example, Fisher Score [9], only uselabeled training data for feature selection. With suf-ficient labeled training samples, supervised featureselection is reliable to train appropriate feature selec-tion functions because of utilization of class labels.However, labeling a large amount of training sam-ples manually is unrealistic in real-world applications.Recent works on semi-supervised learning have indi-cated that it is beneficial to leverage both labeled andunlabeled training data for data analysis. Motivatedby the progress of semi-supervised learning, muchresearch attention has been paid to semi-supervisedfeature selection. For example, Zhao et al. propose asemi-supervised feature selection algorithm based onspectral analysis. A common limitation of the existingsupervised and semi-supervised feature selection al-gorithms is that they evaluate the importance of eachfeature individually, ignoring correlations betweendifferent features. To address this problem, some state-of-the-art algorithms are proposed to take featurecorrelations into consideration for feature selection.For example, [10] and [3] implement their methods ina supervised way and Ma et al. design their approachin a semi-supervise way in [5].
Another limitation of current feature selection al-gorithms is that they select features for each taskindividually, which fails to mine correlations amongmultiple related tasks. Recent researches have indi-cated that it is beneficial to learn multiple relatedtasks jointly [11], [12], [13], [14]. Motivated by this fact,multi-task learning has been introduced to the field ofmultimedia. For instance, Yang et al. present a novelfeature selection algorithm which leverages sharedinformation from related tasks in [6]. Nevertheless,they design their algorithm in a supervised way.
The semi-supervised algorithm proposed in thispaper combines the strengths of semi-supervised fea-ture selection and multi-task learning. Both labeledand unlabeled training data are utilized for featureselection. Meanwhile, correlations between differentfeatures are taken into consideration to improve theperformance of feature selection.
Fig. 1. The Illustration of general process of applying the proposed approach for video classification.
We illustrate how the proposed algorithm worksfor video classification in Figure 1. First, we representall the training and testing videos as feature vectors.Then, sparse coefficients are learnt by exploiting re-lationships among different features and leveargingknowledge from multiple related tasks. After selectingthe most representative features, we can apply thesparse coefficients to the feature vectors of the testingvideos for classification.
We name our proposed algorithm Semi-supervisedFeature selection by Mining Correlations among mul-tiple tasks (SFMC). The main contributions of ourwork can be summarized as follows:
1) We combine semi-supervised feature selectionand multi-task learning into a single framework,which can select the most representative featureswith an insufficient amount of labeled trainingdata per task.
2) To explore correlations among multimedia data,we leverage the benefit of manifold learning intoour framework.
3) Since the objective function is non-smooth anddifficult to solve, a fast iterative algorithm toobtain the optimal solution is proposed. Experi-mental results on convergence demonstrate thatthe proposed algorithm converges within veryfew iterations.
The rest of this paper is organized as follows: Sec-tion 2 summarizes the overview of the related work.A novel Semi-supervised Feature Selection by MiningCorrelations among multiple tasks is proposed in sec-tion 3. We present our experimental results in section4. The conclusion of our work is discussed in section5.
2 RELATED WORK
In this section, we briefly review the related researchon feature selection, semi-supervised learning andmulti-task learning.
2.1 Feature selection
Previous works have claimed that feature selection iscapable of selecting the most representative features,thus facilitating subsequent data analysing tasks [15][16] [17].
Existing feature selection algorithms are designed invarious ways. Classical feature selection algorithms,such as Fisher Score [9], evaluate the weights of allfeatures, rank them accordingly and select the mostdiscriminating features one by one [18]. Althoughthese classical feature selection algorithms gain goodperformances in different applications, they havethree main limitations. First, they only use labeled
3
training data to exploit the correlations between fea-tures and labels for feature selection. Labeling a largeamount of training data consumes a lot of humanlabor in real-world applications. Second, the mostrepresentative features are selected one by one, thusignoring the correlations among different features.Third, they select features for each task independently,which fails to leverage the knowledge shared bymultiple related tasks.
To overcome the aforementioned limitations, re-searchers have proposed multiple feature selectionalgorithms. l2,1-norm regularization has been widelyused in feature selection algorithms for its capabilityof selecting features across all data points with jointsparsity. For example, Zhao et al. propose an algo-rithm which selects features jointly based on spectralregression with l2,1-norm constraint in [19]. Nie et al.adopt l2,1-norm on both regularization term and lossfunction in [10]. Yang et al. propose to select featuresby leveraging shared knowledge from multiple re-lated tasks in [6]. However, their algorithms are alldesigned in a supervised way.
2.2 Semi-supervised learning
Semi-supervised learning has shown its promisingperformance in different applications [20], [21], [22],[23], [24], [25]. With semi-supervised learning, unla-beled training data can be exploited to learn datastructure, which can save human labor cost for label-ing a large amount of training data [26], [27], [28], [29].Hence, semi-supervised learning is beneficial in termsof both the human laboring cost and data analysisperformance.
Graph Laplacian based semi-supervised learninghas gained increasing interest for its simplicity andefficiency [30]. Nie et al. propose a manifold learn-ing framework based on graph Laplacian and com-pared its performance with other state-of-the-art semi-supervised algorithms in [31]. Ma et al. proposea semi-supervised feature selection algorithm builtupon manifold learning in [5]. In [32], Yang et al.propose a new semi-supervised algorithm based ona robust Laplacian matrix for relevance feedback.Their algorithm has demonstrated its prominent per-formance. Therefore, we propose to leverage it in ourfeature selection framework. These previous works,however, independently select features for each task,which fails to consider correlations among multiplerelated tasks.
2.3 Multi-task learning
Multi-task learning has been widely used in manyapplications with the appealing advantage that itlearns multiple related tasks with a shared represen-tation [11] [12] [33]. Recent researches have indicatedthat learning multiple related tasks jointly alwaysoutperforms learning them independently. Inspired
by the progress of multi-task learning, researchershave introduced it to the field of multimedia anddemonstrated its promising performance on multime-dia analysis. For example, Yang et al. propose a novelmulti-task feature selection algorithm which improvesfeature selection performance by leveraging sharedinformation among multiple related tasks [6]. In [6],Ma et al. apply knowledge adaptation to multimediaevent detection and compare its performance withseveral state-of-the-art algorithms. Despite of theirgood performances, these classical algorithms are allimplemented only with labeled training data.
3 METHODOLOGY
In this section, we describe the approach of our pro-posed algorithm in detail.
3.1 Problem Formulation
Suppose we are going to select features for t tasks.The l-th task contains nl training data with ml datalabeled. We can formulate the regularized frameworkfor feature selection as follows:
minWl
t∑
l=1
(loss(Wl) + αg(Wl)) + γΩ(W ), (1)
where Wl is feature selection matrix for the l-th task,W = [W1, · · · ,Wt], loss(Wl) is the loss function whichevaluates consistency between features and labels,g(Wl) is a regularization function, Ω(W ) is a regu-larization term which is used to encode the commoncomponents of different feature selection functions, αand γ are regularization parameters.
To step further, we first give the definitions ofFrobenius norm and trace norm. Given an arbitrarymatrix M ∈ R
a×b where a and b are arbitrary num-bers, its Frobenius norm is defined as ‖M‖F . Thedefinition of its l2,1-norm is:
‖M‖2,1 =a∑
i=1
√√√√b∑
j=1
M2ij , (2)
and the definition of its trace norm is:
‖M‖∗ = Tr(MMT )1
2 , (3)
where Tr(·) denotes the trace operator. In the liter-ature, there are many approaches to define the lossfunction. Following the works in [5] [6], we adoptthe least square loss function for its simplicity andefficiency. Recent works [10] [18] claim that minimiz-ing the regularization term ‖Wl‖2,1 makes Wl sparse,which demonstrates that Wl is especially suitable forfeature selection. Motivated by the works in [34] [6],we propose to leverage shared knowledge amongmultiple related tasks by minimizing the trace normof W . The objective function is given by:
4
minWl
t∑
l=1
(loss(Wl) + α‖Wl‖2,1) + γ‖W‖∗ (4)
State-of-the-art feature selection algorithms are im-plemented through supervised learning and selectfeatures for each task independently. In our work,we want to incorporate multi-task learning and semi-supervised learning into (1). We propose to leveragesemi-supervised learning by adopting the Laplacianproposed in [32]. We adopt this Laplacian because itexploits both manifold structure and local discrimi-nant information of multimedia data, thus resultingin better performance.
To begin with, let us define Xl = [x1l , · · · , x
nl
l ] as thetraining data matrix of the l-th task where ml data arelabeled and nl is the total number of the training dataof the l-th task. xi
l ∈ Rd is the i-th datum of the l-th
task. Yl = [y1l , · · · , yml
l , yml+1
l , · · · , ynl
l ]T ∈ 0, 1nl×cl
is the label matrix and cl denotes class number of thel-th task. yil |
nl
i=1 ∈ Rcl is the label vector with cl classes.
Yli,j = 1 if xil is in the j-th class of the l-th task while
Yli,j = 0 otherwise. For unlabeled datum xil , y
il is set
to a zero vector. For any d, we define 1d ∈ Rd as a
column vector with all the elements equal to 1, Hd =I − 1
d1d1Td ∈ R
d×d as a matrix for centering the databy subtracting the mean of the data. Note that Hd =HT
d = HdHd. For each data point xil of the l-th task,
we construct a local clique Nlk containing xil and its
k− 1 nearest neighbors. Euclidean distance is used todetermine whether two given data points are within knearest neighbors in the original feature space. Gi
l =i0l , i
1l , · · · , i
k−1
l is index set of samples in Nlk. Sli
denotes selection matrix with its elements (Sli)pq = 1if p = Gi
lq and (Sli)pq = 0 otherwise.Inspired by [32], we construct the Laplacian matrix
by exploiting both manifold structure and local dis-criminant information. Denoting Lli = Hk(X
Tl Xl +
λI)−1Hk, we compute the Laplacian matrix L as fol-lows:
Ll =
nl∑
i=1
SliLliSTli
= [Sl1, · · · , Slnl]
Ll1
· · ·Llnl
[Sl1, · · · , Slnl
]T .
(5)Note that Manifold Regularization is able to explore
the manifold structure possessed by multimedia data[31] [35] [36]. By applying Manifold Regularization tothe loss function in (1), we have
argminW,b
t∑
l=1
Tr(WTXlLlXTl W ) + α(‖Wl‖2,1
+β‖XTlLWl + 1nl
bTl − YlL‖2F )) + γ‖W‖∗,
(6)
where Tr(·) denotes trace operator, XlL and YlL are
labeled training data and corresponding ground truthlabels of the l-th task.
To make all labels of training data contribute to theoptimization of Wl, we introduce a predicted labelmatrix Fl = [fl1, · · · , fln
l] ∈ R
nl×cl for the trainingdata of the l-th task. fli ∈ R
cl is the predicted labelvector of xli. According to [20] [5], Fl can be obtainedas follows:
argminFl
Tr(FTl LlFl) + Tr((Fl − Yl)
TUl(Fl − Yl)), (7)
where Ul is the selection diagonal matrix of the l-thtask. The diagonal element Ulii = ∞ if xli is labeledand Ulii = 1 otherwise. In the experiments, 106 is usedto approximate ∞.
Following the work in [5], we incorporate (7) into(6). At the same time, all the training data and corre-sponding labels are taken into consideration. There-fore, the objective function finally arrives at:
minFl,Wl,bl
t∑
l=1
(Tr[(Fl − Yl)TUl(Fl − Yl)] + Tr(FT
l LlFl)
+α(‖Wl‖2,1 + β‖XTl Wl + 1nl
bTl − Fl‖2F )) + γ‖W‖∗
(8)From (8) we can see that the proposed algorithm
is capable of evaluating the informativeness of allfeatures jointly for each task with the l2,1-norm andthe information from different tasks can be transferredfrom one to another with the trace norm.
3.2 Optimization
The proposed function involves the l2,1-norm andtrace norm, which are difficult to solve in a closedform. We propose to solve this problem in the follow-ing steps.
By setting the derivative of (8) w.r.t bl to 0, we get
bl =1
nl(Fl −XT
l Wl)T1nl
(9)
Substituting bl in (8) with (9), we obtain
minFl,Wl,bl
t∑
l=1
(Tr[(Fl − Yl)TUl(Fl − Yl)] + Tr(F T
l LlFl)+
α(‖Wl‖2,1 + β‖XTl Wl +
1
nl
1nl1Tnl(Fl −X
Tl Wl)− Fl‖
2F ))
+ γ‖W ‖∗
⇒ minFl,Wl
t∑
l=1
(Tr[(Fl − Yl)TUl(Fl − Yl)] + Tr(F T
l LlFl)
+ α(‖Wl‖2,1 + β‖HnlX
Tl Wl −Hnl
Fl‖2F )) + γ‖W ‖∗
(10)
where Hnl= Inl
− 1
nl
1nl1Tnl
is a centering matrix. Bysetting the derivative of (10) w.r.t Fl to 0, we have
2UlFl − 2UlYl + 2LlFl + αβ(2HnlFl − 2Hnl
XTl Wl) = 0
5
Algorithm 1: Optimization Algorithm for SFMC
Data: Training data Xl|tl=1∈ R
d×nl
Training data labels Yl|tl=1 ∈ Rn×c
Parameters γ, α and βResult:
Feature Selection Matrix Wl|tl=1 ∈ Rd×cl
1 l = 1 ;2 while l ≤ t do3 Initialise Wl|tl=1
∈ Rd×cl ;
4 Compute the Laplacian matrix Ll|tl=1 ;5 Compute the Selection matrix Ul|
tl=1 ;
6 Hnl= Inl
− 1
nl
1nl1Tnl
;
7 Pl = (αβHnl+ Ul + Ll)
−1 ;8 Rl = XlHnl
(Inl− αβPl)Hnl
XTl ;
9 Tl = XlHnlPlUlYl ;
10 end11 Set r = 0 ;12 Set W0 = [W1, · · · ,Wt] ;13 repeat14 l = 1 ;15 Compute the diagonal matrix as:
Dr = (1/2)(WrWTr )−1/2 ;
16 while l ≤ t do17 Compute the diagonal matrix Dr
l
according to Eq. (16) ;18 Update W r
l by
W rl = (Rl +
αβD
rl +
γαβ D
r)−1Tl ;
19 Update F rl by F r
l =(αβHnl
+ Ul + Ll)−1(αβHnl
XTl Wl + UlYl) ;
20 Update brl by brl =1
nl
(Fl −XTl Wl)
T1nl
;
21 l = l + 1 ;22 end23 Wr+1 = [W1, · · · ,Wt] ;24 r = r + 1 ;25 until Convergence;26 Return the optimal Wl|tl=1 and bl|tl=1.
Therefore, we have
Fl = (αβHnl+ Ul + Ll)
−1(αβHnlXT
l Wl + UlYl) (11)
Denoting Pl = (αβHnl+ Ul + Ll)
−1 and Ql =αβHnl
XTl Wl + UlYl, we have
Fl = PlQl (12)
By substituting Fl into (10) with (12), we can rewritethe objective function as follows:
minQl,Wl
t∑
l=1
(Tr[(PlQl − Yl)TUl(PlQl − Yl)]
+ Tr(QTl P
Tl LlPlQl) + α(‖Wl‖2,1
+ β‖HnlXT
l Wl −HnlPlQl‖
2F )) + γ‖W‖∗
(13)
As Tr(QTl P
Tl UlYl) = Tr(Y T
l UTl PlQl) and
Tr(αβWTl XlHlPlQl) = Tr(αβQT
l PTl HlX
Tl Wl),
the objective function can be rewritten as follows:
minWl
t∑
l=1
(αβTr(WTl XlHnl
(Inl− αβPl)Hnl
XTl Wl)
−2αβTr(WTl XlHnl
PlUlYl) + α‖Wl‖2,1) + γ‖W‖∗(14)
Denoting Rl = XlHnl(Inl
− αβPl)HnlXT
l , Tl =XlHnl
PlUlYl and Wl = [w1l , · · · , w
dl ], the objection
function becomes:
minWl
t∑
l=1
(αβTr(WTl RlWl)− 2αβTr(WT
l Tl)
+ αTr(WTl DlWl)) + γ‖WT DW‖∗,
(15)
where D = (1/2)(WWT )−1/2 and Dl is a diagonalmatrix which is defined as:
Dl =
1
2‖w1
l‖2
. . .1
2‖wd
l‖2
. (16)
By setting the derivative w.r.t Wl to 0, we have
Wl = (Rl +α
βDl +
γ
αβD)−1Tl (17)
As shown in Algorithm 1, an iterative algorithm isproposed to optimize the objective function (8) basedon the above mathematical deduction.
3.3 Convergence Analysis
In this section, we prove that Algorithm 1 convergesby the following theorem.
Theorem 1. The objective function value shown in (8)monotonically decreases in each iteration until convergenceby applying Algorithm 1.
Proof: Suppose after the r-th iteration, we haveobtained F r
l , brl and W rl . According the definition
of Dl and D, the convergence of Algorithm 1 corre-sponds to the following inequality:
t∑
l=1
Tr[(F r+1
l − Yl)TUl(F
r+1
l − Yl)] + Tr((F r+1
l )TLlFr+1
l )
+ α(d∑
j=1
‖(wr+1
l )j‖22‖(wr
l )j‖2
+ β‖XTl W
r+1
l + 1nlbr+1
l
T− F
r+1
l ‖2F )
+ Tr((W r+1)Tγ
2(W r(W r)T )−
1
2Wr+1)
≤
t∑
l=1
tr[(F rl − Yl)
TUl(F
rl − Yl)] + Tr((F r
l )TLlF
rl )
+ α(d∑
j=1
‖(wrl )
j‖22‖(wr
l )j‖2
+ β‖XTl W
rl + 1nl
brlT − F
rl ‖
2F )
+ Tr((W r)Tγ
2(W r(W r)T )−
1
2Wr)
(18)
6
Following the works in [10] [6] [5], we have:
t∑
l=1
(Tr[(F r+1
l − Yl)TUl(F
r+1
l − Yl)] + Tr((F r+1
l )TLlFr+1
l )
+ α(d∑
j=1
‖(wr+1
l )j‖+ β‖XTl W
r+1
l + 1nl(br+1
l )T − Fr+1
l ‖2F ))
+γ
2Tr(W r+1(W r+1)T (WW
T )−1
2 )
≤
t∑
l=1
(Tr[(F rl − Yl)
TUl(F
rl − Yl)] + Tr((F r
l )TLlF
rl )
+ α(d∑
j=1
‖(wrl )
j‖+ β‖XTl W
rl + 1nl
(brl )T − F
rl ‖
2F ))
+γ
2Tr(W r(W r)T (W r(W r)T )−
1
2 ).
(19)
We can rewrite (19) as follows:
t∑
l=1
(Tr[(F r+1
l − Yl)TUl(F
r+1
l − Yl)] + Tr((F r+1
l )TLlFr+1
l )
+ α(d∑
j=1
‖(wr+1
l )j‖+ β‖XTl W
r+1
l + 1nl(br+1
l )T − Fr+1
l ‖2F ))
+γ
2Tr((W r+1(W r+1)T )−
1
2 ) +γ
2Tr(W r+1(W r+1)T (WW
T )−1
2 )
−γ
2Tr((W r+1(W r+1)T )−
1
2 )
≤t∑
l=1
(Tr[(F rl − Yl)
TUl(F
rl − Yl)] + Tr((F r
l )TLlF
rl )
+ α(d∑
j=1
‖(wrl )
j‖+ β‖XTl W
rl + 1nl
(brl )T − F
rl ‖
2F ))
+γ
2Tr((W r(W r)T )−
1
2 ) +γ
2Tr(W r(W r)T (W r(W r)T )−
1
2 )
−γ
2Tr((W r(W r)T )−
1
2 ).
(20)
According to Lemma 1 in [6], we have:
γ
2Tr(W r+1(W r+1)T (W r(W r)T )−
1
2 )− γTr((W r+1(W r+1)T
≥γ
2Tr(W r(W r)T (W r(W r)T )−
1
2 )− γTr(W r(W r)T )
(21)
By deducting (21) from (20), we arrive at:
t∑
l=1
(Tr[(F r+1
l − Yl)TUl(F
r+1
l − Yl)] + Tr((F r+1
l )TLlFr+1
l )
+ α(‖W r+1
l ‖2,1 + β‖XTl W
r+1
l + 1nl(br+1
l )T − Fr+1
l ‖2F ))
+ γ‖W r+1‖∗
≤
t∑
l=1
(Tr[(F rl − Yl)
TUl(F
rl − Yl)] + Tr((F r
l )TLlF
rl )
+ α(‖W rl ‖2,1 + β‖XT
l Wrl + 1nl
(brl )T − F
rl ‖
2F ))
+ γ‖W r‖∗)).(22)
Eq. (22) indicates that the objective function valuedecreases after each iteration. Thus, we have provedTheorem 1.
Having Theorem 1, we can easily see that thealgorithm converges.
4 EXPERIMENTS
In this section, experiments are conducted to evaluatethe performance of our algorithm on video classifi-cation, image annotation, human motion recognitionand 3D motion data analysis, respectively. Additionalexperiments are conducted to study the performancew.r.t. influence of number of selected features andparameter sensitivity.
4.1 Experiment Setup
We use four different datasets in the experiment,including one video datasets CCV [37], one imagedatasets NUSWIDE [38], one human motion datasetHMDB [39] and one 3D motion skeleton dataset Hu-manEva [40]. In order to demonstrate advantages ofour algorithm, we compare its performance with thefollowing approaches.
1) All Features: We directly use the original fea-tures without feature selection as a baseline.
2) Fisher Score: This is a classical feature selec-tion method, which evaluates importances offeatures and selects the most discriminating fea-tures one by one [9].
3) Feature Selection via Joint l2,1-Norms Mini-mization (FSNM): Joint l2,1-norm minimizationis utilized on both loss function and regulariza-tion for joint feature selection [10].
4) SPEC: It uses spectral graph theory to conductfeature selection [19].
5) Feature Selection with Shared Informationamong multiple tasks (FSSI): It simultaneouslylearns multiple feature selection functions ofdifferent tasks in a joint framework [6]. Hence, itis capable to utilize shared knowledge betweenmultiple tasks to facilitate decision making.
6) Locality Sensitive Semi-supervised Feature Se-lection (LSDF): This is a semi-supervised featureselection based on two graph constructions, i.e.within-class graph and between-class graph [41].
7) Structural Feature Selection with Sparsity(SFSS): It combines strengths of joint featureselection and semi-supervised learning into asingle framework [5]. Labeled and unlabeledtraining data are both utilized for feature selec-tion. Meanwhile, correlations between differentfeatures are taken into consideration.
In the experiments, a training set for each dataset israndomly generated consisting of n samples, amongwhich m% samples are labeled. The detailed settingsare shown in Table 1. The remaining data are used astesting data. We independently repeat the experiment5 times and report the average results.
We have to tune two types of parameters inthe experiments. One is the parameter k that spec-ifies k nearest neighbors used to compute graphLaplacian. Following [5], we fix it at 15. The otherparameter is the regularization parameters, α, βand γ, which are shown in the objective function(8). These parameters are tuned in the range of10−6, 10−4, 10−2, 100, 10+2, 10+4, 10+6 and the bestresults are reported. Linear SVM is used as classifier.Mean average precision (MAP) is used to evaluate theperformance.
4.2 Video Classification
First, we compare the performances of different al-gorithms in terms of video classification task usingColumbia Consumer Video dataset (CCV) [37]. Itconsists of 9, 317 web videos over 20 semantic cat-egories, in which 4, 659 videos are used as trainingdata and 4, 658 videos are used as testing data. Thesemantic categories include events like ”basketball”and ”parade”, scenes like ”beach” and ”playground”,and objects like ”cat” and ”dog”, based on which wegenerate three different classification tasks. Since theoriginal videos of this dataset have not been availableon the internet, we directly use the STIP featureswith 5, 000 dimensional BoWs representation pro-vided by [37]. We set the number of selected featuresas 2500, 3000, · · · , 4500, 5000 for all the algorithms,and report the best results.
We show the video classification results when dif-ferent percentages of labeled training data are usedin Table 2. From the experimental results, we can getthe following observations: 1) The performances of allthe compared algorithms increase when we increasethe number of labeled training data. 2) The proposedalgorithm consistently gains the best performance. 3)With 5% labeled training data, our algorithm signifi-cantly outperforms other algorithms. For example, forsubject 2, our algorithm is better than the second bestalgorithm by 6.6%. Yet the proposed algorithm gainssmaller advantage with more labeled training data.
4.3 Image Annotation
We use NUS-WIDE dataset [38] to test the perfor-mance of our algorithm. This dataset includes 269648images of 81 concepts. A 500 dimension Bag-of-Wordsfeature based on SIFT descriptor is used in this exper-iment. We take each concept as a separate annotation
task, thus resulting in 81 tasks. It is difficult to reportall the results of these 81 tasks, so the average resultis reported. In this experiment, we set the number ofselected features as 250, 275, · · · , 475, 500 and reportthe best results.
We illustrate the experimental results in Table 3.From the experimental results, we can observe thatthe proposed method gains better performance thanthe other compared algorithms. We give the detailedresults with 1%, 5% and 10% labeled training data.It can be seen that the proposed algorithm is morecompetitive with less labeled training data.
4.4 Human Motion Recognition
We use HMDB video dataset [39] to compare the algo-rithms in terms of human motion recognition. HMDBdataset consists of 6,766 videos which are associatedwith 51 distinct action categories. These categoriescan be categorized into five groups: 1) General facialactions, 2) Facial actions with object manipulation, 3)General body movements, 4) Body movements withobject interaction, 5) Body movements for humaninteraction. Therefore, in this experiment, the fivegroups are considered as five different tasks. Heng etal. claim that motion boundary histograms (MBH) isan efficient way to suppress camera motion in [42] andthus it is used to process the videos. A 2000 dimensionBag-of-Words feature is generated to represent theoriginal data. We set the number of selected featuresas 1000, 1200, · · · , 1800, 2000 for all the algorithmsand report the best results.
Table 4 shows the experiment results of humanmotion recognition. From Table 4, we observe thatour method outperforms other compared algorithms.This experiment can further provide evidence thatour algorithm is more advantageous with insufficientnumber of labeled training data.
4.5 3D Motion Data Analysis
We evaluate the performance of our algorithm interms of 3D motion data analysis using Human-Eva 3D motion database. There are five differ-ent types of actions in this database, includingboxing, gesturing, walking, throw-catch and jogging.Following the work in [43] [44], we randomly select10, 000 samples of two subjects (5, 000 per subject). Weencode each action as a collection of 16 joint coordi-nates in 3D space and obtain a 48-dimensional featurevector. Joint Relative Features between different jointsare computed on top of that, resulting a feature vectorwith 120 dimensions. We combine the two kinds offeature vectors and get a 168-dimensional feature. Inthis experiment, we consider the two subjects as twodifferent tasks. The number of selected features aretuned from 100, 110, · · · , 160.
The experiment results are shown in Table 5. Table 5gives detailed results when 1%, 5% and 10% training
8
TABLE 2PERFORMANCE COMPARISON OF Video Classification (MAP ± STD) w.r.t. 1%, 5% AND 10% LABELED
data are labeled. From the experiment results, we canobserve that our algorithm consistently outperformthe other compared algorithms and obtains moreperformance gain when small number of training dataare labeled.
4.6 Comparison with Other Semi-Supervised Fea-ture Selection Methods
In this section, experiments are conducted on CCV tocompare the proposed algorithm with two state-of-the-art semi-supervised feature selection algorithms.Following the above experiments, 1%, 5%, 10%, 25%,50% and 100% training data are labeled in this experi-ment. We show the experiment results in Figure 2. Wecan observe that our method consistently outperforms
both LSDF and SFSS. Visible advantages are gainedwhen only few training data are labeled, such as1% or 5% labeled training data. From this result, wecan conclude that it is beneficial to leverage sharedinformation from other related tasks when insufficientnumber of training data are labeled.
4.7 Parameter Sensitivity
We study the influences of the four parameters α,β, γ and the number of selected features using CCVdatabase with 1% labeled training data. First, we fixγ and the number of selected features at 1 and 3500respectively, which are the median values of the tunedrange of the parameters. The experimental results areshown in Figure 3. It can be seen that the performance
9
TABLE 4PERFORMANCE COMPARISON OF Human Motion Recognition (MAP ± STD) w.r.t. 2%, 5% AND 10%
of our algorithm varies when the parameters (α and β)change. More specifically, MAP is higher when α andβ are comparable. Then, α and β are fixed. Figure 4shows the parameter sensitivity results. Note that theshared information among multiple feature selectionfunctions W1, · · · ,Wt by the parameter γ. From thisfigure, we can see that mining correlations betweenmultiple related tasks is beneficial to improve the per-formance. We can also notice that better performancesare gained when the number of features is around
3500 and 4000.
5 CONCLUSION
In this paper, we have proposed a new semi-supervised feature analysis method. This method isable to mine correlations between different featuresand leverage shared information between multiplerelated tasks. Since the proposed objective functionis non-smooth and difficult to solve, we propose an
10
TABLE 5PERFORMANCE COMPARISON OF 3D MOTION DATA ANALYSIS (MAP ± STD) w.r.t. 1%, 5% AND 10%
Fig. 2. PERFORMANCE COMPARISON OF 3D MOTION DATA ANALYSIS w.r.t. PERCENTAGE OF LABELEDTRAINING DATA. WE CAN OBSERVE THAT THE PROPOSED ALGORITHM YIELDS TOP PERFORMANCESON BOTH THE TASKS. (a) SUBJECT 1. (b) SUBJECT 22.
10^−610^−4
10^−21
10^210^4
10^6
10^−610^−4
10^−21
10^210^4
10^6
0
0.02
0.04
0.06
0.08
αβ
MA
P
(a)
10^−610^−4
10^−21
10^210^4
10^6
10^−610^−4
10^−21
10^210^4
10^6
0
0.1
0.2
0.3
0.4
αβ
MA
P
(b)
10^−610^−4
10^−21
10^210^4
10^6
10^−610^−4
10^−2110^2
10^410^6
0
0.1
0.2
0.3
0.4
0.5
αβ
MA
P
(c)
Fig. 3. MAP with different α and β while keeping γ and feature numbers fixed on CCV database. (a) SUBJECT1. (b) SUBJECT 2. (c) SUBJECT 3
iterative and effective algorithm. To evaluate per-formances of the proposed method, we apply it todifferent applications, including video classification,image annotation, human motion recognition and 3Dmotion data analysis. The experimental results indi-cate that the proposed method outperforms the othercompared algorithms for different applications.
REFERENCES
[1] L. Yu and H. Liu, “Feature selection for high-dimensional data:a fast correlation-based filter solution,” in Proc. ICML, 2003, pp.856–863.
[2] L. Talavera, “An evaluation of filter and wrapper methods forfeature selection in categorical clustering,” in Proc. IDA, 2005,pp. 440–451.
[3] S. H. Yang and B.-G. Hu, “Discriminative feature selection bynonparametric bayes error minimization,” IEEE Trans. Knowl.Data Engin., vol. 24, no. 8, pp. 1422–1434, 2012.
11
25003000
35004000
45005000
10^−610^−4
10^−2110^2
10^410^6
0
0.02
0.04
0.06
0.08
# of featuresγ
MA
P
(a)
25003000
35004000
45005000
10^−610^−4
10^−2110^2
10^410^6
0
0.1
0.2
0.3
0.4
# of featuresγ
MA
P
(b)
25003000
35004000
45005000
10^−610^−4
10^−2110^2
10^410^6
0
0.1
0.2
0.3
0.4
0.5
# of featuresγ
MA
P
(c)
Fig. 4. MAP with different γ and the number of features while keeping α and β fixed on CCV database. (a)SUBJECT 1. (b) SUBJECT 2. (c) SUBJECT 3
[4] Z. Ma, F. Nie, Y. Yang, J. R. Uijlings, and N. Sebe, “Webimage annotation via subspace-sparsity collaborated featureselection,” IEEE Trans. Multimedia, vol. 14, no. 4, pp. 1021–1030, 2012.
[5] M. Zhigang, F. Nie, Y. Yang, J. Uijlings, N. Sebe, and A. G.Hauptmann, “Discriminating joint feature analysis for multi-media data understanding,” IEEE Trans. Multimedia, vol. 14,no. 6, pp. 1662 – 1672, 2012.
[6] Y. Yang, Z. Ma, A. Hauptmann, and N. Sebe, “Feature selec-tion for multimedia analysis by sharing information amongmultiple tasks,” IEEE Trans. Multimedia, vol. 15, no. 3, pp. 661– 669, 2013.
[7] H. Liu and L. Yu, “Toward integrating feature selection al-gorithms for classification and clustering,” IEEE Trans. Knowl.Data Engin., vol. 17, no. 4, pp. 491–502, 2005.
[8] J. Yang and C. J. Ong, “An effective feature selection methodvia mutual information estimation,” IEEE Transactions on Sys-tems, Man, and Cybernetics, Part B, vol. 42, no. 6, pp. 1550–1559,2012.
[9] R. O. Duda, P. E. Hart, and D. G. Stork., Pattern Classification.John Wiley & Sons, 2012.
[10] N. Feiping, H. Huang, X. Cai, and C. H. Ding, “Efficient androbust feature selection via joint 2, 1-norms minimization,” inProc. NIPS, 2010, pp. 1813–1821.
[11] R. Caruana, “Multitask learning,” Machine Learning, vol. 28,no. 1, pp. 41–75, 1997.
[12] A. Argyriou, T. Evgeniou, and M. Pontil, “Convex multi-taskfeature learning,” Machine Learning, vol. 73, no. 3, pp. 243–272,2008.
[13] A. Argyriou and T. Evgeniou, “Multi-task feature learning,”in Proc. NIPS, 2007.
[14] S. Wang, X. Chang, X. Li, Q. Z. Shen, and W. Chen, “Multi-task support vector machines for feature selection with sharedknowledge discovery,” Signal Processing, 2015.
[15] Z. Zhao, L. Wang, and H. Liu, “Efficient spectral featureselection with minimum redundancy,” in Proc. AAAI, 2010.
[16] S. H. Yang and B.-G. Hu, “Discriminative feature selection bynonparametric bayes error minimization,” IEEE Trans. Knowl.Data Engin., vol. 24, no. 8, pp. 1422–1434, 2012.
[17] S. Xiang, F. Nie, G. Meng, C. Pan, and C. Zhang, “Discrim-inative least squares regression for multiclass classificationand feature selection,” IEEE Trans. Neural Netw. Learning Syst.,vol. 23, no. 11, pp. 1738–1754, 2012.
[18] Y. Yang, H. Shen, Z. Ma, Z. Huang, and X. Zhou, “L21-normregularization discriminative feature selection for unsuper-vised learning,” in proc. IJCAI, 2011.
[19] Z. Zheng and H. Liu, “Spectral feature selection for supervisedand unsupervised learning,” in Proc. ICML, 2007, pp. 1151–1157.
[20] X. Zhu, “Semi-supervised learning literature survey,” Com-puter Science, University of Wisconsin-Madison, Tech. Rep.,2006.
[21] R. G. F. Soares, H. Chen, and X. Yao, “Semisupervised classi-fication with cluster regularization,” IEEE Trans. Neural Netw.Learning Syst., vol. 23, no. 11, pp. 1779–1792, 2012.
[22] F. Wang, “Semisupervised metric learning by maximizingconstraint margin,” IEEE Transactions on Systems, Man, andCybernetics, Part B, vol. 41, no. 4, pp. 931–939, 2011.
[23] I. Cohen, F. G. Cozman, N. Sebe, M. C. Cirelo, and T. S. Huang,“Semisupervised learning of classifiers: Theory, algorithms,
and their application to human-computer interaction,” IEEETrans. Pattern Anal. Mach. Intell., vol. 26, no. 12, pp. 1553–1567,2004.
[24] M. Guillaumin, J. Verbeek, and C. Schmid, “Multimodal semi-supervised learning for image classification,” in Proc. CVPR,2010, pp. 902–909.
[25] X. Chang, F. Nie, Y. Yang, and H. Huang, “A convex formu-lation for semi-supervised multi-label feature selection,” inProceedings of the Twenty-Eighth AAAI Conference on ArtificialIntelligence, July 27 -31, 2014, Quebec City, Quebec, Canada., 2014,pp. 1171–1177.
[26] Y. Yang, D. Xu, F. Nie, J. Luo, and Y. Zhuang, “Rankingwith local regression and global alignment for cross mediaretrieval,” in Proc. ACM Multimedia, 2009, pp. 175–184.
[27] Y. Liu, F. Nie, J. Wu, and L. Chen, “Efcient semi-supervisedfeature selection with noise insensitive trace ratio criterion,”Neurocomputing, 2012.
[28] X. Chang, H. Shen, S. Wang, J. Liu, and X. Li, “Semi-supervisedfeature analysis for multimedia annotation by mining labelcorrelation,” in Advances in Knowledge Discovery and Data Min-ing - 18th Pacific-Asia Conference, PAKDD 2014, Tainan, Taiwan,May 13-16, 2014. Proceedings, Part II, 2014, pp. 74–85.
[29] T. Jebara, J. Wang, and S.-F. Chang, “Graph construction andb-matching for semi-supervised learning,” in Proc. ICML, 2009,pp. 441–448.
[30] Y. Yang, Z. Ma, F. Nie, X. Chang, and A. G. Hauptmann,“Multi-class active learning by uncertainty sampling with di-versity maximization,” International Journal of Computer Vision,pp. 1–15, 2014.
[31] F. Nie, D. Xu, I.-H. Tsang, and C. Zhang, “Flexible manifoldembedding: A framework for semi-supervised and unsuper-vised dimension reduction,” IEEE Trans. Image Process., vol. 19,no. 7, pp. 1921–1932, 2010.
[32] Y. Yang, D. Xu, F. Nie, S. Yan, and Y. Zhuang, “Image clusteringusing local discriminant models and global integration,” IEEETrans. Image Process., vol. 19, no. 10, pp. 2761–2773, 2010.
[33] J. Chen, J. Zhou, and J. Ye, “Integrating low-rank and group-sparse structures for robust multi-task learning,” in Proc. ACMSIGKDD, 2011, pp. 42–50.
[34] G. Obozinski, B. Taskar, and M. I. Jordan, “Joint covariate se-lection and joint subspace selection for multiple classificationproblems,” Statistics and Computing, vol. 20, no. 2, pp. 231–252,2010.
[35] Y. Yang, Y.-T. Zhuang, F. Wu, and Y.-H. Pan, “Harmonizinghierarchical manifolds for multimedia document semanticsunderstanding and cross-media retrieval,” IEEE Trans. Mul-timedia, vol. 10, no. 3, pp. 437–446, 2008.
[36] Y.-Y. Lin, T.-L. Liu, and H.-T. Chen, “Semantic manifold learn-ing for image retrieval,” in Proc. ACM Multimedia, 2005.
[37] Y.-G. Jiang, G. Ye, S.-F. Chang, D. Ellis, and A. C. Loui,“Consumer video understanding: A benchmark database andan evaluation of human and machine performance,” in Proc.ICMR, 2011, p. 29.
[38] T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng,“Nus-wide: A real-world web image database from nationaluniversity of singapore,” in Proc. of ACM Conf. on Image andVideo Retrieval (CIVR’09), Santorini, Greece., July 8-10, 2009.
[39] H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre,“Hmdb: a large video database for human motion recogni-tion,” in Proc. ICCV, 2011, pp. 2556–2563.
12
[40] S. Leonid and M. J. Black, “Humaneva: Synchronized videoand motion capture dataset for evaluation of articulated hu-man motion,” Brown Univertsity, Tech. Rep. CS-06-08, 2006.
[41] J. Zhao, K. Lu, and X. He, “Locality sensitive semi-supervisedfeature selection,” Neurocomputing, vol. 71, no. 10, pp. 1842–1849, 2008.
[42] H. Wang and C. Schmid, “Action recognition with improvedtrajectories,” in Proc. ICCV, 2013.
[43] N. Huazhong, W. Xu, Y. Gong, and T. Huang, “Discriminativelearning of visual words for 3d human pose estimation,” inProc. CVPR, 2008, pp. 1–8.
[44] Y. Yang, D. Xu, F. Nie, S. Yan, and Y. Zhuang, “Image clusteringusing local discriminant models and global integration,” IEEETrans. Image Process., vol. 19, no. 10, pp. 2761–2773, 2010.
![Page 1: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/1.jpg)
![Page 2: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/2.jpg)
![Page 3: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/3.jpg)
![Page 4: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/4.jpg)
![Page 5: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/5.jpg)
![Page 6: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/6.jpg)
![Page 7: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/7.jpg)
![Page 8: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/8.jpg)
![Page 9: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/9.jpg)
![Page 10: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/10.jpg)
![Page 11: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/11.jpg)
![Page 12: Semi-supervised Feature Analysis by Mining Correlations ... · analysing tasks [1], [2], [3]. Consequently, feature se- ... tion functions because of utilization of class labels.](https://reader033.documents.pub/reader033/viewer/2022050116/5f4cf6fba7130c672449f01d/html5/thumbnails/12.jpg)
