Embedded Feature Selection For Multi-modal Tasks

Nowadays,with the development of feature acquisition and extraction,multi-modal data extensive exist in many practical applications.Efficiently analysis these data is one of the focus areas in the theory of machine learning.These tasks are usually confronted with the issue of curse of dimensionality.As most of the existing feature selection methods can only handle single-modality data,in this work we consider the feature selection problem for multi-modal tasks.In many practical applications,large amounts of data are unlabeled,and the label information is much expensive to obtain.In this situation,how to select the discriminative features in unsupervised scenarios is the key issue.So we develop a new algorithm,called Cluster Structure Preserving Unsupervised Feature Selection(CSP-UFS).During the feature selection process,the discriminant analysis is introduced to preserve the cluster structure of the original data,which is obtained by spectral clustering.Since different modalities may be complement and reinforce each other,we simultaneously utilize the correlation information between multiple modalities.We then select features via a structural sparsity regularization model.We design a new optimization algorithm to solve the objective function.Experimental results on five public data sets demonstrate the effectiveness of the proposed algorithm.Although we cannot obtain plenty of labeled samples,a small set of labeled samples are available.In this context,the application of a semi-supervised approach is very suitable,and it is interesting to explore semi-supervised feature selection algorithm for multi-modal tasks.We develop a new algorithm,called Semi-supervised Feature Selection via Structured Sparsity(SFS-SS).For each modality,we utilize manifold regularization to explore the geometry of the marginal distribution of the original data.Then we use the joint structured sparsity regularizations to learn the feature importance for the classification tasks from booth group-wise and individual point of modalities.We design a new optimization algorithm to solve the objective function.Experimental results on five public data sets demonstrate the effectiveness of our algorithm.The feature selection algorithm for multi-modal tasks is the focus of this paper.We consider sufficiently the characteristics of multi-modal data.Unsupervised feature selection method and semi-supervised feature selection method are proposed in this paper,and experimental results show that the proposed algorithm is indeed a good solution.