Multi-label Feature Selection Based On Manifold Learning And Sparse Regression

In text classification,image annotation,bioinformatics and other fields,instances often show the characteristics of multi semantics.The traditional single-label learning framework is no longer applicable.Multi-label learning believes that each instance can have multiple class labels at the same time.Therefore,it can be used to process multisemantic data.Multi-label data are usually high-dimensional.The existence of a large number of redundant features not only increases the computational cost of multi-label learning algorithms,but also influences the performance of classifiers.Feature selection is an effective method for the dimensionality reduction of high-dimensional data.Sparse feature selection adds a sparse regularization term on the regression coefficient matrix of the regression model and achieves feature selection through the coefficient analysis of this matrix,which has better performance than the traditional feature weighting methods.Nevertheless,the linear relationship between data and labels that this kind of methods assume in the regression model does not hold in most cases.Some multi-label feature selection methods introduce the idea of manifold learning to the sparse regression model and map data to the low-dimensional manifold,so that the selected features can reflect the intrinsic structure of data.However,how to construct the low-dimensional manifold and how to constrain the manifold structure to benefit the classification still need further research.In this thesis,multi-label feature selection is studied based on manifold learning and the sparse regression model.The construction and constraint methods of the manifold are discussed.The main research contents are as follows:(1)Recent advances in multi-label feature selection and the theories and methods of multi-label learning are reviewed.The sparse regression model and manifold learning are discussed in detail with the focus on the related multi-label feature selection methods.(2)A method named multi-label label-specific feature selection based on graph Laplacian(LSGL)is proposed.In LSGL,the adjacency matrix is constructed based on the distribution information of data on each label.The low-dimensional manifold on each label is obtained through Laplacian eigenmaps.Then,the low-dimensional manifold is introduced into the regression model with the sparse regularization term.Finally,the regression coefficient matrix is obtained through optimization.This matrix can be used to evaluate the importance of features and get the corresponding labelspecific features of each label.Experimental results show that the label-specific feature subset gained by LSGL can improve the performance of the classifier.(3)A method named multi-label feature selection via manifold regularization and dependence maximization(MRDM)is proposed.MRDM maps data to the lowdimensional manifold space using the spectral regression.The method adds structure constraint and dependence constraint on the manifold space so that it not only has the similar structure information with the data space,but also has a strong dependence with the label space.Two constraints are added into the sparse regression model.The objective function is solved through iterative optimization to obtain the regression coefficient matrix and the feature subset.Experimental results show that MRDM achieves better performance than multiple multi-label feature selection methods.