Subspace Learning And Non-convex Sparse Regression Based Feature Selection

The progress of information technology has led to the emergence of massive high dimensional data in various fields.High-dimensional data often contains a large amount of redundant features and noise,which seriously affects the efficiency of the learning tasks and reduces the accuracy of the algorithm models.Therefore,how to use the dimensionality reduction methods to reduce the dimension of data effectively has become a research hotspot.Feature selection,as an effective dimensionality reduction method,has received extensive study and attention for it retains the semantic information of the original feature space.At present,feature selection methods have achieved good effectiveness.However,there are still some issues.For instance,the manifold structure information is not preserved sufficiently.Additionally,the manifold structure preservation strategies are not robust to noise and the sparse regularization terms are not very effective.Considering these issues,some feature selection algorithms are proposed in this thesis to overcome the above problems effectively.The research contents of this thesis are listed as follows:1)A novel algorithm called sparse and low-redundant subspace learning-based dual-graph regularized robust feature selection(SLSDR)is proposed.First,based on the framework of subspace learning-based graph regularized feature selection,SLSDR extends it by introducing the data graph.Since SLSDR constructs graphs in both data space and feature space,the geometric structures of the data and feature manifolds can be preserved simultaneously to guide the process of feature selection.Then,the inner product regularization term is used in SLSDR to guarantee the sparsity of the rows of feature selection matrix to select the representative and low-redundant features.Additionally,the l_2,1-norm is used to constrain the residual matrix of subspace learning to ensure the robustness to outlier samples.Experimental results on twelve benchmark datasets show that the proposed SLSDR has great performance.2)A novel algorithm called subspace learning for unsupervised feature selection via adaptive structure learning and rank approximation(SLASR)is proposed.First,adaptive manifold learning strategy is introduced into the framework of subspace learning.Therefore,not only the local geometric structure information,but also the global reconstruction information is well preserved.Benefiting from the application of adaptive manifold learning strategy,the learning of the similarity matrix and the low-dimensional embedding is completed in one step.So the similarity matrix becomes more accurate and is more robust to noise.Then,the rank regularization constraint is imposed on the laplacian matrix,making manifold structure more accurate.Additionally,the l_2,1/2-norm is used to constrain the projection matrix to select the most sparse and robust features.Experimental results fully show that SLASR is more efficient than the traditional algorithms based on fixed graphs.3)A novel algorithm called adaptive dual graphs and non-convex constraint based embedded feature selection(DNEFS)is proposed.First,by using the framework of sparse regression,DNEFS preserves the manifold structure information in both data space and feature space.Meanwhile,by introducing the principle of information entropy,the local manifold information in dual graph can be learned and updated adaptively.So the manifold information can be more accurate.Then,a novel non-convex regularization term is used in DNEFS to ensure the sparsity of rows.Concretely,this regularization term consists of the difference between l_2,1-norm and Frobenius norm,and is written as l_2,1-2-norm.By using this novel regularization term,DNEFS can handle redundant features more efficiently.Experimental results on six benchmark datasets show that the performance of the proposed DNEFS outperforms that of the comparison algorithms.