Research And Improvement Of Feature Selection Algorithms Based On Sparse Learning

Feature selection plays a key role in many machine learning problems,which is used to speed up the learning process,improve the generalization ability of the model and solve the dimensionality disaster problem.In particular,as a significant data preprocessing method,robust and practical feature selection methods can be used to choose meaningful features and get rid of redundant ones.Feature selection refers to the extraction of the best feature subset from the original feature set.The objectives of feature selection include:establishing simpler and more understandable models and obtaining clean,understandable data.With the development of sparsity research,both theoretical and experimental studies have shown that the sparsity is one of the inherent attributes of real world data and embedded feature selection models have also applied sparsity regularization.Recently,Sparse learning technology are widely used to improve the performance of algorithms.We focus on multi-class feature selection using structured sparsity regularization,which can select the features across all the classes with joint sparsity.The problem of feature selection needs to be considered from two aspects:the discriminant ability of features and the correlation between features.Because of the good discrimination ability of Linear Discriminant Analysis(LDA)in feature selection,several researchers took advantage of LDA structures to construct objective optimization functions.And some effective and robust feature selection methods were proposed by introducing the _ul-norm regularization term.Most feature selection methods rank all features according to certain evaluation criteria,so as to select the highest ranked features.However,the relevant features often have strong similarity,which will lead to a large correlations between the selected high ranking features and redundancy problems between the features.In this paper,we apply self-weighted linear discriminant analysis(SLDA)method and introduce _ul-norm to SLDA problem.To solve the redundancy problem between the selected features,we introduce the redundancy matrix A in the AGRM(a novel auto-weighted feature selection framework via global redundancy minimization)framework to our constraint optimization problem.Besides,we also introduce adaptive redundancy matrix S,and process adaptive redundancy matrix S as an optimization variable instead of setting adaptive redundancy matrix S to a prior.In order to solve the proposed optimization problem,we design an effective algorithm based on the augmented Lagrangian method to solve the above constrained optimization problem,where the global optimal solution can be gained.In multi-classification task,both theory and experiment demonstrated the superiority of our feature selection algorithm.