Research On Multi-Glass Feature Selection With Modified Extended Elastic Net

Feature selection is a basic procedure and an important task of pattern recognition,and has been widely applied in various fields,such as image processing,text mining,computational neuroscience,bioinformatics,and so on.In the practical applications of pattern recognition,observations(also called samples)usually contain a lot of features.Unfortunately,for a specific recognition task,these features are not all relevant valid features.There are redundant and irrelevant noisy features among them.The presence of them probably results in incorrect classification.To solve this problem,the feature selection technology arises at the historic moment.Feature selection aims to select relevant valid features from all features of observations,and eliminate the irrelevant and redundant features at the same time.Feature selection,on the other hand,also can avoid the dimension disaster cased by too many features,and reduce the time cost of pattern recognition.The higher feature dimension,the more highlighted the importance of feature selection.In order to select better feature subsets in which features are most relevant and least redundant,researchers proposed various optimization schemes,such as minimum redundancy maximum relevance feature selection(mRMR)method,global redundancy minimization(GRM)method,uncorrelated Lasso(ULASSO)method,and so on.On the basis of previous researches,from a new point of view,a new optimization feature selection method is proposed in this paper,and the corresponding effective algorithm is proposed at the same time.The main researches in this thesis contain the following three parts:(1)Compared with the frequently-used least absolute shrinkage and selection operator(LASSO),the elastic net(EN)model shows superior performance in the case of the number of selected features is equal.Especially,when the number of features is larger than the number of samples,the performance of EN is more satisfactory than that of LASSO.With this in mind,in this paper,the correlations between features and class labels are regarded as constraints and are integrated into the elastic net model in the form of weight.Thus,two-class feature selection via discriminative elastic net(TFS_DEN)method is proposed.By the means of weighting on regression coefficients,the regression coefficients,corresponding to the features that are highly correlated with class labels,are increased in the procedure of optimization.Otherwise,the regression coefficients will be shrunk.As a result,the distinction of regression coefficients is amplified.The selected features are highly correlated with class labels,which is benefit for improving recognition accuracy.In this part,two kinds of correlation measurement are chosen,and corresponding different four forms of weight are given.For TFS_DEN,not only an effective iteration algorithm is proposed,but also the corresponding convergence proof is proposed in this part.Experiments with respect to TFS DEN is performed on several two-class datasets,and the results illustrate that the performance of TFS_DEN is better than some state-of-the-art feature selection methods.The weakness of TFS_DEN is this method can only handle feature selection problem which contains only two classes of samples.(2)TFS_DEN is extended to multi-class case,which makes it applicable to more practical problems.Thus,multi-class feature selection via discriminative extended elastic net(MFS_DEEN)method appeared.Due to the complicated characteristic of multi-class case,the forms of weight in TFS_DEN are not applicable any more.Therefore,new forms of weight are given for MFS_DEEN,which are different from that in TFS_DEN.Simultaneously,new effective iteration algorithm and the corresponding convergence proof are proposed for MFS_DEEN,which is different from that in TFS_DEN.Experiments with respect to MFS_DEEN are performed on several multi-class datasets,and the results proved the effectiveness of MFS_DEEN.(3)A multi-class feature selection via adaptive extended elastic net(MFS_AEEN)method is proposed,it's a horizontal expansion of MFS_DEEN.MFS_AEEN is weighted by another kind of data-dependent weight,and can be solved by directly using the iteration algorithm of MFS_DEEN.The experiment results on several datasets proved that MFS_AEEN has good feature selection performance as well.