?-Minimum Absolute Deviation Distribution Regression Recursive Feature Elimination

In the fast-developing Internet era,advances in network technology have accelerated the digital transformation in various fields,and data acquisition methods are becoming more and more convenient.The dramatic increase in the amount of data and feature dimensions has made information more and more abundant,and the amount of calculations has also increased.Therefore,how to select features is particularly critical.Feature selection has attracted wide attention as an important step in machine learning algorithm processing,because it can effectively reduce the feature dimension,avoid "overfitting" and "dimensional disaster",and effectively improve the calculation speed and generalization performance of machine learning algorithms.This paper studies feature selection methods based on regression problems.First,this paper proposes a new support vector regression algorithm,that is,?-minimum absolute deviation distribution regression(?-MADR).We use the recent support vector regression theory to define two statistics,absolute regression deviation mean and absolute regression deviation variance,and introduce them into the ?-SVR to obtain the original optimization problem of ?-MADR.For optimization,we propose a dual coordinate descent(DCD)algorithm for small sample problems,and we also propose an averaged stochastic gradient descent(ASGD)algorithm for large-scale problems.The experimental results on both artificial and real datasets indicate that our v?-MADR has significant improvement on generalization performance with shorter training time compared to the popular ?-SVR,LS-SVR,?-TSVR,and linear ?-TSVR.Then,we implemented a new feature selection algorithm based on the ?-MADR algorithm,that is,?-minimum absolute deviation distribution regression recursive feature elimination(?-MADR-RFE).This method uses the fitting result of the ?-MADR algorithm as a criterion for evaluating the importance of features,and gradually eliminates features that contribute little to the regression model.The experimental results on real datasets show that our feature selection method has better performance,while being less sensitive to parameters compared with other feature selection methods such as PCA,stepwise,LASSO,?-SVR-RFE.