Feature Selection In High-Dimensional Statistical Learning Problem

We focus on feature selection problem for high-dimensional machine learning problems.Many methods,such as sparsity regularization and stepwise methods are proposed to deal with the challenge occurred due to high dimensionality.In this work,we generalize the greedy inverse scale space(GISS)method,which is originally proposed for compressed sensing,for solving sparse machine learning model.GISS method is known to be efficient for solving sparsity promoting regularization,which is suitable for large-scale signals.We discuss the application of GISS method for linear regression and logistics regression.A detailed evaluation on the performance of different methods is presented.Numeric example,both on simulated examples and real dataset demonstrate the efficiency and accuracy of GISS method for sparse learning.Finally,as it is expected,GISS method achieves the comparable accuracy while outperforms the other methods in terms of computational efficiency.