Exploration And Generalization Of Several Stepwise Variable Selection Algorithms

Several stepwise variable selection algorithms have been explored and gen-eralized under the framework of linear regression model in this paper. Firstly, we introduce the orthogonal forms of classic forward selection, backward elim-ination algorithms and a refined version of classic stepwise regression method, named adaptive "Fo-Ba" algorithm. And simulations show that the refined adaptive "Fo-Ba" algorithm, which is much more practical than the original one, performs nearly perfectly when the sample size is not too small and the noise is not too strong. Secondly, this paper tells the wonderful story about the idea and process of "LARS" and introduces a new algorithm called "back-ward quasi-LAR elimination algorithm(BQLAR)", which can compete with LARS in most situations and even perform better with some special settings. Thirdly, as it is well known that LARS with proper modification can generate the whole solution path of Lasso, we show a similar outcome that with some modification BQLAR can give the solution path of Lasso inversely. Finally, this paper generalizes BQLAR to the situation of partial correlation and finds that the new method combined with the least square algorithm can receive fine predicting results in our simulations.