| When a linear regression problem contains many predictor variables, a regression equation based on a few variables is more accurate. However, if the regression coefficients are very close to each other, it is not simple to choose a few dimensions to represent the data information. To solve this problem, we propose a novel model fitting method and implement an efficient computational algorithm. Our method incorporates both shrinkage and average procedures to obtain model estimates. This model selection method is developed using the approach of hypothesis testing with orthogonal covariates. The technique of little bootstrap (Breiman 1992) and its variants have been used to make the procedures data-adaptive and fully automated.;We compare the shrinkage alone and our proposed methods from both numerical and theoretical perspectives. Extensive simulation results are presented to show that the shrinkage-average method always has smaller Model Errors than shrinkage alone. Several real data examples, including a Brain MRI, a Thorax X-ray, and a large-scale fractional factorial design problem, are used to illustrate the proposed procedure. We derive the asymptotic approximation of the Mean Square Error (MSE) of the proposed method and it is shown that the MSE of our proposed method is smaller than that of shrinkage alone. Shibata type of asymptotic optimality of the shrinkage method using the little bootstrap type of criteria functions is established. This result includes asymptotic optimality for stepwise selection with orthogonal covariates. |
