INVESTIGATION OF PARAMETERS AND INTERACTION EFFECTIVENESS OF RIDGE ESTIMATORS: A MONTE CARLO STUD

The question of the appropriate use of ridge regression emerged with the advent of ridge regression. This Monte Carlo study compared ordinary least squares regression and principal components regression accounting for 100 percent of the trace with three forms of ridge regression: Hoerl-Kennard-Baldwin, Lawless-Wang, and McDonald-Galarneau. The criteria for comparison of solutions were: error of the beta vector, variance of beta, mean square error, shrinkage of R('2) upon cross-validation, and the value of the cross-validated R('2).;The orientation of the beta vector with respect to the eigenvector associated with the largest eigenvalue of X'X was calculated for each sample to examine the interaction of this statistic with type of regression solution.;Ridge regression, particularly the Lawless-Wang solution, clearly outperformed ordinary least squares and the principal components solution using 100 percent of the trace on the criteria of stability and error of the beta vector. Thus, whenever interpretability of coefficients is important to an investigator, ridge regression offers a desirable alternative to ordinary least squares. For prediction or hypothesis testing, ordinary least squares is preferable to the ridge solutions considered here.;The principal components solution using all components was equivalent to the ordinary least squares solution in production of R('2),(' )Y, and MSE but different in variance and error of the coefficient vector. The variance of the coefficient vector in the principal components solution was small for the first component, increasing dramatically as components with smaller eigenvalues were added, supporting the use of a cutoff in a principal components solution.;The relative performance of each regression solution was dependent upon the intercorrelation among predictor variables in the sample as well as upon the sample R('2) for any particular value of average absolute intercorrelation among predictor variables. The orientation of the beta vector, appeared to be of little value in selecting a regression method.;For samples with high multicollinearity when interpretability and stability of coefficients is important to an investigator, Lawless-Wang ridge regression proved a superior solution to the ordinary least squares solution, principal components accounting for 100 percent of the trace, Hoerl-Kennard-Baldwin ridge regression and McDonald-Galarneau ridge regression in this study.