A Comparison Study Of Relative Importance Methodologies In Linear Model

Object This simulation study compared several predictor relative importance methods,including traditional methods like squared correlation coefficients(denoted r~2), squaredstandardized regression coefficients(denoted β~2), Product Measure(β_r), and new methods whichnamed Dominance Analysis and Relative Weight. We observed the impact of sample-size andsample procedure on relative importance estimators. Implement a huge-scale simulation to evaluatethe difference between Dominance Analysis, Relative Weight and other methods, takingcollinearity, numbers of predictors, simple-size as the experimental factors.Methods This study introduced existed importance methods, using PROCSURVEYSELECT procedure to sample a fixed population for1000times, to evaluate the stabilityof relative importance methods. To explicate the difference between Dominance Analysis andseveral importance methods, we conducted the study within a principal component pattern matrix.Three principal components were defined by eight experimental factors, including validity of thepredictors, collinearity and number of predictors. Then the population correlation matrices used inthe relative importance analysis was generated from the pattern matrix which defined byexperimental factors. Eight factors are used to generate lambda weights comprising our principalcomponent pattern matrix. Using the PROC FACTOR procedure and IML module to generatesimulation data, difference between importance methods was obtained by minus estimators andconduct a regression taking the mean τ value as criterion.Result The sum of squared correlation coefficients’ estimator is bigger than model R-square,squared standardized regression coefficients’ sum is smaller. In contrary, sum of the ProductMeasure, Relative Weight and Dominance Analysis are extremely close to model R-square. Whenthe sample size small than1000, the estimator have obviously variation, but the variation decreasedwhen the sample size rise up. The major disadvantage of Product Measure is this method mayproduce negative value (229/2400). The distribution of estimator difference between r~2andDominance Analysis is disperse, performing a left-skewed distribution. The β~2’s estimator havemany abnormal value, which didn’t happened in other methods. The results indicated that the meancriterion validity accounting for2%to13%of the predicted variance in the average Kendall’s taucriterion. The collinearity accounted for4%to25%of the predicted variance in the average Kendall’s tau criterion. The sample-size accounted for20%to77%of the predicted variance in theaverage Kendall’s tau criterion. The number of predictors accounted for14%to60%of thepredicted variance in the average Kendall’s tau criterion. In contrary, the standardize deviation ofcollinearity was an unimportant factor accounting0%to0.07%of the predicted variance across thevarious discrepancy criteria.Conclusion The most important factors effecting relative importance estimators is sample-size and number of the predictors, the collinearity and predictor’s validity is also important, butneither of predictors set have nearly no effect on estimators. The squared standardized regressioncoefficients have a high probability of producing abnormal value, squared correlation coefficientsproducing negative value as well. Researchers introduced Relative Weight and DominanceAnalysis as prior methods in many studies, but we find that these two methods didn’t have sameestimator result, the difference between Relative Weight and Dominance Analysis is tiny butcannot be ignored, in mostly experiment conditions (63958/64000), the Dominance Analysisestimators is bigger than the Relative Weight.