A Monte Carlo study of two nonparametric statistics with comparisons of Type I error rates and power

Scope and method of study. The Mann-Whitney (MW) test and the Kolmogrov-Smirnov two sample test (KS-2) are nonparametric statistical tests used to detect whether there is a general difference between two samples when the two underlying population distributions are distribution-free. The focus of this study was to examine and compare Type I error rates and statistical power between the MW and the KS-2 tests when the two samples had different population variances or various degrees of kurtosis and skewness. This study also compared Type I error rates and power, if applicable, when the two samples were of different sizes. Simulations in SAS program were conducted to simulate various conditions to examine Type I error rates and statistical powers for these two nonparametric statistical tests. There were 15 population distributions, 12 sets of sample size combinations, and 7 different ratios of standard deviation. Exactly 20,000 replications per condition were executed for a total of 1380 conditions.; Findings and conclusions. The study revealed that the KS-2 test is smaller than the MW test in comparisons of the type I error rates in unequal sample sets. The MW test had slightly more statistical power the KS-2 test under the condition of small and equal-sized samples. Moreover, when population variances vary between two samples, the KS-2 test has more statistical power than the MW test. Furthermore, the power of the KS-2 test exceeded the power of the MW test in large sample settings when either one of the following conditions existed: (1) The difference in the Skewness ratoss in populations between the two samples was more than 0.5 with the same kurtosis and variance. (2) The difference in the Kurtosis ratios in populations between the two samples was more than 2.0 with the same skewness and variance. Theoretical and practical implications, limitations of the study, are discussed, as well as recommendations for future research.