False Discovery Rate Control For Multiple Test Of Pairwise Comparison

In 1995, Benjamini&Hochberg proposed a new error rate FDR(false discovery rate) and a new procedure for multiple hypothesis testing: B-H procedure. The original proof for B-H method assumed the independence between the test statistics. However,the statistics in the pairwise setting is dependent. In 1999, Williams, Jones and Turkey proposed a conjecture, they believed the B-H method can control the FDR in the pairwise setting. In fact, FDR control of the B-H procedure in pairwise comparisons was extensively studied in simulations. There is, however, no theoretical proof for the validity of the B-H procedure for testing pairwise comparisons. A fact in real application is that the true p-values of the hypothesis tests are typically unknown. It is common to use the standard normal distribution, Student’s t distribution or the Bootstrap menthod to estimate the p-values.In this paper, I prove that, under certain conditions, the B-H method controls the false discovery rate(FDR) asymptotically with p-values estimated from N(0, 1), Student’s t distribution or the Bootstrap menthod. And the conclusion is still holds in the unbalanced cases in which the variances and sample sizes in different groups may be not equal. Finally, I conduct simulations to verify the conclusions. Furthermore, I apply the B-H method to pairwise comparison of the entrance scores of different districts in Shanghai, 2012 and get some results, such as, the average level of entrance examination in Xujia Hui is signi?cantly higher than in Fengxian.