Topics in multiple comparisons: Weighted Hochberg and gatekeeping procedures

Multiple comparisons problem occurs when one considers a set, or family, of many statistical inferences simultaneously. The probability of making at least one type I error (referred to as the familywise error rate or FWER) is inflated when one considers the inferences together as a family. Failure to compensate for multiple comparisons can have adverse consequences. For example, in assessments of drug efficacy, drug could be approved when actually it is not better than placebo if the drug is compared with the placebo on multiple endpoints or for multiple subgroups of patients.This thesis makes contributions to the area of multiple comparisons in the following aspects. First, we consider different ways of constructing weighted Hochberg-type step-up procedures including closed procedures based on weighted Simes tests and their conservative step-up short cuts, and step-up counterparts of two weighted Holm procedures. It is shown that the step-up counterparts have some serious pitfalls such as lack of familywise error rate control and lack of monotonicity in rejection decisions in terms of p-values. Therefore an exact closed procedure appears to be the best alternative, its only drawback being lack of simple stepwise structure. A conservative step-up shortcut to the closed procedure may be used instead, but with accompanying loss of power. Simulations are used to study the familywise error rate and power properties of the competing procedures for independent and correlated p-values. Although many of the results are negative, they are useful in highlighting the need for caution when procedures with similar pitfalls are used.We also study reverse fixed-sequence multiple testing procedures in the context of superiority and noninferiority tests. An application of this procedure would be when comparing several doses of a drug with a control, superiority of each dose is tested first noninferiority of a dose is tested only if superiority can not be established which is the reverse of the common practice. Two-stage reverse fixed-sequence procedure is constructed based on the Bonferroni procedure. The Dunnett procedure is also proposed if the usual normality and homoscedasticity assumptions are satisfied. We also constructed a Hochberg-Bonferroni reverse fixed-sequence procedure which is more powerful than the Bonferroni-based procedure but it requires that the test statistics be multivariate totally positive of order two. The Hochberg-Bonferroni procedure can be extended to multistage situation with nested hypotheses. Rigorous proofs for the strong control of the familywise error rate are provided for all these procedures.Lastly, we consider a clinical trial with a primary and a secondary endpoint where the secondary endpoint is tested only if the primary endpoint is significant. The trial uses a group sequential procedure with two stages. The FWER of falsely concluding significance on either endpoint is to be controlled at the nominal level alpha. The type I error rate for the primary endpoint is controlled by choosing any alpha-level stopping boundary, e.g., the standard O'Brien-Fleming or the Pocock boundary. Given any particular alpha-level boundary for the primary endpoint, we study the problem of determining the boundary for the secondary endpoint to control the FWER. We study this FWER analytically and numerically and find that it is maximized when the correlation coefficient rho between the two endpoints equals 1. For the four combinations consisting of O'Brien-Fleming and Pocock boundaries for the primary and secondary endpoints, the critical constants required to control the FWER are computed for different values of rho. Numerical studies indicate that the O'Brien-Fleming boundary for the primary endpoint and Pocock boundary for the secondary endpoint generally gives the best primary as well as secondary power performance among the four combinations of these two boundaries. A clinical trial example is provided to illustrate the proposed procedure.