| Bayesian techniques have been advocated for analysis of clinical trials, but because of the complex nature of these methods, many traditional researchers remain skeptical of their utility. We performed a retrospective parallel comparison of frequentist and Bayesian group sequential analyses on data from four published clinical trials. We chose three widely accepted methods for our frequentist analyses (Pocock, O'Brien-Fleming and Lan-DeMets) and implemented Bayesian sequential analyses with two different prior distributions, one informative, and the other non-informative. Using bootstrap simulation, we determined an 'average answer' for all situations and compared them directly. We estimated Type I error rates for our Bayesian analyses using repeated bootstrap sampling and simulated the Null situation. Thus for the data-generated distribution represented by these trials, we were able to compare the relative virtues of these techniques and provide a classical evaluation of both the Bayesian and frequentist methods. We found that it was difficult to make direct comparisons between the two types of analyses because they inherently ask and answer different questions. However, we saw that in the case of the t-test, outcomes as measured by significant effect sizes were similar, but for Cox proportional hazards models, the two types of analyses often resulted in different outcomes. Bayesian outcomes were quite dependent upon investigator decisions. Type of analysis should be chosen based on the nature of the particular question of interest rather than solely on the assumption of the merits of one type of analysis over the other. |
