Contributions to the analysis of experiments using empirical Bayes techniques

This work is composed of two chapters. Both chapters contribute to the field of the analysis of physical experiments by addressing some practical limitations and offering alternatives to the existing methodology. The first chapter primarily addresses the issue of how to estimate the many factorial effects in highly fractionated designs. This is achieved through the application of nearly objective Bayes techniques. These techniques employ a functionally induced prior for the model parameters that have the highly desirable property of incorporating the concepts of effect hierarchy and effect heredity. The second chapter addresses a common "second step" in industrial settings, where often the entire purpose of the experiment is that of finding the optimal factor settings. Optimization experiments require the determination of settings for all of the factors so that a desired response can be achieved. With this as our primary objective, we make the case for an alternative to the standard practice: estimation followed by the use of statistical testing or the application of model selection algorithms, and finally the optimization of some reasonable parsimonious model. Instead, we propose the estimation techniques described in the first chapter in addition to a method of determining significance based on a criteria directly related to the problem at hand.;In the first chapter we focus on the estimation of a large number of effects from an experimental design with only a small number of runs. A full factorial experimental design over even a moderate number of multi-level factors may become infeasible to carry-out since the number of runs increases very rapidly with the number of factors. As a result, highly fractionated designs are employed in practice. However, while now the frequentist analysis may be carried out on this reduced run size, other problems are introduced. For instance, we can only estimate a small subset of the factorial effects. The quantity of effects we can estimate is limited by the degrees of freedom available from this reduced run size. In addition, special techniques must be employed to resolve aliasing.;Bayes techniques have been suggested to address these issues. However, the common hierarchical model Bayesian approach to the design and analysis of experiments is typically encumbered by the daunting task of specifying a prior distribution for the large number of parameters in the linear model. Such a prior should also reflect a belief in the well known experimental design properties of effect hierarchy and effect heredity. Recently it has been proposed that we may specify a functional prior on the underlying transfer function. Through this functional prior, we are able to reduce the task of prior parameter specification to that of only a few hyper-parameters. When carefully selected, this functional prior may also incorporate the properties of effect hierarchy and effect heredity. Previously, this functionally induced prior was developed for two level experiments. Here we have extended these concepts for three and higher level designs. These designs play a very important role in industrial experiments.;The prior specification for multi-level factors requires that an interesting distinction be made between qualitative and quantitative factors. Such a distinction was not necessary in the case of 2-level factors. However, the Gaussian process functional prior assumption that we employ enables us to seamlessly integrate this aspect of multi-level factors in the modeling through the choice of an appropriate class of correlation functions. The application of the methodology is demonstrated with the analysis of two real world examples.;In the second chapter, we focus on what to do next, after estimation, in the case of an optimization experiment. Again, cost constraints may require that an experimental design's run size be kept small. In many such cases, not having enough data may be solely to blame for not being able to conclude an effect's significance via a standard frequentist statistical test. This is particularly troublesome in an optimization experiment, where we wish to determine the optimal settings for all of the factors based on the experimental output. Another problem associated with frequentist hypothesis testing is that the choice of a significance level, alpha, tends to be completely arbitrary and has little connection to the real world problem.;A convenient property of the empirical Bayes estimates obtained in the first chapter is that they already incorporate information about uncertainty through the prior specification and the data. These estimators can be characterized as shrinkage estimates. In this chapter, some special known cases of the empirical Bayes estimator are discussed. For instance, connections are drawn to the so-called James-Stein estimator as well as the Beta Coefficient Method of Taguchi. Discussion of these special cases allow us to fully appreciate the functionally induced prior empirical Bayes estimator that is recommended here for the purpose of analyzing experiments.;After obtaining the empirical Bayes estimates, for an optimization experiment, it may not be desirable to perform additional statistical hypothesis testing or model selection. Instead, we may wish to use these estimates to determine factor settings which balance the goal of optimizing the response with the cost of changing factors from their current settings. Simulation results provide support for the conclusion that the recommended procedure is superior to frequentist estimation and hypothesis testing, with respect to a metric that should be of particular interest in optimization experiments. On average, the proposed techniques dictate factor settings that yield response values closer to our objective. Finally, we complete the analysis of a real world optimization experiment that is first visited in chapter one.