Analysis of experiments to validate computer models with binary output

The purpose of this research is to examine techniques to analyze computer validation experiments where the physical experiments and computer simulation runs both result in binary responses. The main objective of these studies is to compare the output of a computer model with the corresponding outcomes of physical trials to ensure the computer model is operating appropriately. It is assumed the cost of physical trials is high, which restricts the number of physical trials in the experiment. This dissertation examines four possible modeling techniques of varying degrees of complexity and difficulty, with the central goal of estimating the difference of failure probabilities between the computer simulation and the actual physical process across the range of covariates that define the physical test and act as inputs to the computer simulation.;The first proposed method is to fit a Generalized Linear Model (GLM), relating the failure probabilities to some linear function of the covariates. Choosing an appropriate linear form can be difficult, especially given the small sample size of physical trials. To circumvent this, we propose a Bayesian methodology that draws from the computer experiments literature, using a Gaussian Stochastic Process (GSP) as a prior distribution on the unknown failure functions. This method is capable of modeling a much wider variety of functional forms than the GLM, but is much more complex and difficult to implement, and so we also examine two approximations. The first is an Empirical Bayes-like approach that estimates unknown GSP parameters, as opposed to using a hierarchical approach and applying prior distributions to these parameters. The second involves ignoring the binomial nature of the observed data and assuming that the observation error is actually part of the GSP. The relative performance of these methods is examined across a number of differing true failure functions.