Efficient perfect and MCMC sampling methods for Bayesian spatial and components of variance models

Bayesian hierarchical models give rise to complicated posterior distributions. Monte Carlo and Markov chain Monte Carlo (MCMC) methods are used to estimate expectations with respect to these complicated distributions. When there is enough structure in the problem, it is often possible to design algorithms that are much more efficient than "off-the-shelf" MCMC algorithms. This thesis investigates and develops efficient Monte Carlo and MCMC algorithms for inference for some important Bayesian hierarchical models.A major focus of the work presented here is on studying computation for Bayesian models used for modeling areal (spatially aggregated) data via spatial Poisson models (Besag, York and Mollie 1991). These models use Gaussian Markov random fields to model spatial correlation in disease mapping applications. The usual Gibbs sampling and univariate-update Markov chain Monte Carlo methods exhibit very poor convergence and mixing properties for these models. The work here describes various systematic MCMC block sampling techniques to solve this problem.While block sampling methods are often effective, the theoretical work needed to rigorously assess the accuracy of MCMC based estimates is prohibitively difficult for most realistic problems, and hence ad hoc "convergence diagnostics" techniques are typically used in practice. If the samples drawn are i.i.d. rather than dependent, these issues are easily resolved however, i.i.d. or exact sampling methods are generally not considered to be practical for complicated, multivariate continuous distributions. This thesis proposes a systematic method for producing heavy tailed proposal distributions that can be used in exact sampling schemes for Bayesian disease mapping models. These proposal distributions allow the implementation of the rejection sampling and perfect tempering (Moller and Nicholls, 1999) algorithms to draw i.i.d. samples from the posterior distributions of interest. Exact simulation algorithms are also studied for some Bayesian variance components models, and their application to the widely used Bayesian one-way ANOVA model is successfully demonstrated.The thesis concludes with a discussion of ideas for further automating the exact simulation methods developed here, along with possible extensions of exact sampling to more complicated models.