Bayesian machine learning

Statistical machine learning has seen a lot of progress in the last two decades. Rather than being a purely algorithmic approach to machine learning, the statistical methodology of Bayesian learning is distinguished by its use of probability to express all forms of uncertainty. Learning and other forms of inference can then be performed by simply applying the rules of probability. Results of Bayesian learning are expressed in terms of a probability distribution over all unknown quantities. These probabilities can be interpreted only as expressions of our degree of belief in the various possibilities. In contrast, the conventional frequentist strategy takes the form of an estimator for unknown quantities, with possibly some optimality criterion in mind.; We extended several machine learning methods (neural network, support vector machine, and relevance vector machine) to the Bayesian paradigm. We developed a hierarchical Bayesian neural network for binary and bivariate binary data. The main applications of our developed methodology are in staging and classification of prostate cancer. Several applications using clinical and gene expression microarray covariates along with simulations show empirically that our proposed method is more accurate in predicting stages of prostate cancer compared to other methods (Neal's Bayesian neural network, logistic regression, and the classical neural network). We also proposed a Bayesian support vector machine (SVM) and a relevance vector machine (RVM) to tackle the problems of nonlinear regression and classification in "high dimensional low sample size" data. Our Bayesian SVM and RVM constructed on reproducing kernel Hilbert space theory does not require an additional projection into the sample space, and gives a stronger learning algorithm than the classical one. Its virtue over the standard methods is that it produced a richer class of models that provide better predictions and it has the unique ability to quantify prediction errors. Several applications on near-infrared spectroscopy data and gene expression microarray data indicate that our Bayesian SVM and RVM performs better than most of the available models.