Markov Chain Monte Carlo Bayesian predictive framework for artificial Neural Network Committee modeling and simulation

A logical inference method of properly weighting the outputs of an Artificial Neural Network Committee for predictive purposes using Markov Chain Monte Carlo simulation and Bayesian probability is proposed and demonstrated on machine learning data for non-linear regression, binary classification, and 1-of-k classification. Both deterministic and stochastic models are constructed to model the properties of the data. Prediction strategies are compared based on formal Bayesian predictive distribution modeling of the network committee output data and a stochastic estimation method based on the subtraction of determinism from the given data to achieve a stochastic residual using cross validation. Performance for Bayesian predictive distributions is evaluated using Bayesian methods, while performance for the residual based method is evaluated using conventional statistical techniques.