Bayesian methods for combining and constructing classifiers

This research addresses two issues in the field of statistical classification: improving classification performance by combining outputs from multiple classifiers; and understanding the conditional independency relationships among variables by identifying cliques in a Bayesian network classifier.; Combining classifiers is often found to have better classification performance than the single classifiers on which the combination is based. In this research, we make use of theory and methodology from various fields to provide theoretical and methodological insight as to why and how combinations work. We first interpret and clarify the relationships among various classification performance metrics. Understanding these relationships allows us to justify why combinations outperform single classifiers. We then analyze conditions under which a single classifier contributes to a combination. We apply these principles in a study of how combination affects some business performance measures in multiobjective credit decisions. In addition, we develop a sequential Bayesian updating procedure to construct improved combinations. This procedure captures dependencies between single classifiers and provides an aid in the selection of classifiers for combinations.; A Bayesian network is a powerful tool for representing and manipulating conditional independencies between random variables. In the research, we consider a clique structure defined for augmented naive Bayesian network classifiers. A clique is a group of attributes, and a pair of cliques is conditionally independent of each other given the class variable. The clique structure can help us to gain understanding about the variables and the problems at hand. It can also reduce the complexity of learning a Bayesian network. We take an empirical approach to investigate the existence of the clique structure in various classification problems. Simple clique structures are found in most of the datasets that we consider. We then develop two schemes, differing in design concepts, to identify conditionally independent cliques and construct a Bayesian network classifier. One of the two proposed schemes, the TAN-based scheme, shows promises in rendering reasonable clique structures and achieving competitive classification performance.