| In this research, several modifications to the Fuzzy ARTMAP neural network architecture are proposed for conducting classification in complex, possibly noisy, environments. The goal of these modifications is to improve upon the generalization performance of ART-based neural networks, such as Fuzzy ARTMAP, in classification problems involving overlapping pattern classes or mislabeled examples. One of the major difficulties of employing Fuzzy ARTMAP on such learning problems involves overfitting the training data. Structural risk minimization is a machine learning framework that addresses the overfitting issue. The theory of structural risk minimization reveals a tradeoff between training error and classifier (or hypothesis) complexity in reducing generalization error. Structural risk minimization provides a backbone for analysis as well as an impetus for design of better learning algorithms proposed herein. The first modification proposed herein, called Boosted ART, generalizes upon the Fuzzy ART neural network. It will be shown that both Fuzzy ART, as well as Boosted ART, are universal approximators, which is an important fact in establishing the utility of ART-based neural network architectures. A learning algorithm which is known to be a universal approximator can be applied to a large class of interesting problems with the confidence that a solution is at least theoretically available. A separate modification to Fuzzy ARTMAP, called Boosted ARTMAP, is proposed allowing non-zero training error in an effort to reduce the hypothesis complexity and hence improve overall generalization performance. Boosted ARTMAP is an on-line learner, it does not require excessive parameter tuning to operate, and it reduces precisely to Fuzzy ARTMAP when zero classification error tolerance is requested. Additional modifications, detailed in this dissertation, make use of either Boosted ART or Boosted ARTMAP (or combinations of both) in order to create new neural network architectures. The modifications, proposed in this dissertation, include Structural Boosted ARTMAP, which uses both Boosted ART and Boosted ARTMAP, to perform structural risk minimization learning in an ARTMAP-based neural network architecture. Both empirical and theoretical results will be presented in this dissertation leading to the understanding of why, where and how the proposed architectures are most useful. |
