Neural Network Ensemble Based Multi-class Classification Problem Research

In the tide of the network interconnection, we are facing a growing number ofmulti-class classification problems, such as Remote sensing image analysis, textcategorization, intrusion detection, face recognition, webpage classification, security situationprediction, virus program detection, and so on. Theoretically，due to the distributed processingand learning generalization ability in dealing with binary classification problems，learningalgorithm has been gradually matured over the past few decades.However, for processing amulti-class classification problem, with the increase of the number of desired classificationcategories, classifier’s design become more difficult.Therefor how to deal with the complexityof multi-class classification problem, has gained much attention in the field of machinelearning.In order to solve the multi-class classification problems, usually decompose them intomultiple binary problems. We commonly use One-against-all and One-against-onemethod to solve it.We apply BP neural network and support vector machine classifier as thebase classifier, then using the One-against-all or one-against-one strategy to decompose theoriginal multi-class problem into multiple binary problems respectively, and combined withensemble learning to solve the multi-class classification problem. The main scientificresearchs of the paper are summarized as follows:The first method is based on BP neural network ensemble learning classificationalgorithm.Traditional One-against-all (OAA) decomposition approach performance moredepends on the individual classifier’s accuracy, rather than its diversity.A standard techniqueused to resolve these problem is to decompose the original multi-class problem into multiplebinary problem.In this paper we propose a new learning model applicable to multi-classdomains.The proposed model is an Artificial Neural Network ensemble in which the baselearners are composed by the union of a binary classifier and a complementary multiclassclassifier. Once the classifiers that make up a base model have been built, the next step is todetermine their interrelation. In order to do so, three options have been analyzed: Parallelcombination, serial combination and hierachical combination.The model has a higheraccuracy than other classic integration algorithm on the UCI database and hand-written digitrecognition of multi-class problems, and has the advantage of less storage space andcomputing time.Based on one-class SVM, we proposed an ensemble learning classification algorithm tohandle class imbalance phenomenon in multi-class problem. Firstly, multi-class classification problem with the One-against-all broken down into multiple binary problems, then using theOne-Class SVM classifier classified as a processing class classification.finally,we use theoutput of all one-class SVM by discriminant fuction to predicte the label of testing set.wepropose the classifier,called multiple one-class-SVM classifier (MOCSVMC).This algorithmcan also be combined with AdaBoost algorithm.The experimental results on the UCI databaseshow that the method in dealing with the problems compared to some other classic integrationalgorithm has the advantage in computing speed as well as high classification accuracy.According to multi-class classification problem, we integrated ensemble learning tostudy two kinds of classification models, and applied on the hand-written digit recognitionand UCI data set.The two algorithms we proposed are superior to traditional classicalensemble learning algorithm AdaBoost and Bagging in computing speed as well asclassification accuracy.