Research On Neural Network Ensemble Classification Methods And Their Applications In Parallel Computing Environment

Ensemble learning is a new machine learning paradigm. It integrates the output of weak learning machines to construct a strong learning machine with more effect and quality overall output. The concepts of ensemble learning greatly affect research direction of the traditional machine learning. Ensemble learning improves the effect of machine learning from 3 aspects: statistics,calculation and representation, rather than directly from the improvement of artificial intelligence or learning algorithm. Neural network ensemble(NNE)is refers to a type of ensemble learning algorithm using neural networks as base learning machines. Neural network has been very widely used for all kinds of classification algorithms, so the neural network ensemble classifier is the most commonly used computing framework of classifier ensemble and becomes a long-term active research focus of machine learning and neural computing.Ensemble learning algorithms have two general problems. The first one is generality problem. There was no "universal method" can solve all the problems, we must make a concrete analysis of concrete problems. The second is performance problems. Ensemble learning algorithms require computation multiples of single-machine mode. It may cause performance problems in a real-time system, or in the application of mass data processing systems.According the two above issues, the research goals of this paper include the following two aspects. At first we propose more suitable integration algorithms for low-dimensional data and high-dimensional data classification problems separately. Secondly, we propose optimized ensemble algorithms for parallel calculation algorithm and apply parallel ensemble learning algorithms on parallel computation framework to solve practical problems. The details are as follows:1. In general, neural network ensemble based on fuzzy integral algorithms is using the fuzzy density only related with accuracy, which is incomplete on measure the weight of member neural networks. We propose and design a more effective and comprehensive fuzzy density, which can judge the importance of member neural networks from accuracy, error distance and failure level. Through the study of two spirals classification problems, we prove the validity of neural network ensembles algorithm in typical low-dimensional data classification problem.2. The classification of high-dimensional data requires pre-processing of dimension reduction.This paper designs multi-feature extraction method to gain mulitiple training set to improve the diversity of member neural network. On the other hand,classical Adaboost algorithm for multi-class classification problems is tend to overlook members which are good at classification for some certain class but have low overall accuracy. Proposed multidimension adaboost(MD-Adaboost) method assigns different independent weight to outputs for different classes in one member, thereby improving the overall performance of the neural network ensemble classifier. The algorithm is used in facial expression recognition(FER) and obtained good results.3. Ensemble algorithms will generate enormous computation in training and classification in massive data.To address this issue, we propose genetic algorithm and Mapreduce based parallel selective neural network ensemble(MSNNE-GA). The algorithm reduce the scale of ensemble classifier through "selection",and effectively shorten the running time of the algorithm by using the Mapreduce framework. Experiments using the algorithm give better result for two spiral problem, and compared with a single classifier, classification speed has been greatly improved.4. The presented MSNNE-GA algorithm is a kind of global optimization algorithms, but its parallelism is not high in using genetic algorithms to select members, and there is loss of time for frequent data exchanges. In addition to global optimization algorithm, clustering is another methods commonly used in selective neural network ensemble.This paper propose the selective neural network ensemble method with K-means and PSO(SNNE-KP), by combining clustering and local optimization algorithm, not only improve the quality of neural network ensemble in diversity and accuracy, but also improve the overall degree of parallelism of the algorithm. In experiments SNNE-KP algorithm shows a much better accuracy on UCI dataset. In addition,Hadoop based parallel computing can effectively improve the construction and classification speed of neural network ensemble.5. Hybrid neural network computing platform(HNetCP) based distributed virtual humanoid robot(VHR) is extended by using Hadoop based neural network ensemble as the robot’s learning modules. The face recognition algorithm is implemented on the module,which accomplish the interaction and control of face recognition and 3D simulation system. Comparison is also done for centralized VHR and distributed VHR...