Research On Neural Network Ensemble Algorithm

Neural network ensemble is a quite active and hot research topic in machine learning and neural computing fields. It is very importantly significant on both theory and practice to design a more effective method of building neural network ensemble model for improving the generalization ability of neural network ensemble model. It is not only helpful for scientists to investigate machine learning and neural computing but also helpful for common engineers to solve real-word problems using neural network techniques.In this paper the neural network ensembles are used for classifying problems and we do some researches on how to produce individual neural network in ensemble, and design a good neural network ensemble algorithm to try and obtain better generalization performance.The main work and innovation in this paper are as follows: We introduce feature selection techniques when separately training individual networks. A neural network ensemble based on cross validation and ReliefF algorithm named CVRNNEn is proposed. First of all, we need to obtain different training sets to train the networks, so the different training sets are obtained by some kind of cross-validation, and then for each training sets, use feature selection to eliminate redundant features in order to reduce the size of the data sets and the interference of redundant features. In this way, we improve the generalization ability of individual networks and increase the ambiguity of the ensemble, and then make the ensemble model to obtain better generalization performance. This algorithm is realized in weka 3.5.6 platform, by applying the CVRNNEn on UCI data sets in weka 3.5.6. Experiments also show that CVRNNEn is able to generate a set of trained networks that is more accurate and diverse than several existing ensemble approaches and to improve the quality of its ensemble.Since the theory of"Many Could Be Better Than All"proposed by Zhi-Hua Zhou behaves remarkably well, recently it has become a very hot topic in neural network ensembles, and many algorithms are proposed by other researchers. In this paper, we propose a CLIQUE based Selective Neural Network Ensemble Algorithm named CLIQUE_SEN. The first to use Bagging algorithm to train all individual networks, then applies CLIQUE clustering to entire set of networks in order to identify the groups of similar network and then eliminates redundant networks inside each cluster. The novel proposed methods is realized in Matlab platform, and applied to several UCI data sets, Experiments show that by selecting an optimal subset of neural networks, it is able to obtain significantly smaller ensemble of neural network while achieving the same or even slightly better generalization performance as when using the entire ensemble, and with a shorter running time than other similar algorithms.