Research On Diversity Issue Of Neural Network Ensemble

To improve generalization capability of neural networks is a fundamental issue in the design and implementation of neural networks for solving regression and classification problems, People usually spend lots of energy and time to conduct experiments on a concrete case for selecting the suitable model, algorithm, and parameters of neural network. How to find another way to avoid the configuration of neural network for improving the generalization of neural network is of great significance. Ensemble learning is expected to be a feasible solution, in which the outputs of a group of trained neural networks are combined to increase the generalization capability. The main work of the thesis is as follows:(1) We investigate ensemble learning by from the aspect of the output sensitivity of neural networks. Nowadays, most neural network ensembles are obtained by manipulating training data and architecture etc. Relying on the diversity of the output sensitivity, we developed three diversity measures and four ways for the selection of the most diverse individuals from a given pool of trained neural networks. Experiments show that our method can decrease the size of obtained ensembles while keep generalization performance.(2) By employing probability technique, we investigate relationship between diversity and accuracy in ensemble learning, We present a new diverse measure in the ensemble learning, which is relied on the error levels, and compare it with ten existing diversity measures, Experiments show that the relationship between diversity and accuracy is nonlinear and the current research results in this field are still not mature . The paper also presents some reasons and analyses for explaining the shortage of the current work and some key issues and directions for further exploring.