Research And Applications Of Evolving Recurrent Neural Networks

This paper presents systemic research about three aspects including structure, function and training algorithms of recurrent neural network (RNN), the combination of evolutionary algorithms (EAs) with RNN, and the application of evolving recurrent neural network in soft-sensing of parameters in a chemical engineering process.Three benchmark problems about times series data and correlative data series are firstly used to identify the modeling capabilities of recurrent neural network in comparison with that of feedforward neural network. Training data structures for modeling, capabilities of various RNN structures are investigated respectively. The simulation results demonstrate powerful representational and prediction capabilities of RNN with simplest training data structure for all of the cases.Based on some previous research results, a general structure of Layered Whole Recurrent Neural Network are presented. Then some typical gradient-based training algorithms available for RNN are reviewed and analyzed. The detailed description of these algorithms as well as their characteristics and mutual relationship are given. As it shows, the restriction about recurrent connections as well as computationally intensive and time consuming training process for most of these algorithms prevent RNN from being widely used. Evolutionary algorithms, as alternatives, are attractive training options for complicated neural network strucctures because they are not constrained to specific network topology, activation function of neural nodes and performance function.As global stochastic search techniques, EAs are distinguished by their reliance on a population of search space positions, rather than a single position, to locate extrema of a function defined over the search space and therefore, hold promise for many kinds of optimizing problems including evolution of the topology and weight distribution of neural networks. The application of evolutionary algorithms to neural network optimization has produced an active field of research. Reports and papers are continuously increasing. An overview of some main issues about evolving neural network shows that genetic algorithm is inappropriate for network acquisition. There have been some existing evolutionary systems for designing ANN, including RNN. Though most of them address the complete problem of network acquisition, they do so in a constrained manner. Furthermore, some of them have no specific coding schemes and pay more attention to structure mutation and local learning than reasonable...