| Because of the existence of feedback connections, recurrent neural networks have good dynamic performance, and are widely used in time series prediction, nonlinear dynamic modeling, and control of nonlinear system. In this paper, two types of recurrent neural networks, Elman neural network and echo state network, have been studied.As a typical locally recurrent network, establishing a simple and optimal structure of Elman network remains a problematic issue. In order to overcome this problem, Pruning Levenberg-Marquardt (PLM) training method, in which sensitivity pruning is added into its training process to obtain the optimal structure, is proposed. In the PLM method, the training process starts with an oversized structure, then the redundant hidden and context neurons are pruned and the configuration parameters are adjusted by mainly using the L-M algorithm. In one pruning operation, the hidden neuron and corresponding context neuron with the least sensitivity are removed. The pruning interval, which is adaptive based on training error, is used to evaluate the success of the last pruning step and the finish of training. After training, the obtained model is named as PLM-ENN.For the new echo state network, to improve its adaptability and generalization ability, an optimization method based on mutual information knowledge is proposed to optimize the input scaling and the structure. The optimization method can be mainly divided into two parts: firstly, the scaling parameters of multiple inputs are adjusted on the basis of mutual information between the network inputs and outputs; secondly, based on the mutual information between reservoir states and network outputs, the output weights are pruned for optimization. The new network after optimization is named as DMI-ESN.Penicillin fed-batch fermentation process, as a typical fermentation process, is strongly nonlinear, time-varying and non-deterministic. The proposed PLM-ENN model and DMI-ESN model are both applied for modeling the process. The testing results show the effectiveness of the two proposed models in predicting the key biological variables. Moreover, the satisfactory results indicate that the PLM-ENN model with sensitivity pruning has an appropriate structure size and performs better than the models without pruning. The comparison results also show that the obtained DMI-ESN model has better adaptability than ESN model without optimization and it is superior to other modeling methods. |
