Research On The Training Methods Of Feedforward Neural Network Based On Nonlinear Filter Optimization

After decades of development, the technique of artificial neural network(ANN) and its related theory are increasingly perfect. With the progress of research, it has been researched and applied in a variety of areas such as fault diagnosis, optimal control, target tracking, financial forecasting and so on. The key to the effect of neural network is training algorithm. The existing training algorithm can obtain ideal effect when problem scales are small and training data are not polluted by noise. But under the background of current application, the problem scales are often large and training data are polluted by noise, the traditional algorithm can not meet the requirements not only in convergence speed but also in precision. Nonlinear filtering algorithm as the optimal estimation of nonlinear system state, has been widely used in areas such as parameter estimation, system identification, etc. It has strong ability to adapt to noise and in global optimization. Through the joint efforts of many experts and scholars domestic and abroad, nonlinear filtering technology has been successfully applied in neural network training and the effect is fairly good.Based on the analysis of neural network and nonlinear filtering theory, two kinds of feedforward neural network training methods based on latest nonlinear filter algorithm are proposed in this article.In existing MLP neural network training method, there has big side effect introduced by first-order rounding error by using extended Kalman filter(EKF) and uneasy to select parameters to solve the positive definite problem by using unscented Kalman filter(UKF). In order to solve these problems, a novel MLP training method based on cubature Kalman filter is proposed in this article. Considering the nonlinear filtering algorithm is implemented under state space model, first establish the MLP state space model by transferring MLP connection weights and bias nodes to state vector and the output of the network as measurement. Then the training procedure of the network is completed after getting the optimum estimate of connection weights vector by using system measurement information combined with CKF algorithm.Finally, a typical nonlinear system equation and Mackey – Glass time series prediction were introduced to validate the proposed method. The results show that compared with existing methods, the new training method not only improves the training precision but also has better efficiency.Considering the Gauss- Newton iteration strategy can make full use of existing information toimprove the estimation precision of nonlinear model parameters, so with the purpose of improving the utilization rate of the known information and further improve the precision of neural network training, a new RBF neural network training method based on iterated cubature Kalman filter is proposed in this article. First the difference of hidden layer activation function between RBF network and MLP is discussed in detail. Second the RBF network state space model is established by conducting the center node of radial basis function and output layer connection weights to state vector. Then introduces ICKF into RBF network training procedure. Finally, a SISO and a MIMO models are introduced to validate the proposed method.The results show that the proposed method can not only improve the training precision of the SISO model,but also effective to MIMO model.