Research And Application Of Two Types Of Improved Recurrent Neural Networks In Time Series Prediction

Recurrent neural networks are widely used in the analysis of various types of time series data,such as machine translation,speech recognition,and time series prediction.Time series are widely used in various fields.Accurate time series prediction is vital to help people make rational decisions and actions.Elman recurrent neural networks and echo state networks are two important types of recurrent neural networks.However,Elman recurrent neural network is prone to long-range dependence problem when the series is too long.Hence,Elman recurrent neural networks are not suitable for too long time series prediction.In addition,Echo state networks only train the output layer weights by a linear model,which avoids the long-range dependence problem,but also brings the problem of poor stability because the weights of the input and hidden layers are generated randomly.In addition,the echo state network trained by a linear model may also lead to poor model prediction for nonlinear data.To address the problems of these two types of recurrent neural networks,this dissertation proposes two new recurrent neural network models for time series prediction.The neural grey system model is a type of neural network model which combines the grey model and multilayer perceptron model and is applicable to smallsample time series prediction.The shortcoming of the neural grey system model is that it does not sufficiently consider the information association before and after the time series.To address the shortcomings of the neural grey system model and the long-range dependence problem of the Elman recurrent neural network,this dissertation proposes an Elman grey memory network to predict small-sample time series by combining the grey system model and the Elman recurrent neural network.The Elman grey memory network takes advantage of the memorability of recurrent neural networks to improve the shortcomings of the neural grey system model and to avoid the long-range dependence problem in small-sample time series prediction.In addition,the backpropagation through time algorithm for the model is presented,and an optimization model is developed to find the optimal parameters of the model and reduce the probability of overfitting.Finally,the prediction performance of the Elman grey memory network and other time series prediction methods are compared and analyzed by 8 sets of numerical experiments.The experimental results show that the proposed model outperforms other prediction methods.To address the problems of echo state networks,this dissertation develops a kernel echo state network model.Firstly,the random weight matrix of the input layer is removed,and a series of neurons constructed by kernel functions are used to map the original data directly into the high-dimensional space,called the kernel function layer.Then,kernel ridge regression is applied to learn the output layer weights instead of other linear models.The parameters associated with the kernel function layer can be estimated using a K-Means clustering algorithm.A benchmark test system is used to compare the proposed kernel echo state network with other echo state models.The results of the comparison showed that the proposed model had better prediction performance and greater prediction stability than the other models.