Research On The Improvement And Application Of Echo State Network

Echo state network, a novel recurrent neural network, uses a dynamical reservoir with random sparse connected neurons to replace the hidden layer, which can express the input by the high-dimensional and nonlinear way. Echo state network only train the output weight, to overcome the existing problem of the traditional recurrent neural network, which include complex training algorithm, slow convergence speed, easily fall into the local minimum, and so on. In order to further improve the performance of echo state network and adaptability, this paper mainly research the improvement of reservoir state update equation of echo state network.For the reservoir state update equation, this paper proposes two improved echo state network. First, on the basis of leaky integrator echo state network(Leaky-ESN), we propose double activation function echo state network(DAF-ESN) model. DAF-ESN utilizes double activation functions to replace the original single activation function in the reservoir state update equation. The model can flexibly modify the reservoir state according to different input signals. Second, we propose variable memory length echo state network(VML-ESN), which can adaptively adjust state update pattern according to the autocorrelation characteristic of input signals. Third, we utilize extend kalman filtering method to optimize parameters of reservoir. Finally, we use two improved echo state network to predict time series, especially the photovoltaic power generation forecasting. Experimental results show that DAF-ESN and VML-ESN can adaptively adjust the reservoir state update equation according to the different types of input signal, to obtain the better prediction performance and adaptability.