Research On Neuron Models And Topologies Of Echo State Network

In recent years,with the rapid development of artificial intelligence technology,echo state network(ESN)show certain advantages in dealing with nonlinear time series and dynamic prediction system,which has attracted extensive attention from researchers at home and abroad.As one of the improved recurrent neural network(RNN),ESN has a large-scale sparse connected recurrent part(dynamic reservoir)as the hidden layer and only the output weight need to be trained,which greatly simplifies the whole training process,overcoming the complex training process and local optimum shortcoming of conventional RNN.Nowadays,it has gradually become one of the most important tools for nonlinear time series prediction.As the medium of information processing,reservoir plays a decisive role in the performance of ESN.However,traditional ESN uses randomly generated reservoir topology and its computation is limited.It is difficult to meet the prediction accuracy requirement of some highly complicated prediction tasks.In addition,we should try to balance the prediction accuracy and the computation complexity in the process of constructing new reservoir computation models.Accordingly,based on the improvement of prediction accuracy for nonlinear time series,this paper carries out the research work including the reservoir topology and the neural models.On the one hand,based on complex network theory,we construct a scale-free highly clustered network and verify the small-world and scale-free characteristics.Then,the scale-free highly clustered ESN is successfully constructed after introducing the structure into reservoir computation.Finally,the effectiveness and superiority of the proposed algorithm are verified via nonlinear autoregressive moving average and financial series prediction tasks.Compared with classical random network,the experimental results show that scale-free highly clustered ESN has stronger computational capability because of its small-world and scale-free features.On the other hand,based on circle structure and leaky integrator neuron,ESNs with low complexity are constructed.In order to compare the performance with the traditional ESN,five different ESNs models are established,respectively: classical random ESN with sigmoid neurons,circle ESN with sigmoid neurons,random ESN with leaky integrator neurons,circle ESN with leaky integrator neurons,circle ESN with leaky integrator neurons and sigmoid neurons.The Mackey-Glass time series prediction task is used to evaluate the performance of different networks from three aspects: prediction accuracy,anti-noise ability and nonlinear approximation performance.The results show that the performance of circle and random structure is equivalent,leaky integrator units have better memory capability than conventional Sigmoid neurons and that the combination of circle topology and leaky integrator units can remarkably improve the performance of ESN on nonlinear time series prediction.