Non-linearity Prediction Using Improved Echo State Networks

Nonlinear system prediction using neural networks appears great efficiency and has abundance of applications. Recurrent neural networks shows more advanced advantages among them against these prediction tasks, although its learning methods have not improved much more for long time.Echo state network is one novel structure of recurrent neural network (RNN) also one novel learning method for RNN as well, it's similar with those bio-neural-networks structurally, and it has the perfect STM capability as one RNN. It employs one large scale RNN as information reservoir called dynamical reservoir, then minimizes the meaning squared error (MSE) during training to get the learning using computing simple regression weight matrix from internal states towards output unit. However, there is one contradiction existing in ESN, it is: to employ nonlinear neuron can raise the nonlinear capability of ESN but reduce the STM of it simultaneously. It has to employ one very large scale DR when face those tough task which require not only high nonlinearity but also nice MC like chaotic time series prediction. This causes the running process of ESN slowing down and becoming more instable during exploitation period.According to the transcendental knowledge theory of ANN, the ESN can employ other neural node to improve the performance, the wavelon using in WNN chosen in this thesis. The internal state space is enlarged when input some tuned wavelon. The SWHESN can predict 46% further than the original ESN without typical deviation but only consuming only 30% time of what ESN do when learning same data sample. We can't forget that ESN has improved the best previous technology by factor 700[1].This thesis shows tri-highlight views:1. We introduce wavelon into RNN which appears in forward ANN traditionally.2. We reduced the diversity between wavelon for the reason to smooth working condition in ESN rather than augmenting them which forward ANN who need larger basic vector function embedded.3. The parameters in echo state networks involved in application are set by expert of echo state networks commonly, which usually waste of computation resource, in this paper we present one method that to optimize the parameters in echo state networks by PSO optimizer when applications.