Research On Echo State Network And Its Applications To Time Series Prediction

Recently,time series analysis and prediction based on echo state network (ESN) has become a vivid research area and more and more attention is being paid by scholars. As a new paradigm in artificial recurrent neural network (RNN), ESN overcomes the complicated training process and local optimum problem of traditional RNN. Nowadays, it is one of the most important tools in time series analysis field. However, we cannot obtain the satisfying prediction accuracy with classical ESN due to its limited approximation capabilities to nonlinear dynamical systems. Accordingly, focusing on prediction accuracy and adaption of reservoir, we attempt to investigate how to extend the classical ESN and how to improve adaption of reservoir in time series prediction. The main work includes the following five aspects:1. To achieve higher prediction accuracy, a multi-step-ahead time series prediction framework is proposed to make the most of the ESN's capabilities to handle simultaneously multiple outputs. In the proposed framework, we formulate a practical implementation method which combines the iterated and direct prediction results by weighted means. Experiment results show that a higher accuracy can be achieved than iterated and direct prediction respectively.2. We propose a time series prediction method called Global Echo State Network with Wavelet Decomposition (GESN-WD). This method mainly focuses on the problem that typical ESN cannot discover multi-scale features of the time series. GESN-WD chooses a specific reservoir matched to its characteristics for every series obtained by wavelet decomposition and determines the output weight in sense of global optimum which can avoid the accumulation of prediction error. Experiment results show that GESN-WD can get more accurate results compared with Local Echo State Network with Wavelet Decomposition (LESN-WD).3. A prediction method based on autocorrelation coefficient and ESN is proposed to determine the embedding dimension and delay time in time series prediction. The autocorrelation coefficient is used to determine the dimension of the input vector. The impact of reservoir parameters on prediction accuracy is also analyzed in terms of simulation experiments. Experiment results show that this proposed method can achieve higher accuracy for some time series prediction task.4. Focusing on time series with strongly nonlinear properties and rich dynamics, a novel framework, called fuzzy echo state network (FESN), is proposed. The main idea of FESN is to replace the linear equations of consequent part of fuzzy IF-THEN rules in the fuzzy TS model with ESNs. Then, a training algorithm based on least square estimation is devised. As an important property, the FESN is proved to be a generality of TS model and ESN. In other words, the TS model as well as ESN is only a special case of FESN in some circumstance. Finally, we prove that if the absolute maximal spectral radius of the reservoirs in FESN is less than 1, the FESN will have echo state property. The above limit is the sufficient condition to guarantee global asymptotic stability (echo state property) of FESN. Experiment results show that FESN can get higher prediction accuracy without decreasing in efficiency.5 Focusing on the adaption of reservoir in practical time series prediction application, an online training algorithm of ESN is proposed. The algorithm combines some key parameters of reservoir and output weight and determines them simultaneously by Extended Kalman Filter in the training process. Experiment results show that the proposed method can avoid parameter selection by cross validation or optimization algorithm and it also can get better prediction accuracy meanwhile.