Association Analysis And Prediction Research Based On Multivariate Time Series

Univariate prediction method is a commonly used technology for time series prediction so far. However, existing results have shown that real-word systems are always complex and may contain multiple features. Hence, it is sensible to regard the multivariate time series, which are related with each other, as a whole in order to reveal the internal discipline of systems. Nowadays, some exploratory researches have been presented for multivariate time series analysis both at home and abroad. But, these methods did not give sufficient guiding suggestions on how to utilize multiple series reasonably. Meanwhile, there is a lack of discussion on how the model generation capability and parameter selection process can be influenced by the introduction of multiple series as well. To solve the problems mentioned above, this paper will focus on variable selection and model optimization based on the analysis of multivariate time series, and try to find new methods for multivariate time series prediction. It covers:Focusing on the prediction of multivariate time series, sparsely forward orthogonal model and11-norm extreme learning machine are proposed as the structural optimization methods. Although the introduction of multiple series can provide abundant information for modeling, the increment of input variables will also increase model complexity, which may easily lead to over-fitting. Therefore, this paper proposes a sparsely forward orthogonal model to solve the problem mentioned above. By combing PRESS (Predicted Residual Sums of Squares) statistic and incremental selection method, the basis functions can be selected in an adaptive manner. Meanwhile, Singular Value Decomposition is employed to orthogonalize candidate basis functions and reduce the computational complexity of PRESS statistic. In addition, by introducing11-norm regularization, this paper proposes an improved ELM (Extreme Learning Machine). The usage of the11-norm regularization can not only enhance the generalization capability of prediction model, but also force the output weights of some irrelevant nodes to zero, which will reduce model structure. Moreover, surrogate function is employed and the objective function that contains11-norm regularization can be approximated by a more simple form, based on which model parameters can be estimated by using Bayesian method adaptively.Basing on sparse kernel density estimation, simplified mutual information is proposed as the inputs selection method for multivariate prediction model. Due to the introduction of redundant and irrelevant inputs will influence the performance of prediction model, it is necessary to select a proper input subset according to the correlation between multi-variables. Mutual information is an effective correlation analysis method. However, the operation of kernel density estimator that used in the mutual information is very tedious. Hence, based on 11-norm ELM, this paper proposes a new density estimator in order to relax the computational complexity of mutual information. This method regards density estimator as a regression problem and solve it by using11-norm ELM. Compared with kernel density estimator, the formulation of the proposed method is sparser and computationaly lighter. Moreover, because of the feature space obtained by the mapping, there is no need to construct kernel function. Therefore, one does not need to consider how to select type and parameters of kernel function. Furthermore, according to consistent evaluation function, sensitivity analysis and principal component analysis, the paper also proposes three input selection method, which helps to illustrate the importance of the selection process.By introducing Huber loss function and Laplace distribution, robust echo state networks are proposed as the capability optimization method for prediction model. Focusing on the problem that ESN (Echo State Network) is sensitive to outliers, this paper proposes a Robust ridge regression method by using Huber loss function. This method substitute quadratic loss function by Huber loss function in order to enhance the robustness of the’ network. On this basis, weighted least squares method is employed to transform the objective function to a particular form, which can be properly solved by Bayesian method, and reduce the computational complexity of parameter selection. Moreover, based on Laplace distribution, this paper proposes a robust ESN as well. By introducing Laplace distribution, which is robust to outliers, as the likelihood function of model output, the robustness of the prediction model can be enhanced. Meanwhile, in order to utilize Bayesian method to estimate model parameters, a proper surrogate function is constructed, based on which, the Laplace likelihood function is equivalently transformed to a Gaussian form. In this case, Bayesian method can be employed for parameter estimation and effectively solve the problem than derived by introducing Laplace likelihood function.