Multivariate Time Series Prediction

The data collected in different scenarios have different characteristics,such as long and short-term memories,periodical features,temporal features and spatial features.Computation models established in existing works are facing challenges such as high modeling cost,which prevent their wide applications to the time series prediction.A desired time series prediction model should bring low computational cost,wide usages,and significant values of research and application.Based on the above considerations,this thesis proposes three time series prediction methods.The main contents are as follows:1.Aiming at the complex conditions constraining to the application problem,a general approach Conv GRU-TSNet is presented.Conv GRU-TSNet includes linear structure and nonlinear structure.Firstly,the input data should be grouped such that the length of the data passing through the RNN can be reduced.For the nonlinear part,Conv GRU extracts longterm memories and periodical features,generating multiple outputs in which the last group output stores long-term and periodical information.The last group output of Conv GRU is fed into GRU to obtain short-term memory.Then the temporal-spatial module is used to address the temporal features and the spatial features,generating the output of the nonlinear structure.For the linear part,the last group of the grouped data is fed into the AR module to solve insensitive problem of neural network and extract linear features.The sum of such two parts is as the final output of Conv GRU-TSNet.Performing on four open datasets,the metrics(RSE and CORR)are 0.010% to 7.604% better than the baseline models,which proves the effectiveness of Conv GRU-TSNet.2.A model can be applied to most of the time series processing scenarios if it can capture both short and long-term features of a time series dataset.Conv GRU-TSNet increases the number of hyper-parameters by using two modules extracting long and short-term features.It makes the model difficult to apply and deploy.In order to reduce the number of hyper-parameters,x LSTM and x GRU with stronger long-term ability are introduced with an improved gate mechanism.In this thesis,the new gate function xigmoid is constructed to amplify the derivative of the sigmoid function.This improvement is conducive to fully train the parameters of forget gate of LSTM and thus boost the upper bound of the longterm memory.The improved model is powerful at capturing long-term features and can be applied to more complex scenarios.Several extreme tests show that the improved long-term ability is one order of magnitude higher than the baseline models.The loss values of the improved model are one to three orders of magnitude lower than the baseline models,and the number of training iterations required to achieve convergence is only one half to one third of the baseline models.3.xLSTM and x GRU have been effectively improved by xigmoid based on LSTM and GRU.Along the amplifying gradient path,further research shows that adjusting the position of hyper-parameter in xigmoid can enhance the long-term memory capacity of x LSTM.The Zigmoid function obtained based on this idea makes LSTM evolve into the z LSTM model with higher upper bound of long-term memory.Moreover,z LSTM can compute the input vector coupled with multiple system key parameters,and has stronger processing ability to predict the coupling parameters jointly affected by multiple system parameters.z LSTM has been applied to the prediction of coolant flow in nuclear power plant,and z LSTM has obtained the predicted value with 28.7% to 29.52% lower error than baseline methods.To sum up,in order to solve the problem of long-term memory requirement faced by multivariate time series,this thesis studies a variety of methods step by step,and gradually proposed multiple models of multivariate time series prediction in complex and longtime scenes.Researches proposed in this work proves the superiority of the proposed methods by multiple experiments and comparison to baseline models.These methods have advantages on engineering applications such as multivariate time series prediction in complex scenarios.A deep learning-based solution is then provided for intelligent control of complex longtime processes in industrial control.