Research On Imputation And Forecasting Of Multivariate Time Series Based On Transformer

Multivariate time series can be used as signals to predict the future state of complex systems.For example,in power electronic equipment,life prediction can be made by collecting the data of each indicator at different time of the equipment.Missing values reduce the information that can be mined in the sequence and increase the difficulty of time series modeling.Therefore,it is of great practical importance to construct incomplete multivariate time prediction models that account for missing factors and fill in the missing values of incomplete multivariate time series.Using multivariate time series as the study subject,this work investigates incomplete multivariate time series prediction and missing value filling in real-world applications to provide more accurate prediction and filling results.Aiming at the problem of filling missing data in multivariate time series,this paper proposes a method based on Transformer for processing multivariate time series with high miss rate,called Convolutional Transformer Inference Network(CTIN),First,a missing value initialization method is proposed,which uses missing token,timestamp information,and original sequence information to tentatively infer the missing position value.Second,this paper improves the selfattention mechanism for multivariate time series with high miss rate by adding convolution to the self-attention mechanism to enable the model to capture both local and global dependencies.Finally,this paper also designs a weight adjustment mechanism that can adjust the learned weights.This approach can be combined with past and future information to infer missing position values and obtain a complete multivariate time series,and more accurate prediction results can be obtained through the joint training of imputation and forecasting.Relevant experiments show that the imputation accuracy of the model using the above approach is better than that of the MIAM model based on the Transformer architecture on the Air Quality and PhysioNet dataset.The mean absolute error decreased by 13.5%and 2.8%,and the root mean square error decreased by 6.5%and 3.3%,respectively.The mean relative errors were reduced by 12.8%and 5.6%,respectively.Aiming at the limitations of incomplete time series prediction,this paper proposes a less complex global multidirectional Transformer model(GMT)based on Gated recurrent neural network(GRU)and Transformer.First,global GRU are employed to capture the mixture information of all variables and local dependence.Second,the multidirectional Transformer is used to take into account the relationship between variables and long-term time dependence and reduce the effect of error accumulation in recurrent neural networks on the prediction results.The model using the above approach was used to perform the classification task on the PhysioNet dataset,and achieved better accuracy compared to other state-of-the-art models.Compared to MIAM,the current state-of-the-art Transformer model based on imputation and prediction combined,the classification accuracy is improved by 1.3 percent.