Research On Informer-based Long Sequence Time-series Prediction Model

In reality,there are many applications of long sequence time series prediction tasks,such as electricity usage planning.However,with the rapid development of the Internet of Things in recent years,the collected data present characteristics of large scale and high dimensionality,which bring great challenges to the long-sequence time-series prediction tasks.The current related research work,which cannot effectively capture the nearest neighbor dependency and long distance dependency,leads to low prediction accuracy and high training cost.To solve the above problems,this paper proposes an Informer-based long sequence time series prediction model,i.e.,ML-Former.firstly,this paper designs an Embedding-based time series embedding layer for solving the heterogeneity of time series values,temporal information and location information,and extracting features;then,an Informer-based time series encoder-decoder is constructed for capturing the nearest neighbor dependence and long distance dependence.decoder for capturing nearest-neighbor dependence and long-range dependence and reducing the model training cost;finally,a loss function based on decay mechanism for long-sequence timing prediction is proposed to help the model converge.The main contributions of this paper are as follows.1.In this paper,an Embedding-based time series embedding layer is designed.The time series embedding layer uses position encoding to express the temporal and spatial information of relative moments,uses temporal encoding to express the temporal information of absolute moments,and uses the nearest neighbor convolution module to extract the nearest neighbor dependency.Besides,the time series embedding layer solves the long sequence input problem.2.In this paper,an Informer-based time series encoder-decoder is constructed.The time series encoder-decoder uses convolutional block attention module and self-attention mechanism to capture the nearest neighbor dependency and long distance dependency,and reduces the training cost by convolutional block attention module and downsampling module,besides,it uses generative inference to improve the prediction accuracy.3.In this paper,we propose a loss function for long-sequence timing prediction based on a decay mechanism to help the model converge better.Specifically,based on Hidden Markov Model and Focal loss,the loss value is calculated by segmenting the prediction results and assigning different weights,and combining L1 loss function and L2 loss function.Experimental results show that ML-Former improves the accuracy of the long sequence time series prediction task by 35.4% and reduces the GPU memory usage during the model training period by 33.1% compared with the baseline method in three publicly available datasets.