Long Time Series Forecasting Of Exchange Rate Based On Transformer Model

Time series widely exists in various fields of the real world,and the related forecasting tasks play a key role in the development of industry,society and scientific research.Financial time series analysis is one of the more active research areas.Early traditional series forecasting methods rely on the stationarity hypothesis to build models,which have achieved good results in some tasks,but cannot handle time series that are affected by high-dimensional variables such as exchange rates..In recent years,the capital market has continued to develop,and exchange rates have played an important role in international economic relations.Its fluctuations may lead to cross-border financial risks and endanger national economic security.Reliable exchange rate forecasts are required to provide guarantees for risk management.At this stage,the scale and quantity of exchange rate time series data are growing rapidly.The occurrence of various international events has made the financial market increasingly uncertain.Time series prediction models are faced with the need for longer input time series segments and higher prediction accuracy requirements.With the rapid development of current deep learning technology,the Transformer model has achieved excellent performance in many tasks in natural language processing and computer vision,which has attracted attention in the field of time series analysis.Recent studies have shown that Transformer models outperform previous models in dealing with long-term sequence problems,and have a stronger ability to capture long-range dependencies and interactions.Therefore,this paper proposes an exchange rate time series prediction model based on the Transformer architecture.Compared with previous exchange rate time series prediction models,it supports longer input and output of time series segments and provides more accurate prediction results.The main work includes the following two aspects:Based on the task characteristics of exchange rate prediction,this study proposes an improved Transformer model to improve the accuracy.Using event encoding to embed Transformer’s positional encoding,event intensity and time interval information are injected into the model,helping the model learn the impact of historical events and capture more complex event dependencies.The input features are superimposed according to time periods for periodic enhancement,and multi-level sequence associations are established to improve the model’s ability to learn the autocorrelation of historical data.In addition,this study uses wavelet transform to denoise the sequence data.Relevant experiments show that the model using the above method achieves better accuracy than the benchmark method on the 2019USD/JPY exchange rate dataset with a 5-minute frequency,compared with the current state-of-the-art long-term series forecasting model Informer,both are equally accurate.The squared error is reduced by 44.2% and the mean absolute error is reduced by 25.7%.In view of the advantages and disadvantages brought by Transformer’s self-attention parallel structure,this study proposes an improved Transformer architecture to improve performance.Using the convolutional neural network to improve the self-attention unit reduces the time complexity of the self-attention operation by 50%,while ensuring the same or higher prediction accuracy as other Transformer models.An improved self-attention extraction operation is proposed,which uses extended causal convolution instead of regular convolution to connect self-attention blocks,further expands the receptive field of the Transformer model,achieves exponential growth,and effectively reduces computational costs.Relevant experiments show that the prediction effect of the model using the above method is better than that of the Informer model based on the Transformer architecture on the 2019USD/JPY exchange rate data set,and the performance is greatly improved,and the computing and storage costs are lower.The methods proposed in the above two works are independent of the specific X-former prediction model and can be applied to any model based on the Transformer structure.