| Intelligent Transportation Systems(ITS)has become one of the most active research areas due to its potential to promote system effectiveness and decision making.However,as the basis of ITS research,traffic flow data has characteristics such as non-stationary and spatial-temporal dependencies,which makes it extremely challenging to process with deep learning methods.On the one hand,the conventional deep learning methods require a large number of annotated data for the model to imitate and learn.On the other hand,different deep learning models usually need to be designed and trained for different ITS applications,which makes it difficult to get a universal one.Therefore,how to efficiently utilize the large amount of virgin traffic flow data and train a unified model is the main focus of this thesis.The main research contents of this thesis are summarized as follows.(1)Patch-based Self-Supervised Model for Traffic Flow AnalysisA Patch-based Self-Supervised Network(PSSN)for traffic flow is proposed,which can learn traffic flow features in a task-agnostic way,and provide a well bootstrapped pre-training model for a variety of tasks.PSSN tokenizes traffic flow data into subseries-level patches,each of which is comprised of numerous consecutive points.A pretext task is well designed to understand the traffic flow correlations by forecasting patches that are randomly masked.Therefore,PSSN could capture the high-level intrinsic semantics of traffic flow,and provide general-purpose patch embeddings.As a result,by replacing the output layers with a set of untrained new layers and fine-tuning with small-scale task-specific data,PSSN can be deployed for a variety of downstream tasks in ITS applications.Experimental results demonstrate that PSSN can improve the overall performance of various downstream tasks compared to state-of-the-art models.(2)Patch-based Self-Supervised Model for Spatial-Temporal Traffic Flow ForecastA Spatial-Temporal Patch-based Self-Supervised Network(ST-PSSN)is proposed to improve the accuracy of traffic flow forecast.Traffic flow series have not only complex temporal patterns,but also underlying spatial interdependencies.Limited by computational complexity,most existing traffic flow forecasting models only utilize short-term historical data,which directly lead to imprecision of the prediction results.Considering both spatial dependency between sensors and long-term traffic flow series for selfsupervised pre-training,ST-PSSN aims to efficiently learn the complex spatialtemporal patterns and generate robust segment-level representations.Specifically,as the main component of ST-PSSN,the Traffic Flow Spatial-Temporal Transformer(TF-STFormer)could not only model long-range bidirectional temporal-dependencies,but also use both Graph Convolutional Network to model stationary spatial-dependencies and self-attention mechanism to model dynamic spatial dependencies.During the prediction stage,the output layer could be as easy as a multi-layer perceptron,whose inputs are the hidden representations of pre-trained model and short-term traffic flow data.Experimental results demonstrate that the ST-PSSN can extremely improve the prediction performance,compared to other baseline models. |
