| Time series refers to a series of data variables listed in time order.These data variables can be in the form of numbers,vectors,matrices,multimodal data and so on.Time series modeling technology aims to analyze and model the temporal features in these data,and apply to tasks such as representation,detection,recognition,and prediction.In recent years,with the rapid development of digitization and informatization in all walks of life,time series data widely exists in many scenarios,such as daily life and work,complex scientific monitoring,rich Internet records and so on.These various data are constantly accumulating over time.The accumulated massive data contain high-value information about scientific research and application.Therefore,modeling information of time series is an indispensable part of big data analysis and applications.The time series modeling technology is a popular research problem in the fields of artificial intelligence and data mining,which has received wide attention by researchers.However,the characteristics of time series are diverse,and usually have different multi-scale features at different time series scales.The existing time series modeling techniques still lack research from multi-scale perspectives of time series.To this end,this paper systematically conducts research on three core time series scales of short time series,long time series,and multi-time series,and model various multi-scale features at these time series scales.To be specific,this paper uses data mining,machine learning and other methods to explore above three time series scales respectively,and propose four models:a multi-scale description based short time series detection model,a multi interval derivative based short time series recognition model,a periodic attention mechanism based long time series pre-training model,a multi-level time series information collaboration based multi-time series prediction model.The main contributions of this paper can be summarized as follows:First,for the short time series,this paper proposes a neural network short time series detection model based on multi-scale description,which can model the long and short-term multi-scale features in the short window time series.In this kind of short time series,the change of long and short-term features of the time series is very important for target window detection.The long-term features can reflect the background information of the time series,while the short-term features can reflect the changes information of time series.Moreover,the mixed features of different time scales can further highlight the target pattern information.Existing time series modeling methods tend to progressively extract information of different scales and lack hybrid modeling of long-term and short-term features.To solve this problem,this paper proposes a multi-scale description based short time series detection model.Specifically,this model mainly designs a novel neural network structure,which can continuously fuse time series features of different scales to construct a short time series representation.Then this model uses self-attention mechanism to integrate multi-scale fusion descriptions at different time points,which can improve the representational ability.In addition,this paper designs a multi-task learning strategy to compare the representations of different labels through auxiliary task.This strategy can optimize the distance between time series representations with different labels,and improve the representation performance.Finally,this paper conducts comprehensive experiments on two scientific monitoring signal time series datasets to evaluate the performance of the proposed model.Second,for short time series,this paper also proposes a multi interval derivative based short time series recognition model,which can model the pattern features of time series at a fine-grained local scale and achieve accurate recognition of local patterns in time series.The accurate recognition can quickly detect out the time point when the events or patterns start,which is critical to the real-time information acquisition in time series modeling applications.In time series,the early features of target temporal events or patterns usually have unique multi-scale variation characteristics.For example,in user behavior-related time series,the early patterns of emergency events may be rapidly rising and then slowing down or falling.The early pattern of general events may be only a gradual increase.How to use as small length as possible to efficiently and accurately model the multi-scale variation features and recognize target patterns is a key problem.To solve this problem,this paper proposes a multi interval derivative based short time series recognition model.This model designs a metric to select time series which are most relevant to the target event or pattern.Then,based on the selected time series,this paper constructs derivative feature of time series with different intervals which can reflect different scales of variation.For different intervals,this paper also use different trigger thresholds,which can be adjusted flexibly according to the characteristics of the target event or pattern.When all the thresholds are triggered,the target time point is recognized.Furthermore,to improve the accuracy,this paper designs a novel and efficient multi-level attention neural network to improve the recognition accuracy.Finally,this paper conducts comprehensive experiments on real online user behavior time series datasets to evaluate the performance of the proposed model.Third,for long time series,this paper proposes a periodic attention mechanism based long time series pre-training method,which can model multi-scale periodic structures in large-scale long time series and can efficiently,flexibly,generally apply to a variety of time series downstream tasks.On the one hand,in large-scale long-term time series data,the time-varying characteristics usually have multi-scale periodic structure.For example,in a user behavior related time series,it usually has the characteristics of multi-scale and different periods,such as different behavior patterns in a day,in a week,and so on.Existing time series modeling methods are difficult to pay enough attention to the multi-scale periodic structure while modeling long time series information.Meanwhile,the efficiency of long time series modeling is a critical problem.On the other hand,the analysis and application of time series data often include a variety of tasks with different scales and types.How to flexibly adapt to these tasks’ various inputs and outputs is also a key problem.To address these multi-scale problems,this paper proposes a periodic attention mechanism based long time series pre-training method.This method contains a time series aggregation module to aggregate similar information and compress the length of time series.Then,this paper designs a periodic pre-training module to learn a unified representation of time series in each period,and proposes an aggregated time position embedding technique to simultaneously encode the start time and duration of aggregated time series points.Meanwhile,this paper introduces a time position query pre-training task to enhance the representation of the entire time series.Furthermore,in order to deal with input time series with arbitrary lengths,this paper further proposes a fine-tuning module based on a hierarchical selfattention mechanism,which can simultaneously model time series in different periods,and supports various downstream tasks efficiently and flexibly.Finally,this paper conducts extensive experiments and discussions on a real user trajectory time series dataset with three downstream tasks,which can clearly validate the effectiveness,flexibility and accuracy of the proposed models.Fourth,for multi-time series,this paper proposes a multi-level time series information collaboration based multi-time series prediction model,which can model the information correlation between multiple related time series at multi-level scales.In the group of multiple related time series,the temporal regularity information between different time series usually can promotes and complements each other.For example,in the user behavior time series,similar users may have similar temporal regularity,and the information in the high-level time series of a set of similar users can make up the lack information in the individual fine-grained time series.To solve this problem,this paper proposes a multi-level time series information collaboration based multi-time series prediction model.This model first constructs time series with multi-level information,and then designs a mixed input attention module based on attention mechanism to capture the information of different time series and mix the multi-level information.Meanwhile,this paper also proposes an related temporal attention module to model the complex temporal information of the mixed input time series.Finally,this paper conducts comprehensive experiments on a real online marketing time series dataset to evaluate the performance and interpretability of the proposed method. |
