Deep Learning Research For Modeling Incomplete Time Series

Time series is a crucial data representation in the real world,which widely exists in different application fields,such as urban transportation,power grid,financial market,and so on.Due to uncontrollable factors in the real world,such as operational errors,equipment failures,and communication errors,the real-world time series data inevitably contains missing values,thus forming incomplete time series data.However,most of the current time-series modeling algorithms are based on the assumption that the data is complete and cannot deal with the incomplete situation.For this reason,imputation is usually regarded as a pre-processing step,and a two-stage method of imputation first and then modeling is adopted.However,this does not consider imputation and target tasks(prediction and classification)as a whole to optimize jointly and ignores the interaction between them,which leads to the suboptimal solution.Directly jointly optimizing the imputation and target tasks will encounter the problem of error accumulation.Therefore,how to model the incomplete time series data directly and effectively remains a very challenging problem.Based on recurrent neural networks(RNNs),this paper explores the deep combination of the idea with residual neural networks(Res Nets)and generative adversarial networks(GANs).Two new recurrent neural networks are proposed to model incomplete time series data in this paper directly.(1)A novel recurrent neural network with graph-dependency called linear memory vector recurrent neural networks(LIME-RNNs)was proposed in this work.It introduces a residual graph connection to learn a linear combination of previous historical states of RNNs,forming linear memory vectors.This vector integrates over previous hidden states of the RNNs,mining the connection between missing items and previous items,and is used to fill in missing values.Further,a novel prediction loss function is proposed for incomplete time series,which enables the model to predict in the presence of missing data.The efficacy of the model is demonstrated via time-series imputation and prediction of real-world data and synthetic data.(2)A novel recurrent neural network with adversarial and joint learning called adversarial joint-learning recurrent neural networks(AJ-RNNs)was proposed in this work.It adopts joint learning,which integrates time-series imputation and classification within one RNNs model,and can directly classify missing data.Further,to alleviate the bias introduced by imputation value,an adversarial learning strategy is employed to improve the imputation ability of RNNs by using a discriminative network trained to distinguish real and imputed values.The effectiveness of the adversarial joint learning recurrent neural network is verified on the incomplete time series classification of real-world and synthetic data.