Modeling Of Spatiotemporal Sequence Features Based On Tensor Representation

With the progress of science and technology in recent years,the degree of social informatization is getting higher and higher.In the face of the massive images,videos and other tensor-form data that used to be difficult to process,now with the rapid development of machine learning,deep learning and other technologies of data science,people could handle them efficiently and precisely.How to effectively model the complex spatio-temporal feature from tensor-form data is a vital but arduous problem.Taking tensor representation as the starting point,this paper mainly studies the following problems in feature modeling in spatio-temporal sequences:(1)For the problem of tensor sparse representation,how to surpass the disadvantages of traditional iterative algorithms such as high redundancy,slow speed,and poor robustness,and achieve sparse inference rapidly and robustly;(2)For natural spatiotemporal sequence,how to conduct reasonable spatial feature modeling and time-wise relationship capturing according to its structural characteristics,and realize efficient forecast for unknown sequence;(3)According to the structural sensitivity of the existing spatial feature extraction algorithms and operation units,could we combine the tensor representation method to effectively learn the spatiotemporal sequence which have no obvious spatial structure,so as to achieve accurate prediction.Around these problems,the main work of this paper can be divided into three parts:In the first part,aiming at the problem(1),a differentiable tensor sparse inference algorithm is proposed,which realizes the rapid inference of tensor sparse representation in data-driven way,and the convergence condition and error upper bound of the algorithm is also given.Meanwhile,the existence of the proposed theory is verified in numerical experiments,and the effectiveness of the proposed algorithm is verified in reconstruction experiments.The proposed algorithm provides a theoretical basis for the combination of deep neural network and tensor sparse representation.In the second part,we combine tensor sparse representation and compressed sensing,and propose an end-to-end deep predictive network to solve the problem(2),which realizes the feature modeling of natural spatio-temporal sequence and the prediction of future sequence.This method is effective for the prediction of natural spatio-temporal sequence,which owns low-frequency geometry as the main spatial feature and high-frequency detail as the main temporal variation.The effectiveness is verified on two different data sets.In the third stage,aiming at the problem(3),we make a clear definition to the related problem and concepts,and propose a predictive algorithm for feature modeling in ordinary spatiotemporal sequence.Based on tensor representation,the algorithm introduces multi-channel linear operators and an attention mechanism to realize global learning of elusive non-local correlations and adaptive learning of local or non-local features.This method is suitable for the prediction of ordinary spatiotemporal sequence with unconspicuous spatial structure and elusive spatiotemporal correlation,and achieves the state-of-the-art results on two different data sets.