Time-series Deep Learning Algorithm And Its Application Research

The booming development of information technology promotes the distribution of video.However,the proliferation of video data poses great challenges for digital right management,while conventional manual censorship and retrieval cannot handle the amount of online video.Video fingerprint is a technique for solving this problem.Video fingerprinting algorithm extracts a compact identifier?fingerprint?from video,and the distance between fingerprints can indicate the perceptual similarity between video sequences.Focusing on video copy detection,this thesis investigates the design of deep time-series neural network based fingerprinting algorithms.A review of existing deep learning network is presented in Chapter Two,including feedforward neural network,deep generative model and time-series network.In this thesis,we propose two novel deep learning based video fingerprinting algorithms.The network architecture of the fingerprinting algorithm described in Chapter Three is constructed by stacking Conditional Restricted Boltzmann Machine?Conditional Restricted Boltzmann Machine,CRBM?,Denoising Autoencoder?Denoising Autoencoder,DAE?as well as post-processing network.CRBM can simultaneously model the statistical correlations of visual information along the spatial and temporal directions.DAE can discover the information that is invariant to distortions,while the post-processing is trained to balance robustness and discriminative of video fingerprint.In Chapter Four,we introduce a convolutional network and recurrent network based video fingerprinting algorithm.The architecture consists of AlexNet,Long Short Term Memory Network?Long Short Term Memory Network,LSTM?and post-processing network.We fine-tune the AlexNet to extract spatial features from representative fames,and temporal features are extracted by the LSTM network.Post-processing layers are placed on top of the two networks to optimize the robustness,discriminability and orthogonality of output fingerprint,and two cost functions are designed.Compared with traditional video fingerprinting algorithms,our proposed works have ability of learning the optimal feature extractor.In video copy detection experiments,our algorithms outperform state-of-the-arts.The first algorithm's F₁value is 0.982,and those of the second algorithm trained by two different cost functions are 0.956 and 0.945,respectively.Besides,the video fingerprint generated by the second algorithm is the shortest among testing algorithms,and the length of fingerprint is 50.