Research On Temporal Action Detection In Video

In recent years,with the development of multimedia technology and the rapid popularization of digital equipment,video data on the Internet has widely exploded.How to quickly,accurately,and efficiently analyze the massive and unorganized video data has become a significant issue for researchers.As an important branch of machine learning,deep learning technology has made a great breakthrough in the field of image classification and detection,and researchers deveoted to introducing neural networks into the field of video understanding,which contains various video tasks such as temporal action detection,action recognition,video summary,and object tracking.This dissertation aims to address the temporal action detection task.Specifically,temporal action detection is an important task in the field of computer vision,which not only need to locate the precise action interval of each action instance in a long untrimmed video,but also to identify the action lable.Temporal action detection algorithms have broad application prospects in many fields such as medical monitoring and national security.The difficulty lies in two aspects: on one hand,the action localization is sensitive to temporal timestamps;on the other hand,the duration of the action instances may vary greatly.It requires the models to accurately capture long time series information.Based on the deep learning technology,this thesis proposes a Temporal Proposal Optimization(TPO)network for temporal action detection.First,TPO utilizes CNN(Convolutional Neural Network)module to capture the local temporal information,and adopts BLSTM(Bidirectional Long Short Term Memory)and CTC(Connectionist Temporal Classification)modules to capture global temporal information.Then,TPO jointly uses these two types of temporal information to construct boundary probability curve,local action probability curve and global action probability curve.Then TPO constructs candidate action proposals based on the boundary probability curves,and fuses the two action probability score curves to optimize and rank the candidate action proposals.Finally,TPO adderesses temporal action detection.TPO has two advantages:(1)TPO effectively learns the long-term dependence in videos by introducing BLSTM and CTC,(2)the above-mentioned probability prediction curves extract sufficient temporal proposal candidates that can effectively capture the large time-span changes of action instances.Experiments show that TPO achieves promising performances in boththe tasks of proposal generation and the temporal action detection.