| The temporal action detection task aims to identify the category of each action that occurs in the input video and detect the start and end time corresponding to each action.Based on the different labels used in the training phase,temporal action detection can be divided into two subtasks: full supervision and weakly supervision.Most research methods of fully supervised sub-task use sliding windows to obtain a series of candidate regions,and then classify and regress the candidate regions.However,the sliding window method has poor flexibility and needs to set different window sizes for specific data sets.Most of the weakly supervised sub-task methods are based on multi-instance classification results.This classification-based approach fragments detection results.In order to solve the above problems,this paper has conducted in-depth research on two sub-tasks,the main work is as follows:(1)More flexible method of generating candidate regions based on the bottom-up structure.For the spatial-temporal features of video extraction,the convolutiondeconvolution method is used to down-sample the features in spatial domain,upsample the features in temporal domain.Making the feature length the same as the video frames length.Then,using three binary classifiers to generate each location may be the probability of action starting,action occurring,and action ending.Finally,the action start position and the end position with higher probability values are used to generate more flexible candidate regions.(2)A fully supervised temporal action detector with higher performance is obtained by complementary temporal candidate regions.By combining the candidate region generation method proposed in this paper and the candidate region generation method with a predefined window size,a fully supervised temporal action detector that can be trained end-to-end and has better detection performance is obtained.(3)Online temporal class activation maps generation method.Most existing methods of weakly supervised sub-task use a trained classification network to generation temporal class activation maps.This paper explores a method for online generating temporal class activation maps,which is mathematically equivalent to existing methods.The online method,can generate temporal class activation maps in the process of network forward calculation,so that the temporal class activation map mechanism can be more flexibly integrated into other complex networks.(4)Make weakly supervised results more complete through complementary learning.Use the temporal class activation maps erase feature,obtained by forward calculation process of classification network.Using the erased features,an additional classification network is trained to generate complementary temporal class activation maps.The fragmentation problem is alleviated by merging two temporal class activation maps.The improved method proposed in this paper is tested on the corresponding temporal action detection dataset.The experimental results show that the detection performance of the proposed method has been greatly improved compared with the existing methods. |
