Temporal Action Detection Based On Deep Learning

Video detection is the foundation of video analyzing,and action recognition is the key of video detection task.Therefore,this paper mainly focus on temporal action detection task,which is an exploration of action recognition.Compared with the traditional action recognition task dealing with trimmed videos,temporal action detection task pays attention to untrimmed long videos,which makes a distinction between action segments and background segments on the timeline.The data of this task is special because various multi-scale actions appear randomly in every single untrimmed videos.Meanwhile,since action change gradually with time goes by,the main difficulty of this task is how to accurately locate the starting and ending points of actions on the timeline beyond recognizing action categories.According to the untrimmed video data with multi-label and multi-scale duration,this paper propose algorithm innovations on the basis of traditional video recognition method,aiming at locating action temporal boundary and improving the detection results of untrimmed videos.As for the problem of ineffective expression of long actions and inaccurate boundary location of actions during detection,three different temporal action detection methods based on deep learning are proposed to effectively improve the overlap between proposals and Ground True based on accurate detection of action categories.The main methods are as follow:(1)A temporal action detection method with long action seam mechanism is proposed.Based on accurate detection of action categories.This paper propose a clip construction combined with context and a new regression model with long action seam mechanism to get the location of starting and ending points of action,dealing with the inaccuracy of long action detection caused by the limitation of sliding window length.In addition,inspired by phased processing,we extract feature through pre-training network to reduce the calculation of network and improve the efficiency of temporal action detection.(2)A temporal action detection method based on mapping at every unit and global evaluation network is proposed.In order to improve the sensitivity of the detection method to the temporal boundary,this paper adapt one-dimensional convolutional network to map feature sequence to the probability of actions at every unit,which is inspired by the idea of “point-by-point processing”.Then given the mapping results,we train a global evaluation model to evaluate proposal-level confidence based on Long Short-Term Memory network with the thought of “local to global”.Given the sampling feature of the actions,we evaluate the probabilities of the action to improve recall rate of the actions(3)A temporal action detection method based on temporal convolutional network is proposed.This paper proposes a coarse-grained clip construction and the boundary regression model based on temporal convolutional network.We combined two parts of coarse-grained clips by point-by-point processing and the scanning of multi-scale sliding windows,which is to reduce missed detections.Then this method adapt temporal action convolutional network to get accurate detection results.