Algorithm Of Complex Action Recognition Based On Temporal Proposals

Action recognition is an important filed in computer vision.Now action recognition usually aims at classifying the categories of the video which is manually trimmed and only contains one action.However,most videos in real world are untrimmed and may contain multiple action instances with irrelevant background information or activities.In this type of untrimmed long video,it needs to detect not only the action category,but also the precise starting time and ending time points accurately.This task called temporal action localization.Most of the existing temporal action localization methods involve two stages:proposal generation stage and classification stage.Most of the existing temporal action localization methods use proposals and classification.There are several problems in temporal action localization,such as the low proportion of action instances proposals,and the inaccuraty of action recognition.The amount of complex calculation is large,and the processing speed is mostly slow.Therefore,this thesis proposed two algorithms for the problems mentioned before.The main research work of this thesis is as follows:Firstly,we proposed a method with temporal boundary regression for temporal action localization based on two-stream convolutional neural network,which can generate proposals with flexible temporal durations.This method uses a flexible and efficient high-action probabilities clustering algorithm to generate proposals instead of the sliding windows.This method generates proposals by grouping those continuous regions with mostly high actionness probabilities,and uses non-maximum suppression to remove redundancy.The multi-layer perceptron is used for temporal coordinate regression to refine the temporal boundaries of proposals.In order to add the context information to the feature of proposals,athree stages feature is constructed.Experiments were carried out in large data sets THUMOS14 and ActivityNet,which are commonly used to temporal action localization tasks.The mean average precision can reach 30.1% and 33.19%,respectively,which proves that this method can improve the accuracy of action recognition effectively.In addition,this thesis designed a method for fast temporal action localization based on C3 D network,which is mainly for the problem of large computational complexity and low speed of recognition.The method draws on the YOLO and SSD algorithms in the object detection.There is no additional feature extraction and proposal generation.The temporal action localization is performed in an end-to-end network,so the running speed is quite fast,We conduct the experiment on the THUMOS14 dataset with an accuracy of 29.1%.Although the accuracy is slightly lower than the above algorithm,the running speed is greatly improved,reaching 683 frames/second.In this thesis,the temporal action localization task is deeply studied,and two solutions are proposed for different problems.The experimental verified the effectiveness of the proposed algorithm and improves the accuracy of temporal action localization.