| In the computer vision field,action recognition that is the representative research direction.Besides,it also has potential applications in some fields,such as intelligent surveillance,human-computer interaction,intelligent medical care and so on.The key task of action recognition is to extract the motion features of human actions in video.However,it remains a research challenge for scholars to characterize the motion information effectively in videos with large variation and different speed.In addition,traditional handdesigned feature methods are difficult to work and have low recognition rates,the deep learning-based methods can fully exploit the feature information in videos and quickly predict the action categories in videos.But the problem of low recognition rate and high computational cost still exists.Therefore,we investigate the deep learning-based video action recognition method in the paper,the specific research contents include:(1)To address the problems that the methods of action recognition which are based on deep learning only focus on the long-term temporal structure modeling of actions,so that the lack of local information representation affects the recognition accuracy,the large number of network parameters and the complexity of algorithm computation,the paper proposes an action recognition method with long and short-term temporal modeling.Firstly,different temporal flow models are constructed at different time scales to obtain model scores,then model scores of spatial flows are learned along the spatial dimension,which is followed by the construction of a multi-flow action recognition model fusing spatio-temporal features,and finally the action classification results are obtained by fusing the outputs of each flow according to the weight distribution.The method is experimentally evaluated in datasets UCF-101 and HMDB-51,and the results show that the recognition accuracy is improved by4.3% and 8.7% respectively compared with the benchmark model TSN(Temporal Segment Networks,TSN),the computational cost is reduced by 40.1%～68.5%.(2)To address the problems of difficult extraction of fine-grained action motion features and low recognition rate,to further improve the recognition accuracy of actions in videos,the paper proposes ion recognition method based on the motion information feature map.First,we propose a computational method of motion information,which is called motion information image,and explores the best implementation of the method.Finally,the motion information image is able to aggregate multiple frames of motion information into one frame of motion feature map,and it can be combined with existing CNN(Convolutional Neural Network,CNN)to improve the action recognition accuracy and effectively enhance the model’s ability to recognize fine-grained actions in videos.Through extensive experiments on two large-scale human action recognition datasets,NTU RGB+D and NTU RGB+D 120,the results validate that the method improves the action recognition rate by2.2% to 3.0% compared with the baseline model TSM(Temporal Shift Module,TSM)at a limited computational cost and shows a unique advantage in recognizing fine-grained actions. |
