Research On Human Action Recognition Based On TSM

In recent years,computer vision related research has developed rapidly.Human behavior recognition technology is an important task in computer vision.It has important needs and very broad prospects in the fields of smart security,human-computer interaction,and autonomous driving.With the development of deep learning technology,more and more researchers have begun to use deep learning for behavior recognition research,which makes the technology has achieved rapid development.However,because of the complexity of human behavior and the complex background interference in behavior videos,and most of the current mainstream behavior recognition algorithms are unable to balance higher accuracy and lower computing costs,they do not make good use of video multi-modal data.Complementary useful information between.How to design a behavior recognition algorithm with high accuracy and low computational cost is still a big challenge.In view of the above problems,this paper has carried out further research on human behavior recognition technology.The main research work is as follows:This paper embeds the time shift module(TSM)with timing modeling capability into the two-dimensional convolutional network Res Net50 network with excellent performance,and obtains a two-dimensional convolutional network TSM-Res Net50 with the capability of spatial modeling and timing modeling.this network has a computational cost equivalent to the original two-dimensional convolutional network.This paper uses this network as the feature extraction network of the algorithm in this paper,in order to balance the problem of accuracy and computational cost.In order to reduce the influence of factors such as background and light changes in the video,this paper studies the effect of using the skeleton information of the human body for behavior recognition.This paper first uses the Open Pose algorithm to process the original video data to obtain the video data containing the skeleton information of the human body.The TSMRes Net50 network used in this article extracts spatiotemporal features and classifies behaviors.This paper compares this scheme with the scheme using skeleton information and graph convolutional network.The results show that the recognition accuracy of the algorithm in this paper is higher.In order to make better use of the useful information in multi-modal data and improve the accuracy and generalization ability of the model,this paper adopts the network framework of TSN in the classic multi-stream network,and replaces the feature extraction network with the TSM-Res Net50 of this paper.Network,forming a new multi-stream network.This paper calculates the data of two modalities of optical flow and twisted optical flow,and then experimented the effects of the three modal data of RGB,optical flow and twisted optical flow on the network.In this paper,experiments are conducted on different fusion strategies in the network,and the strategy with better effect is selected as the fusion strategy of the network.This paper studies the use of multi-perspective information for behavior recognition,applies multi-stream networks to multi-perspective databases,and explores its feasibility and effects.In order to further improve network performance,through research and analysis of TSM,it is found that TSM is more for short-term timing modeling.Therefore,in order to improve the long-term timing modeling capabilities of the network and allow the network to focus more on the more important In terms of characteristics,this article improves the abovementioned network.In the Segment Consensus module,introduces LSTM and GRU networks with long-term timing modeling capabilities and attention mechanisms.Through continuous adjustments to these structures and experiments,the network performance is finally improved.A certain effect of improvement.