Research On Action Recognition Algorithm Based On Optical Flow And Depth Motion Map

In the field of human behavior recognition,the research on video action recognition algorithm is in full swing at home and abroad.Although the research on motionless image recognition has achieved great success,the research on video action recognition is still a challenging subject.In this paper,in view of the lack of depth information in color information(RGB video frames)that is not easily affected by environmental factors such as lighting,and the lack of recognition ability for actions along the line of sight,a human action recognition algorithm based on optical flow and Depth Motion Map(DMM)was proposed.Its main research contents are as follows:(1)In this paper,the depth sequence is projected on three orthogonal cartesian planes to obtain the DMM feature to represent the motion feature of an action.In addition,in order to reduce intra-class variability,the DMM is adjusted to a fixed size,and the fixed size of each DMM is set to half of the average of all sizes.Due to the high dimensionality of the proposed feature descriptor,the Kernel Entropy Component Analysis(KECA)was used to reduce the dimensionality.Finally,the processed DMM feature was used as the input of the deep flow network channel in this paper.(2)in this paper,a LSTMs network structure is designed by using Long and Short Term Memory networks(LSTMs),which has obvious advantages in controlling and memorizing long sequence historical information.In this paper,the LSTMs network structure is composed of multiple LSTM memory units.At the same time,three spatial features,time features and depth features of the recognition flow output are constructed into the feature matrix,whose feature matrix is divided into multiple time segments according to the time dimension,and then they are input into the LSTMs network layer in order,and the feature matrix of this paper is fused according to the correlation characteristics on the time axis.(3)for the fusion depth information not easily affected by environmental factors such as illumination and RGB video sequence of rich details,first of all,based on the fusion of optical flow information and color information,and at the same depth information from the depth of the synchronous video sequences,and convergence in this framework,the framework for this article provide depth characteristics,to enhance complementarity characteristics;Secondly,the three kinds of feature information are taken as the input of spatial flow network,time flow network and depth flow network based on ResNet101.Then feature fusion is performed by LSTMs.Finally,you send the features into the Softmax layer with full connections to get the probability values for each action category.The experimentalresults show that the recognition effect is better on the challenging UTD-MHAD data set and MSR Daily Activity 3D data set.