Research On Human Action Recognition Method Based On RGBD Data

With the development of network media,surveillance and mobile cameras,human action recognition playing an increasingly important part in many fields,such as video retrieval systems,interacting with a computer,intellectualized monitoring systems and so on.In recent years,RGBD data sets containing depth information have been paid more attention with the emergence of depth cameras.Compared with traditional cameras,depth cameras can not only obtain video data,but also obtain the distance information of objects.However,many current human action recognition methods based on RGBD have the problem of performance degradation due to the change of camera angle.This paper mainly studies human action recognition based on RGBD data in the field of computer vision.The main research contents and contributions of this paper are as follows:(1)In order to deal with the performance degradation caused by the change of camera angle of view,a human action recognition method based on Bi-LSTM and feature fusion is proposed,which does not rely on human skeleton data during model training and does not need to explicitly calculate human posture information.This method integrates the early features of RGB images and depth images of multi-view data sets,and a LSTM network and a Bi-LSTM network are used to obtain the time dimension information of the video sequences,as well as build a cross view decoder network to learn the feature expression against view changes,so as to improve the performance of human action recognition model.The experimental results show that the recognition accuracy of the model reaches 86.5%in the cross view(CV)evaluation of NTU RGB+D data set.(2)Aiming at the problem that dynamic background information and noise affect the performance of the algorithm.Based on unsupervised learning,this paper designs a method of human action recognition based on auto-encoder and feature disentanglement.In the training process,the network only depends on RGB images and masked depth images,without annotation information such as human skeleton data and camera parameters.Firstly,the auto-encoder disentangles the foreground,background and motion of the video sequences,and introduces the attention mechanism to focus on the reconstruction of the foreground region of the image.Then,the masked depth image generated by the masked network is used as the supervision signal to train the whole network end-to-end.Through feature disentanglement,deeper feature expression can be obtained in order to serve the downstream human action recognition task.Finally,the proposed method is tested on NTU RGB+D and MSR Daily Activity 3D data sets to verify the effectiveness of the method.