Research On Behavior Recognition Algorithm Based On RGB-D And Deep Learning

Human behavior recognition algorithms have great research significance and industrial value in virtual reality,intelligent monitoring and unmanned driving.The traditional behavior recognition algorithms are based on the color pictures,which manually designs the feature extraction to extract the shape and color features,establishes feature descriptor and selects the classifier for classification.It will lead to two problems.Firstly,the color pictures have less information entropy,which makes extracted features not to represent the behavior well.At the same time,the generalization of background occlusion and viewing angle changes is poor.Secondly,the traditional feature extraction algorithms are difficult to design and its behavior recognition rate is not high.The new RGB-D data includes color pictures,depth pictures,and skeleton pictures,which are rich in information entropy.But RGB-D multi-source information fusion is still a difficult problem.In addition,experiments have shown that convolutional neural networks have achieved great success in image classification.Therefore,this paper proposes a human behavior recognition algorithm based on RGB-D and deep learning.In order to solve the problem that traditional algorithms design feature extraction difficultly,Faster RCNN is used to extract features and classification.By analyzing the framework of Faster RCNN,the human behavior recognition rate is improved by data enhancement,deleting a layer of the fully connected layer and Dropout strategy.We use the complementarity between RGB-D information to solve the problem that RGB-D multi-source information fusion is difficult.Specifically,the interested region of color pictures is located by using depth pictures and skeleton pictures,which avoids the interference from unrelated regions.In summary,the human behavior recognition optimization algorithm based on RGB-D and Faster RCNN is proposed.Experimental results show that the average recognition rate of the proposed algorithm on the UTKinect dataset has reached 94.70%,which is better than other algorithms and verifies the advantages of the algorithm.In order to solve the problem of less information entropy in color pictures and the poor generalization in the background occlusion and the viewing angle changes,Two Stream CNN is used to fuse the features of the depth pictures and the skeleton pictures.Because depth pictures and skeleton pictures are robust to background occlusion and viewing angle changes.Two fusion strategies are proposed in the network,which fuses features respectively in the fully connected layer and Softmax layer to study the impact of different multi-source information fusion strategies on behavior recognition.The average recognition rates of the two different fusion strategies on the UTKinect dataset are96.20% and 95.70%,respectively.The average recognition rates on the SBU Kinect dataset are 92.70% and 92.10%,respectively,which are better than other algorithms and verifies the robustness of the algorithm.