Research On Behavior Recognition Based On Multi-view Depth Video And Human Skeleton Sequence

With the growing problem of aging society,home service robots have gradually entered the human life and played a key role in caring for the elderly and the disabled people.Intelligent robots need extensive human-computer interaction and intelligent monitoring in their daily work,and the recognition of human behavior becomes the basis of communicative home service robots.Due to the development of depth sensors,it is becoming easier to collect depth videos and extract human skeleton sequences from them.Considering the limited viewing angle of robot airborne sensors and the cost of production,this paper separately describes human behavior based on multi-view depth video and behavioral recognition based on human skeleton sequence development,the specific content is as follows:First of all,during the execution of the mission of the service robot,it is difficult for the airborne sensor to ensure that the positive information of the human body is captured at all times,and the behavior recognition error caused by the change of the viewing angle.To solve the problem,a perspective-independent method for behavior recognition in multi-view depth video via temporal-spatial correlating is proposed.In this paper,the deep convolution neural network is trained by multi-view human body image,and its network fully connected layer is used to map the features of different perspectives to the high-dimensional space independent of the angle of view to form spatial features.Then,the rank pooling and the Fourier time pyramid method are used to model the time information of the video and combine temporal and spatial information for behavior recognition.Experiments show that the proposed temporal-spatial behavior recognition model effectively improves the accuracy of multi-view behavior recognition and it has robust generalization ability.Secondly,based on the human skeleton sequence data,this paper proposes a method of temporal-spatial feature extraction based on joint subgroup for the relationship between different joints of human skeleton for the relationship between different joints of the skeleton,and uses K-means clustering and the vector of locally aggregateddescriptors to aggregate local features to form a global temporal-spatial features.In the end,combined with multi-class optimal margin distribution machine to achieve classification and identification of specific behaviors.Experiments show that the proposed algorithm has good real-time performance while ensuring accuracy.