Research On Human Action Recognition Based On Depth Sequential Features

With the rapid development of artificial intelligence technology,human action recognition research has achieved many results in the field of computer vision,and is widely used in many real-life scenarios such as intelligent monitoring,life entertainment,human-computer interaction,and medical rehabilitation.In general,the research of human action recognition can be carried out based on two main data sources: RGB video sequence and depth video sequence,and the characteristics of human action sequence are described by computer model,so as to realize human action recognition model construction.Compared with RGB video sequence,depth video sequence contains more complex and difficult to extract potential human action information.Therefore,how to extract human action features from depth video sequence and propose more effective human action recognition methods are the key problems to be solved in this field.Due to the complex spatio-temporal characteristics in depth video sequence,on the basis of previous research results of human action recognition,this paper develops human action feature extraction method and classification recognition model in depth video sequence for the key problems to be solved.On this basis,the main research contents and innovation points of this paper are summarized as follows:Firstly,aiming at the complexity of spatial and temporal information and the lack of three-dimensional visual information in human action feature extraction methods,three image model methods of depth action sequence are proposed,which are depth motion history image,depth motion accumulation image and depth motion subtraction image based on depth video sequence.The three action models are projected to the xoy plane,yoz plane and xoz plane of the coordinate axes respectively,then extracting their Hu moment features from the projected images,and finally a complete multi-view depth Sequential human action representation is realized.This feature extraction method discovers the advantages of depth space and time series data of depth video sequence,which not only simplifies the representation of complex human action spatial and temporal information data,but also mining the advantages of three-dimensional data of depth image.Secondly,aiming at the problem that human action segmentation is difficult because of continuous video sequence,this paper proposes a real-time continuous segmentation method based on martingale frame selected by key frames.According to the typical temporal dependence of video sequence itself,this method can segment one action directly without relying on subsequent frames based on existing frame detection and analysis.Therefore,in order to realize the selection of key frames in video sequences,a martingale frame model is proposed to realize the selection of key frames,and then extreme learning machine model is used to segment and classify human action in video sequences.Efficient extraction of key frame and accurate segmentation of human action are the key to building a fast,accurate and lightweight human action recognition model.Thirdly,aiming at the problem of how to improve the recognition rate of human action in depth video sequence,an integrated learning recognition model method of voting strategies for multiple classifiers is proposed.Based on the ensemble learning method in machine learning theory,the function of human action multi-classifier ensemble recognition in depth video sequence is realized by combining multiple individual classifiers.The whole ensemble learning model integrates multiple individual classifiers,and achieves classification verification based on Hard-voting strategy on the temporal features of multi-view depth human action proposed in this paper.On this basis,an evolutionary perception hybrid voting strategy model based on improved genetic algorithm is proposed,which reduces the fitting error of the traditional integrated learning model based on Hard-voting strategy at the decision level.Fourthly,aiming at the problem of multi-view human action recognition based on depth video sequence,a multi-view depth human action representation hierarchical modeling method is proposed.This method is based on the framework of multi-view hierarchical fusion to model the temporal-spatial dependence of depth video sequence,and then realize the feature extraction and classification of human action.Depth video sequence is a typical time series data,considering that the recurrent neural network model can well model the sequence dependence of context information in time series,this paper takes advantage of the spatial advantage of three-dimensional stereo vision of depth image,the depth human action data is transformed into three coordinate planes by coordinate transformation,and the projected human action features are layered and fused in the bi-directional recurrent neural network of long short-term memory units,so as to realize the accurate modeling of the spatio-temporal representation of human action.Fifthly,aiming at the problem of depth time series data acquisition and research,through the research of sensor equipment for depth video acquisition,the characteristics of depth video data are studied,this paper studies two current state-of-the-art depth video acquisition technologies based on binocular stereo vision and Kinect.On this basis,four human behavioral action databases were collected based on binocular stereo vision and Kinect(RGB image,depth image and skeleton data):binocular stereo vision database,basic action database,daily life database and examination database.Efficient and accurate data acquisition and content-rich self-built databases have laid an important foundation for extended application research.In this paper,human action recognition based on depth video sequence is the main research content.Human action feature extraction from deep video sequences,depth video sequence segmentation problem with typical time series dependence,sequence dependency modeling of human action context information in depth video sequence,and spatial feature mining of human action are studied.This paper proposes efficient and accurate human action feature extraction and action recognition methods,which provides new ideas and methods for research in related fields.Furthermore,the results of the quantitative experiments on the human action recognition databases show the effectiveness of the proposed method,and the collection of self-built database provides an important basis for its effective real-life promotion and application.