Research On Methods Of Deep Feature Modeling And Action Recognition Based On Human Skeleton Sequences

With the rapid development of Internet technology,information technology and sensor technology,a large majority of data such as image,video and graph are growing and obtained easily,which widely exist in the field of computer vision and video monitoring system,etc.As the data contains a wealth of information,mining the information implicit in the data is of great application value.The traditional data mining methods mainly use pattern recognition technology to model the data feature,which includes processing modules such as feature extraction,feature selection and classifier design.This tends to result in poor adaptive ability of model.With the development of data-driven technologies such as deep learning,deep models such as convolutional neural network and graph convolutional network can effectively mine the discriminative features of the data.These models can highly match the data,and are effective means of data analysis.As one of the important subjects in computer vision,human action recognition has wide application in security detection,video monitoring and regional security,and video sequences and/or human skeleton sequence based action recognition methods are the key issues.Among them,human skeleton sequence is a typical graph data,and its effective feature modeling can provide a good basis for graph data analysis.The traditional methods usually use the predefined features to classify the data,which usually suffers lower matching degree between model and data,and lack of effective feature modeling methods for complex actions.How to effectively model the features of the human skeleton sequences and mine the implicit information are the key scientific and technical issues in the action recognition,and are of important theoretical significance and application values.Based on the pattern recognition theory and deep learning technology,this thesis focuses on some problems in modeling the deep feature of human skeleton sequence,and puts forward the corresponding solutions.The main research results are as follows:1.For the poor robustness of the model,caused by the occlusion of human joint nodes,in the deep feature modeling of human skeleton sequences,a spatial region inactive network(SRIN)is proposed.This method firstly extracts human skeletons on the basis of video sequences,and constructs the spatial-temporal graph.Then,a spatial region inactive unit is designed to dropout the human joint node.Finally,a graph convolutional network is utilized to extract its features,and action recognition is performed based on these features.The proposed method can effectively solve the occlusion problem in the deep feature modeling of human skeleton sequences,thus enhancing the robustness.2.A variable interval temporal encoding network(VITEN)is proposed to tackle with the problem of poor ability of human skeleton action recognition model.First,a variable interval temporal sampling unit is designed to randomly sample the skeleton sequences.Then,a graph convolution network is utilized to extract the features of the sampled skeleton sequences.This method can simulate the velocity variation of action in real scene,and enable to effectively improve the recognition performance of action recognition model.Meanwhile,it also provides a solution to the over-fitting problem of graph data model,which is valuable for promotion.3.For solving the problem of the recognition performance of the human skeleton sequence action recognition model,restricted by the view change of the skeleton sequences,a multi-view transformation network(MVTN)is proposed.This method constructs a multi-view transformation unit to change the view of the human skeletons,and encodes its output using a graph neural network.The proposed method can effectively improve the recognition performance of the model,and effectively improve the generalization performance.The solution of the above key problems provide new ideas and methods for the graph convolutional network based graph data feature modeling as well as action recognition,and provides technical support for the study of deep feature mining of the graph data.