Study On The Human Action Analysis Model Based On Multi-dimensional Relationship Mining

Human action analysis is the cross fusion of machine vision,pattern recognition,deep learning,artificial intelligence and other disciplines.It has good application prospects of content-based video retrieval,medical assistance,human-computer interaction,video monitoring and other fields.The human body can be seen as a joint system composed of rigid segments connected by joints.The posture of the human body can be described by the three-dimensional spatial position of the skeleton joint,and the action can be seen as the change process of the spatial position of the skeleton joint with time.With the continuous update of depth sensor technology,the action analysis based on human posture has been widely concerned.This dissertation starts with the body posture information and studies the difficult problems in the human action analysis based on video.The goal of this dissertation is the algorithm research of human action analysis based on multi-dimensional relationship mining.The technology of human action analysis has been widely concerned by researchers at home and abroad,and has obtained many excellent results,but there is still a large room for improvement in the performance of the algorithm.Based on the current research status of human action analysis,this dissertation carries out in-depth research,which involves four aspects of human action features extraction,action detection,action recognition and action prediction.The main contributions of this dissertation are summarized as follows:(1)Aiming at the extraction and representation of human action features,this dissertation proposes a multi-dimensional and multi-granularity action feature extraction method based on human skeleton joints,which is used to represent the spatio-temporal context relationship of human action.In order to transform the sparse skeleton joint location map into dense information suitable for neural network processing,this dissertation proposes two kinds of relational dense matrix between skeleton joints.Based on the two relational dense matrices,the spatial domain descriptors based on the relative positions of skeleton joints and the temporal domain descriptors based on dynamic attitude changes are proposed respectively.Then,the action features of the original location,time domain and space domain are divided into three granular levels of global,local and detailed action features according to the structure of human body.(2)In view of the problems of human action boundary detection,this dissertation proposes an action boundary detection model based on context relationship.According to the time relationship between the motions in the video frame sequence,the model introduces the attention mechanism to classify the role of each frame,namely,the start frame,the process frame,the end frame,and the blank frame.Then,the model rationalizes the action scope according to the context logic relation of the classification result,so as to locate the time range of the action and lay the foundation for the action recognition.(3)Aiming at action recognition task,this dissertation proposes an action recognition method based on hard sample mining.In this method,the dilated neural network is introduced to model the action features at multiple scales.Aiming at similar samples that are easy to be confused,this method introduces the mechanism of hard sample mining to enhance the learning pertinently,so as to optimize the result of action recognition.(4)Aiming at the problem of early recognition of human action,this dissertation proposes an action prediction method based on Generative Adversarial Nets for human action early recognition.The prediction model predicts the future motion sequence according to part of the motion sequence,and through the antagonistic learning of the real motion sequence in the future and the generated predicted motion sequence,the reliable and authentic future motion sequence can be finally generated.In order to further improve the accuracy of action recognition by predicting samples,a weakness relearning mechanism is introduced for the model,so as to provide more basis for the early recognition and optimize the accuracy.In view of the above problems in human action recognition based on skeleton joints,this dissertation proposes an effective solution.Experiments are carried out on challenging datasets to verify the effectiveness of the algorithm one by one.This dissertation summarizes the problems to be solved in the research and the next research direction.The dissertation has 70 figures,25 tables and 176 references.