Real-time Detection And Recognition Of Human Behavior And Reconstruction Of Non-Rigid Human Body

Image-based non-contact human body posture,behavior,and shape measurement are key issues in human-machine and environmental engineering,and are necessary conditions for the machine to deeply understand the meaning of human behavior and understand the changes in environmental appearance.This paper starts with the human body posture detection and tracking,human behavior detection and identification,human body dense reconstruction and related nonlinear optimization problems in human-machine and environmental engineering.The human body attitude measurement framework,joint personnel and behavior based on dense key point detection are designed and designed.Detecting the deep convolutional neural network framework,the real-time human body dense reconstruction framework for joint joint motion and embedded motion,the matrix data structure for GPU parallel acceleration,and the specific compression acceleration strategy for deep neural networks.The above algorithm framework has been applied successively.In the "921" project Tiangong 2 on-orbit astronaut human body attitude measurement project,group personnel behavior detection and identification project,real-time multi-person non-rigid dense reconstruction project,provide accurate and efficient measurement algorithm framework for each project,and in the corresponding actual engineering Good results have been achieved in the application.The main results achieved in this paper are as follows:1.A human body attitude measurement framework based on dense key point detection is proposed.The key problem of human body attitude measurement is to obtain the correspondence between model points and image points,and then the human body attitude parameters can be obtained by using explicit model parameter optimization techniques.The traditional correspondence needs to obtain a good initial orientation,and the robustness is poor,which is prone to tracking errors.The existing key detection algorithms based on deep learning are slow,difficult to apply in real time,and can only be used for specific joint positions.Testing is difficult to cope with model deformation with topological transformation.In this context,this paper develops a dense keypoint detection algorithm based on deep convolutional neural network,which can detect dense key points in multiple images in real time,and can cope with human body topological deformation.Based on this algorithm,the design of the human body attitude measurement project of the orbital astronaut,the human body posture measurement project in the surgical operation,and the large-scale human body posture measurement project of the ski jumper were carried out.The in-orbiting astronaut human body attitude measurement program designed in this paper was carried out in the in-orbit experiment with Tiangong No.2 in November 2016.It obtained the in-orbit human body attitude measurement data for the first time in China,and carried out on-orbit long-term for China's manned spaceflight project.Residing provides important underlying data.2.A framework for real-time detection and recognition algorithm for joint personnel behavior is proposed.Personnel detection and recognition and behavior detection in group scenes are the basis for the machine to understand the meaning of behavior in human-machine and environmental engineering.The traditional personnel detection and recognition algorithm is mainly for simple scenes and single-person recognition.It is difficult to deal with mass and complex application scenarios.The traditional behavior detection and recognition algorithm is mainly for the whole video and single-person behavior,which is difficult to be used for each frame.The image is identified by atomic behavior,which makes it more difficult to handle multi-person application scenarios and the recognition of one person's multiple behaviors.In this context,this paper develops a neural network algorithm framework based on end-to-end joint personnel behavior detection and recognition.It can perform personnel identification and behavior recognition on the basis of detecting all personnel in the scene to realize atomic behavior recognition.Corresponds to multiple behavioral actions.On this basis,the program design of the student classroom behavior statistical analysis project in K12 education was carried out,and the embedded front-end hardware development was carried out,and the field project test was carried out in the elementary school classroom.3.A real-time non-rigid human dense reconstruction algorithm framework is proposed.Dense reconstruction of non-rigid scenes is a necessary prerequisite for deep understanding of the human body and scenes in human-machine and environmental engineering.Most of the traditional scene reconstruction algorithms are offline calculations,and can only be reconstructed for rigid scenes.In this context,this paper studies real-time non-rigid multi-person scene reconstruction algorithms,which can estimate real-time multi-person non-rigid motion fields.High-precision scene reconstruction is achieved by high-precision fusion of sequence non-rigid depth images.4.Accelerated strategy design for large,medium and small scale nonlinear optimization problemsMost problems in man-machine and environmental engineering can be transformed into nonlinear optimization problems.The existing nonlinear optimization solution strategy is relatively slow to solve,and it is difficult to deal with real-time applications.In this context,this paper designs a sparse matrix storage method suitable for GPU parallel acceleration solving for small and medium-scale nonlinear optimization problems,which can effectively solve the problem of non-rigid human dense motion field estimation in real time.Secondly,this paper aims at neural network.In the large-scale nonlinear optimization problem,the network compression acceleration research makes the target detection and identification network parameters involved in this paper can be greatly compressed and accelerate the network operation.In this paper,human body attitude measurement,human behavior detection and recognition,non-rigid human scene reconstruction,and nonlinear optimization acceleration algorithm are studied in the human-machine and environmental engineering,and on this basis,for aerospace,medical,sports,The corresponding program design,experiment and project practice of education,social and other applications fully prove the validity and practicability of the algorithm framework of this paper.