Human Pose Synthesis Based On Two-stream Deep Neural Network

In recent years,the application of virtual reality and human-computer interaction has risen sharply,and artificial intelligence algorithms for human joint posture have become an important topic in the field of computer vision.Human pose synthesis is a task of extracting the human motion pattern by observing the human motion sequences,and deriving the realistic poses of other actions.In the current research of human pose synthesis using deep neural network,how to design a reasonable network structure to meet the fusion of multi-modal spatial-temporal coupling features and how to accurately model the dynamic pattern and static pattern of different joint motions have become key issues.Aiming at the above issues,this study is carried out from the aspects of neural network depth,residual connection structure,multi-modal utilization and fusion,etc.The main achievements are as follows:1.A spatial-temporal two-stream network based on temporal consistency(TCNet)is proposed,which solves the problem of spatialtemporal derangement in the fusion process of a two-stream network.Aiming at the problem of weak spatial-temporal coupling in the traditional fusion method of two-stream network,a spatial-temporal coupling reinforcement module is constructed in the fusion process,and the fused feature is re-modeled to strengthen the spatial-temporal co-occurrence modeling of the network.Besides,TCNet effectively combines the advantages of velocity information and position information in short-term and long-term modeling respectively.Experiments show that TCNet has achieved excellent results on H3.6M,CMU-Mocap and 3DPW public datasets.2.A symmetric residual connection structure suitable for human pose synthesis is proposed,which enriches the feature scales in the process of motion pattern modeling and alleviates the gradient disappearance.The designed symmetric residual connection module maximizes the convolutional receptive field.Experiments show that the proposed symmetric residual network(SRNet)improves the utilization of shallow dynamic features,making the generated human pose more accurate by fewer parameters.3.A dynamic pattern-guided multigraph network(DP-MGnet)is proposed to collaboratively model joint dynamic features within similar dynamic pattern,which alleviates the pattern involvement.The network utilized a temporal reinforcement module,which enhances the features of key frames.Experiments shows that the dynamic features of the joints at various scales are extracted,which effectively preserves the motion pattern of the global j oint points,and finally achieves the precise generation of the human pose sequences.