Research On Human Motion Prediction Algorithms Based On Spatial And Temporal Features

With the development of computer technology,intelligent devices are widely used in industrial production and daily life.Human motion prediction is one of the critical technologies to enhance the intelligence of devices.It aims to capture the hidden temporal evolution relationship of human motions and predict the future motion according to historical motion sequences.Human motion prediction is a classical computer vision task applied in many scenarios,e.g.,autonomous driving,human-computer interaction,human tracking,and intelligent sports.However,the human movement is complex in many aspects,e.g.,high dimensionality,joint spatial cooperation,body hierarchy structures,and temporal evolution dependencies.Therefore,it is challenging to preciously capture the temporal dynamic information and spatial dependent features of human motions.Corresponding to the four characteristics of human motion,this paper explores the human motion prediction problem from four aspects:(1)Human temporal context encoding based on the smoothness and low dimensionality of joint trajectories,(2)spatial-dependent information capturing based on the coordination of human joints,(3)multi-scale spatial hierarchical information modeling based on the hierarchical nature of the human movement structure,(4)multiple temporal evolution pattern encoding based on the high temporal correlation of human motion.The main contents of this paper are as follows:1.Human motion prediction algorithm based on joint trajectories in phase spaceThe complicated and high dimensionality nature of human motion brings inherent challenges for dynamic context capturing.First,based on particle dynamics and human kinematics,in this paper,the model converts the complete human pose sequence into a series of joint trajectories,reducing the dimensionality and enhancing the temporal smoothness of the input information.Then,it is proposed to explicitly encode the motion state of the human body at each frame via the inter-frame displacement vector in phase space to improve the temporal continuity of the motion information and reduce the difficulty of prediction at the base level.Finally,local and global motion feature optimization modules were designed to assist the model in capturing related motion information from the sequences.Experimental results show that the proposed algorithm achieves accurate prediction of multiple types of movements,and the generated motion sequences are smoother and more harmonious.2.Human motion prediction algorithm based on semi-constrained spatial dependencies of jointsThere are abundant spatial coordination relationships of joints in human movement.The graph convolutional neural network has a powerful non-Euclidean structure data processing capability and can flexibly define and capture the various joint dependencies in human motion with multiple sub-adjacent matrices.First,following the prior knowledge of human anatomy and kinematics,the untrained sub-adjacent matrices encode the explicit joint spatial coordination relationships in human motion with constraints.Then,the trainable subadjacency matrix provides the flexibility to complement the implicit joint spatial coordination relationships from the input motion data.Based on the above,a semi-constrained human motion prediction model is established.The joint motion information is encoded according to mechanics,which describes human motion from the essence of motion.Due to the differences in joint range of motion and temporal position,a weighted loss function is further proposed to optimize the model.Experimental results show that the algorithm can significantly improve the prediction accuracy and the coordination of the predicted skeleton.3.Human motion prediction algorithm based on multi-scale structural hierarchical spatial featuresAccording to the hierarchical nature of human motion structure,this paper proposes an adaptive multi-scale spatial hierarchical feature extraction model.First,a multi-scale joint feature fusion module is designed to selectively extract information from different structural levels via connections to multi-level hierarchical joint groups,including joints,joint groups,kinetic chains,and kinetic trees.Then,a joint spatial dependent feature extraction function is proposed based on the principle of central joint optimality in the information aggregation process.By emphasizing the effect of the central joint in the information capture process,the crucial motion information is avoided from being overwritten by the correlated joint information aggregated.Experimental results show that this algorithm can significantly improve the nature of predicting inter-joint movements in human sequences and the stability of human structures.4.Human motion prediction algorithm based on a two-stage temporal-spatial information networkTo address highly temporal correlated feature modeling problems in human motion,a two-stage spatio-temporal information capture model based on convolutional neural network and recurrent neural network is proposed in this paper.In the first stage,a spatio-temporal graph convolution module is designed,which provides the model with the ability to encode spatially related features at the local sequence level.In the second stage,a multi-scale temporal feature encoding module is proposed to encode multiple temporal dependent patterns of motion information using a dense parallel multi-scale causal convolutional network.After the above two steps,the problem that traditional recurrent neural network have difficulty capturing long-range temporal dependencies and multiple temporal evolution patterns can be solved,and the ability of the algorithm to capture motion evolution information can be improved.Experimental results show that the algorithm can significantly improve predicted motion smoothness and long-term prediction performance.