Human Motion Modeling And Synthesis

Human motion modeling and synthesis involve two concepts,modeling and synthesis.Motion modeling is to perceive human motion from the environment,synthesis is to synthesize realistic human motion by computer,and the two complement each other.It is one of the hottest topics in computer vision and graphics research area and plays an important role in education,medical treatment,security,entertainment and so on.There are two main challenges: one is to model accurate and stable human motion,and the other is to synthesize realistic and control-lable motion.This paper focuses on human motion and addresses three problems: one is pose estimation in motion modeling,the second is dance synthesis in motion synthesis,and the third is motion retargeting in motion synthesis.Three novel methods are proposed,and the main contributions are as follows:1.For human pose estimation in color images,a Multi-scale Adaptive Structure Net-work(MASN)is proposed to predict human pose.The existing methods directly predict the heatmap of the keypoints with a fixed kernel size,without considering the information of the scale of human body and human structure.The adaptive heatmap is proposed to adaptively represent the keypoint information of different human scales,and the limb region is used to represent the human structure.To solve the problem that it is difficult to predict accurate key-points under complex postures and occlusions,the MASN first learns human body structure at multiple scales,and then predicts keypoints.This method effectively uses the human structure to guide and obtain accurate keypoints.Compared with existing methods,this method achieves better performance on two public datasets.2.For the dance motion in the three-dimensional human motion,a temporal convolution autoregressive model is proposed to synthesize the dance motion.The main difficulty of dance synthesis is the realism,diversity and controllability of dance.Due to the high spatio-temporal complexity of dance movements,existing methods are difficult to synthesize realistic and con-trollable dance.To solve this problem,the model needs to have strong modeling ability and robustness.Therefore,the dilated convolution is used to improve the receptive field,and the gated activation unit and separable convolution are adopted to increase the complexity of the model.Furthermore,the model output is modeled as a probability distribution,thereby further improving the robustness of the model.The experiment first evaluated the modeling ability and robustness of the model in locomotion,and achieved realistic and controllable motion synthe-sis.Then,the autoregressive model takes music style,rhythm and melody as control signals to synthesize dance.Compared with existing methods,this method can synthesize more realistic,diverse and controllable dance motion sequences(in keeping with the music style,rhythm,and melody).In addition,the autoregressive model is improved,and an explicit controllable signal is proposed to achieve more diverse and controllable dance synthesis.3.For motion retargeting between different skeletons,a framework based on recurrent neural network is proposed.The goal of motion retargeting is to transfer the motion of the source character to the target character and keep the content of the motion consistent.The difficulty is that the result of motion retargeting will appear unrealistic motion or abnormal pose when the skeletons of the two characters are quite different and the motion is more complicated.This paper proposes the motion increment model to learn the motion increment of the target character.This method abandons the framework of directly predicting the motion of target character in the existing methods,reduces the search space of the model,and improves the robustness of the model.Compared with existing methods,the motion of target character synthesized by this method is more realistic and has better performance.