Deep Learning Based Real-Time Pose Estimation And Human Animation Generation

Human pose estimation and character animation generation are two essential research contents in computer graphics.In order to create human animations in the virtual world,we take the images as input,extract the human poses,and converts them into an abstract pose data structure.Based on the obtained dataset,we generate the virtual character animation in the runtime based on the captured dataset and player input.With the development of deep learning technology,more and more researchers have begun to use deep neural networks to achieve this task.But there are still many challenges,for example,the accuracy and speed.In this thesis,we propose a new 3D human pose estimation and a GPU accelerated real-time human animation generation system based on deep learning technology.We focus on how to construct a reasonable neural network structure to achieve the goals of high performance and high accuracy and further elaborates on how to improve the requirements of high running speed.And the animation generation part performs specific GPU-based acceleration to achieve real-time speed.The main innovations of this thesis contain two new neural network structures designed for 2D and 3D pose estimation,and one corresponding GPU-based acceleration system.Finally,we compare our systems with related works and confirm the practical effect of the design described in this thesis.In summary,the main work of this article is as follows:1.Real-time 2D pose recognition system based on deep convolutional neural network.The system estimates the poses of multiple people in the input image and output 2D marked points.The entire system achieves real-time running speed on mainstream GPUs with a resolution of 384.2.Based on the design idea of the 2D pose recognition system,a 3D pose estimation system based on a single-purpose is further designed.Thus,we challenge the traditional concept that monocular 3D pose estimation is tough.And our 3D pose estimation system also achieves real-time running speed at a resolution of 384.3.Based on the 3D pose data obtained from the 3D pose estimation,the neural network is used to generate the animation of the character.In order to achieve real-time operating speed,through the analysis of the network structure and hardware architecture,we do the GPU-based acceleration of the network structure,which is initially targeted for the CPU.Our work proves that the character animation generation system based on a deep neural network can run on consumer hardware.Further,we tested the system on the pose recognition data set and different levels of hardware.The 2D pose detection system reaches speeds above 60 fps on mainstream consumer hardware,and the 3D pose estimation system reaches speeds above 24 fps with only 110 mm average error.The real-time animation generation system can reach speeds above 30 fps.The experimental results show that the deep learning-based pose recognition and animation generation system described in this thesis has achieved the set speed and accuracy goals,proving the great potential of deep learning technology in the field of computer graphics.