Research On Key Technologies Of 3D Human Body Reconstruction Based On Deep Learning

Digital human body reconstruction is the key basis for many applications such as virtual reality and augmented reality,and film special effects production.Existing high-precision acquisition equipment can realize high-fidelity digital human body reconstruction,but limited by the high equipment price,it is difficult to promote this type of equipment to the application scenarios of ordinary consumers.In contrast,monocular cameras are cheap and convenient.Therefore,using monocular visible light image input to realize digital human body reconstruction has become a research hotspot in the fields of computer vision and computer graphics.In particular,researchers have actively and effectively explored the regression of images into 3D human models using deep neural networks.However,the current image-based human body reconstruction technology still has the following two challenging problems: one is limited by the reliability of semantic segmentation.However,the 3D human body model reconstructed solely by the deep neural network is prone to incomplete geometric structures such as holes.The second is that the 3D human body model has a complex local geometric structure,and it is difficult to recover the local geometric structure of the human body with high quality by only using the features of the image to predict the 3D human body model.Therefore,starting from the above two problems,this thesis conducts research on the reconstruction of 3D human body models with complete geometric structures and complex local geometric structures based on human body image sequences.The main contents are as follows:1.Aiming at the problem of surface holes caused by multi-view fusion,this thesis proposes a dynamic human 3D reconstruction algorithm based on skin model.By defining the reference space,the algorithm generates the 3D human skin model in the corresponding posture from the pose parameters of the characters in the image sequence,and transforms it into the reference space according to the image information and the information of the human skin model.On this basis,the radiation field is constructed with the information in the reference space,and the neural radiation field is rendered as a 2D image.Finally,the phantom in the neural radiation field is optimized by minimizing the difference between the real image and the predicted image.The algorithm proposed in this thesis establishes the mapping from 2D image to 3D human body model by introducing a parametric human skin model,which effectively solves the phenomenon of holes on the surface of the human body model,thereby improving the integrity of the human body model.2.Aiming at the underfitting problem of complex local geometric structure in human body model reconstruction,this thesis proposes a 3D human body reconstruction algorithm based on multi-scale feature enhancement.According to the characteristics of the complex local geometric structure of the human body,the algorithm designs a multi-scale feature encoding module,extracts the features of the vertices of the human skin through the feature extraction network of different scales,and encodes the vertex features of the human skin at different scales.On this basis,the The vertices of the multi-scale fusion human skin model construct the dense input of the neural radiation field,thereby improving the prediction ability of the geometric neural rendering network for the complex local geometric structure of the human body.Compared with the human body reconstruction method based on image sequences,the algorithm proposed in this thesis improves the network’s ability to predict the local geometric structure through multi-scale local features,and effectively solves the problem of underfitting of the local geometric structure of the human body model.To sum up,this thesis proposes a dynamic human body 3D reconstruction algorithm based on a skin model and a 3D human body reconstruction algorithm based on multi-scale feature enhancement to address the current challenges.By introducing a parametric human skin model,the problem of constructing the dynamic human nerve radiation field is solved,and the structural integrity of the human body model reconstruction result is greatly improved.At the same time,the multi-scale feature enhancement network effectively improves the network’s perception of local features,and realizes the efficient fitting of complex local geometric structures of human body models.