Nonlinear Parametric 3D Human Hand Model And Its Application

3D reconstruction has been a hot topic in the field of computer vision and computer graphics.The human hand is an essential organ for information interaction between hu-mans and the outside world.3D human hand reconstruction occupies an important posi-tion in applications such as augmented reality,virtual reality,and new human-computer interaction.The hand representation model is one of the core problems of 3D human hand reconstruction.The traditional articulated parametric model does not contain prior information of the human hand and has limited representation capabilities,unable to express high-quality human hands or complex hand gestures.To this end,our paper de-signs a novel 3D learning-based hand model-HandVAE,and applies it to the monocular RGB-D/RGB 3D hand reconstruction task.The main contents of this paper include the following items:(1)constructing large-scale high-quality 3D human hand mesh dataset;(2)3D learning-based hand model-HandVAE;(3)HandVAE-based monocular RGB-D/RGB 3D human hand reconstruction model.constructing large-scale high-quality 3D human hand mesh dataset:To reduce the training difficulty of the model and improve the quality of the model,we constructed a new large-scale,high-quality 3D hand data set based on the public hand mesh data set MANO.First,we select a reference mesh in the MANO dataset and use subdivision and remeshing approaches to improve its quality.Then,we use the high-quality reference mesh to fit the other meshes in the MANO dataset.Finally,we use interpolation for data enhancement.The dataset we constructed can be seen as an extension of the MANO data set.3D learning-based hand model-HandVAE:To learn the hand prior information contained in the data,use the VAE framework to design our hand model.At the same time,our novelty is the use of high-dimensional ACAP features that can better handle large-scale rotational deformation to represent 3D human hands.Finally,our model HandVAE learned the mapping of "feature to latent space" and "latent space to feature"of 3D human hands.The HandVAE model contains prior information of the hand data,has the power to generate high-quality hand mesh,is differentiable,and can compute efficiently.In the experiment,we compared it with the traditional articulated model MANO from three aspects of reconstruction tasks,and our model performed better.HandVAE-based monocular RGB-D/RGB 3D human hand reconstruction model:The basic idea of 3D human hand reconstruction is to search for the parame-ter in the hand model's space,which can reconstruct meshes that match the input data.We adopt the newly designed HandVAE model instead of the traditional model,design the neural network frameworks to regress the model parameters according to different forms of input data,and carefully designed a loss function to supervise the reconstructed mesh and input data.Finally,we constructed a monocular RGB-D/RGB hand recon-struction model.In the monocular RGB-D hand reconstruction experiment,our model has achieved good results on public data sets.