Research And Implementation Of 3D Human Joint Reconstruction Technology Based On Monocular Image

3D human model reconstruction has been an important research topic in the field of computer vision.The existing Internet public data contains a large number of human images,and the reconstruction of 3D human models can better understand and describe the human action,morphology,expression and other information in the images.Deep learningbased methods are widely used in the field of reconstructing 3D human models,however,there is a lack of public data sets containing full-body 3D human model labels,which brings a great obstacle to reconstructing full-body 3D human models based on monocular images.In this paper,we construct a dataset with whole-body 3D human model labels and propose an end-to-end whole-body 3D human model reconstruction algorithm.The existing publicly available datasets contain only body model labels,which cannot train end-to-end whole-body human model reconstruction algorithms.In this paper,we borrow the idea of fitting parametric models to construct a dataset with full-body 3D human model labels.First,clear human images in the HUMBI dataset are screened and the coordinates of whole-body key points are extracted based on the Open Pose algorithm;second,the SMPL model is mapped to the SMPLX model to obtain the SMPLX model with the same pose and body shape as the body model in the HUMBI dataset;finally,the key point projection of the obtained SMPLX model is fitted to the estimated whole-body key points to optimize hand movements and facial expressions to obtain the final full-body 3D human model labels.The accuracy results of the whole-body 3D human model constructed in this paper show that it can be used as a real label for end-to-end network training.In this paper,34851 pairs of data are finally constructed,and the dataset includes the original images,their corresponding whole-body keypoints coordinates and the whole-body 3D human model.It is an objective fact that hands and faces occupy a relatively small portion of a 2D human image,and in order to accommodate existing image feature extraction networks,hand and face detail information is lost after scaling the image to a smaller resolution by downsampling the network.Therefore,based on the above constructed dataset,a two-stage reconstruction algorithm for full-body 3D human model is proposed in this paper.Firstly,the body,hand and face images are cropped in the original image to ensure that all three parts have high resolution;secondly,feature extraction networks for body,hand and face are designed to extract the image features of body,hand and face respectively;finally,the features of the above three parts are cascaded and a joint reconstruction network is used to obtain a full-body 3D human model.Based on the constructed dataset for testing,the endto-end reconstructed 3D human model algorithm proposed in this paper is compared with existing methods,and the results show that the accuracy and algorithm operation efficiency of the whole-body human model reconstructed by this method are greatly improved.In addition,the body,hand and face feature extraction modules are also compared with the existing methods in this paper to demonstrate the effectiveness of each module respectively.