Research On Frequency Domain Learning And Completion Method For 3D Face And Body Reconstruction From A Monocular Image

The use of a single image for three-dimensional(3D)face and body reconstruction eliminates the need for image alignment and fusion,making it more practical.However,compared to other body parts,the face has a relatively smaller area and range of motion,leading to its reconstruction being more susceptible to occlusions or lighting conditions.Thus,the independent reconstruction of the face and body can restore their respective detailed features and be better suited for various application scenarios.However,when there are significant occlusions on the face or the body is covered in loose clothing,the reconstruction results may not meet expectations.Therefore,this paper proposes a single-image-based 3D face frequency-domain learning method and further extends its approach to body reconstruction to improve the accuracy of 3D face and body reconstruction.This paper first proposes a weakly supervised spatial frequency decoupling method for facial reconstruction,which utilizes image-level and perceptual-level supervisory losses in the spatial domain and separates frequency information of the input image and rendered image in the frequency domain to construct a frequency-based loss function.Specifically,a spectral weighted wing loss function is designed to achieve balance on different frequency spectrums and better learning of the reconstruction process.Through weak supervision via spatial frequency decoupling,only facial landmarks are used to generate detailed texture and high-quality shape.This study conducted a comprehensive performance evaluation of five facial reconstruction algorithms on two commonly used datasets,AFLW2000 and Now Challenge.The evaluation of qualitative and quantitative indicators showed that our method performed well compared to the majority of facial reconstruction methods,producing high-quality facial reconstruction results.Using manifold harmonics,this paper extends the frequency domain research in3 D facial reconstruction to the body and proposes a frequency-based 3D body completion method.This method linearly interpolates the hole-filled 3D body obtained through normal integration and projects it onto a suitable dimension frequency domain space using manifold harmonic basis.The interpolated points are then replaced through Laplacian mesh editing to obtain a complete and detailed 3D body.Compared to other reconstruction methods,normal integration is not limited by the template body and is free from the accuracy constraints of marching cubes algorithms,resulting in more detailed features.By projecting the body into a low-dimensional frequency domain space for local completion,the method avoids the smoothing phenomenon caused by interpolation and the bulging phenomenon caused by Poisson reconstruction,effectively filling the hole regions while maintaining the consistency of the body mesh smoothness as much as possible.In summary,this paper proposes a weakly-supervised spatial-frequency decoupling method for face reconstruction and a frequency-domain-based 3D human completion method that realizes human body reconstruction and local completion via normal integration and manifold harmonic basis.Experimental results demonstrate that these methods can generate high-quality faces and bodies with more complete and detailed features.