| With the increasing level of science and technology,emerging concepts and technologies such as Virtual Reality,Augmented Reality,and Metaverse have been continuously proposed and developed rapidly.Three-dimensional face is one of the indispensable elements in these frontier technologies.3D face technology has attracted much attention due to its great development potential and growing demand.Face is the most important part of self-expression.3D faces can express more information than two-dimensional face,and has a broader application prospect.3D face reconstruction and generation are two core tasks in 3D face technology.The goal of reconstruction is to restore the real face as much as possible,which is a prerequisite for many computer vision tasks,such as face recognition and face migration.A 3D face is a non-rigid 3D object.Improving the quality of reconstruction is conducive to many computer vision tasks.The virtual face generation is to obtain the face that does not exist in reality.It plays an important role in expanding the threedimensional face database and the use of copyrightless three-dimensional faces.Both tasks are expected to improve the quality and efficiency of 3D face.However,3D face is a non-rigid 3D object which is very difficult to describe.It is a challenging task for its reconstruction and generation.Reconstructing high quality face models is often accompanied by high-cost device,and harsh experimental environments.Some prior model methods based on two-dimensional images can avoid these problems,but they often require huge computational costs,cannot achieve real-time reconstruction,and the diversity of results will be limited.In the generation task,the neural network-based 2D face generation technology has been widely used,but it is not yet very mature in 3D face generation.A generative network based on the internal structure of objects has been proposed,but it is difficult to adapt to the human face due to its non-rigidity.How to combine face data with neural network is one of the problems faced in the current stage of face generation.In order to solve the above problems,this paper has done the following work around 3D face reconstruction and generation:· For the face reconstruction task,a 3D face reconstruction optimization strategy based on RGB-D images is proposed.Using Kinect to capture color images and depth images of large scene in real time.The color map is used for face detection and landmarks detection,then maps the coordinates to the depth map.Then The face is extracted and divided into five key regions.Different optimization strategies are adopted according to the characteristics of different partitions.The optimization algorithm(least squares smoothing)with high smoothing strength is chosen in the smoother areas,and the optimization algorithm(bilateral smoothing)with slightly weaker and edge preservation is used in the areas where more details need to be preserved.In this way,the purpose of eliminating noise while preserving face details is achieved.Experiments were conducted on seven expressions of four experimenters.The experimental results show that the strategy is efficient in obtaining high-quality face models at low cost.· For the virtual face generation task,two 3D face sampling methods adapted to existing generative networks are proposed,the radial ray and geodesic method,and the depth-like image method.Firstly,the results of the previous work are used to collect 3D face data in reality,so as to expand the face data set.Then resample the 3D face data on the expanded database.The sampled face not only retains the basic features of the original face,but also makes the scattered and unstructured point cloud into a matrix form that can be input into the generative adversarial networks.Combined with DCGAN network for virtual 3D face generation task.Seven different expressions are generated and the results are optimized to obtain the final virtual 3D generated face model.And the generated face result is also in the form of a structured matrix.The sampling method in this paper can turn unstructured data into structured data without destroying the face representation.and it does not need to modify the internal structure of the existing mature neural network.It is an idea that can directly apply the two-dimensional neural network results to the threedimensional face generation. |
