3D Face Reconstruction And Its Application In Head Pose Estimation

Head pose estimation is one of the hot research contents of computer vision,and it has a wide range of applications in the fields of human-computer interaction and learner behavior analysis.Although many methods have reported good results on experimental datasets,the performance of these methods has dropped sharply in practical applications.The main reason is that most of the head pose estimation algorithms are trained in a supervised learning way,and they heavily rely on the objective head pose labels of the dataset.However,due to the difficulty of head pose labeling,there are large errors in the head pose labels of the existing dataset.To avoid the dependence of the head pose estimation method on the labels of the dataset,this paper proposes methods to reconstruct the 3D face and estimate head pose via the matching of the 3D face model and the 2D image facial feature points.The main content and contributions are as follows:1)To solve the problem that traditional methods based on morphable model cannot generate 3D faces with personalized geometric shapes and real textures,this paper proposes a face reconstruction method based on morphable model and deep learning.In this paper,a geometric hybrid model of "shape+expression" is constructed,and the facial feature points loss is proposed to train the network to automatically return face parameters from the face image,and finally a personalized geometric model of 3D face with expression is generated.This method has achieved the texture reconstruction error of 0.0413 on the AFLW2000-3D dataset,has improved the 3D face alignment accuracy by 2.8%,and the average cost of reconstructing time is only 27.5ms.2)To solve the problem that the inaccurate model prediction of the supervised head pose estimation method caused by inaccurate labels,this paper proposes a head pose estimation algorithm based on the matching of 3D face model and 2D image feature points.Based on the reconstruction of the 3D face in the research content 1),the 3D facial feature points in the model are extracted,and then the landmark detector is used to detect the 2D facial feature points from the image.Further,this paper constructs a spatial mapping model between 3D facial feature points and 2D facial feature points,and then obtains an unconstrained nonlinear optimization problem about feature points matching.Finally,this paper proposes an iterative optimization algorithm including two-stage solution to solve the optimization problem and calculate the head pose.This method achieved the lowest average absolute errors of 4.78° on Pointing'04 and 5.06°on Pandora,respectively.The combined average error did not exceed 6° on the five datasets,and the error standard deviation was less than 1.This paper proposes a method for reconstructing the 3D face model with the personalized geometric shape and the real texture.Compared with the existing methods,the quality of the reconstructed texture is well,the higher accuracy of 3D face alignment is achieved,and the average cost of time for reconstruction is 27.5ms.Based on the reconstructed 3D face model,this paper proposes a head pose estimation method and compares this method with other current state-of-the-art methods,via rich experimental comparison and analysis,on five widely used public datasets.The experimental results show that this method can achieve stable and good performance on a single dataset.In the cross-data set experiment,the average error does not exceed 6°,the error standard deviation does not exceed 1,and it is more robust.