Research On 3D Face Region Labeling And Pose Estimation

At present,there are many researches and applications on 2D face and 3D face at home and abroad.Most of the research and application are based on the positive head pose and have a good performance.However,when the faces are in other sides like side of the head,the performance needs to be improved.For the research of human face on the side face pose,we must use the 3D face pose estimation.In modern society,the application of 3D face pose estimation is everywhere.It has wide application and huge market demand in many fields such as face recognition system,intelligent security monitoring system,safety assistant driving system,film and television animation,game and virtual reality.In recent years,the problem of semantic markup of the scene has gained wide attention and research,and has achieved good results.However,there are few studies on semantic markup on 3D human faces.According to the idea of semantic markings of the scene,the task of the 3D face region semantic markup can be described as marking the eye,nose,mouth and other areas on the face model.3D face region labeling can improve the 3D face reconstruction,3D face pose estimation,facial landmark estimation,and others,and also can promote the development of 3D face animation.So it is very necessary to study the problem of 3D face region labeling.At present,the 2D face-related technology based on image is mature gradually.Compared to the 2D research of face based on image,the 3D face data has the characteristics of being affected by illumination,pose and etc.,besides it contains the structural information that the 2D image data does not have.In recent years,academics have begun to explore how to use the 3D facial data to improve the performance of system.With the continuous renew and improvement of sensor,3D laser scanner and other hardware devices,it is more convenient for us to get 3D face data including depth info rmation.Taking full advantage of the characteristics of 3D data is one of the key factors in solving many problems of 3D face.We have done further research of 2D information of 3D face,including geometric information descriptors and the depth map,and use the strong fitting ability of machine learning and deep learning algorithm to study 3D face region labeling and pose estimation.What's more,we defined the research method on the basis of combing with the current machine learning methods in the classic random forest and popular depth learning algorithm.The main research of this paper can be listed as follows:(1)Aiming at the problem that the importance of 3D geometric local descriptor importance and the lack of the research on 3D face region labeling,a new 3D face model geometric local feature descriptor for this problem was p roposed in this paper: square tangent plane descriptor.This descriptor consists of the average of the projection distance from the 3D face model vertices to the square tangent plane.Firstly,we calculate a square tangent square area of vertices,and the square is divided into small squares.Then,the other vertices on the model are projected to the square tangent plane,and we calculate the average of the projection distance of the projection point within each small square.The descriptor is encoded by the average distances of each square.The descriptor has good robustness to the noise,model size and model resolution of the 3D face model.Finally,the proposed square tangent plane descriptor is applied to the problem on 3D face region labeling.Using the strong learning ability of the random forest algorithm to effectively label the eyebrows,eyes,nose,mouth and cheek region of 3D face model.(2)A method of three-dimensional face model pose estimation based on depth map and convolution neural network is proposed.Firstly,the depth map data set is obtained by projecting the collected 3D face model data.The depth map data set was used as the training data of the convolution neural network.After obtained the trained model,conducted face pose estimation to three different sources of three-dimensional face model respectively and predicted the angle of the Pitch direction,the Yaw direction,and the Roll direction of the 3D face model.They received good prediction results.