Research On 3D Face Feature Extracting And Pose Tracking

3D face pose tracking is a major research hotspot in the field of human-computer interaction and computer vision, and is also a research direction which extracting an increasing amount of attention in recent years. It refers to the process of calculating a face's pose in 3D dimension according to a sequence of human face image captured from a camera or video.The rotation of the face is usually divided into three types: left-right(yaw), up-down(pitch) and in-plane rotation(roll).Based on the calculating of the three rotation parameters, the common method to face pose estimation can be divided into two categories: the method based on face feature and the method based on a model. This paper chooses the method based on a model to solve the problem of face pose tracking. The method based on a model usually assumes that the face is a rigid object. It calculates face pose parameters based on the feature points matching between consecutive frames. This kind of method is easy to implement and has higher tracking precision.According to the tough situation that only possessing a video image sequence captured from a monocular camera, this paper introduces the general 3D face mesh model. The tracking framework mainly involves three aspects. The first part is the human face feature extraction and tracking. Based on a good study of the feature extraction methods, this paper choose the ASM and SIFT feature extraction method, which respectively are used to complete the initialization and inter-frame estimation of the system. In addition, this paper also presents a fast feature extraction algorithm, namely principal component SIFTS features descriptor PC-SIFT, which can significantly improve the efficiency of feature extraction and quality. The second part is the inter-frame motion inference algorithm. In this paper, we creatively use the space geometry method to calculate the depth of the face feature information, and then put them into the POSIT estimation method algorithm within the framework of the random sampling consensus(RANSAC). The third part is dealing with the robustness.of tracking system. In order to eliminate the cumulative error appeared in the process of continuous tracking, this paper meet both local and global aspect demands. From local side, both the key frames and the adjacent frames are merged in the inter-frame pose estimation part; from the global aspect, designs a dynamic appearance-based model to eliminate error. In the process of the implementation of the whole system, we happen to obtain the adaptive automatic human face detection algorithm, adding the real-time performance and robustness of the system. Finally, the use of MATLAB and C implementation of a face pose estimation and tracking system, so as to verify the feasibility of this algorithm.