3D Human Face Tracking Based On Monocular Video

Thanks to the development of artificial intelligence and machine learning techniques, as an important branch of moving target identification and tracking, the facial tracking technology is widely used in fields like human-computer interaction, virtual reality, visual surveillance and so on. Based on the study of the current situation and development trend of face recognition, aimed at the popular tracking method based on template matching and extremely inadequate, this paper proposes a tracking method based on the combination of SIFT features and template match, which is an effective solution for the two major deficiencies of the traditional template matching tracking method: first, the traditional template matching tracking method needs people to calibrate the camera before using it; second, if the inter-frame displacement is too large, the traditional template matching tracking method can not achieve good results. The majority work of this paper is elaborated as follows:First of all, in the conventional machine learning for face tracking process, face first initialization frame often requires manual labeling. To solve this problem, this paper presents an automatically initialize method for the first frame based on the deep learning. By establishing a stack of sparse self coding with neural networks, we caculate the identity of each node in the hidden layer and the back-propagation method using weights fine-tuning by using a large number of unlabeled samples. After the pre-training for the network, we connect the softmax classifier, and then using a small amount of sample which has been labeled to softmax classifier to do supervised training, so as to form a face tracking classifier which can automatically initialize the first frame. The result reveal that this method significantly improves face tracking in the first frame initialization efficiency.Secondly, to deal with the two limitations of the template matching tracking method, we propose a method based on the combination of the template matching and SIFT features.In order to overcome the first problem, we propose an iterative optimization process, from a few initial frames to estimate the camera unknown focus. In order to alleviate the second problem, we propose a tracking method, which is based on the supplemental information of the combination of the dense optical flow and SIFT features. Optical Flow works well for small displacements, and can provide accurate location information, and when dealing with larger displacement or transformation SIFT feature works better. The experiments take place in three public databases, which are BIWI head pose dataset, BU data sets, and McGill face data sets.The experimental result reveal that the proposed solution we proposed can reach more accurate results than relying solely on the tracking form deformation method or SIFT features. At the same time, because this method circumvented the camera calibration, it can be applied to a larger variety of scenarios, which makes it a more flexible solution compared with the conventional method.