Research On Facial Features Detection And Tracking

We present the novel methods to detect and track facial features in video. The proposed techniques are very useful for electronic entertainment, biometric security system and intelligent user interface. The electronic entertainment industry needs our techniques to generate realistic facial animations from video. The facial animations are used to be manually crafted by artists. Some facial tracking systems have been delivered to synthesize facial animations automatically. But these tracking systems need multiple cameras to capture videos and require people to attach hundreds of markers to their faces which make them inconvenient and expensive. The novel algorithms in this paper can also be used to face and facial expression recognition systems.To detect and track facial features from video are challenging topics in computer vision. It's very hard to model the facial features because they have various apperances for different poses and facial expressions. And the uneven lightings and occlusions in vidoes make them hard to detect and track. So the previous techniques usually require even lightings, frontal and neutral-expression faces in video which prevent them from practical usage. We present some novel methods in this paper which are faster and more robust. Our main contributions are:1. We proposed a robust facial features detection algorithmA novel technique called Directional Projection (DP) is presented to automatically locate features on the human face. DP extends the original Projection methods in three ways: (i) a PCA based face gesture recognition and compensation method is presented; (iii) an automatic threshold and noisy removal method is developed. It outperforms the original projection methods in the occasion where face has planar rotation. It also shows great speed superior to the methods based on nonlinear optimization.2. We unified the face detection and facial features detection proceduresThe techniques for face detection and facial features detection have evolved separately with little overlap. However our brains seem to process the different vision tasks in the interactive and cooperative ways. So we present a pair of new methods for facial landmarks detection. The can share the mutual features with face detection and finds the landmarks by regression or Bayesian probabilistic model. They are fast and slim because of the shared computing. Its accuracy is also promising. Additionally we show the application of the method for human body features detection. The proposed methods are attempts to model the back-projection pathway and the interactions among different vision tasks.3. We developed a smooth facial tracking algorithmFacial Action Units (FAU) tracking is a hard problem for the rigid and non-rigid transformations of human face. The constantly trembling in the tracking result and the tracking failures caused by the absence of constraint remain open problems. This paper presents a novel method to attack these problems by combining the nonlinear data reduction and tracking constraining techuniques. The new method can track facial expressions more smooth and accurate compared with the state of the arts algorithms.4. We proposed a facial tracking method which can recover from errorsA nonlinear data reduction process with limited training examples usually result in a latent variables space with some unpleasant disconnections which may cause a tracking procedure fail at local minimums. This paper presents a novel method to resolve the problem. First the latent variables space is divided by cluster analysis. Second a probabilistic model is formulated to predict the likehood of a sub-space given a tracking error. Third a biased long-range sampling method is consolidated to prevent the tracking procedure from local minimum. The biased jumping in the tracking process seems a consequence of some virtual tunnels that connected the different areas in the latent space. So the new method is named as Tunneled Latent Variables method. It was used to track facial action units in video and the experiment results are encouraging.We have implemented a facial animation system based on the techniques shown in this paper. Our system can track facial expression and generate virtual facial animations with one camera and without any marker.