Research On Video Based Face Tracking And Recognition

Video based face tracking and recognition is one of the key problems in computer vision. As it has a wide range of applications in video conference, human-computer interaction, judicature identification, video surveillance, and entrance controlling, etc. video based face tracking and recognition has got wide attentions of researchers. The goal of video based face tracking and recognition is to imitate the motion sensibility of human vision, empower the machine with the ability of perceiving the moving human faces in the image sequence, and provide an important data source for visual analysis and understanding. Video based face tracking and recognition often becomes very difficult due to the complex variations of human face and backgrounds. Although video based face tracking and recognition has been widely researched and many effective algorithms have been proposed, there are still a lot of difficulties in developing robust tracking and recognition algorithms when there exist variations such as illumination variation, facial expression variation, pose variation and partial occlusions.Video based face tracking and recognition system includes face tracking and recognition, and these two parts can influence each other. In this dissertation, the research is focused on how to improve the robustness and performance of face tracking and recognition. Generally, there are two categories in video based face tracking and recognition according to the types of the combination of data in gallery set and probe set. In the first category, data in gallery set are static face images while that in probe set are face video sequences. In the second category, data in gallery set and probe set are both video sequences. For the first category, the work is focused on how to integrate tracking and recognition into particle filter and update the appearance model during recognition. For the second category, the work is focused on improving the recognition performance in the case of expression variation, pose variation and illumination variation by robust feature extraction algorithm. At the same time, the work makes tracking and recognition module use the same appearance model to improve the recognition performance.The main contributions of this dissertation are summarized as follows:1. A visual face tracking and recognition algorithm based on adaptive feature sub-space has been proposed in the case that the data in the gallery set are static images while that in the probe set are video sequences. In this algorithm, the identification variable and the motion variable are combined into state variable. Particle filter is applied to simultaneously conduct tracking and recognition. The feature sub-space of the identified individual is updated during testing to improve the performance of tracking and recognition.2. Robust locality preserving projection algorithm based on robust statistic technique and path-based similarity measure has been proposed in this dissertation to reduce or eliminate the influence of noise and outliers. In robust locality preserving projection, a total connected graph is firstly constructed, in which a node corresponds to a data point. The similarity between any two data points is computed. Secondly, robust statistic technique is applied to estimate the weight of any data point, the smaller value of the weight for a data, the more likely it is a noise or outlier. Thirdly, similarity matrix is obtained according to path-based similarity measure and the weight of data point. The obtained robust path-based similarity matrix can reflect the genuine similarity between two data points even when noise or outlier exists. Finally, the obtained similarity matrix is utilized into locality preserving projection to compute the projection matrix. Experimental results show that our developed algorithm can improve the recognition performance.3. As class information is a key point for face recognition, a method named supervised robust locality preserving projection has been proposed in this dissertation. In this method, similarity between any two data points is firstly obtained by robust statistic technique and path-based similarity measure. Secondly, this similarity is adjusted according to the class information of the data points to obtain a new similarity matrix, this similarity matrix can reflect the geometric similarity and class information of training data points. Finally, the obtained similarity matrix is utilized into locality preserving projection to compute the projection matrix. Experimental results show that this method can get better recognition results.4. A algorithm named visual face tracking and recognition based on robust locality preserving projection has been proposed in this dissertation. This method is focused on how to improve the tracking and recognition performance when there are variations such as expression variation, pose variation etc. In this method, locally linear embedding algorithm is firstly utilized to embedding face images cropped from the training video into low dimensional space. K-means is applied in the low dimensional space to classify face images into clusters according to the pose and expression. Dynamics is also learned during this step. Secondly, robust locality preserving projection is applied in every cluster to obtain linear feature space to approximate the non-linear sub manifold. During testing, tracking and recognition module share the same appearance manifold. The appearance model of the identified object is used for the computation of tracking likelihood function. Particle filter is utilized to conduct tracking, while Bayesian reference is applied for recognition.According to the requirement of the project of"2010 Shanghai world expo special project", a software toolkit for face tracking and recognition has been developed.