Research On Some Problems In Face Recognition In Uncontrolled Conditions

Face recognition is a classic problem in the fields such as image processing, pattern recognition, machine learning etc. Although many achievements have been made until now, the current face recognition has the following restrictions:(1) Face images are sampled in the controlled environment.(2) The voluntary co-operation from the recognized subject is required. These restrictions seriously hinder the development and popularization of face recognition. However, the demand of face recognition in the uncontrolled environment highly increases along with the increasing demand of the applications such as intelligent video analysis, face matching, face image search etc.However, under uncontrolled conditions, there are complex interference factors such as pose, illumination, expression, occlusion etc, which probably make the intra-person variations much larger than the inter-person variations between face images. This results in a big decline of face recognition accuracy and a failure to meet the requirements of practical applications. This thesis aims to reduce the impact of the complex interferences under uncontrolled conditions. The research is concentrated on and divided into four key steps including face image preprocessing, feature extraction, similarity measure as well as discrimination and classification in face recognition. According to the characteristics of face recognition tasks, the method of reducing the impact of the complex interferences is proposed specially for each key step. The main contributions in this work are:(1) A method of adaptive face verification based on pre-evaluation of face images is proposed. In this method, a pair of faces are first cropped into multiple face image pairs according to the predefined face regions. Then, by evaluating the image gradient differences on key points of the face pair, the image pairs on face regions are adaptively selected. Finally, the selected image pairs are combined to verify the face pair. To select image pairs, three methods for reliability evaluation of image pairs are proposed: the method based on abnormal difference detection, the method based on support vector regression and the method based on stacked auto-encoders deep network. They are suitable for the different cases requiring the different recognition speeds and recognition accuracies. Experiments show, for a face pair, their face regions, which are less affected by the complex interferences, are adaptively selected according to their visual interference conditions differences about occlusion, expression etc. Comparing with the method based on total face and the method of combining all face parts, it can effectively reduce the impact of local interferences.(2) A method of extracting high-level face features by learning feature pooling is proposed. Firstly, a novel algorithm of learning feature pooling is proposed. Aiming at that the information of face structures is not represented and local noise feature codes are not inhabited in the current methods of feature pooling, a novel pooling operation based on Sum-Pooling is defined by introducing two model parameters of pooling weight vector and linear transformation. And an alternate-iteration algorithm is proposed for learning the model parameters of the pooling operation. The high-level face features are learned by learning the pooling operation. Then, the algorithm of learning feature pooling is applied to the local features which are extracted by the sparse coding method. High-level face features are learned on the different sizes of face image blocks. Finally, the high-level features on all blocks are combined to form the high-level face feature representation.Experiments show the proposed method can extract face high-level features on different levels and effectively inhabit the noise feature codes. Moreover, the extracted feature vector is low-dimensional.(3) An algorithm of learning the adaptive distance metric based on visual condition difference is proposed. Firstly, the visual condition differences of a face pair about pose,expression, occlusion, illumination etc are computed by using the position relation of facial landmarks and their local features. The facial landmarks can be detected in face alignment. And the visual condition differences and Mahalanobis distance are combined to define a new distance metric in a feature space. Then, a distance metric learning algorithm is proposed to learn the defined distance metric. And the corresponding optimization problem is solved by the argumented Lagrange method. The optimization procedure learns the distance metric in the feature space. At the same time, it learns how the visual condition differences affect the distance metric and how to adjust the distance according to the visual condition differences. Experiments show, for a new face pair, the learned distance metric can adaptively adjust the distance from their feature vectors according to the visual condition differences between two faces. It effectively reduces the intra-class variations caused by the complex interferences.(4) A method of face verification by optimally training and organizing multiple classifiers is proposed. Firstly, a visual consistency measure method is presented. Then, after cropping the training face image pairs, by utilizing the above method, their sub-image pairs are optimally organized to form the training subsets on different face regions and in the different visual consistency conditions. Next, a binary SVM classifier is trained on each subset by fusing multiple face features. It preserves the classification rule for the specific face part in the specific visual consistency condition. In the verification phase,the classifiers are adaptively selected and optimally organized to verify a pair of faces according to their visual consistency measure results on the total face and face parts. Experiments show the method can adaptively adjust the classification and discriminant rules to determine whether two faces match according to the visual interference condition differences on their total faces and face parts. Thus it can effectively reduce the impact of the complex interferences.