Research On Face Recognition Under Unconstrained Conditions

Face recognition technology has been already widely used in human society such as security systems,corporation manufacturing IT products,personal computers and smart phones.The computer software which is able to recognize faces is very important for the existence of the human society,as it makes the work of various services safer,easier and more efficient.Over the past recent years,numerous methods have been proposed to address the task of face recognition,which have shown computer vision capabilities to surpass those of humans in ideal environments(constrained coditions).Rather than signaling the end of face recognition research,these results have led to a redefinition of the problem,shifting attention from highly regulated,controlled image settings to faces captured under unconstrained conditions.Unconstrained face conditions in particular have often been considered by designing representations that pool information from face image in-the-real-world,thereby accounting for possible misalignments due to pose changes,and image conetent loss due to image blur,noise,low resolution and bad illumination.Such challenging factors greatly degenerate the performance of face recognition,which still needs further research.In addition to the improvement of face recognition performance,a face recognition system needs to identify whether the captured face is from a real person in front of a camera or from recaptured image or video,for more secure identity authentication.Therefore,face anti-spoofing is a new but very important component for a face recognition system during the recognition process.To improve the accuracy,robustness as well as the safety of a face recognition system,this dissertation mainly focuses on the methodologies of face image enhancement,facial landmark detection,facial pose normalization and deep anti-spoofing feature learning,with the following contributions:1.To address the issues that raised by bad illumination on a face image,we have proposed a novel energy minimization based illumination normalization algorithm with the idea of correction on large scale components in Chapter 3.The Logarithmic Total Variation(LTV)technique is applied to decompose the large-and small-scale components for a face image,hence obtaining the small-scale component which mainly contains facial intrinsic features,and the large-scale component that mainly includes illumination field.By minimizing the LTV-based energy function,the illumination bias field could be estimated,which can further be removed for better face recognition.Different with the traditional illumination normalization methods that aim at extracting illumination insensitive features for face recognition,our method is the first attempt to estimate the illumination field for a given face image.Extensive experiments on the popular face illumination databases CMU-PIE,Extended Yale B and CAS-PEAL-R1 have demonstrated that the the proposed method significantly improves the visualization results as well as recognition accuracy for the illumination effected face images.2.Large facial pose and occlusion are challenging issues that may fail facial landmark detection.To address these issues,we have proposed a novel 2D-assisted self-supervised learning method for 3D face reconstruction in Chapter 4.This thesis builds our model by the strong supervision based on landmark self-mapping and the weak supervision based on the generative adversarial learning.Our method is the first attempt to effectively use the unconstrained 2D face images without 3D annotation to improve 3D face model learning.The fitted 3D face model can be used to generate dense 3D facial landmarks.Different with the traditional 2D facial landmarks,the 3D dense landmarks contain rich information of 3D face shape,which can be better used for face alignment to obtain higher accuracy for facial landmark detection.Extensive experiments on the popular 3D face evaluation databases AFLW2000-3D,AFLW-LFPA and Florence have demonstrated that the proposed method achieves promising results in both 3D face reconstruction and 3D facial landmark detection.3.In Chapter 5,we have proposed a novel method for face restoration with arbitrary facial poses and low-quality factors based on deep adversarial learning,aiming to simultaneously address the issues of large facial pose and low quality factor for face recognition.The low quality contains modalities such as low-resolution,bad illumination,noise and blur.Given a face image with arbitrary facial pose and low-quality modality,our model first performs 3D face reconstruction,then automatically rotating facial pose to absolutely frontal in the standard 3D space.The obtained absolutely frontal facial pose is used to guide face frontalization by the well designed high-quality face restoration adversarial networks,to restore frontalized high-quality faces from the given low quality ones under arbitrary facial poses.Our model is able to restore the frontalized high-quality face from the given face image,and meanwhile significantly improving face recognition accuracy.Exten- sive experiments on the three databases CMU Multi-PIE?LFW and IJB-C have demonstrated that the proposed method can significantly improve the performance on both face frontalization and face enhancement.4.In Chapter 6,we focus on secure face authentication with two novel methods proposed.Specifically,this thesis has proposed a deep domain transfer Convolutional Neural Network(CNN)using sparsely unlabeled data from the target domain to learn features that are invariant across domains for face anti-spoofing.Experimental results have shown that this method significantly improves the cross-domain performance of face anti-spoofing only with a small number of unlabeled samples from the target domain.In addition,this thesis has proposed a deep multi-mask based face anti-spoofing method,where face recognition and face anti-spoofing are combined into a single unified framework for the first time.The Total Pairwise Confusion(TPC)loss and Fast Domain Adaptation(FDA)are proposed to enhance the generaliability of face anti-spoofing.Extensive experiments on the popular face anti-spoofing databases CASIA-FASD,Replay-Attack,MSU-MFSD,Oulu-NPU and Si W have demonstrated that the proposed method could obtain satisfying face recognition performance as well as enhance the generaliability for the face anti-spooifng model,therefore making face recognition system more efficient.