Illumination & Pose Invariant Unseen Face Verification Based On Metric Learning

In this paper we focused on the problem of unseen face verification under unconstrained environment. Unconstrained environment refers to the purpose to build an illumination and face pose invariant recognition algorithm. Unseen face verification refers to the problem definition that given two face photos of a person never seen in training set, and make the judgement whether the two photos are of the same never-seen person. This is also known as the’unseen pair matching’problem.The academic value of this problem is that, solving the unseen face verification means being able to solve other face recognition problems. E.g. for a suspect identification system it’s feasible to translate the suspect identification problem into the problem of unseen face verification between the suspect and all the criminal photos within the database with full identities.The application value of this problem is that, given the ability to recognize unseen faces under unconstrained environment, i. e. in a pose and illumination invariant way, the potential market of face recognition can be broadly broadened with more application robustness. E.g. under smart TV home environment, if the smart TV can verify a guest from the second time it passes the TV, without further bothering the guest, then it’11 create a more natural user interface between TV and human. The chararistics for solving this problem is, given that persons from upcoming test set would have never appeared in the training set, and there are so many of them which far exceeds the number of training set, then it’11 not be feasible and realistic to do modeling for any specific person and thus have to compare directly between the two images of faces, i.e. have to learn a’ruler’that can measure the difference between human faces. This idea is called metric learning for face verification.Difficulties of tackling this problem lies in:First, to learn and extract common features of all human faces effectively when training. Second, to ignore similarities between different person when testing. Third, to keep robustness to variations of the same person in different photos. And forth, to do the former 3 points well under an unconstrained environment.To design the algorithm and solve the proposed problem, we applied unsupervised image alignment method called congealing based on SIFT feature, to lower down the negative effect of faces’poses on verification accuracy. Following that, again we computed SIFT feature at every pixel and leveraged PCA and GMM to do the dimension reduction work, in order to compute the Mahalanobis matrix and its corresponding decomposed matrix which is used as a low-rank subspace projection matrix to further lowering down the number of dimensions of the feature. Finally linear SVM is leveraged to complete the verification process classifying the difference between two samples’ compressed feature vector. Experimental results outperformed baseline algorithm[75], though still have room to optimize when compared with state-of-the-art results.