Pose Invariant Face Recognition Based On Face Synthesis And Its Application In The Classroom

Face recognition is one of the most important research field in computer vision.As a direct,discriminative and non-contact biometric feature,face recognition has been widely used in security,electronic certification and transactions.Currently,face recognition in constrained environment has been well developed in research area and industry application.As the research moves along,main focus moves to face recognition in unconstrained environment which is much more common and significant.Unlike well controlled conditions in constrained face recognition,faces collected in unconstrained environment are affected by expressions,occlusion,illumination and poses.Among these,the variation of facial poses is the most direct factor that degrades performance of face recognition algorithm.With different poses,intra-class face variation can dramatically increase and sometimes it exceeds inter-class face variation which makes it hard to extract robust face features.The main focus of this paper is pose invariant face recognition algorithm based on face synthesis.By frontal face synthesis,we can alleviate the effect of pose variation and improve face recognition performance.For pose variation problem,with the aid of 3D face reconstruction,we propose frontal face synthesis network under the framework of Generative Adversarial Network(GAN)which learns from global and local structures of faces.Our main contributions are as follows:(1)We obtain frontal face projection with the help of 3D face reconstruction,dense alignment and weak perspective projection,which can provide global shape and local structure information of faces;we use adaptive encoding of faces based on head pose estimation from 3D face model;(2)We adopt the structure of GAN: the generator learns from global face and local structure details with face priors;we employ a group of discriminators on global face and local areas(eyes,nose and mouth)to improve texture details of synthesized faces;we employ WGAN-GP to help stabilize and accelerate the training process;(3)Face feature supervision network is introduced to push face features of synthesized faces towards the feature space of real faces.This helps a better visual result and it maintains identity preserving ability which can be directly applied into face recognition systems.The face synthesis and recognition experiments on face recognition datasets under constrained and unconstrianed environments show that our method can synthesize faces with decent visual effect and it helps improve the performance of the face recognition algorithm.Then we perform several model comparision experiments to demonstrate the validity of our method.Finally,We conduct a further study of face recognition in the classroom,we test our method in collected face datasets,analyze the potential demands and propose corresponding application design.