Study On Domain Invariant Feature Learning For Heterogeneous Face Recognition

As one of the significant applications of artificial intelligence,face recognition has considerable practical and research value.In recent years,deep convolution neural network-based face recognition has made rapid development.However,it mainly focuses on 2D visible face recognition,which still faces many challenges and problems in real-world unconstrained scenes.With the development of sensor imaging technology and the popularization of face recognition applications,the demand for heterogeneous face recognition(HFR)is growing.However,the apparent cross-modality appearance changes make existing face recognition systems suffer from severe recognition accuracy reductions.In this paper,the HFR problem and its solutions are analyzed,and domain invariant feature learning methods for heterogeneous face images are deeply studied from the three directions: feature representation-based learning,subspacebased learning,and hybrid-based learning.Besides,a 3D-2D face recognition database under unconstrained outdoor scenes is constructed.The main work and contributions of this thesis are included as:1.Investigated the problem of lacking research framework models,and proposed an HFR framework model.Lacking explicit research framework models of HFR,it is easy to cause the understanding of the motivations,associations,and classification of HFR methods are not intuitive enough or even confusing.For this problem,this paper makes preliminary exploration and proposes an HFR research framework model.This model contributes to the understanding and classifying existing HFR methods and provides theoretical support and inspirations for developing new techniques,having instructive significance and effect.2.Studied the measurement and elimination problem of cross-modality differences,and proposed an adversarial domain-invariant feature learning method DIDF.The significant cross-domain difference is a critical issue in HFR.To this end,from the direction of feature representation learning,this paper focuses on the feature extraction process in the HFR framework model and proposes an adversarial domain invariant feature learning method.It optimizes adaptive domain alignment based on adversarial learning and class alignment based on quadruplet metric learning in an end-to-end network,eliminating the distribution discrepancy,reducing the intra-class variation,and enlarging the inter-class separability.Experiments on four(CASIA NIR-VIS 2.0,Oulu CASIA NIR & VIS,BUAA NIR-VIS,and IIIT-D Viewed Sketch)HFR benchmark databases show that the proposed method helps solve the distribution differences and improves the domain invariance and class(identity)discrimination of facial features.3.Researched the problem of identity-irrelevant factors such as the cross-modality difference,etc.,to face recognition,and presented an attention-guided feature disentangling method AgFD.In addition to cross-domain differences,other external factors,such as changes in pose,age,etc.,will also affect the FR results.To solve this problem,this paper studies the feature matching process in the HFR framework model from the direction of subspace learning and presents an attention-guided feature disentangling method.It adaptively decouples the facial representation into identity features and identity-irrelevant features(including modality and all other identity-irrelevant information)in a hierarchical complementary manner.Simultaneously,mutual information-based and total correlation-based adversarial de-correlation learning are introduced to improve the robustness of identity features to all identity-irrelevant factor variations(such as the visual domain,etc.)and the robustness of identity features to themself local dimensional changes.Experimental results demonstrate the recognition accuracy advantages of the proposed method on multiple HFR benchmark databases.4.Realized DIDF-based and DIDF-based 3D-2D face recognition methods,both outperforms existing methods on the public database.For the real-world outdoor unconstrained 3D-2D data lack and recognition problems,constructed a 3D-2D face database WS3D-2D,and proposed an improved feature decoupling-based method FD-3D2 D.Firstly,from the respect view of hybrid learning,this paper studies multiple processes in the HFR framework model and realizes two 3D-2D face recognition methods based on DIDF and AgFD,which outperform existing methods on the public FRGC V2.0 database.Secondly,a 3D-2D HFR database WS3D-2D is collected to solve 3D-2D face data-scarce and mostly captured in the controlled laboratory environment.This database contains high-precision fullface 3D models and 2D surveillance face images from unconstrained outdoor scenes,which are very close to the data in practical applications.So,this database has important research significance and practical value.Additionally,an improved 3D-2D HFR method FD-3D2 D is proposed by improving the quadruplet sampling strategy of AgFD and introducing explicit pose decoupling constraint.Experimental results reveal that this method helps to reduce the influence of training data noise and overcome the FR difficulties of the significant pose change,etc.,in 2D surveillance images.The 3D-2D HFR system based on this work has been successfully applied in many realistic scenes,showing the significance and value of this thesis.Although the works mentioned above aim to study and explore HFR,the theoretical and research methods are generalizable and scalable to some extend.Still,they have reference value and guiding use for other fields,such as cross-age face recognition,cross-pose face recognition,person re-identification,etc.