Facial Semantic Attribute Information Incorporated Heterogeneous Face Recognition Algorithm Analysis

Heterogeneous face recognition means to recognize person identity information between face images with different modalities,which is also defined with cross-modality face recognition.There exist various modalities of face images in real-world scenarios,such as face sketch images are utilized to identify suspects when mugshot photos couldn't be captured;near-infrared face images are utilized in light-sensitive place,even at night;low-resolution ID card images are always utilized in train stations and airports to verify the identity.Thus,heterogeneous face recognition plays an important role in biometric recognition and publish security in the real world.Due to the shape exaggeration distortion and lack of texture information in heterogeneous face images,there exists a large modality gap between cross-modality face images.Thus,traditional face recognition methods couldn't directly in heterogeneous face scenarios,which brings in the challenging problem.Therefore,this thesis aims to incorporate semantic attribution information,to decrease the large modality discrepancy in heterogeneous images and improve the face recognition performance.The main distributions of our thesis are summarized as follows:1.A semantic attribute and discriminative information incorporated heterogeneous face recognition method is proposed.Existing heterogeneous face recognition methods either focus on extracting modality invariant features or transforming different modalities into the same modality to decrease the modality gap.However,there exists a large difference modality gap in heterogeneous face images,like shape exaggeration and distortion,lack of texture.It is difficult to solve mentioned recognition problem only with image information.Thus,we naturally proposed a facial semantic attribute,and discriminative information incorporated heterogeneous face recognition.The proposed method could effectively utilize the modality-invariant discriminative information in facial semantic attributes,to capture more discriminative representation for cross-modality images.Experimental results in several heterogeneous face datasets show that discriminative representation learned from both face images and attribute information would boost the recognition performance.2.A semantic attribute-guided synthesis and local descriptor based heterogeneous face reranking method is proposed.Existing methods only consider the difference of data distribution to decrease the modality gap,ignoring the specific property of heterogenous faces in the real world.The large modality gap like shape distortion is caused by the specific face generation procedures.Considering the heterogeneous face property,the semantic attribute guided synthesis and local descriptor based heterogeneous face re-ranking method is proposed.Firstly,the multiple attribute information is utilized as the clue to synthesis extra cross-modality face images,to eliminate the defect of modality gap;Then,the local descriptors are extracted to capture modality-invariant identity features;Finally,an iterative re-ranking algorithm is utilized to fuse different complementary discriminative information in five face components for the recognition task.Experiments prove the efficiency of the proposed method in heterogeneous face recognition tasks.3.A supervised semantic attribute disentangled representation-based heterogeneous face recognition method is proposed.Existing methods always aim to design a novel loss function or network architecture to directly extract modality-invariant features,or synthesize the same modality faces.Yet the former always lack explicit interpretability,the latter strategy inherently brings in synthesis bias.To solve the problem,the supervised semantic attribute disentangled representation-based heterogeneous face recognition method is proposed,which could explicitly interpret dimensions of face representation.Firstly,the proposed method would construct interpretable representation structure;Then,it could further disentangle semantic attribute information and identity information;Finally,the modalityinvariant discriminative identity information is extracted to recognize faces.Experimental results in multiple heterogeneous face scenarios show the efficiency of the proposed method.4.An unsupervised semantic attribute disentangled representation-based heterogeneous face analysis method is proposed.Existing related methods always would make the reconstructed images blurry,or not disentangle attribute clearly.Especially,heterogeneous faces are captured from the real world,which would make the task more challenging.To solve the mentioned problem,a novel supervised semantic attribute disentangled representation-based heterogeneous face analysis method is proposed.Firstly,the encoder-decoder architecture is used to reconstruct input heterogenous faces;Then,the identity-preserving discriminative disentangled representation is proposed to disentangle facial semantic attribute information;Finally,the unsupervised disentangled semantic attributes would help perform heterogeneous face analysis.Experimental results show the proposed method could achieve superior performance in heterogeneous face analysis tasks.5.A large-scale unreal generated face sketch-based heterogeneous face recognition method is proposed.The existing recognition methods mainly focus on processing heterogeneous faces captured in real-world scenarios.However,the real face sketch generation procedure is costly,which would make collect real face sketch images difficult.To solve the problem,a large-scale unreal generated face sketch-based heterogeneous face recognition method is proposed.Firstly,two unreal face sketch-based recognition scenarios are proposed to mimic the real world;Then,several face recognition models and domain adaption algorithms are considered as benchmarks to evaluate recognition performance.Experimental results prove the research value of the proposed scenarios,and further summary research directs to promote the field development.