Research On Deep Cross-Modal Face Recognition

With the rapid development of image acquisition technology and the rapid popularization of smart terminal equipment,many different modal face images have emerged in people's daily lives,such as near-infrared face images,thermal infrared face images,high and low resolution Face images,etc.The process of identifying these different modal face images is called cross-modal face recognition,such as near-infrared image and visible image face recognition.Cross-modal face recognition has important research significance and wide application value in the field of biometric recognition.In criminal investigation,when the suspect's photo cannot be directly obtained,the suspect's identity information can be effectively confirmed by comparing the sketched face image of the suspect(the portrait created by the painter based on the witness or the victim's description of the suspect's appearance)with the face photo in the face database.In access control security,most of the face images in practical applications are collected under visible light to obtain a face database.By comparing and recognizing the near-infrared face image obtained in real time with the high-quality visible light face image in the database,the problem of visible face image recognition being easily affected by ambient light is solved,and credible identity verification is realized.Cross-modal face recognition faces two challenges: one is the problem of modal differences.The feature distribution of face images of different modalities is quite different.The previ-ous cross-modal face recognition methods are limited by the feature extraction capabilities of manual feature descriptors and the modal independent feature extraction capabilities of traditional convolutional neural networks.The recognition effect of cross-modal face recognition? the second is the problem of category imbalance.Cross-modal face image acquisition is difficult,and the amount of data in different categories is likely to vary greatly,resulting in imbalanced feature distribution,which affects training efficiency and model performance.Therefore,this paper aims at the two challenges of large modal differences and category imbalance in cross-modal face recognition.Based on deep neural networks,this paper proposes a method to effectively reduce modal differences and solve category imbalance.The main innovative work of this article is summarized as follows:1.Propose a face recognition method for single web caricature based on unbalanced categories.There are difficulties in collecting cross-modal face data,and the amount of data in different categories varies greatly.The imbalance of categories in feature extraction will cause the following problems: the amount of data in a category is too large,and it does not contribute too much useful learning information in feature extraction,resulting in low efficiency.A large amount of useless learning information will disrupt the feature extraction process and degrade the performance of the model.In response to this problem,while using the feature map in the convolutional neural network to capture the advanced information in the comic face image,it is proposed to use Focal Loss to solve the problem of category imbalance.By reducing the loss weight of a large number of categories,the model pays more attention to a small number of categories during the training process.The experimental results on the Web Caricature dataset show that the recognition rate has been improved to a certain extent after adding Focal Loss.2.Propose a multi-network cross-modal face recognition method based on modal consistency.Existing methods usually use a network to learn the depth features of two modal face images at the same time.The features learned in this way can better maintain the common characteristics within the modal,but ignore the private characteristics between different modalities.It affects the accuracy of feature representation,and then affects the accuracy of face recognition.In response to this problem,this paper designs a new multi-modal face feature learning network framework,which consists of two branched convolutional neural networks,and the two networks extract facial features of two modalities respectively.A consistency loss function used to punish the distance between two features of different modalities under the same identity is proposed,and the two branch networks are combined through this loss function.In addition,the center loss is also used to minimize the difference of the same class while keeping the features of different classes separable.The validity of this method is verified by CASIA NIR-VIS 2.0 dataset and Web Caricature dataset.For the Web Caricature dataset,the recognition rate of this method is significantly improved compared with the previous method.3.Propose a multi-network cross-modal face verification method based on modal consistency.In addition to face recognition,face verification is more widely used in daily life.Face verification is the process of comparing the current face with the face database and verifying whether it matches.Face verification is generally used in the fields of finance and information security.Therefore,this chapter conducts face verification experiments on the basis of the previously proposed network structure.The two networks extract the facial features of the two modalities,calculate the cosine similarity,and train to obtain the threshold.In the training process,a triple loss is newly added,which makes the reference samples of the same type closer to the positive samples,and the reference samples of different types are farther away from the negative samples.In the test phase,if the similarity of two faces with different modalities is greater than the threshold,they are judged to be the same identity,otherwise they are not the same identity.Use this method to perform experiments on CASIA NIR-VIS 2.0 dataset,Web Caricature dataset and NJU-ID dataset,and get better experimental results.