Research On Heterogeneous Face Image Synthesis And Recognition In Unconstrained Environment

The heterogeneous face image includes face images with different expressions obtained by different imaging methods.These different modal face images provide a more diverse perspective for the description and analysis of target characters.Heterogeneous face recognition is a technique to find the commonality of these angles to match the heterogeneous images of the same person.Broadly speaking,the research content of this technology also includes heterogeneous face image synthesis aiming at realizing the mutual conversion of heterogeneous images.As an important branch of this technology,the sketch-photo face recognition and its included sketch-photo face synthesis technology can be widely used in criminal investigation,law enforcement,digital entertainment and other fields,and has good application value and academic significance.However,the current related research work is mainly concentrated on constrained conditions,and the unconstrained environment is still a huge challenge facing the technology.To this end,this thesis takes sketch images and visible light photos as the research objects,focusing on the research on the synthesis and recognition of heterogeneous face images in scenes of age change and partial occlusion.Combining generative adversarial networks and attention mechanisms,methods are proposed from two aspects of image conversion and image recognition to overcome the effects of unconstrained environment on image synthesis effect and recognition accuracy.In terms of image conversion,this thesis proposes a cross-age sketch-to-photo conversion method based on double dual generation adversarial networks.By establishing a double dual generation adversarial network,we learn the mapping and inverse mapping between the source and target age domains,and the mapping and inverse mapping between the sketch domain and the photo domain,respectively.Therefore,the transformation and reconstruction of the image are realized under the two attributes of age and style,and the double duality is realized.With the help of the reconstructed images under the two attributes,the reconstruction identity loss and the complete reconstruction loss are established to restrict the image generation process.Use age estimation software to add age tags to the four public datasets and the web crawling data respectively.Experiments after comparison with related methods show that the converted images obtained by the algorithm in this thesis can accurately complete the age spanning while style conversion,and effectively retain the identitycharacteristics,with a certain generalization ability.In terms of image recognition,in order to reduce the effect of image information loss caused by partial occlusion on sketch-photo face recognition,this thesis proposes a occlusion robust sketch-photo face recognition method based on ternary differential siamese network combined with attention mechanism.This method uses the encoder to project the input image into a common subspace,and completes the processing and matching identification of the occlusion area in the subspace to reduce the modal difference.The proposed ternary differential siamese network constructs a mask generator by introducing an attention mechanism.By inputting the differential signal between the complete face and the occlusion face feature,the corresponding relationship between the image occlusion area and the destroyed feature is learned,so as to discover and discard these useless features in the recognition stage.The content loss function and the modal loss function are introduced as constraints to make the network eliminate the modal interference and locate the feature elements which deviate from the real value due to occlusion more accurately.Using image modification software to artificially synthesize occlusion blocks in two public datasets,the corresponding experimental results show that the algorithm architecture of this thesis can effectively identify sketch and photo face images that contain partial occlusions.