Research Of Face Sketch-Photo Synthesis Technology Based On Generative Adversarial Network

Face sketch-photo synthesis technology has been widely applied in various aspects of daily life.In digital entertainment,face sketch-photo synthesis technology can automatically realize the sketching of face photos,and these sketch-style photos are popular among users now.In public security and criminal investigation,face sketch-photo synthesis technology can automatically generate corresponding face photos from the sketches of criminal suspects that drawn by professional artists,assisting investigators in identifying the suspects’ identities and effectively improving the efficiency of criminal investigations.Consequently,research on face sketch-photo synthesis algorithms is of great significance.Existing face sketch-photo synthesis methods commonly face a problem:the performance in the Face Sketch to Photo Synthesis(FS2PS)direction tends to be inferior to that of Face Photo to Sketch Synthesis(FP2SS),particularly manifested in the severe aliasing distortion and style errors in the generated face photos.In response to the above problems,this paper focuses on FS2PS as the main research direction and proposes two face sketch-photo synthesis methods and a multi-stage generative model for generating face photos from face sketches.The specific contributions can be summarized as follows:(1)To address the problems of the severe aliasing distortion and heavy noise in generated face photos caused by the neglect of the semantic difference between face sketches and photos in mainstream methods,a face sketch-photo synthesis method utilizing intermediate semantics enhancement is proposed.The algorithm implementation is mainly divided into two stages:in the first stage,the model learns the mapping between the sketch domain and the photo domain,generating intermediate semantics that is semantically closer to the photo domain;in the second stage,the model utilizes the intermediate semantics to supplement and revise the sketch semantics,achieving semantic enhancement and subsequently reducing the semantic difference between the two domains.Based on this,an semantic matching loss is proposed to further improve the accuracy of intermediate semantics.Compared to mainstream methods,this method can effectively resolve the problems of the severe aliasing distortion and heavy noise,generating face photos with stable facial structure and clear,delicate details.This method achieves the best results in both subjective scores and objective metrics,as well as obtaining the highest face recognition rate.(2)To address the problems of style errors and the aliasing on facial components in generated face photos which commonly exist in mainstream methods,a face sketch-photo synthesis method that utilizes both face attributes and face semantics as generation constraints is proposed.In the preprocessing stage,each face photo in the dataset is labeled with color attributes,obtaining face attribute vectors.Face attribute vectors effectively constrain the color attributes and the generation style of the generated face photos,while face semantics can effectively constrain facial structure during generation.Since the face auxiliary information of the same face in different modalities are relevant,the constraint features obtained by fusing face attributes and face semantics can exert stronger constraint effects during the generation of face photos,thereby constraining the model to generate face photos with correct styles and no significant aliasing.In the comparative experiments of subjective scores and objective metrics,this method achieves the best results and the highest face recognition rate.(3)Current mainstream methods generally adopt single-stage models or build multi-stage models by simply stacking single-stage networks,resulting in generated face photos that are often not sufficiently clear and detailed.To address this problem,a multi-stage generative model based on face auxiliary information is proposed.The first-stage model learns the overall mapping between the sketch domain and the photo domain to generate the intermediate semantics which are then used for semantic enhancement.In the second-stage model,face attribute vectors and a feature fusion module are introduced.The enhanced face semantics and face attribute vectors,as two kinds of face auxiliary information,are fused in the feature fusion module to obtain constraint features.Embedding constraint features into the latent face features can effectively constrains the generation style and facial structure of the generated face photos.Finally,this method generates clear and delicate face photos with rich and realistic details.Compared with mainstream methods,the face photos generated by this method exhibit superior subjective perceptual quality and achieve the best results in all objective metrics,having a distinctly competitive advantage.