Research And Implementation Of Key Technologies Of Face Photo-Sketch Synthesis

Face photo-sketch synthesis is a kind of heterogeneous image synthesis problem,which refers to the automatic generation of a face sketch from a photograph of a face,or the automatic generation of a face photo from a sketched face image.The application of face photo-sketch synthesis in criminal investigation plays a huge role.Therefore,the research on this issue is of great significance.This article discusses face photo-sketch synthesis from three perspectives: the first is a method of face photo to sketch synthesis(FP2SS);the second is a method of face sketch to photo synthesis(FS2PS);the third is constructing a dual model and training face photo-to-sketch generator and face sketch-to-photo generator simultaneously.(1)In the research on FP2 SS,a global method based on the dictionary atom is proposed,which is divided into two steps: atom-anchored neighborhood construction and pseudo sketch generation.The specific way to construct the atom-neighborhood is as follows: Firstly,the training face photos and sketches are divided into patches then all collected to form global face photo-sketch patch pairs.Secondly,all face photo patches are extracted to "multiple features" and learned a global dictionary.Thirdly,for each dictionary atom,its K closest photo-sketch patch pairs are clustered,namely atom-anchored neighborhood.The steps for generating a pseudo-sketch is as follows: for face photo patch,find the nearest atom-anchored neighborhood,then find the nearest trained face photo patch in the neighborhood,so the corresponding face sketch patch is output.In addition,in order to increase details,similarly construct high-frequency atoms to define the neighborhood,output high-frequency pseudo-sketches,and fuse with the original pseudo-sketches to obtain more detailed face sketches.(2)In the study of FS2 PS,a method based on generation adversarial network is proposed.In the training process,a FS2 PS generation network,a style calculation unit,and a discrimination network are constructed.In the structure of the generated network,the normalization layer of the last three residual network blocks introduces input-related parameters to ensure the consistency of the generated result and the input.The generating network and the discriminative network are confronted with each other during training,forming a GAN loss.The style calculation unit does not include the parameters to be trained,but calculates the loss of the feature image patches of the pseudo-face photo and the numerous style reference face photos,and form a style loss.The total loss for training includes the GAN loss,the loss of style,and the loss of measuring the difference between fake photos and target photos.In the generation process,the face sketch is directly input to the trained FS2 PS generation network,and a fake face photo is output.(3)A dual face photo / sketch generator with lantent space is proposed,and trains FP2 SS and FS2 PS models(i.e.face photo-to-sketch generation model and face sketch-to-photo generation model)at the same time.In training,based on the idea of dual learning,input the result of photo-to-sketch generator into sketch-to-photo generator,and compare the pseudo-photo with the original photo to form the dual-photo-loss.Similarly,in the reverse direction,the dual-sketch-loss is drawn.In addition,a feature map network is designed to project the images in input domain and the target domain into a common lantent space.And in the lantent space,the image from input domain and the image from target domain are similar,so form a lantent space loss.The total loss value includes dual loss,lantent space loss,generation confrontation loss,etc.In the process of generating a pseudo image,two pre-trained generators independently implement the functions of generating sketches / photographs respectively.Through experiments on the face photo-sketch database,the feasibility and rationality of the three proposed methods are verified,which can produce better results of face sketch or photo.Compared with other common algorithms,it generates The results are outstanding both subjectively and objectively.