Facial Sketch-Photo Transform With Application To Sketch-based Face

Facial sketch photo transform plays an important role in many aspects of life.In digital entertainment,human face caricature is sought after by the majority of users,which can be automatically done by facial sketch photo technology.In many criminal investigation cases,there are often only some witnesses' testimonies and the suspects' portrait sketches drawn by professionals according to their testimonies as references.Facial sketch photo transform technology can provide new technical ideas for better identification of sketch portraits.Due to the large differences between sketch images and photographic images,transition between them is extremely challenging.Most of the existing face sketch photo transform methods usually use the style transfer method or artificial modeling method to transform between face sketch and photos,but ignore the details of the face attribute;or only focus on the global information of the image and ignore the local features.These make the images generated by the existing methods lack details,which is not conducive to sketch face recognition.In view of the above problems and challenges,the main work completed in this paper is as follows:(1)Proposing an attribute-guided facial sketch-photo transform method.Due to the lack of information and subjectivity in the sketch image,in order to make up for the lack of information in the sketch image,this paper selects 21 kinds of face attributes as additional information and input them into the generator together with the sketch image to get the corresponding photo image.In this paper,Conditional Cycle GAN is used as the generator,and the loss function to control the edge is further introduced in the training to make the final image more accurate.Experiments are conducted on the Sketch Face Library(CUFS)of the Chinese University of Hong Kong to evaluate the quality of generated images and the accuracy of face recognition.Compared with existing methods,the proposed method can obtain comparable image quality,and the recognition rate is higher in both photo and sketch fields.(2)Proposing a facial sketch-photo transform method combined with segmentation.Attributeguided facial sketch-photo transform method is easily affected by background and noise.In this thesis,segmentation information is introduced into the above method to train the sketch photo transform network to focus only on the foreground(person)part,so as to improve the final generation effect.On this basis,this thesis further introduces the face parsing module,the facial features segmentation,the segmentation results into the training process,the design of targeted face local loss function,face local details more rich.(3)Proposing a multi-mode fusion sketch face recognition method.First of all,the existing sketch face database is expanded with the sketch-to-photo method proposed in this thesis,and then the Light CNN face recognition model is fine-tuned to suitable for sketch face recognition.Secondly,multiple modes of face images,such as sketch images and face attributes,are introduced to fuse additional information in different ways to improve the results of face recognition.The experimental results prove that the face recognition method proposed in this thesis can effectively improve the accuracy of sketch face recognition.