Fast And Robust Face Sketch Synthesis

Face sketch synthesis refers to the technique of transforming a face photo to a sketch by specific methods.Face sketch synthesis methods are widely applied in law enforcement and digital entertainment field.In law enforcement,clear face photo of criminal suspect is not available in most time.The only substitute is a pseudo-portrait drawn by a professional painter under the witnesses' description.It is difficult for us to directly compare face photos and face portraits because of their different generation mechanism and information expression.In this case,we can convert all photos in the citizen database into sketches by sketch synthesis technique,and then we compare the pseudo-portrait drawn by the artist with the sketches generated by the database photos to find the suspect.In the field of digital entertainment and mobile internet,face sketch synthesis technology can automatically generate face sketches of different styles.Users can use generated sketches as social account avatars,which can not only express personality in the Internet communication,but also bring fun to life.However,in real scene,due to the varied lighting conditions and complex background environment,sketches generated by existing sketch synthesis methods often contain a lot of noise and artifacts,which greatly affects the quality of the composite results.At the same time,because the neighbor search phase of the sketch synthesis algorithm requires a large amount of distance calculation,it takes a long time to synthesize a face sketch.Based on these two problems,this paper studies the fast and robust face sketch synthesis method,aiming at improving the speed of sketch synthesis and enhancing the robustness of sketch synthesis under different environmental conditions.The main contents and innovations of this paper are summarized as follows:1.A robust preprocessing method for face sketch synthesis is implemented.Normally,the training photos used for sketch synthesis are collected by professional equipment in a natural light,single background environment,and are all frontal pose.However,exposure,dark light,side lighting,and non-frontal face posture may appear in input photos due to the different environmental factors.It is difficult to ensure the training photos and input photos have the same illumination and facial posture,which leads to mismatching in neighbor search phase of sketch synthesis.We adopt a robust preprocessing method to solve this problem.We first warp the input photo to a frontal pose by local affine transformation method.Then we perform side illumination and global illumination processing to make the test photo's statistic luminance consistent with training photos.After synthesizing the final sketch result,we warp the result to original posture by local affine transformation.The whole processing method can be integrated into any existing sketch synthesis method as a preprocessing step,which can effectively improve the quality of synthetic sketch without substantially additional computational cost.2.A deep probabilistic graphical models face sketch synthesis algorithm based on fast neighbor search is proposed.Traditional face sketch synthesis algorithms adopt the pixel intensities as features to match test photos and training photos.These method ignore the facial structural information and are not robust to illumination variations and complex backgrounds.In this paper,a deep probabilistic graphical models face sketch synthesis algorithm based on fast neighbor search is proposed to solve this problem.We adopt the robust preprocessing method to adjust test photos and training photos.The deep features extracted from the deep convolutional neural network are used instead of the pixel intensity features to perform the neighbor matching.We utilize deep probabilistic graphical model to jointly model the sketch reconstruction weight and deep feature weight to obtain the best reconstruction representation of the synthetic sketch.Deep feature representation has a higher dimension than the pixel luminance feature,which leads to more complex distance calculation.We propose a fast neighbor search method to accelerate the speed of nearest neighbor matching.Experimental results verify the robustness and rapidity of the proposed method.3.A semi-supervised face sketch generation algorithm based on generative adversarial networks is proposed.Model-driven face sketch synthesis algorithm is usually unable to train a generative model with strong representation ability because of lacking sufficient image training data,which makes the synthetic sketches contain much noise.This paper proposes a semi-supervised face sketch generation algorithm based on generative adversarial network to solve this problem.Firstly,an image collage method is proposed to copy the face area of the input photo into the artist-style portrait's face area seamlessly.Then iterative local style transfer network is utilized to make the pasting face area have the same style texture as the portrait.We train a generative adversarial network using the pasting images generated by the image collage method and the style harmonization results from style transfer network to obtain a generative model.In the test phase,we first adopt the robust preprocessing method to adjust the input photo,then copy the face area of input photo into portrait.We put the pasting image into generative adversarial network,the output is the final stylized sketch.Experimental results verify the effectiveness of the proposed algorithm.