Research On Methods Of Heterogeneous Facial Image Synthesis

Heterogeneous images refer to images with different forms,such as high-low resolution images,face photo-sketch images,visible light-infrared light images,and normal-dark light images.Heterogeneous images often describe the same scene or the same target using different methods,giving the target a richer information expression.There exists both redundancy and complementation among these images that help people understand the natural property of things more deeply and comprehensively.In practical applications,heterogeneous images have specific usages.When an image with the desired form is difficult to obtain,it can be synthesized based on the existing forms.That is the problem of heterogeneous image synthesis.This manuscript mainly explores the synthesis of heterogeneous images,focusing on the face photo-sketch synthesis methods and aiming at alleviating the problems in the existing synthesis methods and improving the quality of the images.The main contributions of this article are summarized as follows.1.To solve the problem that the existing face sketch synthesis methods cannot synthesize lowresolution input images,a dictionary-based multiple estimation traditional nonlinear image interpolation method is proposed to increase the resolution of input images.First,a subspace clustering method based on dictionary atoms is proposed to deal with the uneven distribution of image patches.Second,the nonlinear mapping between high and low-resolution image patches is learned rather than linear mapping.Finally,multi-estimation is performed to reconstruct images to improve interpolation performance.Experimental results show that compared with other traditional image interpolation methods,this method can achieve better interpolation results,and improve sketch synthesis performance of low-resolution photos to a certain extent.2.To solve the problem that existing heterogeneous face image synthesis methods have many restrictions on the input face images,a free face sketch synthesis method based on the global search is proposed.Existing synthesis methods can only synthesize a single front face image with a fixed size,and the input images also need key point calibration for content alignment.This is mainly because of the local search with location information is used.To break through the limits of input images,extending the search area from the local to the global is proposed.First,a variety of features are extracted to train a global dictionary,and the dictionary atoms are used to subspace image patches.Second,the initial sketch patches are synthesized in the matched subspace.The same idea is applied to enhance high-frequency information and texture details.Experimental results show that,compared with other data-driven synthesis methods,this method can synthesize free-size face images without alignment or even multi-face images,and can obtain better synthesis quality and higher face recognition rate.3.The results of patch-wise data-driven methods suffer from severe blurring effect and face structure missing,while the results of image-wise model-driven methods always suffer from heavy noise and unreal sketch textures.To solve the problems,a multi-scale attention generative adversarial network for face sketch synthesis is proposed to take both advantages and avoid the drawbacks.To avoid the disadvantages caused by dividing images into patches,a generative adversarial network with residual network blocks is adopted as the main framework.To make the synthesized sketches more realistic and natural,patch-wise feature loss from a pre-trained neural network is adopted as the main loss.Meanwhile,multi-scale features are extracted and a channel attention mechanism is applied to emphasize the important features,further help the network produce more detailed sketches.Experimental results demonstrate that this method can achieve better performance in both visual evaluations and quantitative evaluations than other both patch-wise and image-wise synthesis methods.4.To solve the problems of identity information missing and color mixing that brings from the cyclic generative adversarial network,a single feature encoder guided cyclic generative adversarial network for face photo-sketch synthesis is proposed.An additional feature encoder is introduced into the cyclic framework to improve the synthesis performance.The single feature encoder encodes both domains of photo and sketch images in a unified manner.It encodes the essential features and common information of the paired heterogeneous images.Besides,both of the intra-domain and interdomain feature coding losses are proposed to guide the training of two generators.Experimental results show that,compared to other cyclic generative adversarial network-based methods,the method can synthesize sketch and photo images with good subjective performance,effectively maintain identity information and greatly alleviate the color mixing problem.