Super-Resolution Reconstruction For Face Images With Missing Region

Face super-resolution is one of the research hotspots in the field of face image research.Although good results have been achieved,it is still unable to reconstruct the input image with very low resolution,large attitude change and large area occlusion.To solve these problems,a new super-resolution algorithm based on deep learning is proposed in this paper.The main contents are as follows:The super-resolution algorithm in this paper is divided into two steps.Firstly,a lowresolution face image inpainting network is used to repair the input image.Because the lowresolution inpainting network adopts the encoder-decoder structure,and the middle feature layer is connected by the fully connected layer,which will promote the communication between unknown and known regions and obtain better restoration effect of low-resolution image.Experimental results show that the introduction of the low-resolution image inpainting network will greatly reduce the overall difficulty of the problem.Then an upsampling network is used to upsample the restored low-resolution face images,and complete high-resolution face images are obtained.However,MSE-based deep models usually generate smooth high-resolution face images.In order to improve the visual effect of generated images,the adversarial loss is added.Experiments show that adversarial loss can reduce the image smoothing effect and make the image more real.Although the adversarial loss is introduced,the special face priori information is not taken into account.In order to enhance the semantic information of the face structure of the generated image,this paper proposes a new semantic regularization(semantic structure loss).Train a landmark detection network first,and make L2 loss for the key points of the generated image and the corresponding real image.The experimental results show that the visual effect of the generated image is more realistic because of the face priori regularization?Experiments show that the proposed algorithm is superior to other comparison algorithms,and 0.44 dB higher than the second in PSNR and 10.5%higher in SSM.Since PSNR and SSIM cannot objectively judge the visual effect of generated image,this paper uses face alignment and face segmentation to further evaluate the visual effect of generated images.Moreover,the proposed algorithm not only has good super-resolution effect for low-resolution images with large area occlusion in the center,but also can obtain good repair effect for random large-area occlusion,which proves that the algorithm has strong robustness.