Progressive Large-scale Super Resolution Method For Face Images

Face image super-resolution(FSR)aims to enhance the resolution of low resolution(LR)and low-quality face images by numerical calculation method,which can effectively improve the visual quality and identification of face image.FSR has important application value in the field of face recognition and surveillance video.Face images are highly structured and have unique identity attributes,how to recover face images with fine details and identity preservation is a very challenging topic.In recent years,FSR methods based on deep learning have attracted great attention and shown obvious advantages over traditional methods.To compensate for the loss of high-frequency facial information in the process of image degradation,many methods introduce facial prior information to assist highresolution image reconstruction to achieve great performance improvements.However,there are still some problems with the FSR method using facial prior: 1)Most of the methods directly predict the prior information from LR,low quality or rough face images,and it is difficult to estimate the accurate prior information.2)Face prior information estimation and image feature recovery are usually optimized on a scale factor as a multi-tasking learning problem,the structural changes of face components cannot be fully captured and utilized,especially when the upscaling factor is large.3)Many methods only improve the image resolution through transposed convolution or sub-pixel convolution layer at the end of the network,which not only leads to serious artifacts but also increases the difficulty of training.To solve the above problems,we first proposed a parsing prior guided progressive network for large factor FSR,which uses the progressive reconstruction strategy and attention mechanism to effectively extract face prior knowledge to assist the facial detailed recovery.To further make use of the face prior,we also proposed a multi-stage cascaded recurrent convolutional neural network for FSR to improve the image quality.The specific work is as follows:1)A parsing prior guided progressive network for large factor FSR is presented,which introduces progressive tactics in both model design and network training.By performing 2×upsampling progressively,the parsing prior is predicted in the image features of different resolutions and then is fused with corresponding resolution image features to better preserves facial component structure.The progressive training method adds new layers to the model step by step,which greatly reduces the difficulty of network training and more stably recovers high-quality face images.Compared with ther state-of-the-art methods,the experimental results show that the proposed method improves the PSNR value on the CelebAMask-HQ and the Helen datasets by 0.57 dB and 0.4259 dB,and the MS-SSIM value by 0.0044 and 0.0078,respectively.2)A progressive cascaded recurrent convolutional network for large factor FSR is proposed.Specifically,a novel multi-stage cascaded convolutional neural network is developed to progressively obtain high magnification face images,where the first stage of the network achieves an initial 2×magnification image,and the following other stages,adopting the recurrent structure,sequentially generate the corresponding 4×,8×and possibly larger factor SR images through multiple independent iterative modules.The deep features and parsing priors of the face are extracted in parallel in each stage of the network and integrated to improve the deep representation ability of the network.The training of the whole network is supervised in an end-to-end way by the weighted sum of multiple losses.Compared with other state-of-the-art methods,the experimental results show that the proposed method improves the PSNR value on the CelebAMaskHQ dataset by 0.3491 dB.In this paper,we propose two kinds of FSR networks for the problems of how to extract the facial parsing prior effectively and how to fully integrate the prior knowledge.Through the use of progressive strategy,feedback connection,and attention mechanism to significantly improve the quality of face image.We provide a new and effective scheme for low-quality face images in the surveillance and mobile camera scenes.