The Research On Deep Models And Algorithms For Ultra-Low Resolution Face Hallucination

With the development of Internet technology,we have entered an era of data explosion.Face image is one of the most concerned data in people's daily life.Our demand for high-resolution face images is getting higher and higher.Ultra-low resolution face hallucination technology reconstructs very low resolution face images into high resolution for subsequent face recognition or other processing.Face halllucination technology can be understood as super-resolution technology for face images.The research history of this subject has been more than 20 years.From global and partial face methods to some comprehensive methods based on face structure characteristics,it has an ideal hallucination result for 4? reconstruction of a normal low-resolution face.Howerer,when the scaling factor increases to 8 ?,the performance of most approaches degrades.In 2012,Alexnet's success sparked a wave of deep learning research.Deep learning can fully exploit the characteristics of large-scale data sets,and ultra-low resolution face hallucination methods based on deep learning become possible.The main work of this paper is the algorithm research of deep models for ultra-low resolution face hallucination.Its core ideas and innovation points are mainly presented in the following:1.Based on the VDSR super-resolution network and incorporating Googlenet's Inception idea,a natural image super-resolution method called MP-VDSR(Multi-Path Very Deep Convolutional Networks for Image Super-Resolution)is constructed.The experimental results show that the multi-path structure can increase the ability of feature extraction without increasing the the amount of parameters.It has achieved stable improvement over VDSR in terms of standard PSNR and SSIM.However,when the network is directly applied to the ultra-low resolution face hallucination,the resultant image is too smooth and lacks details.2.Combining the Encoder-Decoder structure in the seq2 seq task,the Resnet module with up-and-down sampling function is used to construct the encoder and decoder respectively,and an ultra-low face hallucination method based on EDV(Encoder-Decoder-Vector)and EDF(Encoder-Decoder-Feature)network is proposed.The experimental results show that the main structure of the face can be reconstructed,and the details are also obviously enhanced.However,compared with the high-resolution original image,there is still a large space for enhancing high-frequency details such as the corners of eyes and the corners of the mouth.3.In order to further enhance the facial details,the adversial training mechanism is introduced.Under the framework of WGAN-GP and WGAN-LP,the EDV and EDF network are used as generators to construct an enhanced ultra-low resulution face hallucination method based on the regularized WGAN.The experimental results show that the network reconstruction results are effective in terms of standard PSNR,SSIM and image generation score FID.Finally,we augment the data for training set,and the network is retrained to enhance the applicability of the network to ultra-low resolution face images in the real world.