Research On Deep Learning Face Super-resolution Technology Based On Parsing Map Prior

Face super-resolution(face hallucination)is the technology recovering highresolution face images with high-frequency details from existing low-resolution face images,which has important academic research and practical application value.In computer vision,deep learning shows its great abilities.Deep learning based face hallucination methods perform well,but there are still some drawbacks in the design of the network and the use of unique prior knowledge of face images.Firstly,the existing methods with prior knowledge,always first super-resolve low-resolution face images to generate the intermediate results,then get the prior knowledge from the intermediate results,then reconstruct the super-resolution results with the intermediate results and prior knowledge.In this way,the prior knowledge obtained directly depends on the quality of the intermediate result s.Once the quality of the intermediate results is poor,the obtained prior knowledge is also biased.Secondly,in natural image super-resolution,channel attention and spatial attention have been proven to improve the performance of image reconstruction.Face image as a special kind of natural image,we assume that attention mechanism also has the ablitily to improve the performance of face reconstruction,while existing face superresolution methods don't make full use of attention mechanism.Thirdly,the structure of existing face super-resolution network is usually pre-upsampling or postupsampling.The features in the network are only in low resolution or high resolution space.Then,the diverity of features is low which has influence on the performance of the network.To solve the problems of the existing methods,this paper proposes two face super-resolution methods.1)Firstly,this paper proposes a parsing map guided multi-scale attention network for face hallucination.For the first problem,in order to avoid parsing map being affected by intermediate results,we directly extract face parsing map from lowresolution face images,and then use low-resolution face images and parsing map to generate high-resolution face images.For the second problem,we introduce channel attention and spatial attention to the face super-resolution problem,and improve by proposing a multi-scale attention mechanism to extract multi-scale features.For the third problem,we propose Fish SRNet network,which generates features in different resolutions to improve the resolution diversity of feature maps.The features are not limited in low-resolution space or high-resolution space,and the purpose of improving the performance of face super-resolution network is finally achieved.2)Secondly,we propose a deep recurrent face super-resolution network with parsing map's guidance.This paper finds that in the multi-scale attention face superresolution network guided by parsing map,the introduction of parsing map does not improve the objective evaluation index,so this paper d oes some comparative experiments to verify the validity of the parsing map,and finds a simple and effective method of using parsing map.The method is to interpolate the low-resolution face image and the parsing map to the same size as the high-resolution face image,and concat them together as the input of the network.In addition,this paper designs a recursive face super-resolution network to improve the performance of the network without increasing the network parameters.