A Face Image Super-resolution Algorithm Based On Lightweight Neural Networks

The face super-resolution algorithm can be widely used in intelligent security and other fields,and is one of the current research hotspots.There are still some aspects that need to be improved in the existing related works: complex models have low inference efficiency and are difficultly deployed on mobile devices;most of the lightweight models can only ultra-resolve a image to its larger version of one specific scaling factor;labelling face prior information is difficult.In view of the above problems,the thesis proposes a super-resolution algorithm for face images based on lightweight neural networks and progressive training,which mainly includes the following two innovative works:(1)A three-stage model which is improved on Residual Feature Distillation Networks is proposed,that can ultra-resolve a very low resolution face image to its larger version of three scaling factors after progressive training.The model structure first includes an improved residual feature aggregation block to extract image features,and then includes three upsampling blocks corresponding to three stages to ultra-resolve a image to a specific scaling factor at each stage.The comparison experiments show that: our model is not only lightweight but also has good super-resolution performance.(2)A joint loss based on pixel loss,face semantic segmentation loss,and perceptual loss is proposed to train our model.By using a pre-trained neural network for face semantic segmentation to generate face parsing maps,and obtaining face semantic segmentation loss by calculating the cross-entropy loss between the reconstruction images’ parsing maps and the real images’ parsing maps;by using a pre-trained neural network for face recognition as an image feature extractor,and obtaining perceptual loss by calculating the feature difference between the reconstruction images and the real images.The ablation experiments show that:using face semantic segmentation loss can help the model reconstruct realistic facial features;using perceptual loss can improve the visual sensory quality of reconstruction images.