Research On Face Image Super-resolution Reconstruction Algorithm For Video Surveillance

Video surveillance,which is an important part of the security program system,plays a very important role in maintaining public order and protecting the people's personal and property safety.The human face is one of the most effective identity identification features,which is one of the most important pieces of evidence in criminal investigations.However,limited by different imaging conditions such as camera quality,light occlusion,weather conditions,camera distance,and motion blur,the resolution of human faces captured by surveillance equipment is difficult to meet the needs of detection.Face image super-resolution(SR)aims to comprehensively use signal processing,statistical learning,optimization theory and other tools to reconstruct high-resolution faces that are difficult to obtain directly by imaging systems from one or more low-resolution face images in the same scene.The deep learning-based SR methods have been widely explored in the current face superresolution field.Based on the excellent nonlinear mapping ability of deep convolutional networks,these methods can directly learn the mapping between low-resolution images and high-resolution images.However,in the real world application scenarios,due to nonuniform degradation factors and limited computing resources of image processing equipment,the deep learning-based face SR methods still have the following four challenging problems:(1)the extremity quality degradation in high-resolution face images;(2)the identity information loss under strong noise and blur interference;(3)the high dependence of the face SR model on high-performance computing equipment;(4)the performance degradation of the video face SR caused by the complex temporal dependence between frames.Focusing on these challenges,this article has carried out a systematic study and obtained the following research results:1.A face image SR algorithm based on multi-category prior fusion is proposed.Existing face super-resolution methods aim to achieve input low-resolution images by learning the mapping of sample data sets.Due to the typical ill-posed SR process,when the input image resolution is low,there will be serious performance degradation in image details and texture of the reconstructed images.To solve this problem,we propose a face image SR algorithm based on face prior integration.Firstly,the semantic attribute information and pixel-level texture and contour information are analyzed and extracted from the input lowresolution image.Then,a multi-information fusion model is constructed by combining the pixel-level and semantic-level information.The experimental results show that the face image SR method based on face prior integration is superior to the current methods in the very low-resolution face image SR task.2.A SR algorithm of noisy face images based on capsule representation is proposed.Most of the existing face SR methods are carried out under the assumption that the input image is noiseless.When applied to the real scene,the input image polluted by noise will lead to a sharp decline in model performance and there will be obvious identity information confusion in the reconstructed face image.To solve this problem,we proposed an SR algorithm for noisy face images based on capsule representation.This method describes the human face from three aspects: semantic features,probability distribution and attributes information.Based on the complementarity of the three information,a comprehensive face information representation model with high noise robustness is constructed,and the face SR is completed by attribute deconstruction.The experimental results show that the SR algorithm based on capsule representation not only has a better image reconstruction effect but also is superior to the current methods in the identification task of reconstructed face images.3.A lightweight face image SR reconstruction algorithm is proposed.Although the image SR technology has made significant progress in recent years,the current face image SR algorithms are difficult to be widely applied due to the high computational resource requirements of deep learning methods.To solve this problem,we first constructed a lightweight face image SR network based on wavelet transform according to the sensitive property of wavelet transform to image details.Then,considering the low computational resource dependence of the binary neural network model,we proposed a model binarization algorithm for the face image SR reconstruction task,which can significantly reduce the high demand for storage and computing resources of the face SR model.Experimental results show that the efficiency of the proposed lightweight face image SR network based on wavelet transform is better than state-of-the-arts.Moreover,the model binarization method for the face image super-resolution task has also achieved initial success in the field.4.A face video SR algorithm based on motion-adaptive compensation is proposed.The existing video SR methods usually use frame fusion to complete the mapping process from multiple low-resolution images to one high-resolution image.However,due to the limited modeling ability of the existing methods for the time-sequence dependence between video frames,the network performance will be reduced with the increase of the input frames.To solve this problem,we propose a face video SR algorithm based on motion-adaptive compensation.This method can make motion compensation in the feature domain by forgetting,retaining and enhancing,and feedback to the network in an adaptive way.The ability of time-dependent modeling is significantly enhanced,which advances network performance.The experimental results show that the face video SR reconstruction method based on motion-adaptive compensation is superior to the current methods in the video SR task with complex temporal dependence between frames.