Low-quality Face Super-resolution Algorithm Based On The Joint Framework Of Deep Learning And Manifold Learning

With the more attention to public security,video surveillance system has been widely used in the field of criminal investigation and public security,and the highresolution display and recognition of face images is becoming more important.However,unfavorable factors existed in the imaging devices and the real environment lead to poor quality of the captured face images,which are seriously affected by noise and blur.Face super-resolution technology can restore corresponding high-quality face images using observed low-quality face images without changing the existing conditions.Therefore,it is particularly crucial for the actual low-quality faces to develop an efficient face super-resolution technology to improve the facial clarity.Existing super-resolution algorithms include manifold-learning based methods and deep-learning based methods.The former is based on consistent representation of highquality and low-quality face manifold structures online,and the reconstructed image is directly synthesized from high-quality face samples.The latter is based on the mapping network between the low-quality and high-quality faces offline,and the reconstructed image is predicted by low-quality face image and the mapping network.When the face is affected by strong noise and blur,the accuracy of consistent representation in the former methods decreases.As a result,such as the locality-constrained representation algorithm in this kind of methods synthesizes interference information while restoring the facial details.For the latter methods,the performance of the mapping network decreases in both removing noise and restoring details.As a result,such as the deep convolutional network algorithm in this kind of methods removes the noise,however,the facial details of the reconstructed face are not clear.In view of the advantages and disadvantages existed in the above-mentioned methods on low-quality face super-resolution problem,this paper proposes a lowquality face super-resolution algorithm based on the joint framework of deep learning and manifold learning.Considering the respective advantages of locality-constrained representation algorithm and deep convolutional network algorithm,the designed joint enhancement network learns facial details from the reconstructed image by localityconstrained representation algorithm adaptively,which is used to compensate for the loss of facial features after removing noise by the deep convolutional network,this is a way of learning noise information and texture information separately to restore the high-quality face images.And on this basis,considering the influence of image deep features on deep-learning based face similarity comparison,a discriminant constraint network is introduced to improve the enhancement of face details by the joint enhancement network,so as to further enhance the similarity between reconstructed face and the high-resolution face image.Experiments show that,compared with the locality-constrained representation algorithm,the proposed super-resolution algorithm improves the average values of PSNR,SSIM and FSIM by 1.2dB,0.05 and 0.02 on the simulated CAS-PEAL-R1 data,and 0.3dB,0.01 and 0.01 compared with the deep convolutional network algorithm.On the real low-quality face data,compared with the locality-constrained representation algorithm and the deep convolutional network algorithm,the average face similarity score increases by 0.05 and 0.02 respectively.The improvement of subjective and objective results shows that the proposed algorithm can effectively reconstruct facial features,improve facial clarity and similarity with high-resolution face images,which is of great significance for the practical application of face super-resolution.