Research On Image Super-Resolution Algorithm Based On Image Prior And Convolutional Neural Network

The research objective of image super-resolution is reconstructing high resolution image from low resolution image.Currently,the resolution of widely used display terminations in the market has reached 4K.However,due to cost and technical limitations,the resolution improvement speed of image acquisition equipments is lagging behind that of terminations.It is required transforming low resolution images received by terminations into high resolution images to achieve resolution matching.Therefore,image super-resolution is a very important algorithm module in visual terminal products and is one of the current hot topics in image processing research.Image super-resolution is a typical reconstruction problem under "incomplete information" conditions.The information available to solve the problem includes:1)human prior knowledge about the laws of image data;2)instance data that exists in pairs of low resolution and corresponding high resolution images.At the beginning of the rise of deep learning methods,scholars proposed purely data-driven deep super-resolution networks.However,high image dimension and abundant information content make the purely data-driven methods face the problems of high training sample requirements and operational complexity.With the rapid development of deep learning methods,the knowledge-data joint-driven deep network method,which utilizes both prior knowledge and information contained in data,has become the consensus of academia.Non-local and multi-scale priors are milestone priors in the field of image processing and describe the main rules of image data.Following the consensus of knowledge-data joint-driven methods,this thesis attempts to incorporate non-local and multi-scale priors into the convolutional neural network(CNN)and investigate image super-resolution algorithms based on prior knowledge and CNN.The main work includes:(1)An image super-resolution algorithm based on dual domain non-local(DDNL)network is studied.In order to reduce the difficulty of capturing and combining global features in existing deep super-resolution networks,the DDNL network introduces an additional prior network branch into the backbone network for image super-resolution calculation.The prior network branch firstly uses the nonlocal module to obtain non-local features of the image and gradient domains,and then processes them with multi-layer channel attention modules.This enhances the features in both spatial dimensions and channel dimensions.To better integrate the dual-domain non-local features into the main branch of the network,a nonlocal residual dense connection structure is designed to guide the network to use this prior information to learn deeper features and further promote the network to generate more refined texture structures.Experimental results show that the DDNL network can obtain better objective evaluation indicators and subjective visual effects,and obtain more significant improvement in super-resolution performance especially for images rich in texture content.(2)An image super-resolution algorithm based on multi-scale dual attention(MSDA)network is studied.Currently,most image super-resolution networks use fixed-size convolution kernels to extract local features,resulting the extracted feature information scale is relatively single.Thereby the MSDA network constructs a multi-scale module that can obtain features of different scales at the same level,providing multi-scale feature information for deeper feature learning.The multiscale module is a multi-branch structure composed of dual attention blocks and dilated convolutions,which can not only enhance the features extracted by each branch from dual dimensions,but also partly reduce the number of network parameters.Aiming at the problem image super-resolution network trained only with per-pixel loss is prone to make the reconstructed image over-blurred,the loss function of MSDA also includes content loss,gradient loss and region-level perceptual loss.Among them,region-level perceptual loss uses a pre-calculated binary mask to process the input of a discriminator,so that the discriminator result only acts on the texture area,avoiding the generation of artifacts in the flat region of the reconstructed image.Experimental results show the MSDA network has achieved certain improvement in objective indicators and visual quality with fewer parameters.