Research On Image Super-resolution Based On Deep Networks With Attention Mechanism

As an important branch in the field of computer vision,image super-resolution makes use of computer technology to restore the highresolution images from the low-resolution images.It has important applications in satellite maps,video surveillance,traffic detection and other fields.To overcome the problems in various downsampling methods and the ill-posed issue in reconstructed images,this paper has designed some image super-resolution methods to effectively improve the quality of reconstructed images based on deep learning: reconstruction attention based convolutional neural network,reconstruction attention based asymmetric convolutional neural network,multi-Scale attention based U-shape convolutional neural network and two-Stage coordinating deep convolutional neural network for super-resolution.The details are as follows:1.Reconstruction attention based convolutional neural network for image super-resolution.Most existing deep networks with attention mechanism don't consider the problem that each feature in the part of image reconstruction has different influence on the final result.To solve this problem,we have proposed a reconstruction attention mechanism and constructed Reconstruction Attention based Convolutional Neural Network(RAN)for image super-resolution.The proposed method firstly uses multi-scale convolution kernels to extract features at different scales;then the channel attention is introduced to select the features;finally the reconstruction attention is proposed,that is,the high-resolution feature outputs from each unit are weighted and combined to obtain the final high-resolution image.Therefore,the proposed multi-scale convolution kernels can help the network to extract features at different scales,the channel attention can select the important feature information in the same convolution layer,and the reconstruction attention can select the important reconstructed feature layers.Experimental results show that the proposed method has better reconstructed effects than some existing super-resolution methods.2.Reconstruction attention based asymmetric convolutional neural network for image super-resolution.As the extraction ability of the square convolution kernel is limited in the existing deep convolutional neural networks,we have adopted the asymmetric convolutions to replace the traditional square convolutions.At the same time,the different modules in the reconstructed part are not considered in detail in existing networks,we have proposed the concept of dividing the reconstructed part of the network and constructed Reconstruction Attention based Asymmetric Convolutional Neural Network(RAAN)for image super-resolution.Firstly the proposed method uses asymmetric convolution kernels with different sizes to extract features at different scales;secondly the channel attention is introduced to select the important image channels;then the corresponding reconstructed images are obtained by methods of main reconstruction,residual reconstruction and the upscaled reconstruction.Finally,the final part of network is added with the above three images together to obtain the final reconstructed image.Therefore,the proposed structure of asymmetric convolution can help the network to extract more image feature information and various reconstructed parts can help improve the quality of reconstructed images.Experiments show that the network has better reconstructed quality than some existing super-resolution methods.3.Multi-scale attention based U-shape convolutional neural network for image super-resolution.As the existing convolutional neural networks rarely consider the image details,we have proposed a multi-scale attention mechanism and constructed Multi-Scale Attention based U-shape Convolutional Neural Network(MSAUN)for image super-resolution.Firstly the method uses U-shaped network to extract image information;secondly the integration of image information is introduced to realize the flow of among different size images information;then multi-scale attention,the weighted combination of images of different sizes,is proposed;finally,the integration of multi-source images is used to obtain the high-resolution image.Therefore,the multi-scale attention proposed in this paper can select important features from multi-scale images.Experiments show that the proposed network can achieve better reconstructed results than some existing reconstruction methods.4.Two-stage coordinating deep convolutional neural network for image super-resolution.As the existing convolutional neural networks can only deal with single down-sampled images,we have adopted the sharing parameters in the feature extraction part,and according to the data,we divide the network into two phrases to construct Two-Stage Coordinating Deep Convolutional Neural Network(TSCN)for image super-resolution.The method firstly uses three-layer convolutional kernels to extract image's shallow features;then four simple unit structures are used to extract the deep features of images;finally,the separating outputs are used to train the different down-sampled data.The experimental results show that the proposed super-resolution method can effectively coordinate the data of all aspects and achieve better reconstruction effects under multiple downsampled data.