Research On Image Super-Resolution Reconstruction Based On Self-Attention Mechanism

Image super-resolution reconstruction aims to improve the resolution of low-quality images and restore high-resolution images with rich texture details.With the continuous development of deep learning,image super-resolution reconstruction networks based on convolutional neural networks often increase complexity to continuously improve performance,resulting in huge computational costs.Moreover,convolutional operations are locally limited by the receptive field,but establishing long-distance dependencies is crucial for image super-resolution reconstruction.The self-attention mechanism proposed by Transformer performs well in learning global information of images.Recently,Swin IR constructed an image restoration model based on the window based self-attention mechanism of Swin Transformer,achieving the most advanced level at that time with fewer parameters.However,the fixed window size is difficult to balance between computational cost and performance.This thesis proposes three end-to-end image super-resolution reconstruction methods to address the limitations of Swin IR,such as limitations of fixed window,unstable training,blurry reconstruction details,and high computational costs.(1)A multi-scale window based self-attention method for image super-resolution reconstruction is proposed.Firstly,a multi-scale window based self-attention is designed to improve the limitations of fixed window size.Different scales of window groups are set in the channel direction,and the parallel results of multiple branches of window self-attention are concatenated and fused to learn diverse features under different scale windows,flexibly adjusting the relationship between long-distance dependency modeling performance and computational cost.Secondly,to enhance the stability of the model during training,Swin Transformer layer is set as a post-normalization structure,effectively reducing the oscillations.Finally,a post-processing strategy is proposed to integrate the weights of several best-performing models during the testing phase,unleashing greater potential during the training process of the model.The experimental results show that the evaluation metrics and reconstruction effects of this method are improved to a certain extent,and the training curve is more stable.(2)A combining multi-scale window based self-attention and frequency features method for image super-resolution reconstruction is proposed.Based on the Octave convolution,a frequency feature processing and fusion module is designed to simplify low-frequency components,retain high-frequency components that represent edge details,and balance the intra-group and inter-group information transmission of high and low-frequency groups.This module is cascaded with multi-scale window based self-attention to fuse frequency features into long-distance feature extraction,further improving the performance and efficiency of the model.Meanwhile,dilated convolutions with multiple dilation rates are designed to expand the receptive field and more fully learn the shallow information distribution of the input image.Compared with other advanced algorithms,the experimental results show that this method achieves significant improvements in reconstruction metrics and produces images with rich and clear texture details.(3)A method for image super-resolution reconstruction based on lightweight self-attention is proposed.A lightweight self-attention mechanism is designed to compress the key and value matrices,and sample and retain representative features,reducing the computational cost by half compared to the original self-attention.To verify the effectiveness of the lightweight self-attention,a lightweight model for image super-resolution reconstruction is constructed by combining the lightweight components of the first two methods.Compared with other advanced lightweight methods,this method not only improves the reconstruction metrics but also reduces the number of parameters and improves the inference speed compared to the Swin Transformer-based lightweight method.