| With the rapid development of the information age,image super resolution has been widely used in image compression,security monitoring,medical image system,satellite remote sensing and many other fields.With the rapid development of the information age,image super resolution has been widely used in image compression,security monitoring,medical image system,satellite remote sensing and many other fields.In practical applications,due to the limitations of equipment,sensors and other reasons,the obtained low resolution image greatly affects people’s understanding of the image,and people expect to get a high resolution image.Therefore,it has become a hot topic in the field of computer vision to study super resolution reconstruction methods to improve image quality.After a large amount of literature research,this paper summarizes three problems that need to be solved by image super resolution algorithm,including weak adaptive ability of processing different features,poor reconstruction effect of real image with unknown degradation mode and model lightweight.From the perspective of dynamic convolution,divergence to convergence method and Swin Transformer,corresponding solutions are proposed.The main focus of the following three aspects of work in practical applications,due to equipment,sensors and other reasons,the obtained low resolution image greatly affects people’s understanding of the image,people are more expected to get a high resolution image.Therefore,it has become a hot topic in the field of computer vision to study super resolution reconstruction methods to improve image quality.After a large amount of literature resea rch,this paper summarizes three problems that need to be solved by image super resolution algorithm,including weak adaptive ability of processing different features,poor reconstruction effect of real image with unknown degradation mode and model lightweight.From the perspective of dynamic convolution,divergence to convergence method and Swin Transformer,corresponding solutions are proposed.The work mainly focuses on the following three aspects~*:(1)For synthetic images,common super-resolution models are ineffective in dealing with the reconstruction of low-resolution images with different contents.Based on the idea of Dynamic convolution(Dy-conv),a Locally Enhanced Dynamic Back Projection of Image Super-Resolution Networks(LE-DYBN)based on dynamic convolution is proposed.Firstly,rich texture features are obtained by shallow feature extraction module.Then,the local enhancement module is used to enhance the detail features by combining the local sensitive hash algorithm and non-local attention.At the same time,the self-similarity of the image is fully utilized to further improve the accuracy of the model to estimate the high-resolution initial image.Finally,dynamic convolution is introduced,and the weight of convolution is adjusted adaptively according to different input images,and multiple parallel convolution cores are dynamically aggregated according to the calculated weight values.The nonlinear mapping relationship between low resolution image and high resolution image is learned by dynamic back projection module,and finally the adaptive ability and reconstruction effect of the model are improved.Compared with the baseline model DBPN(Deep Back-projection Networks),LE-DYBN improves the peak signal-to-noise ratio by 0.42d B on Set14 data set(×2).The results of subjective and objective experiments on four public data sets show that the proposed method has good performance.(2)For images in the real world,the degradation process is unknown and complex,and there may be different degradation modes of different parts of the same image.In this paper,a Dynamic Multi-Scale Enhancement for Super Resolution(DME-SR)algorithm for real world image is proposed by combining the idea from divergence to convergence.Firstly,a multi-scale feature extraction module based on residual blocks in divergent networks is used to obtain rich high-frequency feature information.Then,a selective nuclear network is designed to fuse multi-scale features,which dynamically combines features and preserves the most original feature information at each spatial resolution.Secondly,in order to generate the results closer to the real high-resolution image,the results generated by the divergent network are fused in the convergence part.Considering the different contribution weights of different regions to the final results,the divergent predictions are further combined one by one by pixel weighting to make the details of the reconstructed results more accurate.The results of subjective and objective experiments in public data sets show that the proposed algorithm DME-SR can generate SR output with more real and natural textures,and has good feasibility and effectiveness.(3)In view of the problems such as complicated calculation of image super resolution model,large number of parameters,and difficult deployment in mobile devices such as mobile phones,this paper is oriented to the needs of practical engineering.A Blueprint Separable Swin Transformer Lightweight Image Super Resolution(BSST-SR)algorithm is proposed.In order to extract rich feature information and make use of the excellent performance of blueprint separable convolution in efficiency standard convolution separation,a shallow feature extraction module based on blueprint separable convolution i s designed in this paper,which can extract rich feature information and reduce redundant information.Swin Transformer is introduced into the deep feature extraction module to complete content-based interaction between image content and attention weight,and long-term dependence model can be realized by shifting window mechanism.In order to further enhance the extracted deep features,two effective attention modules are introduced in this paper to enhance the capacity of the model without increasing the amount of computation,so as to achieve the purpose of a lightweight model.The subjective and objective comparison experiments with the mainstream algorithm on two public data sets show that the proposed algorithm BSST-SR can greatly reduce the number of model parameters while maintaining stable performance,and has good performance. |
