| Rainfall scenes such as rain streaks,raindrops,and rain fog in rain images can cause serious interference with the background information in the images,which not only affects the visual effect,but also reduces the reliability and accuracy of some outdoor computer vision systems.Therefore,how to effectively remove the rain streak information from degraded images and recover the corrupted background texture information is a problem that deserves to be studied.In order to solve the single image rain deraining problem,this paper designs image deraining algorithms using the advantages of both convolutional neural network and visual Transformer techniques.First,to address the problems of insufficient deraining and poor generalization in common convolutional neural network-based deraining methods,an end-to-end image deraining network is designed,focusing on learning multi-scale feature information to generate clear and rain-free images.Then,to address the problems such as lack of non-local information modeling capability of convolutional neural network and over-parameterization of Transformer which increases the complexity of the system,we further explore the combination of Transformer and convolutional neural network to design a hybrid lightweight image deraining method.Finally,experiments on many benchmark datasets demonstrate that our proposed method outperforms the state-of-the-art methods in terms of quantitative metrics and qualitative analysis,and has good generalization capability,while the computational cost and number of parameters are much smaller than other advanced method.The details of the study are as follows:(1)An end-to-end image deraining algorithm based on convolutional neural networks,called combining multiscale learning and attention mechanisms densely connected network for single image deraining,is proposed.Specifically,Multiscale segmentation attention module and Dense Net are employed to construct the overall frame-work,where the multiscale segmentation attention module aims to use the attention mechanism to learn the feature maps of rain areas,and the Dense Net helps to enhance the feature reuse.Furthermore,considering the important in-formation on multiscale features,a multiscale feature learning module is proposed,where the re-parameterization VGG is used to extract different scale feature maps and effectively characterize the rain streaks features.(2)A lightweight image deraining algorithm combining Transformer and convolution,called non-linear recursive Conv-Transformer network for single image deraining,is proposed.First,a feature extraction module based on the dual branching of convolutional network and Transformer is designed to combine the local modeling capability of convolutional network and the non-local modeling capability of Transformer to capture the global context;second,based on the architecture of residual recursive network,we use a nonlinear projection module to implement constrained recursion and employ channel attention to fuse the multi-branch residual features,thus achieving network lightweighting.(3)A Transformer image deraining algorithm based on a lightweight encoder-decoder architecture,called channel transformer based on full convolutional decoder for single image deraining,is proposed.First,a novel channel Transformer module is designed to obtain global contextual information,in which deeply separable convolution is applied to efficiently learn multi-scale local features,and the Transformer encoder is formed by stacking this module.Second,a decoder based on a full convolutional architecture is designed to achieve feature progressive fusion and feature recovery using mask attention and reverse bottleneck convolution,significantly reducing computational complexity and GPU memory requirements. |
