Fusing Deep Learning Model And Attention Mechanism For Image Deraining

The purpose of the image deraining task is to remove rain from rainy images captured outdoors for obtaining clear rain-free images.The deraining task is of great significance for improving the quality of video surveillance and upgrading the accuracy of object detection and semantic segmentation in rainy environments.However,the rainy images acquired in different rainy conditions usually have rain streaks of various directions and sizes.The existence of rain streaks will seriously degrade the image,block the key object structure and detail information in the background scenes,and affect the implementation of subsequent computer vision tasks.The image rain removal method can eliminate the rain streaks in the image data and restore the scenario content to improve the visual quality.Nevertheless,rain streaks are complex and variable in the real world.It is difficult to completely eliminate rain even a small amount.The existing image deraining algorithm can upgrade the degeneration of rainy images to a certain extent,but the model-driven methods based on hand-crafted assumptions can only remove a small amount of rain in the image,and most of data-driven are not sufficient in feature extraction and utilization.There is much room for promotion in the derained results of these methods.How to obtain clean rain removal results and improve image quality has become a basic and difficult problem in computer vision.Considering existing important limitations in the image deraining task,we propose three targeted works to solve the obstacles faced and obtain satisfactory derained results.Firstly,an image rain removal approach based on a recurrent context-aware multi-stage network is proposed in the first work to deal with complex rain streaks in heavy rainy scenarios for completely deraining in different conditions.Then,to solve the problem of detail loss in the derained outputs,the second work introduces an image deraining method based on multi-scale enhancement and aggregation encoder-decoder network.Finally,a semi-supervised rain removal method based on a residual network is presented in the third work to improve the network generalization ability on real-world images.The main contributions of this paper are as follows:Firstly,considering that rain streaks in heavy rainy scenes are usually complex and overlap each other,the limited network receptive field makes it difficult to completely remove rain streaks.In order to completely eliminate the rain streaks that cover the scene information,an image deraining method based on a recurrent context-aware multi-stage network is proposed to effectively utilize the contextual information and expand the receptive field for obtaining the rain-free image stage by stage.Secondly,to recover details that were lost after the removed rain streaks,a rain removal approach based on multi-scale enhancement and aggregation encoder-decoder network is developed to capture multi-scale features,and the encoder-decoder network structure is optimized to effectively transmit informative features from the encoder stage to the decoder stage,and finally,the blocked background content will be accurately predicted.Thirdly,the residual network is applied to obtain effective features,and the advantage of the semi-supervised method is employed to generate high-quality clean images.A semi-supervised deraining approach based on residual network is presented to better fuse supervised and unsupervised learning parts.SSIM loss constraint is employed for the supervised learning part.Total variational regularization and Gaussian likelihood term function are applied to constrain the unsupervised learning part.The K-L regularizer is used to constrain the whole network.Finally,the proposed method can improve the generalization capacity of the network and better process rainy images in the real world.