Research On Deep Learning-based Single Image Rain Removal Under Complex Scenario

Single image rain removal technology mainly studies how to extract rain information from rain images with complex distribution,and recover the clean background,which is of great significance for improving the ability of low-level visual perception and highlevel cognitive understanding.Compared with the traditional single image rain removal method,deep learning-based single image rain removal technology has made breakthrough progress.However,due to the diversity and complexity of rain distribution,as well as the increasing complexity of application scenarios,it still remains some challenging problems in single image rain removal research which need to be solved urgently:(1)Most of the existing rain removal methods adopt the supervised learning mode,whose synthetic paired data is selected as the training data,but it maintains a great gap between the rain distribution of the synthetic data and the real scenario data,which results in the poor generalization in real scenarios;(2)Most of the existing studies focus on removing single type of rain degradation,which cannot deal with the multiple types of rain degradation(such as rain streaks,raindrops,and rain fog)at the same time,resulting in insufficient robustness in real scenarios;(3)Most of the existing studies only consider the rain removal task but ignore the interaction and connection with high-level visual tasks,which results in poor performance of rain removal results on high-level tasks subsequently.In this dissertation,the above three challenging problems can be summarized as the poor generalization of rain removal in complex scenarios,the poor robustness of rain removal for multiple types of degradation,and the lack of collaborative interaction between rain removal and high-level tasks.These three problems are closely related and need to be researched progressively,i.e.,from a single rain degradation scenario to a variety of rain degradation scenarios,and then to the low-level and high-level task collaboration scenario.In this dissertation,three innovative strategies are proposed systematically for the mentioned problems.The main contributions in this dissertation are summarized as follows:(1)To solve the problem that the deep supervised rain removal method relies too much on synthetic paired data and the poor generalization of the model caused by the large distribution difference between synthetic data and real data,this dissertation focuses on the unsupervised deep rain removal mode based on the unpaired data.Chapter 3 of this dissertation systematically proposes a single image rain removal and rain generation method Derain Cycle GAN,which is based on an unsupervised rain streak attention detector.This method makes full use of Cycle GAN’s cyclic structure and has a great transfer learning ability to carry out unsupervised training learning;An unsupervised rain streak attention detector(URAD)is designed to enhance the ability of rain streak information detection,which takes both rain images and rain-free images into consideration.Under the unsupervised mode,this method can learn the distribution of real scenario rain images,so that Derain Cycle GAN has a stronger generalization ability.In addition,a new synthesis mode is proposed to automatically generate diversified rain streaks.The generated rain streaks have more complex shapes and orientations,which can help the existing supervised methods to better generalize to real rain images.(2)To solve the problem that most existing deep rain removal methods can only deal with a single type of rain degradation(such as rain streaks or raindrops),and the lack of robustness of rain removal in real scenarios,this dissertation also focuses on the rain removal methods of various types of rain degradation.Chapter 4 of this dissertation systematically proposes a robust single image rain removal method Rad Net,which can satisfy various types of rain degradation.First of all,this method designs a lightweight robust attention module RAM,which adopts a general attention mechanism to pay more attention to rain streaks and raindrops simultaneously;In addition,a multi-scale deep refining module DRM based on dual path residual dense blocks is proposed to precisely remove rain streaks.Rad Net has strong robustness and can handle not only different types of rain degradation,including raindrops,rain streaks,or both,but also different data paradigms,including single type,superimposed type,and blended type.(3)To solve the problem that most of the existing deep rain removal methods only focus on the low-level restoration effect,ignoring the interaction with high-level semantic information,which results in an insufficient performance of the restoration results in high-level tasks subsequently,this dissertation also focuses on the collaborative interaction between rain removal and semantic segmentation task.Chapter 5 of this dissertation systematically proposes a single image rain removal method SGINet guided by the high-level semantic segmentation information.A three-stage rain removal mode is designed innovatively.Firstly,based on the designed full-resolution module FRM,the coarse-derained image without context semantic loss is predicted.Then,the pre-trained segmentation extraction module SEM is used to extract the semantic information from the coarse-derained image.Then the semantic interaction module SIM is used to realize the rain removal under the guidance of semantic segmentation information.By linking the tasks of rain removal and semantic segmentation,this method can achieve better performance of rain removal and high-level semantic segmentation simultaneously.Besides,a synthetic rain image dataset Cityscapes＿syn and a real rain image dataset Cityscapes＿real are also constructed based on the Cityscapes dataset,which can be used to simultaneously evaluate the performance of image rain removal task and semantic segmentation task.