Research On Image Rain Removal For Subsequent Tasks

In rainy scenes,rain streaks can cause severe occlusion of the acquired background image,resulting in blurred image information and reduced background visibility,which further affects the accuracy of subsequent tasks(object detection,image segmentation).Therefore,the single image rain removal task is a pre-processing process for the subsequent high-level vision tasks,and the performance of the subsequent high-level vision tasks can be further enhanced through rain removal pre-processing operations.Research on datadriven deep learning image rain removal algorithms has become mainstream,while the performance of deep learning algorithms mainly relies on the quality of training data and the design of network models.This thesis proposes a lightweight rain removal algorithm and rainy image data generation technology for pre-processing algorithms deployment and generalization requirements of subsequent tasks,and designs a prototype system.The specific contents are as follows:(1)Lightweight single-image rain removal algorithm incorporating visual saliency.To address the problem that existing data-driven image rain removal algorithms often pursue complex and large model design to improve rain removal performance,which leads to an excessive amount of parameters and slow running speed,and is not beneficial to outdoor device deployment.In this thesis,the dilation convolution and the Convolutional Block Attention Module(CBAM)are used to form a lightweight single-image rain removal algorithm.Among them,the dilation convolution with different dilation factors can extract the contextual information of the input rainy image at different scales,gradually extracting the spatial information from local to global.the CBAM attention mechanism can assign different weights to the features in terms of channel and spatial dimensions to guide rain removal,meanwhile making the network more easily embedded into resource-constrained hardware devices.Extensive experiments are conducted on some typical rainy scenarios in synthetic and real datasets.The experiments show that the proposed lightweight image rain removal algorithm incorporating visual saliency has a much smaller covariate size than most baseline rain removal algorithms,and also achieves a similar rain removal performance to the baseline algorithm.(2)Dual joint adversarial rain rendering algorithm based on convolutional sparse coding.Most of the existing mainstream data-driven image rain removal algorithms train their models on synthetic datasets,which have a single pattern of the rain streaks,and the quality of the synthetic rain image will limit the performance of the image rain removal algorithm and further affect the execution of subsequent high-level vision tasks on rainy.In this thesis,we improve the performance of the image de-rain algorithm from a data perspective and propose a prior knowledge-based guided rain rendering algorithm.The initial rain streak is generated using sparse noise and Gaussian motion fuzzy kernel convolution,and the initial rain streak is used as the input of the generative adversarial network,which can facilitate network convergence by enabling the network to incorporate physical prior knowledge under the guidance of the initial rain streak.At the same time,the rain image pairs generated by the dual generator and the real rain image pairs are discriminated twice between true and false,and rain streaks with different directions,thicknesses and scales can be generated recursively.Extensive experiments show that the rain rendering model proposed in this thesis can accurately generate rain streak types with different thicknesses,directions and scales that are consistent with the statistical distribution of the rain dataset.Joint training of the generated rain dataset with the benchmark dataset can significantly improve the performance of existing image rain removal algorithms.(3)A prototype system combining single-image rain removal algorithm and subsequent high-level visual tasks is proposed.In this thesis,a detection system for subsequent highlevel visual tasks in rainy traffic scenes was designed using Py Qt.The system can perform a single image rain removal task,subsequent high-level visual tasks on the input rainy images,as well as processing both the image rain removal task and subsequent high-level visual tasks simultaneously.The system directly outputs the rain-removed image along with corresponding results of object detection and image segmentation.