Single Image Rain Removal Based On Generative Adversarial Network

Different weather conditions,such as snow,rain,fog,etc.,have a huge negative impact on images or videos.Single-image rain removal has always been a challenging and ill-posed problem,and can be widely applied in real life,such as: object tracking and detection,intelligent driving and so on.At first,single-image rain removal is mainly based on the image decomposition model.The shortcomings of this kind of algorithms lie in ignoring the low-frequency component and hardly distinguishing the atoms of background and rain streaks.The de-rained image is blurring or remaining rain streaks.With the excellent performance of neural network in various applications of image processing,scientific community directly use rainy images as the input of different neural network models,to achieve nonlinear mapping from rainy images to rain-free images.Compared with traditional methods,the algorithms,based on neural network,have improved the effectness.Howerver,due to high dimensionality of the input,the trained models are tend to be under-fitted and easily neglect the details of images,which lead to various problems,such as color distortion and texture distortion.This paper is based on generative adversarial network single-image rain removal model.To reduce its high input dimensionality,we improve the rain removal model from two different aspects: first,dividsion into image blocks.That means,input image is directly divided into multiple blocks with same size,and each block is used as the input to network,to reduce the input dimensionality.Meanwhile,to maintain the continuity of color and texture among blocks,we employ the weighted sum of adjacent blocks to predict current block.The weights are defined by similarity and distance between the current block and adjacent ones,which is learned from the notions of bilateral filter and non-local mean algorithm.Then,the difference between predicted and ideal block is added to loss function of the generative network,to finetune it during training.Finally,the result is generated by sliding window,whose size is the same as block when training.Second,multi-scale model.We take use of Laplacian Pyramid,to build a multi-scale model from the bottom to top,to reduce the dimensionality.The advantages of Laplacian Pyramid are that information on each scale does not overlap with each other and little information is loss.At each scale,a generative adversarial network is separated trained,to realize nonlinear mapping from rain component to rain-free one on this scale.Large scales contain texture details in the image,and the smallest maintain the lowest frequency part of the image.Compared to the original image,the amount of information contained in each scale is so small,that trained rain removal model can take care of texture details better.Finally,from the top to bottom,the sum of two adjacent scales propagates step by step,to fuse the derained result with good visual quality.Single-image rain removal algorithms,proposed in this paper,do not require any prior knowledge,preprocess or post-process,which ensures the integrity of the whole framework.The experimental results clearly show that,compared with previous de-raining algorithms,our proposed algorithms can efficiently remove raining streaks and accompanying veiling effects,while preserving texture and edges of clean background as much as possible and avoiding color distortion.For synthetic rainy image databases,our proposed algorithms are better than previous methods.As for the PSNR quality index,we can improve 2dB?7dB.For natural rainy images,we can find that,derained images,which are generated by our models,have better visual quality,whether from texture or color aspects.In addition,this paper suggests that our proposed rain-removal models can be directly migrated to snow removal.