Single-Image Rain Removal Via Multi-Scale Cascading Image Generation

In the field of computer vision,the task of single-image rain removal is a very challenging topic.Under rainy days or other harsh conditions(such as snow,haze and fog),the images or videos we get will often be unclear or obscured.Some important image data will therefore become unusable.Therefore,the work of image rain removal presents very important significance.It can be an essential pre-processing step for the task of image retrieval,target detection and tracking,and unmanned vehicle automatic driving systems.For the traditional image processing,the single-image rain removal mainly depends on the image decomposition as high frequency and low frequency bands for the training of the rainy and rainless dictionary.However,these methods and algorithms cannot accurately separate the rainwater from the image.It often creates blurry effects.In the last decade,the structure of deep learning has been greatly improved the performance of the tasks in computer vision.Researchers have used the neural network to learn the map of rain layer on rainy image and obtained good results.The current deep learning-based methods mainly focuses on learning a rain layer subtracting by the origin rainy image.Considering that these methods only use L1/L2 norm to train and haven't taken multi-scale process into account,it will lead to color distortion and loss of detailed textures.In some cases,rain is still not completely cleaned up.Secondly,for the scenes with heavy rain or varying rain shapes,the current typical methods cannot deal with these situations well.The rain streaks cannot be completely removed.With regard to these problems,in this paper,we propose a multi-scale cascading image generation de-raining model.First,we adopt the idea of generation adversarial network(GAN)to directly generate a de-rained image instead of subtracting a learned rain layer from the original rain image,which can reduce the blurring of the image details and can maintain higher consistency with the colors and textures of the original image.In addition,we have noticed that on a rainy picture,the shape and the length of rain streaks are often variable.Accordingly,it is not rational to analyze rainy image from a single scale.We should observe and process rainy image across the scale space.Thus we propose to construct a multi-scale encoderdecoder model in the GAN based network to obtain promising rain-removal results.In the encoder,we input the rainy image to extract features through multi-scale convolution block.In the decoder,through the deconvolutional neural network,we will use the features obtained from the encoder to generate three de-rained images from coarse scale to fine scale.In particular,we have fused the high-level features at coarse scales during the generation of fine-scale de-rained image.This top-down method helps to optimize the de-raining performance at the finest scale.In addition,for the model training,we combine L2 loss,adversarial loss and perceptual loss to build a hybrid objective function,which improves the consistency of the generated results with the ground truth across the whole scale space.Moreover,we use a multi-task decision mechanism to constrain the results at different scales to be consistent with the ground truth,which will greatly improve the accuracy of our de-rain model.Our proposed rain removal method in this paper doesn't require prior knowledge and image pre-processing.The trained model can conduct rain removal in real time.The experimental results demonstrate that our methods achieves better performance than other state-of-the-art algorithms on multiple test sets with 1-4 dB improvement of PSNR.