Image De-raining Via Dual-branch Information Fusion And Joint Spatial-frequency Feature Extraction

The purpose of single image de-raining is to remove a range of image degradation problems caused by rainfall in an image,where the removal of raindrops and rain streaks are two common phenomena in the image de-raining task.For example,images with raindrops or rain streaks taken in bad weather conditions can degrade the image quality.Therefore,it is very necessary to remove rain for a single image.This thesis summarizes and analyses the current research status of image de-raining.We propose an image de-raining algorithm combining multi-scale feature extraction and dualbranch information fusion and an image de-raining algorithm combining joint space-frequency feature extraction and multi-level feature reconstruction based on deep learning and other techniques follows.1.Image De-raining by Combining Multi-scale Feature Extraction and Dual Branch Information Fusion(MSDBF)In the multi-scale structure framework,an image de-rain algorithm(MSDBF)combining multi-scale feature extraction and dual-branch information fusion is proposed.The algorithm extracts image features from coarse to fine in a multi-scale input way,and proposes a twobranch information fusion module.The module takes cross-branch bidirectional interaction subblocks as the core,combines attention mechanism,and makes comprehensive use of the advantages of feature map channel information and spatial information,and realizes the highlevel and effective integration of the global and local information of the image at the corresponding scale.In order to realize residual restoration of "rainfall interference" from low resolution to higher resolution,supervised enhancement is carried out on the extracted singlescale depth features,and more valuable "rainfall interference" information is transmitted to the adjacent feature extraction sub-network with higher resolution,so as to obtain clear restoration images.Based on two datasets of image raindrops removal and image rain streaks removal,our algorithm is compared with other image raindrops removal and rain streaks removal tasks,which shows the effectiveness of the algorithm for image raindrops removal and rain streaks removal.2.Image De-raining by Combining Spatial-frequency Joint Feature Extraction and Multi-Level Feature Reconstruction(SFEMFR)Based on U-shaped network structure,an image rain removal algorithm(SFEMFR)is proposed,which combines spatial-frequency feature extraction and multi-level feature reconstruction,to alleviate the limitations of single feature extraction.The whole network uses the space-frequency joint feature extraction module as the encoder and decoder,in which the residual module combining the fast Fourier transform and the convolution process is the main feature extraction part,so as to achieve the purpose of combining the global feature extracted from the frequency domain and the multi-scale local feature extracted from the spatial domain.In order to retain the high frequency and detail information better while recovering the clear image,a feature refinement reconstruction module is added after the output of the decoder,and the features of the output of the encoder and decoder at different stages are gradually integrated from bottom to top,so as to promote the refinement of the image features and get a clearer restored image.Based on four datasets of image raindrops removal,image rain streaks removal,image noise removal and image blur removal,the algorithm in this chapter is compared with its corresponding image restoration tasks,which verifies the effectiveness of the algorithm not only for image raindrops removal and rain streaks removal,but also the feasibility and applicability of this algorithm for other restoration tasks.3.Application of Image De-raining ModelIn order to verify the practicality and effectiveness of the two algorithms in this thesis,it is tested from two aspects respectively.On the one hand,the real rain outdoor scene images are tested to show that this algorithm has certain rain removal effect on the rain-containing images in natural scenes.On the other hand,for more advanced vision tasks,such as target detection,the algorithm can be used as a pre-processing step for rain image processing,improving the performance of subsequent tasks.