Video-based Rain Removal With Model Of Rain Line Movement

Rainfall,as a common bad weather,usually causes visual degradation of image blur and detail missing in the captured videos.Nowadays,outdoor computer vision system has been applied in the fields of automatic driving and traffic supervision.This low-quality video will lead to a decrease in the accuracy of tasks such as pedestrian detection and lane line recognition,thereby reducing the reliability of outdoor computer vision system.In recent years,the research on video deraining based on deep learning has achieved much progress by training on the synthetic datasets.However,the existing synthetic dataset and the video deraining networks are built on highly simplified video rain model– they assume that the rain streak layers of different video frames are uncorrelated.This way,the continuity of rain layers is not considered,which reduces the performance of the rain removal network in practice.To solve this problem,this thesis designs a video rain synthesis model by explicitly considering the model of rain motion,with which we develop a recurrent disentangled deraining network(RDD-Net)to improve the performance of video deraining.The new rain video synthesis model considers the continuous motion between adjacent frames of the rain layer,and simplifies the implementation process by using continuous sampling.Based on this model,we construct a rain video dataset that contains 80 video clips.Based on the synthesis model,RDD-Net treats each adjacent frame of a considered frame as a separate source to extract inter-frame temporal information,and then fuses it into the spatial feature of the considered frame through the residual structure.After that,a disentangled temporal module is used to predict the rain streak layer,the rain motion from the adjacent frames to the considered frame,and the rainfree background layer from the fused features.Finally,an adaptive recovery module is included to calculate the weight of each initial prediction result based on the attention mechanism and predict the final recovery result.Comprehensive experiments are conducted on three public datasets and the proposed synthetic dataset.Quantitative and qualitative evaluations verify the effectiveness of the method.The practicability of the method is also verified by visually examining the results on real rain videos.A set of ablation studies are also conducted to justify the usefulness of each module of the proposed network.