Research On Image Recovery Algorithm Under Bad Weather Conditions

Image quality can be degraded by many factors during imaging,for example,image noise caused by the incomplete imaging systems,some pragmatic and robust solutions have been proposed to solve this kind of image degradation.This thesis brings special at-tentions to the image degradation caused by the changed propagation media.Specifically,haze,rain and snow under bad weather conditions will be captured by cameras inevitably.Both the colors and contrasts of images can be affected by these weather factors.Espe-cially,rain and snow occlude some image contents,so that the information conveyed in the images will be changed irreversibly.These degradations can impair the visual qual-ity of images.More importantly,they will ruin the useful image information,so that the performance of some outdoor vision systems and some computer vision algorithms will decrease,sometimes even deactivate them.Hence,the image enhancement/image recov-ery under bad weather conditions possesses practical applicable values.Apparent improvements of image enhancement under bad weather conditions have been achieved by existing algorithms.But these achievements mainly focus on the de-hazing task.While,the researches on the more complex rain and snow weather are rel-atively fewer.Existing deraining or desnowing methods mainly utilize filters,image de-composition and deep learning,their limitations are the following two aspects:(1)some traces of rain/snow still remain in the final deraining results;(2)some image details are misregarded as rain/snow and removed.This work deeply studies the characteristics of rain/snow in single color images and the degradation principles caused by them,so that a clear rain/snow-free background can be obtained via conventional optimization methods or deep learning based methods.The major contents and innovations of the thesis are as follows:1.This thesis proposes an accurate rain pixel detection method via sufficiently an-alyzing the characteristics of rain streaks.To remove rain streaks in single images,this work derives an linear relationship to formulate the influence of rain on the background pixels from the physical imaging process.By combining the detection of rain streaks and the linear model,we build and optimize a convex loss function to determine the parame-ters in the model,then the rain-removed results can be obtaind from the inverse expression of the linear model if rainy images are given.2.This work decomposes a rainy/snowy image into a rain/snow-free low-frequency part and rainy/snowy high-frequency part via a guided filter,then tries to discriminate rain/snow in the high-frequency part to recover degraded image.However,for the rain/snow pixels with high intensities,rain/snow-free low-frequency cannot be obtained only by a guided filter.Hence,this work introduces rain/snow pixel detection to help image decomposition.To extract image details in high-frequency domain,we learn an over-complete dictionary and design a 3-layer identification and reconstruction mechanism for non-rain/snow dictionary atoms.The clear image is obtained by adding the reconstructed image details and the low-frequency part.3.Not only rain streaks but also haze-like effect will appear in a rainy image,es-pecailly for the heavy rain condition.The haze-like effect is caused by the accumulation of tiny raindrops,which will lower the contrasts of an image and shallow the object col-ors.The degradation by haze-like effect is difficulty for conventional methods.Hence,this work builds a deep neural network to simultaneously remove rain streaks and haze-like effect via supervision learning.Specifically,we build a new model to formulate the influence of rain streaks and haze-like effect on an image,based on which,we design a two-branch network structure to learn the parameters of our rainy model,so that the back-ground can be obtained by our model in an inverse way.Besides,another subnetwork is designed to control the strength of haze-like effect in the final deraining results to obtain different visual effect.4.Rainy image can be considered as the scattering result of rain to atmospheric light and reflection light of background.This work decomposes the influence of rain into two parts according to the size of raindrops.One is the apparent rain streaks caused by large raindrops,and the other is the haze-like effect caused by the accumulation of tiny raindrops.By modeling the transmission of these two parts,we build a more complete model to formulate the form of rainy images.Then,three subnetworks are built to learn the parameters of our rain model to enhance the degraded rainy images.5.Due to the more apparent characteristics of rain in gradient domain,this work fo-cuses on combining the information in spatial and gradient domain via GAN to generate better deraining results.Specifically,we utilize gradient-assistant encoding in the gen-erator and also optimize generator in both spatial and gradient domain to generate more favorable deraining features.For discriminator,gradient acts as the condition to provide more accurate rain/non-rain information to enhance its accuracy.In terms of network structure,we revise ASPP to relieve the abnormal deraining results.Experiments show that our method improves the deraining performance substantially.6.This work studies the intrinsic priors of rainy images and develops corresponding intrinsic loss functions to constrain the training of our deraining network via a maximum likelihood estimation.Moreover,we design an anxiliary decoder structure to boost our encoder to learn more favorable deraining features for our main decoder.Accordingly,an information sharing operation is applied to fuse the deep features from different-scale spaces to boost our performance further.Moreover,our auxiliary decoder facilitates sep-arately studying the properties of deep features from different scale spaces and their re-spective contributions to deraining performance.