Incorporating Conventional Priors And Deep Neural Networks For Image Restoration

Image restoration is a typical ill-posed problem in the digital image processing and computer vision community.Capturing digital pictures may suffer from some inevitable disturbance,such as camera shake,object motion,depth variation,and hazy/foggy weather,which generates de-gradated(e.g.,blurred or hazy)images.Image blur or haze offers human undesirable poor pictures and affects the performance of the following computer vision tasks,such as object detection and target tracking.Thus,it is of great significance to recover clear/clean images from de-gradated ones by some specific image restoration approaches.Recent success in image restoration problems mainly comes from image priors and deep convolutional neural networks(CNNs).Image priors aim at exploiting the statistical difference between the de-gradated images and clear ones and designing a response function that is capable of distinguishing clear images from de-gradated ones.By embedding the response function as a prior into the algorithm,it favors clear/clean images over de-gradated ones during the optimization.On the other hand,deep CNNs learns a mapping function from de-gradated images to clear/clean ones.After training on a large scale of paired data,the networks can recover the clear images by given a de-gradated one.In this thesis,we incorporate the image priors and deep CNNs for image restoration,mainly deal with image deblurring and image dehazing problems.The contributions of the dissertation are summarized as follows:Firstly,we propose a deep regression neural network for motion-kernel size estimation by formulating the problem as a regression task and constructing a deep CNN to solve it.Given a blurred image,the network can predict the width and height of the blur kernel,respectively.However,conventional regression network applies fully-connected layer,which makes the input size of the network fixed.It limits the ability of predicting arbitrary sizes.To address this problem,we propose to use a relative scheme for labeling the training data,which enlarge the range of the network predicting kernel sizes.Numerous experimental results demonstrate that the proposed network can precisely predict the kernel size of an input image,thus offers a reliable input of conventional blind deconvolution algorithms,which can improve their performance as well as the efficiency.Sencondly,we propose a generic and robust data-driven discriminative prior for blind image deblurring.The work is motivated by the fact that a good image prior should favor sharp images over blurred ones.In this work,we formulate the image prior as a binary classifier using a deep convolutional neural network.The learned prior is able to distinguish whether an input image is sharp or not.Embedded into the maximum a posterior framework,it helps blind deblurring in various scenarios,including natural,face,text,and low-illumination images,as well as non-uniform deblurring.However,it is difficult to optimize the deblurring method with the learned image prior as it involves a non-linear neural network.In this work,we develop an efficient numerical approach based on the half-quadratic splitting method and gradient descent algorithm to optimize the proposed model.Furthermore,we extend the proposed model to handle image dehazing.Both qualitative and quantitative experimental results show that our method performs favorably against the state-of-the-art algorithms as well as domain-specific image deblurring approaches.Next,we propose a deep neural convolutional network that exploits the depth prior information for dynamic scene deblurring.Given a blurred image,we first extract the depth map and adopt a depth refinement network to restore the edges and structure in the depth map.To effectively exploit the depth prior,we adopt the spatial feature transform layer to extract depth features and fuse with the image features through scaling and shifting.Our image deblurring network thus learns to restore a clear image under the guidance of the depth prior.With substantial experiments and analysis,we show that the depth information is crucial to the performance of the proposed model.Extensive quantitative and qualitative evaluations demonstrate that the proposed model performs favorably against the state-of-the-art dynamic scene deblurring approaches as well as conventional depth-based deblurring algorithms.At last,we propose an effective semi-supervised learning algorithm for single image dehazing.The proposed algorithm applies a deep Convolutional Neural Network(CNN)containing a supervised learning branch and an unsupervised learning branch.In the supervised branch,the deep neural network is constrained by the supervised loss functions,which are mean squared,perceptual,and adversarial losses.In the unsupervised branch,we exploit the properties of clean images via sparsity of dark channel and gradient priors to constrain the network.We train the proposed network on both the synthetic data and real-world images in an end-to-end manner.Our analysis shows that the proposed semi-supervised learning algorithm is not limited to synthetic training datasets and can be generalized well to real-world images.Extensive experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art single image dehazing algorithms on both benchmark datasets and real-world images.