Research On Convolutional Neural Networks For Image Denoising And Super-resolution

With the popularity of portable smartphones,digital images have become an important carrier for people to perceive,process,analyze and share information.Due to the various factors in real life,such as the imaging system,the storage capacity and network bandwidth,digital images are often corrupted by noise and have low spatial resolution.Image denoising and single image super-resolution(SISR),as two classical and yet active low-level vision research topics in the field of digital image processing,can not only improve image visual perception quality,but also improve the accuracy of subsequent high-level semantic analysis.Generally,model-based methods and discriminative learning methods are the two dominant strategies for solving image denoising and image super-resolution.Typically,those two kinds of methods have their respective merits and drawbacks,e.g.,model-based methods are flexible for handling different image restoration problems but are usually time-consuming with sophisticated priors for the purpose of good performance;in the meanwhile,discriminative learning methods such as convolutional neural networks(CNN)have fast testing speed and usually have good performance due to end-to-end training but their application range is greatly restricted by certain degradation models.To address these issues,we in this dissertation will design an efficient denoising network,propose flexible denoising and super-resolution networks,and solve image super-resolution for complex degradation from the perspectives of function regression,maximum a posteriori and optimization algorithm.The main research contents and contributions are summarized as follows.(1)Model-based denoising methods generally achieve high performance at the sacrifice of computational efficiency.In the meanwhile,discriminative learning based denoising methods have shown promising results toward bridging the gap between computational efficiency and denoising quality,however,their performance are inherently restricted to the specified forms of prior.We take one step forward by investigating the construction of feed-forward denoising convolutional neural networks(Dn CNNs)to embrace the progress in very deep architecture,learning algorithm,and regularization method into image denoising.Specifically,residual learning and batch normalization are utilized to speed up the training process as well as boost the denoising performance.With the residual learningstrategy,Dn CNN implicitly removes the latent clean image in the hidden layers.We then further analyze the connection between Dn CNN and MAP inference guided discriminative learning method.Our analysis reveals and our experiments verify that it is possible to train a single Dn CNN model to tackle with several general image denoising tasks such as SISR and JPEG image deblocking.(2)Existing discriminative learning based denoising methods mostly learn a specific model for each noise level and lack flexibility to deal with spatially variant noise,which limits their applications in practical denoising.To address these issues,we present a fast and flexible denoising convolutional neural network,namely FFDNet,with a tunable noise level map as the input.The proposed FFDNet works on downsampled sub-images to speed up the inference.In contrast to the existing discriminative denoisers,FFDNet enjoys several desirable properties,including(i)the ability to handle a wide range of noise levels(i.e.,[0,75])effectively with a single network,(ii)the ability to remove spatially variant noise by specifying a non-uniform noise level map,and(iii)faster speed than benchmark BM3 D even on CPU without sacrificing denoising performance.(3)Existing CNN-based SISR methods mostly assume that a low-resolution image is bicubicly downsampled from a high-resolution image,thus inevitably giving rise to poor performance when the true degradation does not follow this assumption.Moreover,they lack scalability in learning a single model to deal with multiple degradations.To address these issues,we propose a dimensionality stretching strategy that enables a single convolutional super-resolution network to take two key factors of the SISR degradation process,i.e.,blur kernel and noise level,as input.Consequently,the proposed superresolver can handle multiple and even spatially variant degradations,which provides a highly effective solution to practical SISR applications.(4)While model-based methods and discriminative learning methods have their respective merits and drawbacks,recent works have revealed that,with the aid of variable splitting techniques,denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other image restoration tasks.Such an integration induces considerable advantage when the denoiser is designed via CNN.However,the study of integration with fast deep denoiser prior is still lacking.To this end,we train a set of fast and effective CNN-based denoisers and integrate them into model-based optimization method with half quadratic splitting algorithm to solve other image restoration tasks such as SISR.(5)Existing CNN-based SISR methods are mainly designed for the widely-used bicu-bic degradation,there still remains the fundamental challenge for them to super-resolve low-resolution image with arbitrary blur kernels.In the meanwhile,plug-and-play image restoration has been recognized with high flexibility due to its modular structure for easy plug-in of denoiser priors.Inspired by this,we propose a principled formulation and framework by extending bicubic degradation based deep SISR with the help of plug-andplay framework to handle LR images with arbitrary blur kernels.Specifically,we design a new SISR degradation model so as to take advantage of existing blind deblurring methods for blur kernel estimation.To optimize the new degradation induced energy function,we then derive a plug-and-play algorithm via variable splitting technique,which allows us to plug any super-resolver prior as a modular part.