Deblurring Methods For Out-of-focus Images Based On Conventional Model And Deep Neural Networks

Image deblurring is a hot research topic and an important branch of image restoration.It plays a prominent role in image processing and computer vision applications such as security surveillance,intelligent transportation,medical imaging,industrial manufacturing,broadcasting,entertainment,etc.Image deblurring aims to recover the sharp image from a blurred observation.In general,the causes of image blur can be roughly divided into two categories:out-of-focus blur and motion blur.The ubiquitous image degradation usually leads to difficulties in object identification and scene interpretation.In addressing the problem of image blur,different methods are required for different types of blur.In recent years,important advances have been made in the area of image deblurring.However,current image deblurring methods still face challenges such as the inaccurate description of the blur kernel for out-of-focus blurry images,low robustness in estimating the parameters of the out-of-focus blur kernels,occurrence of artifacts in guided image deblurring,and difficulty in restoring details for out-of-focus and motion blur images.This thesis mainly focuses on the problems of out-of-focus image deblurring and guided image deblurring,and carries out innovative research in two aspects: the establishment and parameter estimation of out-of-focus blur kernels,and guided image deblurring using deep neural networks and image fusion techniques.This research includes the following four aspects:1.Aiming at the problem of inadequate accuracy in depicting the out-of-focus blur kernels,this thesis proposes a generalized Gaussian(GG)out-of-focus blur kernel.It is motivated by the theoretical analysis of the out-of-focus blur and the practical observation of real blur kernels.The causes of out-of-focus blur based on an optic model include focusing errors and diffraction phenomena.Traditional out-of-focus blur kernels like disk and Gaussian blur kernels,are approximations of ideal optic models and suitable for different levels of out-of-focus blur.To deblur images using appropriate blur kernels,this thesis proposes a disk-Gaussian blur kernel conversion formula based on the expressions of disk and Gaussian blur kernels.This formula can be used to convert between the two types of blur kernels depending on different levels of out-of-focus blur.This thesis verifies the effectiveness of the conversion formula on real outof-focus blurry images.Furthermore,to explore the shape of real out-of-focus blur kernels,we collect real blurry/sharp image pairs for blur kernel estimation.The observation of the real blur kernel shape shows that real out-of-focus blur kernels have the characteristics of Gaussian and disk blur kernels under different degrees of blur.Given that the GG blur kernel can describe a variety of shapes including Gaussian and disk blur kernels with different parameters,the proposed method of using the GG function to depict out-of-focus blur kernels is feasible and consistent with the optical theory of out-of-focus blur.2.Aiming at the problem of the limited performance of out-of-focus image deblurring based on traditional out-of-focus blur kernels,this thesis proposes an out-of-focus image deblurring algorithm based on single-parameter GG blur kernel.Motivated by capturing blurry/sharp image pairs on real scenes,we analyze the statistics of estimated real blur kernels.Observing that the blur kernels have specific edge widths inspires us to simplify the GG function to a single-parameter model.Experimental results show that the single-parameter GG blur kernel is more accurate in depicting out-of-focus blur compared to disk and Gaussian blur kernels.Compared to the original two-parameter model,the single-parameter GG blur kernel greatly reduces the computational complexity and estimation difficulty.Furthermore,we propose a GG blur kernel estimation algorithm using image patches containing step-like edges,followed by non-blind image deblurring to obtain sharp images.Experimental results validate that the proposed single-parameter GG blur kernel based deblurring algorithm is more robust than disk and Gaussian blur kernel based algorithms with biased estimated parameters.Synthetic experiment results show that the proposed algorithm can accurately estimate the blur kernel,and outperforms the compared algorithms.Experiments on real scenes illustrate that the proposed algorithm is capable to deblur real out-of-focus blur scenes with noise and estimate the most reasonable blur kernel and the best deblurring performance.The proposed algorithm is also the fastest in the computation time comparisons.3.Aiming at the problem of the difficulty in recovering details using single image deblurring methods and removing artifacts in guided image deblurring methods,this thesis proposes a fusion framework for guided image deblurring based on multi-modal image pairs,called Guided Deblurring Fusion Network(GDFNet).The object is to integrate the compensation information in multi-modal image pairs by applying image fusion into out-of-focus image deblurring.GDFNet,constructed by fully trainable convolutional neural networks,fuses the pre-deblurred single image and guided image deblurring streams and integrates their structures and details according to the fusion weights.GDFNet adopts a blur/residual image splitting strategy to enhance the representation ability and employs a 2-level coarse-to-fine reconstruction strategy to improve the fusion and deblurring performance by supervising its multi-scale output.Experimental results show that GDFNet effectively fuses the information among pre-deblurred streams and obtains better image deblurring quality than traditional optimized-based algorithms.Experiments on how the features change during training show that the ability to recognize significant structures improves as training continues,which eventually benefits the image deblurring results.The comparisons on computation time show that GDFNet has advantages in terms of speed.4.Aiming at the problem of the difficulty in recovering details and robustly in motion image deblurring,this thesis extends the application of GDFNet to motion blur images deblurring.Motion blur widely exists in daily life and professional scenarios.Compared with out-of-focus blur,the prior information of motion blur is hard to acquire.Therefore,deblurring motion blur images is more difficult and more likely to produce artifacts.This thesis replaces the pre-deblurred streams of GDFNet for out-of-focus deblurring with streams designed for motion deblurring,which makes GDFNet suitable for motion blur image deblurring.Experimental results on multiple multi-modal image pair datasets show that GDFNet effectively integrates the compensation information in motion blur images and guided images with advantages on all image quality assessment metrics.Compared with representative image deblurring and fusion algorithms,GDFNet produces sharper deblurred images with fewer artifacts.Ablation studies further verify the effectiveness of the loss function and architecture design of GDFNet.The extended application of GDFNet in motion blur image deblurring reflects its value.