Research On Dynamic Scene Image Deblurring Using Deep Neural Networks

Images are one of the most important ways for humans to transmit information in modern society.In the actual scene,due to human operations,scene limitations,natural environment,sensor limitations and other factors,the image will inevitably be blurred.This blur not only seriously affects the clarity and visual quality of the image,but also directly affects the wide application of the image in precision medicine,intelligent transportation,video surveillance,aviation,civil industry and other fields.Therefore,the research on image deblurring technology has high socio-economic value.Dynamic scene blur is caused by camera shake,scene depth variations,multiple object motions or occlusion in motion boundaries during the exposure time and other factors.Because the reasons for blur formation are very complicated,different regions or pixels in the same dynamic scene image undergo different blurring processes and degrees,and the blurring presents nonlinear characteristics.Therefore,dynamic scene deblurring is extremely difficult,and it is one of the most challenging problems that have long existed in the field of image deblurring.For the dynamic scene image deblurring,researchers have proposed various characteristic network models based on deep learning technology,and have made some progresses.However,due to the complex causes of dynamic scene blur and the limitations of deep neural networks,the existing network models generally have the problems of unsatisfactory deblurring effect and poor robustness,especially for the region or objects with serious blurring in dynamic scene images.Based on the in-depth analysis of the characteristics of dynamic scene blur images and the deep neural networks,this paper starts a systematic exploratory research on the dynamic scene deblurring methods with the design of the deep neural network structures as the starting point.Three effective methods for dynamic scene images deblurring are proposed.The main work of this paper is as follows:(1)In order to solve the problem that the ability of the existing network models to learn nonlinear blur features of complex dynamic scenes is limited,which leads to poor effect of dynamic scene deblurring,this paper proposes an end-to-end multi-width activation and multi-receptive field network method to fully explore the potential of the deep network model and improve the learning ability of complex blurry features.Firstly,this method builts a multi-width activation feature extraction module,in which the core-designed multi-width activation residual module uses a 1×1 convolutional layer to amplify the number of feature channels before the activation function,which solves the problem that the number of feature channels is less before the activation function in the existing network model,so that the network model can extract wider nonlinear activation features and learn more nonlinear blur features of dynamic scene images.Secondly,this method builts a multi-receptive field feature extraction module,in which the designed multi-receptive field residual module uses different convolution kernels to obtain different receptive fields,so that the network model can capture more nonlinear blur features of dynamic scene images from distant pixel positions,which solves the problem of single receptive field in the network structures.Finally,this method designs a multi-scale feature fusion module,in which the designed learnable fusion structure can adaptively fuse multi-scale features and complex nonlinearblur features from different modules by using convolution,element-wise multiplication and addition operations,and reconstruct high quality dynamic scene images.(2)In view of the problems of strong edge information loss and saturation of network model performance in the existing methods based on multi-scale and scale-recurrent framework,this paper proposes a dynamic scene image deblurring method based on progressive down-sampling and adaptive guidance network.Firstly,this method proposed network framework is able to avoid the loss of the strong edges and other high frequency information of the blurry image.The framework mainly includes three stages,in which the input image of each stage is the original image without any preprocessing and scaling sampling operations.At each stage,the convolutional neural network with learning kernel is used to reduce the spatial dimension of the original images,which can better integrate the global information and retain the necessary details.Most importantly,this design can enable the network to obtain more information that is essential for image deblurring,such as strong edges,thereby helping the proposed network model to learn more effective the mapping relationship between dynamic scene blur image and clear image.Secondly,this method designs a multi-scale blending activation residual block for solving the problem of network performance saturation caused by a single activation function.The core of this module is to better learn the complex nonlinear features of dynamic scene blur images and effectively improve the multi-scale feature extraction ability of the network model by blending the activation functions of PRe LU and ELU and the convolution kernel of different receptive fields.Finally,this method designs a multi-supervised strategy to solve the problem of the sensitivity of the neural network to the input data.This strategy forces the output latent sharp image of each stage and the label image to calculate the mean-square-error(MSE),so that the network model can obtain more robust and effective features and deblurring mapping relationships.And the network training is more stable and convergence is faster.(3)Aiming at the existing methods based on deep neural networks,which are generally unable to deal with the problem of dynamic scene blur with severe local or global blur,this paper proposes a multi-stream attentive generative adversarial network method.Firstly,According to the same dynamic scene image,the blur degree of different regions or objects in the same dynamic scene image often varies with different pixels,this method proposes an attentive guidance module for generating an attention map that can guide to focus on the severe blur regions or objects and their edges or texture structures.Therefore,without increasing the number of convolutional layers or the size of convolutional kernel and so on to expand the receptive field,the network model can improve the deblurring ability of the severely blurry regions.Secondly,this method designs a multi-stream and multi-scale feature extraction strategy,which mainly uses attention flow and multi-scale flow to solve the limitations of the single network structure in extracting multi-scale features.Based on this strategy,this method designs an effective feature extraction module,which is composed of multi-scale feature extraction and attention feature extraction modules,which can effectively extract multi-scale features and attention features of regions or objects with severe blur.Combined with the above-mentioned specific designs,the proposed multi-stream attentive generative adversarial network in this paper can further improve the quality of dynamic scene blur images.In order to verify the effectiveness and robustness of the three methods proposed in this paper,some experiments were conducted on multiple publicly authoritative datasets.The experimental results prove that the proposed three methods in this paper achieve better results in both objective and subjective indicators,and also reduce the time consumption of dynamic scene image deblurring.