Research On Natural Image Restoration Methods Based On Learnable Image Diffusion

Digital images are an important source of human access to information.With the popularity of smartphones with high-definition cameras,and imaging devices widely used in many fields such as urban security,satellite remote sensing,huge amount of image da-ta is generated in real time.These image data are collected,analyzed and processed to obtain the information about real world.However,in the imaging,transmission,and storage processes,digital images are inevitably affected by various degradation effects such as noise,compression,and blur.The quality of the image has a significant impact on the accuracy of image-based analysis and understanding.Therefore,the image restoration problem is a widely studied problem as a preprocessing process for image analysis and understanding.This paper studies several key technologies involved in the representa-tion of image prior model,learning efficiency of image prior model and computational efficiency of image restoration.The main contributions and innovations of this paper are summarized as follows:(1)A Trainable NonLocal Reaction Diffusion for image denoising is proposed.The image diffusion model is one of the commonly used models for processing image restoration tasks,e.g.,the Trainable Nonlinear Reaction Diffusion(TNRD)model[1].Since the TNRD model is a local model whose diffusion behavior is completely dominated by the information in the local spatial range,the TNRD model is tend to produce artifacts in the homogeneous region of the clean image,and over-smooth the highly textured region of the clean image.Especially in the case where the noise level is high,the above phenomenon is more prominent.At the same time,the Nonlocal Self-Similarity(NSS)image prior model can effectively extract the self-similar structural information of the image,it benefits the restoration methods to suppress artifacts and avoid over-smooth the textures.To This end,this paper proposes to embed the NSS image prior model into the TNRD model to improve the above-mentioned shortcomings of TNRD.This model is called Trainable NonLocal Reaction Diffusion(TNLRD).The experimental results show that the trained TNLRD model can produce visually more reasonable restored images with more textures and fewer artifacts.Compared with the state-of-the-art image denoising methods,the TNLRD model proposed in this paper has obtained competitive performance in terms of Peak Signal-to-Noise Ratio(PSNR)and Structural SIMIlarity(SSIM).(2)A fast nonlocal image diffusion by stacking local filters is proposed.Most of the image denoising methods are designed to provide good denoising images with little or no consideration for computational and storage efficiency,for example[2].As high-definition imaging devices are becoming popular,these denoising methods cannot be well extended to high-resolution images,due to excessive computation time or memory over-flow.To this end,this paper proposes a fast nonlocal image diffusion by stacking local filters.Based on the distribution of the distance distrs between the similar patches with respect to their reference patch,we conducted an analysis from a semantic aspect,and found that in artifact and natural object images the large distrss occur while their contribution to the overall distribution is small.Therefore,we get the proposed model by replacing the nonlocal filters in Trainable NonLocal Reaction Diffusion(TNLRD)with local filters which are coined as 2nd layer filters.Experimental results show that the proposed model is more computational efficiency than TNLRD.In 4096 x 4096 resolution,the proposed method runs about 6 times faster than TNLRD via a single-thread CPU implementation,and about 67 times faster via a GPU implementation.Furthermore,the proposed model achieves competing denoising performance compared with TNLRD in terms of PSNR and SSIM.(3)A generic diffusion process for image restoration is proposed.Image restoration problem is a typical illposed problem.Methods based on regularization are com-monly used.In addition to manually designed 2,such as the total variation model,image prior models can be learnt in a data-driven manner via machine learning.Image prior learning based on machine learning can generally be divided into two maj or categories,generative learning and discriminative learning.On the one hand,the generative model can be transferred to various image restorations,but the resulting methods provide inferior restored image,compared with the discriminative model.On the other hand,the discriminative model is usually trained for a specific image restoration problems and will fail in untrained problems.In order to solve this problem,this paper proposes a generic diffusion process(genericDP)that shares the diffusion term among multiple denoising problems.The genericDP model is trained to deal with denoising problems with multiple noise levels image denoising problems by sharing the diffusion term.Compared with the original training method,the trained genericDP model can provide similar image denoising performance and much higher training efficiency.The trained diffusion term is transferred to the non-blind image deconvolution problem.Note that the training samples of the non-blind image deconvolution problem are not provided during the training phase of the genericDP model.The experimental results show that the trained diffusion term can be used as a general image prior model to deal with non-blind image deconvolution problem.And the resulting non-blind image deconvolution method obtains the competitive recovery performance,compared with the state-of-the-art methods.(4)A high-level task aided low-level optimizer is proposed.The l2 loss function is the most commonly used loss function when training an image restoration model in a discriminative way.Models trained using the l2 loss function tend to over-smooth the details of the image.Although the PSNR is higher,the visual quality of the restored image is not very good.Image restoration models trained via the Generative Adversarial Nets(GAN)provides restored images with richer texture,the training of GANs is very unstable,and it is very sensitive to the network structure and training hyper-parameters.To this end,this paper proposes a High-level task Aided Low-level Optimizer(HALO)to improve the training stability and the visual quality of the restored image.The experimental results show that the training via HALO is more stable.And in HALO framework,the adj ustment of low-level vision model,high-level vision network and the training hyper-parameters is easy to understand when diagnosing the training process.While improving the robustness of the high-level vision network to the noisy input,the trained denoising model via HALO provides restored image with more reasonable details and higher visual quality.