Real Image Denoising Based On Residual Network By Using Generative Adversarial Network For Data Augmentation

Image denoising is one of the most fundamental operations in image restoration,and its basic objective is to recover the corrupted image from the noisy one effectively while preserving the original details and the edge information as far as possible.Although image denoising methods based on convolutional neural network(CNN)have achieved the state-ofthe-art in Gaussian reduction,their performances are still very limited on real photographs even beyond the reach of traditional methods,such as BM3 D and WNNM.One of the biggest issues with building a CNN-based denoiser is to collect paired training data,which is hard to derive because the nose-free photographs require a long exposure and strict calibration.For this reason,methods based on CNN commonly assume that the noise is additive white Gaussian noise(AWGN),making it just highly engineered for the task of Gaussian denoising and difficult to extend to real-world image denoising.In this paper,we propose a two-step framework for real-world photographs denoising based on CNN.Firstly,for the deficiency of realistic ground truth images,we train a Generative Adversarial Network(GAN)GAN to estimate the noise distribution over the existing denoising benchmark and construct a synthetic dataset,which is more consistent with the real noise distribution.It can be seen as a data augmentation schema,designing to improve the diversity of the data and avoiding overfitting.Secondly,since noise levels of the real photograph usually are unknown,it is necessary to obtain the noise level of the noisy image as a priori in order to improve the robust generalization ability of the denoising network.Therefore,in this work we propose a blind denoising model for real photographs by combing noise estimation with blind denoising network,and trains it by the extended dataset,which is consisted of the real images and the synthetic images constructed in the first step.We firstly design a residual denoising network referred to DRNet.For the integration of noise estimation with CNN-based denoiser,we introduce a noise estimation module in front of the network.Based on the observation that nonlocal self-similar patches from the noiseless image ofen lie in a low-dimensional subspace,we can obtain the eigenvalues of the covariance matrix in redundant dimensions as the standard deviation,providing a flexible way to handle unknown noise levels with a single model.However,this method simply assumes that the noise in each local patch is additive zero-mean Gaussian distributed.Although it can improve the performance of the model with unknown noise levels,we find the proposed DRNet is still very limited on some real photographs,because the realistic noise actually is signal-dependent,non-Gaussian,and spatially variant.Then we further improve the method of noise estimation and design a separated network to estimate the noise level of each pixel,making the single model has the ability to inherit the different noise levels at the same time.Besides,we design a dual-domain residual denoising network(D3Res Net)to preserve high-contrast features in a spatial domain and learn prior information in a wavelet domain.Experiments also show that the proposed D3 Res Net is able to improve the performance of image denoising the detail features.In addition,an adjustable interaction strategy is employed to fine-tune the image noise level and enable the model to solve different requirements of image denoising effect in practical application.