Unsupervised Learning Methods For Image Restoration

In a deep learning based frameworks,two main aspects are critical for the model success,one is the model architecture and second is training stage.For training a deep learning model,large labeled data set need to be at hand.In terms of image restoration,pairs of degraded/ground-truth images are needed to perform the training process by minimizing the mean squared error(MSE)between the network estimation and the ground truth(degradation-free)images.That being said,in many cases ground-truth data can be hard to acquire or technically consuming,one example is in medical imaging,where 3D MRI needs hours of acquisition for a single high quality volume.Reducing time acquisition leads to degradation perturbation,which is harmful for medical diagnose.Recently,an interesting property has been noticed in regards to image restoration tasks(in the context of image denoising)can be robust to normal perturbations in groundtruth data,meaning that if the ground-truth data used during training had to be altered by a zero mean Gaussian degradation then this would not alter significantly the restoration performance.It has been also shown that in many cases we can achieve similar restoration performance,said differently we can train a deep learning based restoration model by mapping two different noisy instance of the same sample and learn to achieve denoised outcome(this method is hereafter called noise to noise training).This sort of behavior is rather appealing,and motivate us to explore more unsupervised training methods.In this work we propose to approach the issue of training deep learning based restoration models in case of no ground-truth data set.Previous works on unsupervised deep learning training focused on the task of image denoising.We revisit unsupervised deep learning training in the context of image deraining.Image deraining is still a developing discipline because of a number of open problems:· Most deraining algorithms rely on synthetic data sets to compare and evaluate their performance.Although it is relatively easy to synthesize a large number of training pairs,models trained on synthetic data sets fails to generalize well to real rain images since synthetic rain does not match well real rain distribution.· Collecting real rain/rain-free pairs simultaneously is unfeasible.Despite some recent works that proposed to construct large scale real rain data-sets by taking advantage of rich rain temporal information(multiple frames),the generation process can be tedious and time consuming–especially since it includes human supervision.In spite of the data collection and ground-truth generation difficulty that real rain image data sets are known for,little work has been done to explore unsupervised training methods for training single image deraining models.In this work we propose to explore no ground-truth training for single image deraining.More explicitly we show that it is possible to train single image deraining models using only rain images by training models to map rain images to rain images.The main contributions of this work include:1.The introduction of rain to rain training–an unsupervised training method for single image deraining that relies only on rain images,and does not assume any prior knowledge about the rain distribution.2.We also introduce a method of selecting adequate training pairs– here named least overlapping training pairs which proves to enhance rain to rain training perfor-mance to reach supervised performance.3.Using different data sets and different training settings,we could identify different training scenarios where rain to rain training can achieve similar,better or worse deraining performance compared to supervised training.Going back to the denoising task,previous works assume having access to pairs of noisy data to achieve unsupervised training.In our work,we assume that there is access to only single instances of noisy images and develop our own unsupervised training method.Our goal is to design a single image no ground-truth training method for image denoising,most existing deep learning based methods relay exclusively on training a discriminative learning model using ground truth data,which is obviously the default approach to adopt if available.We know that training is not only restricted to ground truth data,and noisy target can also achieve satisfying denoising performance.But it is still it is not yet clear how can we use this property to our advantage in case no pairs of noisy images is available.Indeed it is rare to find datasets of that include pairs of noisy data;and in fact the most common case is to find data of only single noise instance captions,here again noise to noise mapping while being highly relevant,does not meet an optimal practical training requirement,since we would still need pairs of noisy data.It would be advantageous to find a way train exclusively on noisy data,yet,it is still crucial to have training pairs,in this case non-local self-similarity suits perfectly our need.Indeed it is rare to find datasets of that include pairs of noisy data;and in fact the most common case is to find data of only single noise instance captions,here,noise to noise mapping while being highly relevant,does not meet an optimal practical training requirement,since we would still need pairs of noisy data.It would be advantageous to find a way train exclusively on noisy data,yet,it is still crucial to have training pairs,in this case non-local self-similarity suits perfectly our need.Natural images exhibit redundancies across different spatial position in the image,it is thus possible to extract training pairs from the given data using similar patches,which enables us to train a denoising model using only corrupted data with no corresponding ground truth versions.To that end,we combine the non-local self-similarity prior with noise to noise mapping,we also design an appropriate convolutional neural network(convolutional neural network)architecture for that matter,our training model is composed of two networks.Inspired by the way block-matching and 3D filtering(BM3D)exploit non local self-similarity,we propose a patch group based CNN model;a first network takes as inputs groups of N similar patches of n are formed after performing a block matching on the noisy images(with respect to a reference patch in research window),these N patches are then fed to the neural networks and denoised jointly.the resulting estimates are then fed to a second CNN network,this second network would provide the final estimation of the clean reference patch.