Unsupervised Remote Sensing Image Super-Resolution Based On Deep Learning

High-resolution remote sensing images play an important role in resource exploration,environmental monitoring,urban and rural planning,and change detection.However,due to the limitations of signal transmission bandwidth and imaging sensors,the images obtained by remote sensing imaging devices typically have low spatial resolution.Improving the resolution of remote sensing imaging equipment is restricted by the current manufacturing level.Besides,the imaging system will be disturbed by noise,blur,and so on,which will degrade the quality of the acquired image and impair the resolution.Overcoming the limitations of imaging system is time-consuming and extremely expensive,thereby rendering image super-resolution(SR)technology a feasible and economical approach for improving the resolution of remote sensing images.Image super-resolution refers to the estimation of high resolution(HR)image from one or more low resolution(LR)observations of the same scene,for which digital image processing techniques are typically employed.Super-resolution technology can effectively improve image resolution through image processing techniques without improving existing imaging equipment.Given the reasons that the high cost of improving imaging equipment and the restriction of the imaging system,it is of great significance to research on remotely sensed image SR.Deep learning(DL)has developed rapidly in image processing.And these years have seen an upsurge of using DL models for single image super resolution(SISR)since they achieved the state-of-the-art results.And image super-resolution has also benefited from its development.But current DL-based methods of remote sensing image SR typically rely on HR images as references,considering it is difficult to obtain HR remote sensing images in the real world.In addition,it is impossible to obtain high-and low-resolution image pairs with strict registration in degradation process.In view of the above problems,this paper adopts the way of unsupervised learning,which introduces the image degradation.In unsupervised SR,HR remote sensing images are not required as training labels and is more in line with practical applications.In this paper,to realize unsupervised image SR,a method is first proposed by introducing the average pooling to downsample SR image involving the generative adversarial network(GAN).Then,the problem that the complex degradation of real remote sensing images does not match with the bicubic downsampling process adopted in the research leads to the unsatisfactory SR results.Therefore,a multi-degradation Degrader aided SR method is proposed.Finally,to solve the problem that the SR results of large scale factor(×8)is relatively poor in unsupervised SR,a method based on variational autoencoder is proposed to improve the reconstruction result,especially that of large scale factors.The main research contents of this paper are as follows:(1)An unsupervised remote sensing images super-resolution method based on generative adversarial network and average pooling is proposed.The average pooling operation is used to downsample the SR image to the same size as the LR image,so as to calculate the loss of this two,thereby achieving the unsupervised SR.The Generator extracts the features of the image,learning the prior information of the external dataset,and then obtains the SR image.Utilizing the average pooling as the degradation method,the SR is downsampled,as well as the LR image is discriminated by the discriminator.And the comprehensive loss function with image loss,perceptual loss,adversarial loss and total variation are calculated.The experimental results show that the method can better improve the structural information of the SR image under the constraint of comprehensive loss,and the peak signal-to-noise ratio(PSNR)result can also be effectively improved.(2)A multi-degradation aided unsupervised remote sensing image super-resolution method is proposed.In the study,the simple bicubic degradation does not match the complex degradation process of remote sensing images in the real world,resulting in the unsatisfied results.this method trained a convolutional neural network named Degrader through a synthetic image dataset containing downsampling,blur and noise.The Degrader degrades the SR image to avoid using HR labels.Beisdes,the Degrader can learn multiple degradations from synthetic data while acquiring the external prior information of remote sensing image datasets.The training of the reconstruction network is also for testing,making the method a“plug and play”style.In this process,the internal prior information of a testing image is explored.Combined with the Degrader,the LR image is used to calculate the loss function to update the parameters of the reconstruction network.As the training process is completed,the SR image is also obtained.The SR results of this method on six evaluation metrics can exceed the current unsupervised remote sensing image SR methods,and some of the metrics such as PSNR and SSIM exceed the results of most supervised SR methods.Even the best PSNR can be achieved when involving more complex degradations.In addition,convincing visual effects and satisfying blind image evaluation index NIQE are also obtained in the reconstruction of real remote sensing images obtained by Jilin-1.The above results show the effectiveness of the proposed method.(3)An unsupervised remote sensing images SR method based on variational autoencoder is proposed.This method is developed on the basis of the method aided by Degrader,to solve the jagged effect and color distortion,and also to improve the results of larger factors like ×8.This method introduced a variational autoencoder.First,the image is mapped into the latent variable space through the encoder,and the bicubic interpolated image is replaced with latent code as the input of the decoder,which removes the jagged effects caused by the repeated input of interpolated image.By alternately training the encoder and decoder,the latent code are constrained to the image space as well as the manifold space of HR images through the loss function,thereby improving the SR results.In addition,by modifying the network structure of Decoder with removing part of the BN layers,the color distortion in SR image is cleared.Finally,a regularization strategy is applied to the latent code when dealing with the scale factor 8.The experimental results show that this method can greatly exceed the results of the current unsupervised reconstruction methods on six evaluation metrics.Compared with the previous method,the numerical results such as PSNR are greatly improved,and also the gap between supervised and unsupervised method are reduced.In real remote sensing images,it has achieved the best NIQE and visual results so far.