Remotely Sensed Imagery Downscaling Using Generative Adversarial Networks

Accurately and scientifically measurement of land use can provide not only scientific decision-making basis for human economic and social activities,but also important data for the geoscientific system,environmental and climatic dynamic monitoring,and other related researches.To meet the needs of high-quality quantitative remote sensing research,how to improve the spatial resolution of remote sensing imagery by downscaling techniques has become a hotspot.At present,there are three main ways to in the downscaling field according to different data sources and application scenarios,namely spatial-temporal fusion,spatiotemporal fusion and super-resolution mapping.Theoretically,no matter which the technical route is based,the information process in downscaling involves temporal,spatial,and spectral resolution.Due to the diversity of remotely sensed imagery,and the complexity of temporal,spatial and spectral information,existing downscaling methods still have large limitations,and need to be further developed in theory and practice.The main difficulties of remotely sensed imagery downscaling are as in the following aspects:Firstly,due to the interaction between temporal,spatial and spectral information during the observation process using remotely sensed imagery,the dynamic monitoring of land use from a single data source often has only one advantage alone in temporal and spatial resolution;Secondly,the downscaling process of remote sensing imagery is a typical inversed under-determined problem,which may have more than one solutions,and this uncertainty may be inevitably large.Finally,traditional downscaling methods are relatively simple and have many limitations;while emerging machine-learning-based(including deep-learning)methods have made some achievements,however,no targeted improvements are made for complex geographic problems,which limits the application of these downscaling methods.Therefore,this doctoral dissertation mainly focuses on remotely sensed imagery downscaling technique,and will make a thorough discussion on the theory and practice use of the generative adversarial networks for coping with the inconsistent problem of information processing with the temporal,spatial,and spectral resolution of remote sensing imagery at the complex geographic environment.The main work and contributions of this dissertation are as follows:(1)This dissertation systematically analyzes the background and significance of the research on downscaling using remote sensing imagery,and takes the improvement of spatial resolution as a breakthrough point Motivated by this,this thesis mainly focuses on spatial-spectral fusion,spatiotemporal fusion and super-resolution mapping as the main researches,expounds on the relevant theories and methods of remote sensing image spatial downscaling.Furthermore,difficulties and challenges in the current research are pointed out,and the typical models are discussed and summarized.(2)For the spatial-spectral fusion downscaling technique,a pansharpening model based on a stacked generative adversarial network is proposed.Traditional pansharpening is only a simple fusion technique of panchromatic and multispectral bands,which can not break the maximal resolution limitation of input data(usually panchromatic band),and the downscaling efficience is not high.This model complementarily makes full use of the spatial information from panchromatic bands and spectral information from multispectral bands,and combines the super-resolution reconstruction and pansharpening organically into one end-to-end deep-learning model.On the one hand,the spatial resolution is further improved by super-resolution reconstruction using the panchromatic band;on the other hand,the downscaling panchromatic band is fused with multispectral band in a PCA-based network.This model breaks the limitation from traditional pansharpening models,in which the produced maximal spatial resolution can only reach the same as the spatial resolution of the input panchromatic band.(3)For the spatiotemporal fusion downscaling technique,a spatiotemporal reflectance fusion model based on a deep convolutional generative adversarial network is proposed.In spatiotemporal fusion,temporal information is coupled with convnetional spectral and spatial information,which increases the complexity of spatial downscaling research.The traditional spatiotemporal reflectance fusion model is relatively simple to deal with the coupling information of temporal,spectral and spatial resolution,which makes them very sensitive to the spatiotemporal change,especially in the area of abrupt change.Thus the efficiency of traditional spatiotemporal downscaling is lower.To solve this problem,the classical spatiotemporal fusion framework and deep convolution super-resolution reconstruction are integrated into a end-to-end deep-learning model,which significantly improves the accuracy of temporal-change information extraction and spatial downscaling.The model can remain better spectral correlations between multispectral bands,and it is an important spatiotemporal model for one-pair case.In addition to being stable in heterogeneous regions,the performance of this model is also excellent in rapidly and abruptly changing flood areas.(4)For the super-resolution mapping downscaling technique,a super-resolution mapping model based on a relativistic average generative adversarial network is proposed.In the traditional super-resolution mapping model,the error of spectral unmixing will directly affect the accuracy.At the same time,the traditional description algorithms about spatial distribution are not accurate.In order to alleviate the errors caused by spectral unmixing and mixed pixels,an end-to-end deep-learning model is used to train nonlinear relationships of spectral unmixing error,spatial downscaling and the assignment from land cover fraction to and category.This model can eliminate errors from spectral unmixing by adding Guassian noises to conditional learning,smooth realistic spatial shape by relativistic average discriminating mechanism,and learn the non-linear fraction-to-category conversion relationship from deconvolutional layers.This model solves the inaccuracy of prior spatial information described by traditional algorithms,errors from spectral unmixing and the allocation process of the fraction to land covers,and significantly improves the accuracy of super-resolution mapping.(5)In this study,the remotely sensed imagery downscaling model system based on generative adversarial networks is integrated and proposed to enhance its study in the application of impervious surface and water bodies in the Sihu Basin of Jianghan Plain.These models take the Sentinel-2 images in 2020 as input,and further realize downscaling.The mapping results in this study contain rich spatial information,and the accuracy is greatly improved compared with the hard classification results,which verifies the high-precision and large-scale application values of the proposed downscaling models in this dissertation.