Remote Sensing Image Spatial-spectral And Spatial-temporal Fusion Based On Deep Learning

With the rapid development of modern sensor technology and surveying,mapping and remote sensing technology,the spatial,spectral and temporal resolution of satellite images has been continuously improved.This advancement provides important data support for military,environment,land resource,agricultural and other important fields.Due to the limitation of physical characteristics of sensors and the consideration of signal-to-noise ratio,there are irreconcilable contradictions among spatial,spectral and temporal resolution.Meanwhile,remote sensing images with high spatial,spectral and temporal resolution play an indispensable role in many remote sensing applications(such as urban change monitoring).Therefore,remote sensing image fusion technology emerges and it fuses images form a single or multiple sensors according to application purpose,so as to obtain the required high-resolution remote sensing images.Based on the deep learning theory,remote sensing image spatial-spectral fusion and spatial-temporal fusion are studied respectively to satisfy the requirements of different applications.The research contents mainly include:(1)The feasible of deep learning theory for remote sensing image fusion is studied.The high-level features of images can be fully extracted by deep learning,and the features of input images can be extracted and fused to obtain high-resolution remote sensing images.Deep learning has been widely used in image super-resolution reconstruction,and the fusion of remote sensing images also involves the improvement of spatial resolution.Therefore,the study on the application of deep learning in remote sensing image fusion has certain significance and value.(2)Remote sensing image spatial-spectral fusion based on deep learning is studied.Spatial-spectral fusion needs to coordinate the contradiction between spectral resolution and spatial resolution.The data sources are often multispectral images and panchromatic images from the same sensor.In this paper,a deep two-branch convolutional neural network is constructed to extract the features of multispectral and panchromatic images respectively,and it further learns to integrate extracted features to obtain the final fusion image.In this paper,convolution layers of different depths are set to extract the features of corresponding input images,and the influence of network depth on the final fusion effect is fully analyzed.At the same time,residual structure is used to make network learning the most effective features,so as to improve the fusion ability of the model.(3)Remote sensing image spatial-temporal fusion based on deep learning is studied.Compared with spatial-spectral fusion,spatial-temporal fusion focuses on coordinating the contraction between temporal resolution and spatial resolution,the data sources are generally from different sources.The images in this part of the study are from Landsat-8 and Sentinel-2 satellites.The research results of the spatial-temporal fusion of these two satellites are not as abundant as those of Landsat and MODIS.Therefore,this paper proposes a fusion framework based on deep learning for Landsat-8 and Sentinel-2 images.The traditional spatial-temporal fusion methods often require one or more image pairs form different sensors at the same time to learn the mapping process,while the method studied in this paper can overcome this limitation and make full use of the currently available images to obtain fusion results,which is more practical in actual application scenarios.