Research On Missing Information Reconstruction Of Remote Sensing Imagery Employing Temporal-Spatial-Spectral Deep Feature Learning

Due to the actual working conditions and interference of atmospheric environment,remote sensing images often suffer from information missing issue,such as dead pixel and thick cloud covering.This kind of discontinuous data is difficult to be used in subsequent processing and application.Therefore,how to reconstruct missing data is an indispensable and important task in the field of remote sensing image processing.In this paper,feature dimension information of remote sensing image in spatial,spectral and temporal domain is fully mined under the framework of deep learning theory for reconstructing missing information of remote sensing image.The main work and innovation points are listed as below:1)A missing information reconstruction method based on the spatial-temporalspectral with deep convolution neural network is proposed.Considering the complexity and nonlinear of information missing issue in remote sensing image,and combining the spatial-temporal-spectral complementary information,the proposed method utilizes the nonlinear feature expression ability of deep residual convolution network learning and improves the robustness of the model by multi-scale convolution unit.Meanwhile,dilated convolution is introduced to improve the size of receptive field,to mining the feature information in spatial,spectral and temporal domain.Finally,this spatialtemporal-spectral deep convolutional neural network(STS-CNN)is constructed to train and learn when the loss function is convergent.Simulated and real experiment results show that STS-CNN method can effectively reconstruct typical missing information such as dead pixel in MODIS Aqua 6-th band,ETM+ SLC-Off,and thick cloud covering in multi-type remote sensing images.2)A progressively spatio-temporal patch group learning framework for cloud and cloud shadow removal is presented.Firstly,for obtaining the mask data,cloud and cloud shadow detection of multi-temporal imagery is implemented.Secondly,the spatial and corresponding multi-temporal patches with masks are stacked and sorted as the patch group fashion.Thirdly,a spatio-temporal patch group recovering model is developed to reconstruct original information of covering areas with a global-local deep convolutional neural network.Finally,all the ergodic patches are weighted aggregated with integrity measure,then updated spatial data and its mask are regenerated through progressive iteration.A series of real and simulated experiments have been carried out to validate the availability of presented method in Sentienl-2 MSI and Landsat-8 OLI data,with single/multiple temporal imageries in small/large-scale scenes.