Multispectral Images Pan-sharpening Based On Deep Residual Neural Network

With the continuous improvement of sensor technology,remote sensing images with different spatial resolution,spectral resolution and temporal resolution have been continuously provided by satellites on different earth observation platforms.Due to the limited incident light energy,there are critical tradeoffs among the signal-to-noise ratio(SNR),spectral resolution,and spatial resolution.Therefore,satellites sensors often only measure a high-resolution grayscale(PAN)image and several low-resolution multispectral(LRMS)images when the signal-to-noise ratio is constant.Pansharpening is a technique for fusing multispectral images with panchromatic images,which aims to use the panchromatic images to improve the spatial resolution of multispectral images.Pan-sharpening has great research significance as a fundamental and significant preprocessing step in remote sensing tasks.In recent years,deep learning methods have achieved remarkable results in the field of computer vision,and scholars have begun to introduce them to solve related tasks in the field of remote sensing.Based on this,this paper integrated the specific knowledge of spatial and spectral in the deep learning framework,and constructed a nonlinear deep learning model to achieve pan-sharpening of multispectral images.The following work has been done in this paper:(1)According to the sequence of the pan-sharpening algorithm,this article summarized and classified the research status at home and abroad.The analysis found that the method based on component replacement has clear edge details but severe spectral distortion,and the method based on multiresolution analysis solves this problem but has the phenomenon of spatial information degradation.Through quantitative evaluation of the experimental results,it is found that the pan-sharpening model based on deep learning has insufficient generalization ability and the deep network is prone to network degradation.(2)In view of the shortcomings of the traditional pan-sharpening method,this paper proposed an improved pan-sharpening algorithm for multispectral images based on a deep residual network(ResNet).The algorithm implemented an end-to-end deep network,and the model automatically learned the mapping relationship between input and output on WorldView-3 and GF-2 simulation data.Experimental analysis shows that the improved algorithm effectively prevented the gradient explosion or disappearance of deep networks and network degradation.(3)In order to made the deep residual network suitable for solving the pansharpening problem,this article used the ASPP module instead of the pooling layer to reduce the loss of spatial details,and used transposed convolution to learn the parameters to reconstruct the image size.Comparative experimental analysis with a variety of classic models proved that the improved network trained in the high-pass domain effectively preserves the spatial details of WorldView-3 data,while reducing the spectral distortion of the pan-sharpened image and having better generalization ability.In this paper,the satellite datasets are used to perform experiments on the multispectral pan-sharpening algorithm.The results show that the proposed method is advanced and superior in spatial and spectral information retention.