A New Pan-Sharpening Algorithm Using Convolutional Neural Network

With the development of sensor technology,more and more remote sensing images can be acquired.In order to integrate the advantages of multiple sensors,we need to fuse remote sensing images on different sensors,and the fusion of panchromatic image and multi-spectral image is one of the research hotspots.The key of remote sensing image fusion is to keep the spectral information of multi-spectral image intact while injecting the spatial information of panchromatic image.Various fusion algorithms have been proposed to solve this problem.However,due to the limitations of the existing algorithms,it is difficult to achieve a balance between spatial maintenance and spectral maintenance.For this reason,this paper studies the fusion algorithm based on convolutional neural network.Based on the previous research,this paper proposes two new image fusion algorithms by improving the structure of the existing network.The main work of this paper is divided into the following aspects:First,aiming at the problem that the scale difference between panchromatic image and multi-spectral image is too large,a deep pyramid network is proposed to reconstruct the fusion image in a multi-scale way,which alleviates the difficulty brought by the large scale difference to the network learning.In order to further play the advantages of pyramid structure,we are providing corresponding supervision for each layer of the pyramid,which is conducive to the network convergence to a lower error.Secondly,for the upsampling problem,the existing algorithms usually adopt the traditional upsampling method and use the same interpolation strategy in each channel,without considering the differences between different channels and different ground objects.In order to solve this problem,neural network is used to learn the upsampling method of each channel.Finally,in the previous work,spatial information injected from panchromatic images is extracted manually,and the features extracted manually are not necessarily required for multi-spectral images.For this purpose,a parallel network structure is designed in this paper.Among them,panchromatic guidance network generates the feature map to be injected into the multi-spectral image,and the multi-spectral upsampling network fuses the above feature map to generate the final high-resolution multi-spectral images.In addition,experiments are carried out on the geoeye-1 data set,and the effectiveness of the proposed algorithm is proved by comparison with a large number of algorithms.