Research On Remote Sensing Image Fusion Algorithm Based On Deep Learning

With the continuous launch and increase of the number of satellites,remote sensing data also began to grow massively.Sometimes,the ground object features can not be obtained from single remote sensing data.Therefore,it is necessary to fuse a variety of remote sensing data.the requirement of remote sensing image in spatial resolution and spectral resolution is higher,but due to the limitation of sensor hardware,it is difficult to obtain remote sensing image with high spatial resolution and spectral resolution at the same time.In view of the fact that the existing fusion methods can not take into account the spatial and spectral information of the image at the same time,as well as the spectral distortion phenomenon,the improved algorithm is proposed in this paper,the main work is as follows:(1)In order to solve the problem that most of the existing deep learning algorithms need to make a simulated data set because there is no real high spatial resolution multispectral image as a label,and the fusion results have the problems of dangerious spectral distortion and lack of spatial details,a deep learning algorithm using real remote sensing image for training is proposed.Firstly,a feature enhancement layer is added on three-layer convolutional neural network which is based on super-resolution convolutional neural network to make the extracted detail features more obvious;Then,a loss function based on no reference quality evaluation function is designed,which includes spectral quality evaluation function D_?and spatial quality evaluation function D_s;Finally,a new label making method is designed to make the label contain panchromatic image(PAN)and multispectral image(MS)with different spatial resolution.The feasibility and effectiveness of the proposed method are verified by using 4 bands gaofen-1(GF-1)satellite data.From the objective quality evaluation and supervisor visual evaluation of the experimental results,the network output results are better than the traditional method and deep learning method.In addition,due to the spatial loss and spectral loss of the network output can be minimized when calculating the loss,the spectral information of the original MS and the spatial detail information of the PAN are fully preserved.(2)In order to solve the problem that most of the super resolution reconstruction based deep learning algorithms have spectral distortion and need to produce the simulated data of PAN and MS at the same time,a remote sensing image fusion algorithm based on spectral learning is proposed.Firstly,the original PAN is reduced to the size of the original MS,and using the reduced PAN as training data which simplify the process of making the simulation training set of most current deep learning algorithms.Secondly,the spectral quality evaluation function SAM is used to control the spectral loss of network output.Finally,after the training,the original PAN is used as the network input to predict the high spatial resolution MS.The validity of the method is verified by the simulated data experiment and real data experiment of 8-band worldview-2 satellite.From the objective quality evaluation and supervisor visual evaluation of the experimental results,the network output results are far better than the traditional method and deep learning method in the aspect of spectral information retention,In the aspect of spatial detail preservation,it is as good as the traditional method and deep learning method.(3)In order to solve the problem that the existing deep learning based remote sensing image fusion algorithms can not fusion without multispectral images,and the steps of making simulation data set are complex,a PAN colorization based remote sensing image fusion algorithm is proposed.Firstly,the autoencoder network with skip connection is used as the image generation model.Then,a loss function based on fusion quality evaluation function is designed,which includes spectral quality evaluation function SAM and spatial quality evaluation function UQI.Finally,concatenating the up-sampled MS and the original PAN as label.The priori MS and PAN are used for training.After training,the fused image can be obtained even when the MS is missing.By this way,the steps of making training set are simplified.The validity of the method is verified by the real data experiment of 8-band worldview-2 satellite.From the objective quality evaluation and supervisor visual evaluation of the experimental results,the network output results are better than the traditional methods and deep learning methods in terms of spectral information preservation and spatial detail preservation.