Research On Algorithms Of SAR And Multi-spectral Image Fusion Based On Convolutional Neural Network

In recent years,with the vigorous development of remote sensing technology in various countries,the number of remote sensing satellites is increasing,and their spatial,temporal and spectral resolutions are greatly improved.They play an increasingly important role in military reconnaissance,resource investigation,disaster monitoring and other fields.As a single sensor can only provide limited amount of information,in order to make full use of multi-source sensor data,image fusion arises at the historic moment,in which synthetic aperture radar(SAR)image and multi-spectral image fusion is one of the hot research topic.The key of SAR image and multi-spectral image fusion is how to extract the spatial information of SAR image and how to inject the information without causing spectral distortion of the fusion image.Due to the limitations of the traditional fusion algorithm,it is difficult to achieve a balance between spectral preservation and spatial detail enhancement.Therefore,this paper studies the fusion algorithm of SAR and multispectral image based on convolutional neural network.1.Aiming at the problems of color distortion and spatial detail blurring in the fusion results of SAR and multi-spectral images,a fusion algorithm of SAR and multi-spectral images based on dual branch convolutional neural network is proposed.The algorithm realizes the fusion of SAR and multi-spectral images by designing the dual branch network of spectrum preserving and detail enhancing,and by introducing the fusion scheme of feature extraction,fusion and reconstruction;At the same time,the spectral loss function and the detail loss function are designed to control the SAR information injection,so that the fusion results can get a better balance between the spectral preservation and the spatial detail enhancement.2.Aiming at the difference of the deep features of SAR and multi-spectral images in different spatial locations and channel dimensions,a fusion algorithm of SAR and multispectral images based on attention model is proposed.The algorithm designs the spatial attention model and the channel attention model to give different weight values to the points of different spatial positions and to the feature maps of the different channels in the deep features to achieve the recalibration of the deep features,so as to make the deep features better fusion.We use Sentinel 1 satellite SAR data and Landsat-8 satellite multi-spectral data to carry out experiments,and compared with IHS,Wavelet,IHS?NSST?SR,NSCT?AVG,RSIFNN methods.The experimental results show that: compared with the existing fusion algorithms,the proposed algorithm has a significant improvement in both subjective and objective evaluation,and achieves good results in spectral information preservation and spatial detail enhancement.