Research On Multi-Spectral Remote Sensing Image Fusion Algorithm Based On Convolutional Neural Network

In recent years,remote sensing technology has developed rapidly,people obtain remote sensing satellite data from remote sensing technology platforms,and data analysis and processing are carried out by remote sensing instruments,which continuously provides people with a large amount of scientific data and dynamic information,and realizes people’s multidimensional dynamic monitoring of national land space.With the development of sensor technology,the ability of remote sensing satellites to acquire remote sensing images is also improving.The remote sensing images acquired by the sensors contain different information about the ground and ground objects.For example,panchromatic(PAN)images are rich in spatial details,low-resolution multispectral(LRMS)images contain rich spectral information,and high-resolution multispectral(HRMS)images have both rich spectral and spatial information.Remote sensing images are also used in many aspects of life,especially HRMS images with both high spatial resolution and multispectral characteristics are of great importance and practical application in target detection and target classification.However,due to the limitation of existing technology,commercial remote sensing satellites such as Worldview-2 and Quick Bird cannot acquire HRMS images.To solve this problem,the pansharpening technique in the field of remote sensing image fusion is born,which fuses PAN images and corresponding LRMS images,integrates spatial and spectral information,and finally generates HRMS images.The main innovative work of this paper is as follows:(1)In this paper,an end-to-end attention-based dual-residual remote sensing image fusion network is proposed for the pan-sharpening of MS images and PAN images,and combines cross-residual feature blocks and coordinated attention mechanisms to obtain highquality,high-resolution multispectral images.The network consists of three main stages,two feature extraction stages,and one feature fusion stage,and the stitching of the channel domain features in these three stages is achieved by cross-stage fusion.In the feature extraction stage,cross-residual feature blocks are used to connect the encoder and decoder stages of the MS feature extraction module and PAN feature extraction module.These different features are used to guide the training of each,improving the network’s expressiveness.In the feature fusion stage,the coordinated attention mechanism and the SE attention mechanism are incorporated to encode the channel relationships and long-term dependencies with the precise location information obtained,enabling the network to locate the region of interest more precisely and allowing the features extracted in the first two stages to be better fused with the original image to reduce the occurrence of loss of spectral information and spatial detail information.(2)In this paper,a dual-residual multi-stage remote sensing image fusion method based on local binary patterns is proposed.The proposed network consists of three modules:1)MS residual feature extraction module;2)PAN residual feature extraction module;3)image reconstruction module.These three modules are connected through the attention module and cross-stage fusion.In the MS/PAN residual feature extraction module,local texture features of MS images are extracted and input to the network using a modified local binary pattern to reduce distortion of spatial information in this paper.And the crossresidual feature blocks are used between the MS/PAN residual feature extraction module to extract different features between the encoder and decoder,using these complementary features to help train the two networks.The CBAM attention module is used in the image reconstruction module to emphasize useful features and suppress redundant features from the channel level and spatial level to focus the network on key features to improve the quality of the output fused images.