Multi-Scale Dense Pan-Sharpening Method Based On Relative Average Generative Adversarial Network

With the popularization of China’s remote sensing industry,remote sensing image fusion has become a research hotspot in recent years.Due to the physical characteristics of satellite sensors,high-resolution multispectral images cannot be obtained.Therefore,the purpose of this study is to fuse low resolution multispectral images with high-resolution panchromatic images to obtain high-resolution multispectral images,a process known as panchromatic sharpening.The existing panchromatic sharpening methods have problems such as unstable performance,distorted fusion of image spectrum and spatial information,and visual artifacts.In response to these issues,the specific research content of this article is as follows:1.To solve the problem of incomplete extraction of depth image features during panchromatic sharpening,which can easily cause information loss during the convolution process.And to solve the problem of neglecting the spectral correlation between different channels in multispectral images,causing spectral distortion in the fusion results,and ignoring the differences in spatial features in different regions,resulting in blurred texture details in the fusion results.This paper proposes a panchromatic sharpening relative average Generative adversarial network(FAP RaGAN)based on attention mechanism.Taking the relative average generation countermeasure network as the basic backbone network,firstly,multi spectral images and panchromatic images are input into the network in two branch structures to solve the problem of ignoring the spectral difference between the two as a single network input,which leads to spectral distortion;In the feature extraction module,the improved dense connection structure maximizes the preservation of intermediate layer features and eliminates redundant parameters;The extracted features are fused using the principle of pixel addition,and then a residual triple attention module is proposed as the feature refinement module.By adjusting the attention allocation on different channels and spaces,the indirect correspondence between channels and weights is eliminated;In addition,an additional channel attention module is proposed to provide rich feature representations by assessing the importance of cross dimensional interaction during accelerated training,thereby obtaining refined features;Finally,the image reconstruction is carried out according to the pan-sharpening detail injection model,which improves the spectral quality and detail recovery ability of the fused image.2.To solve the problem of incomplete extraction of image features at different scales during the panchromatic sharpening process,and the inability to fully utilize all the features extracted by the network,resulting in poor fusion results and ability to restore details.This paper proposes a multi scale Dense Pan Sharpening Relative Average GAN(MDP RaGAN)based on multi-scale density and attention mechanism.In the dual-stream input structure,multiscale feature extraction and dense connections are combined to increase the connection between different layers.In addition,a local dense connection structure is added to the multi-scale dense block,so as to reduce the loss of source image spatial information and spectral information,and realize feature reuse.Then,this paper designs an overall dense connection structure,which allows direct connection from the current multi-scale dense block state with all subsequent multi-scale dense blocks,fuses multi-level features,avoids the loss of extracted detailed information during propagation,and further extracts more advanced features.In the feature refinement and image reconstruction modules,experiments are still carried out using the existing pansharpening structure to obtain fusion images.Based on Quick Bird satellite images and World View-2 satellite image data sets,this article verifies the effectiveness of FAP-RaGAN network and MDP-RaGAN network.The experimental results show that FAP-RaGAN and MDP-RaGAN networks have strong practicability in both low resolution and full resolution experiments.