Research On Remote Sensing Image Fusion Method Based On Deep Learnin

With continuous development of remote sensing technology,there is a growing demand for high-resolution multi-spectral(HRMS)images.However,due to limitations in remote sensing instrument hardware,information transmission,and reception,current sensors can not directly capture HRMS images.Instead,they can only obtain low-resolution multispectral(LRMS)images and high spatial resolution panchromatic(PAN)images.Therefore,it is significantly valuable to investigate the generation of HRMS images that exhibit enhanced spatial details and spectral characteristics by LRMS and PAN images.In recent years,computer vision has witnessed substantial advancements,leading to the application of generative adversarial networks(GANs)in remote sensing image fusion.This thesis explores a GAN-based method for remote sensing image fusion and conducts a series of experiments utilizing image datasets from three remote sensing satellites including Quick Bird,Gao Fen-2,and World View-2.The specific contributions of this study can be summarized as follows:(1)To address the common issues of color blurring,excessively smooth edges,and poor visual perception in generated images during remote sensing image fusion,this thesis takes the fusion problem as a coloring problem and presents a supervised remote sensing image fusion method based on Generative Adversarial Networks(named PAN-GAN).The proposed method utilizes the LRMS image as the reference image and simulates the PAN image using the grayscale reference image.It incorporates the blurred reference image as input to the generator,which independently extracts texture detail features from the former and spectral features from the latter,and fuses them for image reconstruction.By introducing perceptual loss,jointly optimizing adversarial loss and pixel loss,the reconstructed image exhibits spectral and texture detail features that closely resemble those of the reference image.Experimental results demonstrate that the proposed method outperforms common methods,yielding HRMS images with more realistic spectral and spatial texture details.Moreover,the incorporation of perceptual loss further enhances the quality of the reconstructed images and provides empirical evidence for the effectiveness of the proposed method;(2)Due to the practical unavailability of HRMS images,supervised image fusion methods often rely on the Wald protocol,which limits their effectiveness to reconstruct high spatial resolution images.Consequently,unsupervised remote sensing image fusion methods based on GANs have garnered significant attention.However,existing unsupervised methods suffer from shortcomings in the design of loss functions,resulting in suboptimal overall effects for generated HRMS images.To address this issue,a GAN-based unsupervised remote sensing image fusion method is proposed,utilizing PAN and LRMS images as inputs.The generator extracts spatial and spectral features from the input images and fuses them to generate HRMS images.Concurrently,spatial and spectral discriminators are employed to calculate the spatial detail difference(spatial loss)with the PAN image and the spectral difference(spectral loss)with the LRMS image for the generated HRMS image.By introducing perceptual loss and image quality loss and combining them with adversarial loss,spatial loss,and spectral loss to form a fusion loss function for generator optimization,HRMS images with rich spatial information and spectral features can be obtained.Experimental results show that the proposed method yields HRMS images with sharper edge and texture spatial features compared to existing methods,which highlights the effectiveness of the model in remote sensing image fusion.