Multisource Remote Sensing Image Fusion For Pansharpening Based On Image Filtering And Convolutional Autoencoder

Satellites collect extremely important data about the Earth,which are used for various purposes and benefits,including environmental monitoring,weather forecasting,and cartography.Although satellites are useful in many ways,they are costly to develop and operate.It is also challenging to maximize the value of data acquired from accessible satellites,such as when integrating the output of several sensors.A proper illustration of that is the fusion of multispectral images with a low spatial resolution but a high spectral resolution and panchromatic images with a high spatial resolution but a low spectral resolution.Multispectral imagery helps distinguish land surface characteristics and landscape patterns.It can help obtain rich spectral information while acquiring geometric information of the scene to fully reflect the essential attributes of features and the subtle differences between various features.In contrast,panchromatic images have a single band that combines the visible bands’ information.It produces a single intensity value per pixel,as in a greyscale image.Due to the limitation of the resolution in multispectral and panchromatic images and how to more effectively combine the image’s spatial-spectral resolution information are all key solutions to improve fusion results that positively affect remote sensing applications.As such,this thesis,based on the research on the development status,conducts an in-depth exploration of image fusion for pansharpening based on image filtering and convolutional autoencoder techniques.These techniques are referred to as pansharpening,which represent the main focus of this dissertation as explained below:(1)This dissertation introduces remote sensing image fusion’s definition,development status,demonstrates its applications,and the current advancement of remote sensing multispectral pansharpening.In addition,the dissertation also introduces the evaluation criteria for image fusion methods.(2)A pansharpening approach using kernel-based image filtering is proposed;The dissertation proposes a technique based on image filtering utilizing a bilateral filter to generate high-frequency details from the panchromatic image.The various types of side window-guided filters are employed to enhance the multispectral band from panchromatic image and then use these filters to adjust spatial data misfortune when images are combined.Experimental results demonstrated that the proposed method provides consistent results concise with reported by the previous research in terms of subjective and objective assessments on remote sensing data.(3)A pansharpening method based on a convolutional autoencoder is also proposed in this dissertation.First,the autoencoder network is trained to reduce the difference between the degraded panchromatic image patches and reconstruction output original panchromatic image patches.The intensity component was developed by adaptive intensity-hue-saturation(AIHS)then was delivered into the trained convolutional autoencoder network to generate an enhanced intensity component of the multispectral image.The pansharpening was accomplished by improving the panchromatic image from the enhanced intensity component using a multiscale guided filter;then the semantic detail is injected into the upsampled multispectral image.Real and degraded datasets are utilized for the experiments,which exhibit that the proposed technique can preserve the high spatial details and high spectral characteristics simultaneously.Experimental results demonstrated that the proposed study performs state-of-the-art results in terms of subjective and objective assessments on remote sensing data.(4)In addition,this dissertation proposes a panchromatic and multispectral image fusion method based on a multi convolutional autoencoder(CAE).First,an original panchromatic(PAN)image is constructed from its spatially degraded version.Then,the relationship between the original PAN image and its degraded version is utilized to reconstruct the high-resolution MS image.An intensity component of MS image is obtained using an Adaptive Intensity-Hue-Saturation(AIHS)and reconstructed using the aforementioned relationship.Two types of remote sensing datasets are adopted,and the effect of the patch size with the overlapping pixel on spectral and spatial distortion is considered.After training CAE,the low-resolution MS image and its intensity component are given to the trained network as input to obtain MS image and intensity component with better details.Eventually,the fused image is obtained by using a component substitution framework.Experiments affirm that the proposed method has better results compared with some exiting methods from objective and visual aspects.(5)Finally,a technique based on the non-subsampled contourlet transform(NSCT)and convolutional autoencoder(CAE)is proposed in this dissertation.The NSCT is used to decompose the MS and PAN images to high frequency and low-frequency components using the same number of decomposition levels.A CAE network is trained to generate original low-frequency PAN images from their spatially degraded versions.Low-resolution multispectral images are then fed into the trained convolutional autoencoder network to generate estimated high-resolution multispectral images.Similarly,another CAE network is trained to generate original high-frequency PAN images from their spatially degraded versions.High-resolution multispectral images are then fed into the trained convolutional autoencoder network to generate estimated highresolution multispectral images.The final pan-sharpened image is accomplished by injecting the detailed map of the spectral bands into the corresponding estimated high-resolution multispectral bands.The proposed method has been tested on QuickBird datasets and compared with some existing pan-sharpening techniques.Objective and subjective results demonstrated the efficiency of the proposed method and its usability for Quick Bird data.