Remote Sensing Image Fusion Algorithm Based On Sparse Tensor And Multi-view Features

Remote sensing image fusion is designed to use low resolution multi-spectral images and panchromatic images to obtain multi-spectral images with both high spatial resolution and high spectral resolution.In recent years,remote sensing image fusion technology is widely used in agriculture,military,medical,natural resources and other professional fields.At present,the existing remote sensing image fusion methods use the spatial details in the panchromatic image to be injected into the low resolution multi-spectral image,or fuse the low resolution multi-spectrum images band by band.However,these methods only extract the spatial information simply under a certain transformation or view,and then inject into the multi-spectral images,not only ignores the comprehensive expression of the image under different views,but also does not take into account the relationship between the spectral bands.Resulting in the distortion of the spectral information of the multispectral image after fusion.Tensor is a high-order generalization of vectors and matrices,and is well suited for the joint description of multiple spectral images.In addition,multi-view modeling is based on multiple views to model the object,and can improve the learning performance by using multiple functions of the object.In this paper,tensor representation and multi-view modeling are used to reduce the spectral distortion of the fused image.The sparse tensor nearest neighbor embedding fusion,multi-view sparse tensor nearest neighbor embedding fusion,and tensor fusion with deep features are studied.The specific work includes: 1.A remote sensing image fusion method based on sparse tensor nearest neighbor embedding is proposed.Aiming at the spectral distortion phenomenon existing in the existing fusion algorithm,the multi-spectral image is constructed as a tensor block,and the tensor sparse representation of the multi-spectral image under the multi-mode dictionary is established.It is assumed that the embedding relation of low-resolution multi-spectral blocks in low-dimensional manifold can be mapped to high-dimensional manifold,and then N-way Block OMP and sparse tensor of the neighboring embedded theory are used to solve sparse coding coefficient.The high resolution multi-spectral image is obtained by weighting the high resolution panchromatic image with sparse coding coefficients.Because the tensor makes full use of the correlation between the bands of the spectral image and the statistical properties of the high order data,the proposed method can better maintain the correlation between the bands in the fusion,thus reducing the spectral distortion.The experimental results show that the fusion results of this method are 0.6 ~ 0.8 higher on the SAM metric,than the current recovery method.2.A multi-view sparse tensor neighbor embedding fusion method is proposed.Considering that in a single view,different image blocks may have the same gradient feature.In order to represent the feature of the image block more effectively,we form the multi-view modeling analysis of the image,select the gray features,texture features and edge features of the original image construct multi-view model,from the multi-view expressed the essential features of image blocks.It is assumed that under the fusion framework of Nonsubsampled Contourlet Transform(NSCT),the band-pass coefficients contain most of the spatial information and a mount of spectral information,and multi-mode dictionaries are constructed using the multi-view features of the image.N-BOMP is combined with the sparse tensor of the neighboring embedding theoretical model to solve the band-pass coefficients,then obtain the high resolution multi-spectral image after inverse transformation.Because the tensor form makes full use of the correlation between the band coefficients,and the multi-view method improves the reliability of the multi-mode dictionaries atoms,the proposed method achieves better fusion results.The experimental results show that the sparse tensor fusion method under the multi-view frame improves the spatial resolution and spectral resolution of the multi-spectral image,and obtains the advanced remote sensing image fusion results.3.A multi-spectral and panchromatic image fusion algorithm based on sparse tensor and deep features is proposed.Due to the multi-view sparse tensor nearest neighbor embedding fusion method,the manual selection of multi-view and the removal of redundant view during construction are discussed.In order to obtain the deep description of the image automatically,Convolutional Auto-Encode network(CAEs)is designed to realize the hierarchical semantic extraction of the image features.Assuming that the low-resolution multi-spectral image block and the panchromatic image block can obtain the tensor structure coefficients under the multi-mode dictionary respectively,the high-resolution multi-spectral image can be obtained by multiplying the tensor structure coefficient with the multimode dictionary.The use of the CAEs network extract the deep features automatically,and increase the reliability of multi-mode dictionary atoms,reduce the manual selection of multi-view process,better capture the high-frequency details of the image.From the large number of experiment results and the average line graphs can be clearly seen: the method proposed both in the visual effects and numerical indicators have achieved good results.