Multispectral Image Terrain Classification Based On Feature Adaptive Fusion

The fusion classification of multispectral(MS)image and panchromatic(PAN)image is a research hotspot in remote sensing image processing.Multispectral image and panchromatic image are the imaging results of the same scene at the same time,but due to the difference of imaging technology,they present a kind of appearance difference,but the essence is the same.The common multispectral image is the remote sensing image data with four spectral bands(RGB + NIR).Its characteristic is that the spectral information is more abundant than panchromatic image.Panchromatic image has only one band,and its advantage over multispectral image lies in more prominent spatial details.Multispectral image and panchromatic image play an important role in urban planning,water conservancy and agriculture,resource detection,geological disaster detection,military reconnaissance and other fields.In this paper,three problems in the field of multispectral and panchromatic image fusion classification are explored and studied.The specific plan is as follows:(1)Aiming at the problem of huge external differences between multispectral image and panchromatic image,we proposed an adaptive weighted IHS algorithm based on intensity hue saturation(IHS)in Pan-sharpening method.Multispectral image and panchromatic image absorb each other’s unique information to reduce the difference between data.In order to quantitatively compare the performance of the algorithm,we combine the strategy with residual 18 network to form a hybrid classification model.Experimental results on two datasets demonstrate the effectiveness of our network.(2)Aiming at the problem that feature fusion of multispectral image and panchromatic image in deep learning model is not efficient enough.We use the idea of feature correlation for reference,and propose a feature fusion module based on feature correlation.The module calculates the covariance matrix of the input features to obtain the correlation between the feature channels,then generates a mask vector according to the correlation,and finally realizes the weighted fusion of the input features by the product of the mask vector and the corresponding channel.This module is a plug and play module.We first verify the effectiveness of the module,then combine the module with the first strategy,and learn from the idea of feature pyramid,propose an adaptive hybrid fusion classification network.Experimental results on two datasets demonstrate the effectiveness of our network.(3)Aiming at the problem of low efficiency of the previous non end-to-end classification model,we propose a spatial spectrum fusion strategy by studying the spatial and channel attention mechanism.They are used to fuse the shallow spatial and spectral features of the network.By combining this module with the full convolution network,we integrate the fusion and classification of multispectral images into an end-to-end network.At the same time,in order to extract more robust features,we also replace the convolution in the full convolution network with the residual block.Experimental results on two datasets demonstrate the effectiveness of our network.