Hyperspectral Image Classification Based On Self-supervised Learning And Multi-Structure GAN

Hyperspectral image classification is one of the challenging topics in remote sensing image processing,which can effectively realize the fine recognition of ground objects.However,limited training samples greatly restrict the classification performance of hyperspectral images.Generative adversarial networks(GANs)are known for their powerful generative power,which forces the generator to learn the distribution of real samples through adversarial optimization between the generator and the discriminator to obtain high-quality generated samples.At the same time,the discriminator also improves its classification ability in the process of continuous discrimination.Therefore,GAN is applied in hyperspectral image classification to alleviate the problem of insufficient training samples.In hyperspectral images,the categories of ground objects are diverse and the class distribution is imbalanced,which easily leads to GAN falling into mode collapse and makes the generated samples corresponding to the categories with a small number of samples lack diversity.At the same time,limited by the influence of imaging conditions,the phenomenon of "different objects with the same spectrum" is easy to appear in hyperspectral images,making it difficult to effectively distinguish some confusing samples.In view of the above problems,how to effectively design generators and discriminators suitable for the hyperspectral image classification task,so that GAN can alleviate the problem of mode collapse and realize the effective distinction of hard samples while generating diverse samples are the research focuses of this thesis.In addition,the number of hyperspectral image bands is large,and the use of full-band images for classification poses a huge challenge to the network operation efficiency.The existing methods reduce the dimension of the hyperspectral image first to decrease the computational cost of the network,but the image after dimension reduction will lose some information.Therefore,how to process hyperspectral images to retain less dimension while maximizing the use of important information in the original image is also one of the research focuses of this thesis.In order to solve these problems in a targeted manner,three different GAN methods are proposed for hyperspectral image classification.The main research contents of this thesis are as follows:(1)A GAN method based on a multi-branch generator and a multi-granularity discriminator is proposed.In this method,a generator with a multi-branch structure is designed for the problem of unbalanced class distribution.The generation advantage of the categories with a large number of samples is weakened by the re-division of the training samples,and then the adaptive fusion module is used to fuse the results of multi-branch generation,forcing the generator to generate samples with better diversity.The multi-granularity discriminator adopts the recurrent neural network as the basic structure,and gradually extracts the regions that should be paid attention to by the spatial attention mechanism.Finally,the multigranularity discriminant features are used to realize the classification.The proposed method is validated experimentally on three different hyperspectral image data sets,and the experimental results show that the proposed method is effective and superior.(2)A GAN method based on self-supervised learning and a divide-and-conquer discriminator is proposed.According to the self-supervised learning architecture,this method designs an upstream clustering task based on the encoder-decoder network and a downstream classification task based on a divide-and-conquer discriminator.The clustering task uses a large number of unlabeled samples to learn its clustering centers in the highdimensional space.After the training is completed,the cluster network will be transferred to the classification task as a clusterer,and then the labeled samples in the downstream classification task will be accurately divided.Aiming at the problem of "different objects with the same spectrum",a divide-and-conquer discriminator is designed in this method,which sends these easily confused hard samples to the expert branches for re-classification,so as to improve the classification accuracy of hard samples.Experimental results show that the proposed method is effective and superior to other comparison algorithms.(3)A GAN method based on sample enhancement and masked encoding and decoding is proposed.Aiming at the problem of insufficient information extraction of training samples,this method designs a learnable sample enhancement way to maximize the extraction of effective information in the original image.A spatial weight map is generated by using spectral attention and spatial attention.Then,the hyperspectral image after principal component analysis is multiplied with the spatial weight map to obtain the enhanced input image.The method also proposes a masked encoder-decoder generator,a certain proportion of spatial blocks in the input image are occluded by masking,and only a small number of visible spatial blocks are learned and the generated samples are finally obtained.Compared with the method of generating samples by deconvolution with random noise,this method directly learns from real samples,which can not only reduce the generative difficulty of the generator but also improve the quality of generated samples.Experimental results demonstrate the effectiveness of the proposed method.