Joint Classification Based On Deep Learning For Hyperspectral And LiDAR Data

Compared with single data source classification,multi-source remote sensing data joint classification can comprehensively use remote sensing data information obtained from different sources to achieve more accurate and comprehensive earth observation,and is widely used in environmental monitoring,Precision agriculture,geological survey,military and security fields.Due to the different sources and structures of multi-source data,this poses many challenges for feature extraction and joint classification of multi-source data.This article takes deep learning as the research foundation and designs three deep network models for joint classification of multi-source remote sensing data from the perspectives of multi-scale feature extraction,attention module design,and fractional Gabor convolution.The effectiveness of the proposed method is verified through classification experiments on multiple sets of public datasets.The main research contents of this paper are as follows:(1)A Multi-scale Feature Extraction Network(MsFEN)based joint classification method for hyperspectral and LiDAR data is proposed to address the problem of reduced effective information and reduced target detection ability in remote sensing data caused by excessively deep network models.This method uses selection kernel convolution to adaptively select the size of the convolution kernel,effectively extracting multi-scale features from multi-source remote sensing data,and achieving fine-grained feature classification.Large scale features contain more semantic information,while small scale features contain more detailed information.This method improves the classification accuracy of targets by designing a multi branch network structure and utilizing multi-scale features.Through experiments on publicly available datasets,the experimental results demonstrate that the proposed method has good classification performance in different scales of ground object recognition,improving the feature representation ability of multi-source data.(2)When using deep networks for feature extraction,attention mechanism is introduced,which can achieve efficient feature extraction by assigning weight coefficients and eliminate the impact of redundant features on classification results.A joint classification method for hyperspectral and LiDAR data based on attention mechanism is proposed.Attention modules are introduced between the branches of hyperspectral and LiDAR data,giving lager weight to key features.At the same time,the spatial attention weights obtained through the LiDAR data processing channel are allocated to the hyperspectral data processing branch,achieving complementary features of multi-source data,Improve the efficiency and accuracy of multisource data joint classification.Through experiments on public datasets,the experimental results demonstrate that the proposed method can effectively enhance the expression ability of spatial and spectral features,and enhance the information exchange ability between multisource data.(3)A joint classification method for hyperspectral and LiDAR data based on fractional Gabor convolution is proposed to address the lack of representation of directional features in existing methods that usually only extract features in a single direction.Fractional Gabor convolution can achieve directional rotation in different fractional orders,extract features of hyperspectral and LiDAR data in multi-directional dimensions,collaborate spatial,spectral,and elevation features,and achieve feature fusion and joint classification of multi-source data.Through experiments on public datasets,the experimental results demonstrate that the proposed method obtains rich multi-source data features through multi-directional spatial feature extraction,thereby improving classification performance.