Hierarchical Land-cover Classification Based On Hierarchical Category Structure And Convolutional Recurrent Neural Network

Land-cover is an important variable when investigating the properties of the Earth’s surface.Land-cover can be organized hierarchically through the class-subclass relationships.The upper layers of the hierarchical category structure contain information related to the biotic and abiotic land-surface features at a large scale,while the lower layers distinguish the land-cover attributes related to vegetation,landforms,etc.However,the existing studies have ignored the transmission and dependencies among the different layers of the whole hierarchy.For instance,the determination of class is not only affected by the parent class,but also by the child classes.Therefore,to fully mine the information transmitted between the layers of the hierarchy,and effectively use the temporal,spatial,and spectral information on remote sensing images to classify land-cover is a problem to be considered.Based on hierarchical category structure and convolution recurrent neural network,a hierarchical classification method of land-cover is proposed in this study.Firstly,based on the hierarchical category structure,this study designs an automatic extraction approach for multi-level samples across multiple classification systems.Then,taking the dense MODIS time-series data as input,a hierarchical category structure based convolutional recurrent neural network（HCS-Conv RNN）is proposed.The HCS-Conv RNN constrains the input through the leaf node of the hierarchical structure based input layer,and then constructs the dependencies among the different layers in a top-down manner,to classify the pixels into the most relevant classes in a layer-by-layer manner.A total of 1096 MODIS images of China were used in the experiments in this study.The experimental results show that the overall accuracy（OA）of the classification results of HCS-Conv RNN in level-1 is 0.9218 and Kappa is 0.8009.The level-2 classification OA is 0.6172 and Kappa is 0.5338.The OA and Kappa of level-3 is 0.4853 and 0.4343 respectively.The level-4 classification OA and Kappa is 0.4527 and 0.4137 respectively.As the classification level increases,the land-cover classes become more detailed and the accuracy gradually decreases.The comparative experiments show that:（1）the accuracy at each level of HCSConv RNN is better than that of MCD12Q1,and the difference becomes larger as the classification level increases.（2）HCS-Conv RNN considers the hierarchical relationship between classes and can achieve better accuracy compared with random forest and general deep learning network.（3）The multitask loss function and convolution layer in HCS-Conv RNN can improve the classification accuracy at all levels.The leaf node of the hierarchical structure based input layer can maintain the logical consistency of classification results in hierarchy.Therefore,the HCS-Conv RNN proposed in this study can effectively achieve hierarchical classification of land cover,and has the potential to be used for accurate landcover classification at a large scale.