Deep Learning Based Multi-temporal Crop Mapping And Its Interpretability Analysis

The spatial distribution map of crops is fundamental to monitoring the dynamic growth of crops,evaluating agricultural resources,and supporting food security.Crop classification approaches based on machine learning or deep learning algorithms allow identifying crop types through analyzing multi-temporal remote sensing data.These approaches have been widely applied in crop mapping applications.This paper aims to address the problems of weak generalizability and lack of model interpretability in current researches.This paper develops an end-to-end crop classification method driven by deep learning to facilitate high-performance crop mapping in spatial transfer scenarios,and proposes a multi-perspective interpretation pipeline to reveal the process of extracting temporal features and learning crop growth patterns from multi-temporal and multi-spectral remote sensing data.This study provides new insights for the research on dynamic crop mapping approaches and model reliability examination.The main research contents are as follows:(1)A deep learning approach named Deep Crop Mapping(DCM)based on attention mechanisms and bidirectional long short-term memory networks is developed for dynamic mapping of corn and soybean through extracting temporal features from multi-temporal and multi-spectral remote sensing data.In order to verify the spatial generalizability of the approach,the spatial transfer experiment of crop classification models is performed on the data sets of six sites in the U.S.Corn Belt from 2015 to 2018.Full cross-validation is conducted in end-of-the-season crop classification.The experiment results show that the DCM model achieves an average kappa coefficient of 85.8% in local test and 82.0% in spatial transfer,which is significantly higher than Transformer,Multi-layer Perceptron,and Random Forest models(95% confidence interval).The results of in-season classification experiments show that since the beginning of July,the DCM model has captured important information of key growth stages of crops,achieving higher classification performance than other models.(2)A multi-perspective pipeline is proposed for the interpretation of deep learning-based crop classification models.The pipeline consists of three interpretive approaches: input feature importance evaluation,hidden feature analysis,and soft output analysis.To examine the effectiveness of these approaches,the Attention-Based Long Short-Term Memory(AtLSTM),Transformer,and Random Forest models are developed for corn and soybean classification.The assessment of input feature importance demonstrates that the AtLSTM,Transformer,and Random Forest models usually identify weeks 11 to 20(from early-July to late-August)as the key period to distinguish corn from soybean,and regard shortwave infrared bands as relatively important bands.Hidden feature analysis suggests that the AtLSTM model accumulates useful information over growth periods,while the Transformer model extracts the temporal dependencies that contribute critical information to high-level feature learning.The soft output analysis in the in-season classification scenario shows that the increased length of input time series improves the model’s confidence in the classification results.The further comparison of input feature importance in different sites demonstrates the applicability of the interpretation approach at larger spatial extents with heterogeneous landscapes.