Generating High Spatio-Temporal Resolution NDVI Data Based On Multi-Branch Adaptive Fusion Network

With the development of remote sensing technology,satellite images with spatial and temporal resolutions have become essential data for monitoring and analyzing land surface condition and it dynamics at regional and global scale.High spatial and temporal resolution Normalized Difference Vegetation Index(NDVI)data is the basis for real-time and precise monitoring and assessment of vegetation change.However,due to factors such as cloud contamination,seasonal snowfall,and sensor constraints,most exsiting satellites struggle to achieve high spatial and high temporal resolutions simultaneously,thus preventing the acquisition of high-resolution remote sensing data in both dimensions.The development of effective spatio-temporal data fusion algorithms,combining the merit of sensors across different satellite platforms,can facilitate the generation of high-resolution imagery and meet the data requirements of deepening remote sensing applications.Deep learning provides new feasibility for spatio-temporal fusion algorithms,whereas existing deep learning-based methods still encounter issues such as limited datasets and insufficient accuracy of fusion results.Therefore,it is crucial to improve the accuracy of input data for spatio-temporal fusion algorithms,mitigate the uncertainty inherent in spatio-temporal fusion,and explore more robust deep learning spatio-temporal fusion models for reconstructing high-resolution NDVI data.This area of research holds considerable importance for the scientific community.In this dissertation we propose a Multi-branch Adaptive Fusion Network-based Spatiotemporal Fusion Method(MAFN-STF).Based on MODIS and Landsat NDVI data from a public dataset in two irrigation areas in Australia,MAFN-STF algorithm is validated by generate high spatial and temporal resolution NDVI data.Firstly,a downscaling method based on sensor bias correction is proposed,which constructs a difference set of low spatial resolution image reference pixel and their neighboring pixels,and combines the set with the true sensor bias to form a bias pair.A machine learning model(Light GBM)is then used to learn the relationship between the bias pair,conduct the low spatial resolution data(MODIS NDVI)downscaling to generate a 30 m spatial resolution NDVI as the input for the spatio-temporal fusion model.Secondly,a spatio-temporal fusion algorithm(MAFN-STF)based on a multibranch adaptive fusion network is developed,which is theoretically based on the multi-task learning concept of migration learning.The algorithm constructs a spatial information nonlinear mapping branch,a time-varying information extraction branch,and a texture enhancement branch to learn the spatial difference features,time-varying features,and texture features of the input data,respectively.The Adaptive Fusion Module(AFM)is used to fuse the learned features,and the image structure loss and image element value loss are used as the optimization objectives to obtain a high-resolution NDVI image of the predicted date.Finally,the accuracy of the fused NDVI data is evaluated with observed Landsat NDVI data on the predicted date and compared with the results of four spatio-temporal data fusion methods including Flexible Spatiotemporal Data Fusion Method(FSDAF),Spatial and Temporal Adaptive Reflectance Fusion Model(STARFM),Reliable and Adaptive Spatiotemporal Data Fusion Method(RASDF)and the Enhanced Deep Convolutional Spatiotemporal Fusion Network(EDCSTFN).The main findings are presented as follows:1.By incorporating neighborhood information and machine learning techniques to correct sensor bias,the downscaled MODIS NDVI images exhibits good spatial continuity,effectively avoiding such as homogenization of image pixels and significant jaggedness caused by interpolation-based downscaling.An accuracy analysis,using real observed Landsat NDVI images,revealed that in comparison to downscaled images by nearest-neighbor interpolation,the mean square error of the downscaled MODIS NDVI images generated using the sensor bias correction method decreases by 12.31%,the correlation coefficient increases by 13.01%,the mean absolute error decreases by 10.43%,and the structured similarity index increases by11.47%.Consequently,this correction method solves the problems of sensor inconsistency and discrepancies between sensors originating from the pre-processing of input data,which can lead to unstable spatiotemporal fusion results to a certain extent.2.MAFN-STF algorithm resolves the issue of current fusion models in acquiring two or more ideal priori image pairs as input data simultaneously.By fully learning multifaceted correlation information between remote sensing images,more accurate predicted NDVI images will be generated.The accuracy analysis showed that the fused 30 m resolution NDVI correlation coefficients are 0.9283,0.9187,and 0.8851 in three experimental areas(i.e.,the high heterogeneity area with rapid physical change(CIA),rapid land cover change area(LCG I),and homogeneous landscape area(LCG II)),respectively.The correlation coefficients are0.9283,0.9187,and 0.8851 and root mean square error are 0.0596,0.0657 and 0.0465 respectively.The mean absolute errors are 0.0402,0.0482,and 0.0330,and the structured similarity indices are 0.7245,0.6641,and 0.7220,respectively.MAFN-STF algorithm has better fusion accuracy superior to the other four spatio-temporal fusion algorithms.3.By comparing the time series NDVI data with an 8-day interval and a 30 m spatial resolution simulated by different spatiotemporal fusion algorithms in three experimental areas,it can be found that the NDVI trend predicted by the MAFN-STF model is closer to the observed Landsat NDVI,better reflecting the spatiotemporal variation characteristics of NDVI.The comprehensive average fusion accuracy results show that the MAFN-STF method has higher accuracy in terms of correlation coefficient(0.8957),root mean square error(0.0724),mean absolute error(0.0512),and structural similarity index(0.6719).The time series data generated by the MAFN-STF model has high accuracy and stability under various conditions,making it applicable for various land cover regions.