Research On Remote Sensing Big Data Processing Technology For Global Change Parameters

In recent years,with the increase of human activities,global environmental changes have intensified and sustainable development has become an important issue facing human society nowadays.Meanwhile,the quantification and real-time analysis of global change parameters have gradually become a hot research topic nowadays.However,the quantification of global change-related parameters requires the processing of large-scale or even global-scale remote sensing images,and covers multi-source,multi-temporal and long time series data analysis,which brings great challenges to the traditional remote sensing data storage,management and effective processing.In this paper,we combine cloud computing and big data technologies to conduct an indepth study on the management and processing of massive remote sensing data,and finally design and implement a fast processing system for global change key parameters to provide efficient online processing services for various global change parameters products.Firstly,the problem of unified data integration,management and organization is difficult due to the characteristics of massive and multi-source remote sensing images.In this paper,by analyzing the commonality and characteristics of multi-source remote sensing images,a unified metadata format for geographic information is established to provide standards for multi-source data integration.Meanwhile,a distributed remote sensing data index is established based on Solr Cloud to improve the rapid retrieval capability of massive remote sensing data.Secondly,it addresses the problems of complicated data screening,long preparation time and troublesome pre-processing steps that make the automation limited in the traditional data processing process.In this paper,a global change key parameter processing system is built to carry out distributed/parallel processing of remote sensing data in a fulllink and integrated manner.By sorting out the production processes and algorithms of various global change critical parameter products,standard algorithm modules and production data knowledge base are built to form a complete logical process of remote sensing product production and improve the automation of the system.Various task and resource scheduling strategies are used in the system at the same time to improve the utilization of system resources.Subsequently,this paper implements a large-scale remote sensing image mosaic segmentation method using Spark engine in order to make a research discussion on the direction of fine-grained parallelized processing of remote sensing data.At the same time,the Alluxio in-memory file system is used to realize the data staging function among multiple processes,which reduces the large amount of time occupied by IO in data processing and improves the overall data processing efficiency.Finally,this paper uses multiple high-performance computers to build a fast processing system for global change key parameters,and then conducts experiments and analysis on various modules in the system such as data integration and retrieval,large-scale remote sensing image mosaic and six global change key parameters product production to verify the efficiency and stability of the system.