Collaborative Studies Of Large-scale Geographic Raster Data Parallel Processing

With the improvement of new generation of data acquisition technology, especially high resolution remote sensing satellites, the spatial or temporal resolution, feature data types and cover area of geographic raster data are greatly enhanced, this lead to geometric increase in spatial data amount, and traditional single-process raster analysis methods can’t meet the requirement of large geographic raster data’s process and analysis. Therefore, it is very important that research the methods and systems of big geographic raster data high performance computing to further improve the development efficiency and complex problems solving ability. In order to solve problems faced by big raster geographic data parallel processing, this article pay attention to the big geographic raster data parallel processing method based on high-performance computing architecture. By introducing MPI (Message Passing Interface) and MP (Multi processing) as underlying parallel environment, this research studies the necessary functions and core route of raster data processing algorithms, integrates common part of algorithms, adopts the idea of design pattern, constructs system framework which according to principles of object-oriented programming, proposes a novel Cooperative Big Geographic Raster Data Parallel Processing Framework (CBGRDPPF), combines the parallel program type and complexity characteristics of geographic raster data processing, discusses tasks and techniques of task collaborative method of geographic raster data processing under this framework. This research analysis the impact of different parameters, speed and parallel efficiency, and achieve the optimization of framework, thereby establishing a collaborative parallel development pattern to solve complex geoscience problems, provide a new solution and technical support for the efficient processing of geographic raster data. This article archive following works:(1) Proposed parallel decoupling method for big geographic raster data parallel processing algorithm.Based on MPI with MP parallel environment, this research abstract and encapsulate the common part of geographic raster computing and parallel management, execute functions as parts of loose coupling assembly and execution, separate strong coupling parallel computing system and geological problems effectively.(2) Build a collaborative framework for geographic raster data parallel processingBased on the geographic raster data parallel processing mechanism and parallel decoupling study, this research build a collaborative development mechanism of data block partition, distribution and assemble classes model, establish foundation of geographic raster data collaborative development; Construct core algorithm wrapper class and development strategies, and achieve the separation of code development and algorithmic details to ensure the collaborative work of parallel program development and application analysis effectively.(3) Propose geographic raster data global computing based on decoupling methodTo deal with the problem of the difficulty of load whole data into memory in once when perform big geographic raster data global computing algorithm, this research analysis geological processing algorithm’s principle, based on CBGRDPPF framework using the data of vertical and horizontal division, process data temporary storage strategy, make each parallel process under the condition of limited memory space to process the whole raster data in sequence, greatly reduces the development complexity, thus realize complex geographic tasks parallel computing data efficiently.(4) Construct geographic raster data dynamic computing strategy based on decoupling methodDynamic computing is mainly refers to some raster process algorithms which have dynamic iterative process and the amount of step is unknown, this article, through FCM—a representative clustering algorithm, based on CBGRDPPF framework, through the multi-strategy of data separation, synchronization and broadcast mechanism, realize a dynamic geographical raster data parallel processing method, so as to solve the parallelization problem of uncertain computing task among processes.