The Research And Application For High Performance Computing Of Remote Sensing Data Based On The Integration Of Computing And Storage

The rapid development of high performance computer provides powerful computingresources for massive remote sensing data processing. High performance computing based oncluster and the integration of computing and storage has become the most main computingplatform. To improve the remote sensing data processing speed, it is a necessary choice to designand implement massive remote sensing data parallel processing system in a cluster environment.With the increase of high resolution and sensor type, the growth in TB level of the remotesensing data cause great pressure to the I/O request for remote sensing data processing platformfor high performance, including the pressure to read and write for server disk, the transmissionpressure for server network and so on. In order to maximize the advantage of distributed parallel,on the premise of guarantee system security and stability, this paper adopted the strategy of theintegration of computing and storage to reduce the transmission of data in parallel processingsystem.Based on the analysis of advantages and disadvantages of several kinds of high-performancecomputing method, such as the grid, cloud computing and cluster computing, this paper proposesand implements a high performance cluster architecture based on the integration of computing andstorage, in order to realize the efficient processing of massive remote sensing data, which has beenapplied to the first research project of high resolution information products "Production Line"application system.The work of this paper is mainly manifested in the following: 1. Based on existing cluster system and in-depth study of remote sensing data processingsystem for high performance computing, against to the pressure to processing platform, serverdisk read and write, and the network transmission caused by the massive remote sensing data, thispaper uses the method to store the data from the original one machine to integration architecture ofcomputing and storage. It can minimize I/O transmission for system architecture and make moreuse of the computing resources of storage server and storage resources of computing server, so asto improve the processing efficiency of the platform.2. This paper adopts distributed storage model which is logically static while expansiondynamically physically to implement the storage and management of massive remote sensing data.It introduces the concept of Virtual Disk Space (VDS) to assign stored section to multiple sites,and use the way of cross backups to ensure the stability of the system.3. It plays an important role in large-scale remote sensing image processing to use the twolevels of parallel strategy to maximize the high performance parallel computing.4. The experimental validation. By using the data of different types of satellite for test,compared the validation from multiple angles, it proves that the parallel processing ability andefficiency are improved for the proposed high performance cluster architecture.