Research On Distributed Scientific Data Calculation Framework Based On Remote Object Technology

With the implementing of new generation ground-based and space-based radio telescope, how to deal with massive astronomical observation data is becoming an important problem in the current astronomical research. The MingantU SpEctral Radioheliograph(MUSER) generates observation data in every three milliseconds. In the data processing procedure, MUSER-I generates sixteen images, and the data volume is more than twice of MUSER-I. Therefore, it is important to figure out a way to meet the challenge of processing the MUSER’s massive observational data, for guaranteeing the construction and observation of MUSER. Aiming at the characteristics of MUSER data processing, in this paper, we present a high-performance distributed scientific data computing frame, which is designed for easily rapid deployment. The primary work is as follows:First, the realization of the distributed computing framework scientific data relys on the remote object technology. We contrastively studied on the field of remote object technique. Through analysis and comparsion, we believe that the PYRO(Python Remote Objects) woule have enough characters, which are needed by our research;Second, we designed and developed the frame. Based on openness and scalablility, we designed and developed the frame, to provide some infrastructural support to the design and implemention of astronomical data processing project, especially to MUSER.Third, on the basis of frame, we designed and developed a set of concise APIs. According to the APIs, developers could easily invoke the frame, focusing on the logical design in performing tasks instead of distributed concept.This paper is a basic research in scientific data processing with distributed environment. It presents a valuable reference for MUSER data processing system and the astronomy observation data processing realm. Although there are some issues still need to be discussed and further imporved, we had deployed the frame and completed the basicly initia target.