Space Environment Scientific Data Collection And Key Technologies

This paper is based on The Data Collection Subsystem which is one of the Data and Communication System's six subsystems and the Data and Communication System comes from the Eastern Hemisphere comprehensive space environment monitoring meridian chain project which is called the Meridian Project for short of major national scientific and technological foundation infrastructure projects. The Data Collection Subsystem plays a significant role in the Data and Communication System which is an important part of the whole Meridian Project.The Data Collection Subsystem's main features includes collecting monitoring data from stations of the Meridian Project through network, recording the basic information of scientific data file, standardizing data file's name and storing the original data. According to functional requirements analysis and system architecture design, the Data Collection System can be divided into four parts. The first is the data collection clients including self-developed and standard ftp in the compatibility mode clients which are responsible for manually or automatic collection of the monitoring data files. The second part is the central control server which is responsible for user authentication, scheduling the cluster data collection and FTP servers and forwarding network requests and transmitting data files. The third part is the data collection servers which are responsible for the collection, validation, conversion of data files, and communicating with other subsystems. The last part is the system administration client providing management services for the administrators to monitoring file transfer status, stations'information and etc.Based on the requirements, design and implementation of the Data Collection Subsystem, this paper focuses on the functional requirements, system architecture design, business flow, performance, system's support and operating environment and implementation and the key technologies used to solve problems. In order to schedule the cluster data collection and FTP servers, archive the server performance requirements, forward the external / internal network data, centralized authenticate the clients including the self-developed and standard FTP clients for the consistency of user data and its operation and communicate between heterogeneous systems and so on, the implementation use a variety of appropriate load balancing algorithm, the connection pool technology, resume broken transfer technology, data forwarding method, centralized user authentication technology based on Lightweight Directory Access Protocol combination of asymmetric encryption algorithm(RSA), Web Services technology and NFS file sharing technologies and so on. Considering the system features, this implementation is an optimal solution right now and it's achieved.Taking the convenience, future expansion, good maintainability and rapid system upgrading of the Data Collection Subsystem into account, this paper also conducted a more in-depth research and prospects in the system architecture, usability and performance areas.