Research And Implementation Of Data Management Software Based On Cloud Storage

With the rapid development of China's Beidou industry,the government proposed the construction of ground augmentation system.In the process,many ground stations are created,which are directly communicating with Beidou satellite,and the number of the real-time data flow every day is larger than past.Now a data management center is needed on the basis of the original data processing,which can process and store all the real-time data which is outputted by ground station receivers,and meanwhile can forward and store the system's internal product information.Based on the second phase project's requirements for data analysis and data calculation,the choice of storage needs to combine the distributed platform to do data storage.Based on the above requirements,the design of data management software based on cloud storage is proposed.First,the business of data center and ground augmentation system is analyzed.The system's storage selection combining with the system requirements is given.Thenthe system is divided into six function modules: Beidou receiver data processing,Rinex file cloud storage,Beidou receiver data broadcast forwarding,Rinex file release,Beidou product real-time processing and data center status monitoring after analyzing the requirements and a comprehensive analysis of the system business.In the process of the description,a full description of the function using the system activity map is made,and combining with some requirements in the actual use process,the basic requirements for non-functional requirements are put forward.Secondly,from the level of system a rchitecture design,the system is designed and implemented.The main architecture of the system is designed with multi-thread concurrent processing,message queue and Hadoop,and each part is called by interface to meet the mutual decoupling between the modules.First,when receiving data with multi-thread concurrent receiving and processing,the system meets the requirements of high concurrency and finish receiving and processing the real-time data flow.In the procedure of the data forwarding,the traditional UDP forwarding mode has been updated,a method of the message queue based on Rabbit MQ which use the "the producer-consumer" model for forwarding of data is put forward,which greatly improves the efficiency of data forwarding.In the aspect of data sto rage,a Hadoop platform based on several servers is designed to provide the system a cloud storage platform.Comparing directly to the hard disk storage mode,the cloud storage not only provides a more efficient and safe storage environment,but also well meet the demand for the second phase requirements of the data analysis platform construction.As for the data release part,the system uses a more mature and open source system Hadoop-Over-FTP which publish data services through the process of middle layer.Finally,a test is made for the software in the actual deployment environment and the functional requirements of the module are tested by the method of black box testing,which have verified the stability and reliability of software architecture in the a ctual deployment environment.Non-functional requirements are also tested combining with charts,and the final test results are recorded in the sheet.The test result shows that the system of research and implementation on the current framework can run stably in the actual environment,and can meet the requirements of 150 data base stations' data.The active use time of the system is up to 99.9% or more,which achieves the desired purpose.