China‘s space technology now rapidly improves at exponential rates with the paceof informatization development. Space information industry has also made leaps andbounds. Space data plays a critical role in the exploration of space-time, acting as bothanalytic and practical foundation. However, as the space data generates more and morefaster, storage strategy, query process and analysis of such massive data are bigchallenges.A Hadoop-based data management platform is presented in this paper, whichexploits the HDFS distributed file system and HBase distributed database to providecloud storage and computing services for aerospace test data. We design acomprehensive HBase+HDFS storage structure to accommodate to different datatypes and various user demands, and achieve consistent query and computing serviceson HBase and HDFS. An intermediate infrastructure is implemented to access HBasevia SQL, and multi-threaded approaches are responsible for high-speed data import,query and computing. Besides, we also involve a unified cache layer with an effectivecaching strategy for HBase and HDFS to improve the query speed.Experimental results verify the performance of our data management platform,and show that our platform is capable to meet the project requirements and provideefficient storage and cloud computing services. |