Research And Implementation Of Massive Data Storage System Based On Hybrid Architecture

The development of information technology promote the sustained growth of the data center market size. Energy conservation has become the focal point which now the data center operators pay attention. Data center energy monitoring system is mainly responsible for real-time monitoring and analyzing dynamic environment and IT equipments operation status. Due to massive log data generated by the data center, massive data storage system has become the core issues of data center energy monitoring system. Data center energy monitoring system not only need to provide powerful data analysis, but also requires energy monitoring system data storage systems with high availability and high scalability. Relational database management system not only supports sequential scanning data but also supports random read and write and powerful index search engine. Be different from traditional relational databases, HBase is a kind of column-oriented distributed database, the main advantage of HBase is powerful parallel computing capabilities and high scalability.Based on in-depth research and analysis of relational database management system, NoSQL databases and Hadoop, this paper designed and implemented a massive data storage system based on hybrid architecture. In the storage system, Hadoop is responsible for storing and processing massive data, HBase and MySQL is used for structured data storage.In this paper, an experimental platform based on Hadoop was established, and the performance of data storage and access are tested and analysed. The experiment results show that this storage system with high scalability, storage and query efficiency can solve the messive data storage problem.