| With the rapid development of the Internet, a variety of data showed a blowout growth. This greatly enriches the user's available data channels,but also makes the management of massive data more complex, the need to design a new system to deal with data management challenges. In this paper, the mainstream HBase-based data management platform for in-depth research and analysis, found that the HBase-based data management platform for the complexity of the collection of data that is the use of native API programming data storage is difficult and complex programming, Lack of automation of processing data, low efficiency of retrieval data, and lack of data ease of use.Based on the above problems, this paper designs a new system architecture to solve the problem of data collection, data processing automation, retrieval of data and ease of use in other data management platforms based on HBase system. The main research contents are as follows: First, the design and implementation of HBase-based storage middleware, improve the data storage. The design and implementation of object-oriented way to store data and automatically build the index,improve data collection and storage efficiency. The Second, the design and implementation of the HBase SQL programming interface and visual interface. The design of the SQL parsing module and SQL-ES syntax conversion algorithm achieve a more flexible SQL syntax. The design of Web services that enable HBase to ability of cache improve the processing Data automation, retrieval of data efficiency and ease of use.This paper builds a HBase-based data management platform -theextendNoSQL platform and performs performance testing. And the efficiency of the original API is equivalent to simplify the programming difficulty, improve the efficiency of data retrieval and computing efficiency. In summary, the system can be used in the data management that data collection, storage, processing and application functions more efficient and easy to use. |
PDF Full Text Request