Research And Implementation Of HBase Memory Index System

With the rapid development of computer technology and database technology, the human need to store the amount of data that a great deal of growth, the concept of traditional storage and processing data is applicable is not measured data environment.In order to meet the needs of the current data storage and processing, the data system is based on distributed system.HBase is one of the most popular open source software distributed database.HBase purpose of design is stable to store vast amounts of data, HBase performance in other ways is not our expectations.HBase reads the data line(rowkey) have great dependence, this is particularly restricted HBase under the condition of complex query performance.In the use of traditional database indexing technology can greatly improve the query efficiency of database, in order to guide this paper tries to establish a HBase non rowkey column index to improve HBase under the condition of complex query performance.In this article the index tree Luo Jun adopted in chongqing university professor and learn Chinese zodiac fuping HT tree index is put forward, in order to optimize the index tree of space utilization of HT tree insertion and deletion algorithm optimization.In this article, I will the index tree is stored in the Spark distributed memory computing system, the Spark is a high efficiency calculation software of distributed memory to store the index tree in Spark can quickly improve the processing efficiency of the index.During the implementation of the index system used the secondary indexes design architecture, every query in this design is composed of two parts.Q uery when we can get to the first in the index system rowkey, get to read the data from HBase after rowkey. The thought of the segmented query while in the query conditions have rowkey performance was slightly lower than HBase, but greatly improve the query conditions when there are no rowkey HBase query efficiency, makes the HBase under the condition of complex queries the applicability of the improved.This article also implements a relatively compact index system.Index system by the index middleware and application program interface(API) of two parts.Index middleware is the core of the system, realized the data into the establishment of the index tree, index tree of data query is a query, modify data delete index tree, but also realized the index tree, HBase, Spark and data interaction between the application program interface.Main application program interface with Java language access interface and two kinds of Web Service access interface.