Research On The Implement Way Of Multi-condition Complex Query In HBase

With the coming of the Internet+ era,Internet and datamation have led to an explosion of data.These large-scale and heterogeneous data enormous huge potential and great values,processing it with a efficient way have a great significance.Therefore NoSQL arises at the historic moment.As a representative of NoSQL,HBase is widely used.HBase have used the key-value infrastructure,only supports efficient queries on RowKey,multi condition complex query is not supported.The storage structure is unstructured,if multi condition query is not supported,locate and protect sensitive and critical data will be impossible.So the implementation of multi condition complex query on HBase is important for data processing and information security.This paper is devoted to achieving multi condition complex queries on HBase.The work of the dissertation is partly supported by the National Natural Science Foundation of China(No.61172072,61271308),Beijing Natural Science Foundation(No.4112045),and Research Fund for the Doctoral Program of Higher Education of China(No.20100009110002).The main work of this parper is as follows:Firstly,the technical basis and related characteristics of HBase have been studied.Then this parper studies the existing HBase complex query implementation methods.Typical methods hava been studied,principle,advantages,disadvantages and application scenarios were analysed.Based on the analysis abvoe,a linear index method based on Hilbert space filling curve is proposed to realize HBase's multi condition complex query.Firstly,a RowKey generation model for mapping multiple conditions to primary keys is proposed,and then the overall architecture of the method is designed.After that the query strategy and filter design were proposed,the partition mechanism have been redesign.finally the implement of the method was presented.In order to improve the performance of the method,an optimization method which based on Hilbert index and client cache is proposed in this paper.This paper describes the structure,implementation of Hilbert index and the design and implementation of client caching method from cache structure and hotspot partition and replacement strategy.In order to test the performance of method,this paper test the performance of query,insert,redundancy and load balancing and compare with other ways.Then this paper test the performance of optimization method.The results show that the method can efficiently implement HBase multi-condition complex queries,and it has better performance than other methods.For the optimization method,the performance of insert and query are further improved,especially the repetitive insertion and hot data query,the performance is significantly improved.