The Research On Long Transaction Support For HBase

With the coming era of big data, traditional relational database has not kept up with explosive growth of data, and therefore a new storage scheme come out, which is Key-Value store. As the representative of Key-Value store, more and more attention has been paid to HBase. However, HBase can only provide weak transactional on the BASE level so that the process on long transaction can’t be precisely and orderly, and then it cannot guarantee complete data consistency under the complex business logic. Therefore, research on long transaction support for HBase is of great significance, both academically and economically.In this paper, before the design of long transaction support for HBase, the author study the transactional methods for the existing Key-Value store firstly, especially the long transaction support for HBase. By dividing the methods into two types based on distributed snapshot and centralized snapshot, the author select some typical methods as Percolator and Haeinsa from distributed snapshot and HBaseSI and Omid from centralized snapshot respectively, and then gain the advantages and disadvantages of two types by studying the basic idea, the recovery mechanisms from failures, the limitations and the typical application of four typical transactional methods.By comparing and analyzing the four methods on the basis of principle and performance, a kind of non-intrusive long transaction implementation approach for HBase based on pre-conflict checking has been designed. To avoid the limitations of previous methods, the approach takes priori conflict into consideration, and designs three algorithms as long transaction pre-submit, long transaction submit-process and long transaction snapshot obtain. The approach figures out the storage structure in need of these algorithms, and designs a complete transaction process on this basis. The approach designs a kind of recovery mechanism from servers, clients and the database and a kind of timeout mechanism based on the conflict detection according to characteristics of the approach. At last, from the perspective of the theory, show how the approach supports transaction on the ACID level.Finally, put the approach into practice, and main modules are briefly introduced. Then design experiments to verify the approach from the angle of data integrity and data consistency, and by comparison of various methods based on performance from five aspects, draw the conclusion that the approach is better than other methods when there are so many conflicts in the system. In this paper, there are16figures,7tables and51references.