A High Performance Distributed Index Designed ForRead/Write Isolated& Atch-Update Storage

The distributed Data Base Management System(DBMS) based on Log-Structured Merge-Tree (LSM-tree), use read/write isolated and batch-update storage architecture to manage data. It can effectively solve the problems of high-workload, centralized write and has the capability of the scalable storage of massive data. But such DBMS of this archi-tecture cannot meet the technical requirements for data access, such as high-performance non-primary key query.Index is one of the traditional technologies adopted to improve the performance of the non-primary key query. At present, for the DBMS of read/write isolated architecture where massive baseline data are stored in the disks while incremental data are entire-ly maintained in the memory and updated dynamically, constructing secondary index is rather difficult. The challenges for us are how to efficiently construct index based on the two parts, satisfy the demands for dynamic update and guarantee the consistency of index and data. This paper mainly proposes the index algorithm for read/write isolated architecture which is adapted to load balancing and scalability of the system. The main contributions of this paper are:1. Propose methods to support secondary index construction for massive data under the read/write isolated architecture. The structure of index with the underlying s-torage of distributed system is designed. It guarantees the correctness of index with dynamic update. Then, delay-effective mechanism is presented to avoid block-ing transaction’s process of constructing index, in this way, DBMS’s availability is improved. And two-phase sort and multi-thread parallel dispatch algorithms are presented to speed up index construction of the baseline data.2. Provide efficient index maintenance and query optimization strategies under the read/write isolated architecture. The algorithms of maintenance and query are de-signed. Index is maintained by the means of removing recovery log. And index query is optimized, by using redundant columns to avoid additional overhead of querying original table.3. Demonstrate the validity and the correctness of the index algorithms by implement-ing our methods on Oceanbase which is the typical database using read/write iso-lated architecture. The adequate experiments are carried out. In addition, this index technique has been executed in the one of domestic bank’s UAT environment. By two evaluations, validity and correctness of the algorithms are both proved.