Data Replication In Scalable DBMS

With the rapid development of Internet,the data volume becomes is expanding sharply.The scalability of storage and computing will become increasingly important in database systems.Therefore,distributed database system with good scalability has attracted the attention from industry and academia.An architecture based on log-structure storage(Storage-Structured Storage),which separates read and write,has become a new trend and been applied to distributed database systems,such as Alibaba's open source relational database management system(RDBMS)OceanBase.Data export is one of the common technologies of data replication,which is used in enterprise applications to improve availability,scalability and reliability.In the distributed database system using read-write separation architecture,the data are divided into static data and dynamic data.Since the static data are stored on different physical nodes,data replication becomes an over-consumption operation.This paper mainly analyzes the existing problems of data replication under the distributed database architecture and puts forward an effective solution.The main contributions of this work are as follows:A static data export method for load balancing is designed and implemented.First,owing to directly accessing to the physical storage nodes,our method avoids the limitations of a single middleware merge node.Second,the produce-consumer model takes full advantage of the disk throughput and avoids excessive consumption of memory.Finally,a data export strategy based on multi-copy data is proposed,which further improves the performance of data export.A dynamic data capture method based on log analysis is designed and implemented.On the one hand,we ensure the correctness of log synchronization and log extraction.On the other hand,we leverage log compaction to reduce the log volume,which also reduces the cost of applying of writes.Through the benchmark YCSB,we conduct multiple experiments to verify our proposed methods.We implement our approaches in the open source database system CEDAR.The experimental results show that the data export methods have a good performance in terms of response time and can reduce the system resource consumption effectively.The testing results of data replication method proposed by this paper in CEDAR demonstrate that method greatly improves the efficiency of data export.In the same time,the method proposed by this paper not only equips with reference meaning for same type of scalable database management systems,but also provides the reference for the subsequent data replication technology of scalable database management system.