Research On Virtual Partition Strategies Of A Shared Storage Distributed Database

In a shared-storage distributed database based on the architecture of computing and storage separation,the performance of the system is constrained by such elements as the serial performance,insufficient throughput,and insufficient utilization of storage nodes resource due to the peer-to-peer storage nodes.Improving the utilization ratio of storage nodes can severely affect the read and write performance of the database system.Virtual partition technology can be used to improve the read-write parallelism of the system and the utilization of storage node in the system.The focus of this research is to provide a suitable virtual partition management mechanism and partition load balance scheduling tactic under the shared storage architecture.A virtual partition system of shared-storage database is designed,which integrates virtual partition management,load balance and distributed consensus mechanism.The system mainly consists of 4 modules: compute-layer virtual partition management module,storage-layer write virtual partition management module,load balance module,and distributed consistency module.A set of virtual partition strategies of shared-storage distributed database is given to solve the problems of granularity control of virtual partition,partition maintenance,partition data consistency,and load balance.An adaptive virtual partition management tactic is designed to adjust the granularity of read-and-write partition according to the system running index adaptively,process the information update efficiently and safely when partition is adjusted,and realize the dynamic binding,positioning and cleaning of partition log.A multi-instance distributed consistency protocol of virtual partition is given to ensure the maximum uniform distribution of the partition leaders,improve the parallel degree of system writing,realize the parallel replication among the partitions,and solve the problem of temporary inconsistency of virtual partition data.A load balancing strategy of virtual partition is designed to make full use of the characteristics of virtual partition and load distribution according to the load of nodes.In addition,an asynchronous communication network is designed to reduce the complexity of multi-partition network connection and communication delay.The validity and functionality of virtual partitions are tested with TPC-H.The results show that the virtual partition strategies can improve the read and write performance of the system.When querying a large table,the more nodes there are,the better performance of the query will be.The improvement of write virtual partition performance is not obvious but stable.The load balance strategy can distribute the read-write load well.