Research And Implementation Of Distribute Data Base Based On LevelDB

Due to the rapid development of the Internet,the data scale is exploding.The storage and management of massive data has brought severe challenges to the database system,and the stand-alone database has been unable to cope with it.Traditional relational databases are not suitable in distributed environment because of their relational model and ACID characteristics.The NoSQL database came into being.It abandoned many of the limitations of relational databases,emphasized scalability and availability,and supported the storage of very large data.Based on the key-value NoSQL database LevelDB,this thesis designs and implements a distributed key-value database based on LevelDB by combining with data consistency algorithm Raft,data distribution algorithm and load balancing technology.The main work of this thesis includes:1.Modify the read strategy of Raft algorithm from only allow from Leader node to can read from Follower node,which can reduce the burden of the Leader and increase the throughput when the read load is much larger than the write load,reduce request average latency.Add the pre-election mechanism.The pre-election is initiated first before the official election is initiated,and the election is officially initiated only after receiving most of the replies.solve the problem of discarded log submissions that may occur when the Raft algorithm is in the network partition.2.Implement data distribution strategy based on keyword interval distribution,which provides friendly support for sequential reading and writing;implement load balancing function,dynamically adjust system load through partition splitting and partition migration;add middle layer to realize on single storage instance Multiple users provide logically independent storage.3.Design and implement the prototype system DLeve.The system has the basic functions like Put,Delete and Get,has good availability,scalability and reliability.Finally,the system is fully tested,including basic read/write function test,data synchronization function test,load balance test,read/write test and network performance test.The test results show that the DLevel function is in line with expectations,and the performance is similar to the same type of famous open source database TiKV and Pegasus.Compared with LevelDB,it also has a certain improvement,and has achieved relatively high performance.Tests on network performance show that the network module has good processing power in concurrent scenarios.