The Design And Implementation Of Key-List NoSQL Database

With the rapid development of mobile Internet,No SQL database has been widely recognized by the industry for its high performance,easy expansion,high availability,massive storage and rich types.It has been widely used in the field of Web2.0 application.Since 2009,many No SQL databases have been booming under the promotion of open source organization and large enterprises.At present,the No SQL database mainly consists of four types: key value database,column storage database,document database and graphic database,but it still fails to meet all application scenarios.There are large number of post list data in the Web2.0 community website,such as Baidu's post bar,Tianya forum,etc.These post list data are usually presented on the homepage of the website,they have the features of high concurrency,long list length,need to filter,and need to be dynamically sorted according to the final response time of the user.The traditional relational database and the current popular No SQL database can not solve these problems very effectively.So this thesis specifically designed and implemented a Key-List type No SQL database Mem List to store these list data.Mem List has the following features:(1)On the data storage scheme,this thesis discarded the disk based storage scheme of the traditional relational database,keep all data in memory and write data to disk by persistent operation when needed.This method can avoid common operation to execute disk I/O and improve database performance.(2)In the way of data organization,this thesis discarded the index organization table scheme of the traditional relational database,designed an unstructured storage engine for Mem List,which used the "key space-block chain" model.The key space takes the hash table as the underlying data structure,and each key is both an index and a data.List data is organized as a static block chain,block chains are mixed with the advantages of two-way linked lists and arrays.Block chains can save memory and improve query efficiency.In addition,this article provides a filter function for list elements on the block chain.(3)On the processing scheme of network connection and socket event,this article designed an event engine for Mem List using the Reactor design pattern.Mem List event engine manages events in a thread through I/O multiplexing,this way can greatly improve the concurrency of Mem List and avoid the lock overhead and the process / thread switching overhead.(4)In addition,this article also designs and implements the master-slave synchronization function for the Mem List.This method can be used to backup data on multiple databases and separate database reading and writing.In general,Mem List as a Key-List type No SQL database,supports multiple operations on list data.It has the characteristics of high performance,easy to expand and so on.Mem List is very useful to store the list of posts in the Web2.0 community web site,or to deal with the list data of other business scenarios.