Research On Unified Storage And Access Interface For NoSQL Database

With the emergence of Big Data, new challenges has been brought to the field of information technology. Because of the huge amount of data, a fast increase, various data types, relational databases are hard to meet the needs of large data stores. Comparatively, No SQL databases are rather suitable for large data storage because they have a better reading and writing performance, good scalability, and more flexible data storage. In application platform for building a large data storage and analysis system, coexistence of multiple database schema can give full play to the advantages of the storage system, but the data access mode varies in different database system data model, making it very complicated to build big data storage system. In addition, there are performance problems in using No SQL databases. At the request of high density data transfer, blocking communication patterns can cause database performance degradation. Therefore, following tasks has been carried out in this thesis to solve the above problems:First of all, heterogeneous storage systems at home and abroad are analyzed and compared, and the current situation and problems of this technology are described. Four major data models of the No SQL databases are analyzed, a No SQL database storage and access model is come up with based on the idea of the present data integrated, thus achieving the goal of data access based on RESTful uniform interface. This model unifies the data storage, makes RESTful interface realized through temporary storage system caching data.Secondly, focusing on the performance of the No SQL database, this thesis studies the HBase database, analyzes the communication mechanisms between HBase RPC client and its server. HBase RPC client adopts blocking communication patterns, which will cause the thread to block if the client requests for data are intensive, thus affecting the communication efficiency between subsystems. Therefore, an HBase RPC non-blocking client is designed and implemented through Java NIO.Finally, the system prototype of the uniform storage and interface based on the HBase database is completed. This prototype uses My SQL to store its temporary data source, and Map Reduce to implement its data conversion mechanisms between underlying storage system and temporary data, making the uniform interface realized based on RESTful by JAX-RS 2.0 standard. The feasibility and system performance is testified by deploying and testing the prototype. Experiments show that the uniform interface simplifies manipulation of data resources. Moreover, the non-blocking pattern in HBase RPC client is more efficient than the blocking one under high network loads.