| NoSQL is the general term of non-relationship database technology to meet the growing application requirements of Internet and the network of things. NoSQL database technology is studied and summarized in the aspects of definition, data model, distributed scale and consistency. The internal service mechanism of MongoDB, one of the most popular NoSQL database, are analyzed. On the basis of that, this paper focuses on the strategies of shard deployment and the mechanism of choosing shard key and the solution of disk fragmentation. The anti-paradigm schema design theory and a query optimization algorithm based on semantic association and fine-grained model is proposed. And based on the MongoDB’s application in Web information Crawler System, the policy of storage and query design in non-relational database is discussed. The main research contents and achievements of this paper are as following:(1) Analysis of the storage mechanism and the anti-paradigm schema design in MongoDB. On the basis of the document-oriented data model in MongoDB, the anti-paradigm schema design method is proposed. This method is based on the theory of data dependency and uses the strategy of decomposition or joint data for the different dependencies to achieve the logic schema design of non-relational database.(2) Analysis of the distributed extension mechanism and application optimization in MongoDB. The partitioning algorithm and the load balancing of MongoDB’s auto-sharding are analyzed. The deployment strategy about the shard cluster in the practice is given. On the basis of the shard key in auto-sharding, the mechanism of choosing shard key about partition granularity, write extension and query isolation is given. The causes and the solutions about disk fragmentation are analyzed.(3) Analysis and improvement of query mechanism in MongoDB. The query policy based on the simple pattern and parallel on multi-plan is analyzed. A query improved algorithm based on semantic association and fine-grained model is proposed and improves the efficiency of query processing.(4) Application of Web information crawler system. The requirements of the system database are analyzed and the system solutions of MongoDB and SQL Server are given; the query efficiency of MongoDB’s anti-paradigm data model and SQL Server’s paradigm model is tested. Due to the nest without joins, the speed of MongoDB is faster than SQL Server; the different index solutions for common query are tested, and the high efficiency of composite index in this application is verified. |
PDF Full Text Request