Design And Implementation Of Query Optimization Module In MongoDB Control System

In recent years,with the rapid development of No SQL database and large-scale practical application,more and more enterprises and research institutions,select No SQL database to build internal high-performance systems to cope with the growing internal business requirement and the challenge of storing and searching in big data.Mongo DB is one of the excellent No SQL database and has been used by thousands of enterprises and research institutions in a first-line production environment.Compared with traditional relational database,Mongo DB is not based on relational algebra.Query statements or index types are different from the relational database.The preparation of query statements,the quality of the index design and other factors are very important to affect the performance of Mongo DB query.Therefore,how to optimize the query of Mongo DB according to the characteristics of Mongo DB query and index,has become a problem that needs to be studied.Based on the above situation,this paper designs and realizes the system of optimizing the query of Mongo DB by using process decomposition,influencing factor analysis,grammar checking,index evaluation and index recommendation.(1)This paper first carries out field research on the project background and user requirements,focusing on the current optimization process and management system business background.On this basis,the process of Mongo DB query processing is decomposed.The stages of decomposition are analyzed from three perspectives: bottleneck resource,influencing factors and optimization direction.Combined with reduced data access,return less data and other optimization principles,this paper finally establishes the query predicate check,index evaluation,index recommendation,result set quantitative analysis as the optimization direction of the control system.(2)Log acquisition,judgment and preliminary analysis.Firstly,we read the slow query log of the Mongo DB instance.For each log,we extract the unique identity in the log,calculate the corresponding MD5 value,and use the cache to check for duplicates.After that,we keep the original log without duplication into the database and parse the original log by extracting the key attributes and normalizing the query conditions in the log.Finally,we save the preliminary analysis results to the database.(3)Based on the results of the preliminary analysis,we optimize the query from three aspects: query predicate checking,index optimization and result set quantity analysis.In the aspect of query predicate checking,we carry on the examination of the syntax of the relevant query predicates,which is compatible with the common query conditions.In the aspect of index evaluation,we calculate the index selection rate to evaluate the index filtering ability,and analyze the influence of the index size with the instance memory size.Based on the principle of the influence weight,we evaluate the leftmost prefix matching and index coverage.In the aspect of index recommendation,we recommend index on the basis of the principle of equivalent sorting range In the quantitative analysis of the result set,with calculating the proportion of the number of returned documents to the total number of documents in the collection and combining with the examination of the limit line predicate in the grammar level,we judge the effect of the number of return of the result set.Finally,we save the optimization results to the database.At the end of this paper,the functional and non-functional requirements of the system are tested.The experimental results show that the system can meet the business requirements of the control system.We optimize the Mongo DB query from the angle of management and control platform,which can be used as a reference for the implementation of the relevant operation and maintenance platform query optimization system.