Research And Application Of Data Query Optimization Technology In Large Data Environment

With the rapid development of mobile Internet, the use of mobile terminals is also a sharp increase, huge amounts of data is produced by a lot of the mobile terminal.the data is particularly large volume and has so many categories, which has a wide range of both unstructured and structured data. Moreover, the size of the data sets that composed by these data is beyond the typical database software and processing capacity of the tool.The subject relies on these platforms which composed by the Android App, WeChat public platform and Web services. The application will be popularizing nationwide in the future, there will be a large number of users. Due to the huge amount of future users, Android App will generate vast amounts of structured data, WeChat public platform will generate vast amounts of unstructured data. Through the analysis of these data for the query, to provide users with better service. It is the key to this research that how to store structured data and unstructured data, and enable the data to be effective query analysis.In terms of structured data, according to application characteristics and characteristic analysis on each database, select a MySQL database to store data. According to the large amount of data as well as the structure and characteristics of the data, technology of query optimization includes that the data is perpendicular to the level of segmentation and segmentation, and using the index technology and parallel processing technology. In addition, the characteristics of multiple-table query is slow which have enormous data, this paper proposed a new algorithm to optimize multiple-table queries.In terms of unstructured data, through comparing various features of NoSQL databases and analyzing the characteristics of the data, the paper is to choose MongoDB to store unstructured data. This paper first studied the partitioning strategies of MongoDB and the characteristics of data and the business, and chose the appropriate keys and partitioning strategies. Then the paper combined with the properties of a primary key to establish the right of the document’s structure by studying the general index and geographic spatial index and analyzing of the structure of the document. Finally the paging query is optimized.Through the study, design and implementation, this paper set up the appropriate data store structure and query optimization for mobile terminals which provide solutions for the storage and query of data on mobile terminals. Finally, the above work has been verified by experiment.