Research And Implements On Optimization Of Big Data Query Performance

Nowadays,as data in dll fields are constantly increasing,the pressure of database is becoming higher and higher.From user's point of view,the database query efficiency play an important role in personal experience.From the perspective of the database itself,with the enlargement of data and network users,transaction query speed,transaction throughput and transaction concurrency query under the original architecture becoming more and more dependent on the scale of the hardware and cluster configuration.In the process of query,the importance of the cache is becoming more and more obvious,Therefore,the most important problem is how to bring the cache in to full play to improve the query speed,and how to expend database concurrency number or increase the throughput of database with much smaller cost.Therefore,this thesis focuses on database transaction concurrent query and hot point cache technology under the big data query.In recent years,there has been a lot of research on big data query,the research on database concurrency number is mostly focused on database connection pool,database distribution technology,and how to use cache to improve the query speed and further expend indirectly the concurrency number in per second.Concerning cache technology,most research is focused on its architecture,storage format,and elimination approaches.For issues of improving the database query speed through the use of cache contents,and improving database concurrency number through the good manager of query content,there are only limited research.To solve the problem of the database concurrency number limitation in big data query,and to improve the big data query speed through cache content.This thesis designs the SQL-Combines mechanism and the hot data prefetching mechanism.Between them,SQL-Combines mechanism enables different kinds of SQL query to merger in a highly efficient way,thus easing the competition of database resources among different SQL query approaches and improving the whole query speed.As for hot data prefetching mechanism,it is used to obtain future possible hot data,and put them in cache through a proper prefetching mechanism,in order to improve the future query speed.Simulating real access scenarios based on Pareto Law and establishing real platforms and structures in servers,we conduct various contrast experiments and confirm the effectiveness and superiority of SQL-Combines mechanism as well as the hot spot data prefetching mechanism.Finally,the thesis concludes with an assessment with the whole research and points out possible improvement.