Research On OLAP View Materialization Method Based On Reinforcement Learning

Traditional database and data management technologies are facing more and more challenges in the era of big data.The environment for the big data is becoming more and more complex,while traditional databases are static systems,which means that traditional databases cannot dynamically optimize their performance based on the performance of historical queries,nor can they specifically optimize their performance based on the size of a user's data and the characteristics of the query load.OLAP database has always been the mainstream of data processing in the field of big data,as the amount of data that the database needs to process continues to increase,the database system is required to process faster.The query load of large data applications is changing rapidly,which requires the database system to dynamically adjust the system to get the best running state.During the decades of artificial intelligence development,machine learning has successfully supported a series of applications involving image,voice and text data,and most traditional databases lack data learning capabilities.Recent years have seen a surge in approaches that explore artificial intelligence to power traditional databases,making databases more adaptive and intelligent.Specifically,they can be automatically optimized according to historical metric statistics and current query workload,which significantly improves the database performance and relieves the trivial routine maintenance suffering.This paper focuses on the benefits of materialized views in OLAP databases.Based on the study of traditional view materialization,this paper finds that static methods may degrade performance when data distribution and query load change.Therefore,considering the dynamic nature of view materialization,the following dynamic view materialization methods based on reinforcement learning are designed in the dynamic environment,and this paper introduces the calculation method of view revenue and the specific process of data feature processing.By collecting user queries and system performance logs,learning and predicting user query patterns and behaviors using machine learning,automatically pre-calculating or materializing the data required for large data query and analysis,the reinforcement learning model is used to interact the view with the query load,and the views that can improve the overall query efficiency are selected for materialization.In this way,the selection of intelligent materialized views is achieved.After obtaining the view that can improve the query efficiency,the materialized view is compressed by the method of quotient cube equivalent compression data,which can further improve the overall performance of the database while compressing the materialized view data.In the last,through experiments,it is verified that the view materialization method based on reinforcement learning can save more costs.The physical space occupation of the compressed materialized view is reduced.At the same time,it is verified that the dynamic view materialization method based on reinforcement learning designed in this paper is superior to other methods based on comparison,and can better achieve view adaptive and improve overall performance.Finally,the experiment preliminarily proves that the more accurate the cost model,the better the solution of the view selection model.