Massive Data Query Optimization And Their Application In Power Inspection System

With the rapid development of internet technology, the data in web applications is larger than ever. How to access and process the massive data has became an important issue in information technology. There have been long standing problem that it is very difficult for mobile company to detect device errors and aging problems in base stations. Due to lack of effective management of those base stations, much electrical power has been wasted and even theft, which may even seriously affect the quality of communication services and the expenses of the mobile company. Even though detailed electrical consumption information could be provided by the power and environment centralized supervision system, it seems quite burdened to make statistics and analysis due to the large volume data produced every day. Thus, those unordered data could provide quite limited support to operation decisions on power management.This paper presents a cache replacement algorithm LFU-W which based on the weight factor. The improved replacement algorithm can enhance the hit rate. Thus we combine with the index optimization techniques to enhance the mass data query efficiency. In order to tackle problems and difficulties in power management, we pay much attention to the implementation and optimization of power inspection system. Main contributions are as following:(1) We have researched the forgetting curve of Ebbinghaus, and apply it to the calculating of weight factor based on time series. Then propose a cache replacement algorithm LFU-W based on weight factor. According to power data query features, combining index and cache technology could we obtain much more fast response in the system.(2) By data from air conditioners, switch power collected and theoretical consumption data, we came up with the single term comparison model of power consumption and by which, we could detect and make decisions about whether there exist some errors or aging problems of devices.(3) With the help of collected power data, theoretical data, remote data and practical data, we could formulate a total comparison model of power consumption. Moreover, with this model could we examine whether or not there exist inaccurate power data or even power theft.The optimization method based on index and cache technology can greatly enhance the efficiency of massive data queries. The comparison model proposed in this paper could effectively and accurately analysis errors, abnormal situations and even power theft in the system. Practical run-time data has proved its superiority in power management and enhancing the economical interests.