Design And Research Of Power Quality Monitoring System Based On Hadoop Platform

With the development of science and technology,the quality of electrical energy which is an indispensable part of the modern society has become an important indicator of national technological.The amount of data collected by the power quality monitoring system is constantly increasing,because of the expansion of the power grid scale,increased sampling frequency of the acquisition device and the expansion of the monitoring range.However,these data play an important role in the analysis and management of the power grid.Traditional power quality monitoring system is a centralized computing platform based on the scheduling center,which cannot store and process the massive amount of power quality data.Therefore,this paper uses the Hadoop platform to improve the traditional power quality monitoring system,so as to solve the problems that the massive data can't be stored and the processing time is too long.First,the HDFS distributed file system,MapReduce parallel computing framework and HBase distributed database are introduced,and the Hadoop cluster platform is built.In order to solve the disadvantages of the traditional power quality monitoring system,the power quality monitoring system based on Hadoop platform is proposed,and this paper describes the specific architecture and advantages of the system.Then,this paper focuses on the analysis of related technologies for data storage and data processing.The default data distribution strategy and data connection algorithm are improved.In order to improve the data localization,save the network bandwidth resources,improve the efficiency of data query processing,and enhance the performance of the system,the multi-copy consistent hash algorithm and the parallel connection algorithm of HashMapJoin are used in this platform.In addition,the other technologies involved in the system are introduced briefly,as well as setting and optimizing other factors that affect system performance,including load balancing,data block size,and high availability of the cluster.Finally,experiments are designed to verify the feasibility of multi-replica consistency hash algorithm,the rapidity of data storage,the high efficiency of data parallel computing on the algorithm-optimized platform and the reliability of the cluster.The availability of the power quality monitoring system based on Hadoop platform is verified by processing the power quality data.