Evaluation And Promotion Methods For The Value Density Of Electric Power Big Data

The information construction of smart grid has accumulated a large amount of data resources,and how to improve the value density of electric power big data is the focus of current research.Study on the value density of big data lacks of quantitative evaluation index and promotion methods are limited,resulting in limited effect.In view of these problems,focusing on “value density” topic,the definition of value density,evaluation indicators,upgrading methods,and the verification of experimental simulation are studied in this paper.This paper proposes the definition and evaluation index of the value density.And the value density evaluation indicators are calculated from the two dimensions of spatial memory occupancy and time running rate,and also take into account the error of data mining results.Starting from different dimensions,this paper summarizes the technical route of increasing value density into a "three-layer filtering mechanism"--"dirty data" filtering for databases,record "horizontal" filtering based on improved K-means algorithm and field "vertical" filtering based on FP-network model.In-depth research and elaboration are carried out for each layer of filtration.For the first layer of filtering,the common "dirty data" types,causes and corresponding processing methods are also summarized.For the second layer of filtering,the cluster-oriented algorithm is used to achieve record-oriented "horizontal" filtering,which is based on multi-initial clustering center and multi-unit parallel processing.For the third layer of filtering,this paper proposes FP-network algorithm.This not only inherits the advantages of FP-growth algorithm,but also only needs scanning the database once.It also facilitates data maintenance and data updates.And the field-oriented "vertical" filtering is implemented by the association analysis method.Based on the big data platform of a provincial power company,this paper analyzes the daily load forecast of a real distribution network as an example to verify the effectiveness of the value density evaluation index and the performance advantage of the improved algorithm.The results show that the defined indicators can reflect the value density better.The proposed “three-layer filtering mechanism” can effectively improve the data value density,especially the second layer clustering algorithm and the third layer correlation analysis.