Research And Application Of Data Analysis And Data Mining Based On Electric Power Big Data Platform

After more than 20 years of development, data mining has already formed a lot of mature theory, the application also has penetrated into various fields. In recent years, with the rapid development of computer technology and network technology, the amount of data that people are facing exponent ial growth, the tradit ional data mining methods and technology w ill face great difficult ies, how to dig out valuable knowledge from large amount of raw data has become a difficult proble m. In electric power industry, with the in-depth application of power business system and meters popularizat ion and promotion of smart meter, the data of electric power operation data, testing, simulation and so on are exponential growth, associated with mult iple and complex. The combination of data mining technology and big data technology has become a new research direction.In this paper, we analyze the advantages and disadvantages of the apriori algorithm in data mining algorithm, the paper proposed an improved algor ithm based on iterative matrix against the defect of apriori, IM_Apriori a lgorithm use Boolean matrix storage the data sets, use k- itemsets matrix and k-candidate matrix to replace the original set of Boolean matrix to reduce the number of calculat ions, and realize the parallel of the IM_Apriori algorit hm, the IM_Apriori algorithm is imple mented in spark with scala programming language. And analyzed the efficiency of the improved algorithm in theory.Then build a large power data platform, the platform is positioned as data sharing, data analys is, data application development platform, provide services to peopel from data collection, storage, pretreatment, calculation, analys is, visua lizat ion, and other aspects of data analys is. This paper analyzed the construction demand of electric power data pla tform, detailedly designs electric power data platform architecture, expounds the concrete realizat ion of big data platform from the overall architecture, functional architecture, technical architectu- re and other aspects, combined with the concrete technology, detailedly introduces imple mentation procedure of data acquisit ion, preprocessing, data storage, data processing and data display, providing reliable analys is and mining platform for data mining and application in electric power industry.Experiments are respectively carried out in a single envir onment and cluster environment. The results show that the IM_Apriori algorit hm is superior to other algorithms in the execution efficiency.Finally, based on the large data platform, the IM_Apriori algorith m is applied to the electric ity consumpt ion analys is of residents by using the electricit y consumpt ion data of residents. The improved algorit hm is applied to the concrete application.