Data Mining Application Researchon On Smart Grid Data Analysis Based On Spark

As the scale of smart grid system is expanding and the power data within the system is becoming more and more complicated,how to mine useful information from these complex and diverse power data is particularly important.So far,fault prediction and power consumption prediction have been the hot research directions in power data applications.Fault prediction can help power system staff to understand the operation and fault conditions of power equipment in advance,and can take precautionary measures before the fault occurs to reduce the economic loss brought by the failure,the association rule algorithm is one of the most commonly used and effective methods for fault prediction,which has great research value.Electricity consumption forecast is based on the historical electricity consumption information to forecast the electricity consumption in the future.Forecasting the consumption of user’s electricity has a certain guiding significance for the power grid regulation and transport,which is the basis for ensuring the reliable and economical operation of power system.With the continuous development of artificial neural networks,more and more methods for predicting the amount of electricity consumption based on neural networks are proposed.Among them,LSTM is a kind of recurrent neural network,which has a good effect.Aiming at the above background,this paper proposes an Eclat association rule algorithm based on RoaringBitMap structure,which can achieves data compression and storage through RoaringBitMap structure,reduce the memory used by the algorithm and improve the running efficiency of the algorithm.In order to solve the problem of information loss encountered by association rules algorithm in dealing with multi-dimensional time series data,a multidimensional Time-series Association Rules(MTAR)algorithm is proposed.The algorithm is realized based on Spark and the fault analysis and forecasting based on the grid transmission line dataset is completed.Based on the DL4J deep learning framework,the LSTM algorithm based on Spark parallel computing framework is realized,the segmentation operation of time-series data and the distributed training of neural network are completed.The LSTM algorithm is used to predict the value of power consumption.Finally,through the research and development of the BDAP platform,the above two application processes are integrated on the platform.