Research And Application Of Data Mining And Data Processing For Intelligent Power System

With the continuous improvement of the information level of power enterprises,more and more data is accumulated,which shows the trend of information explosion.The explosion of data volume occupies a large amount of computer resources,which causes the congestion of the system,which affects the further growth of useful information,and the management of data is becoming more and more complex.Moreover,the traditional database technology and online transaction processing technology to the statistical analysis of data are very difficult to obtain the desired results from a large amount of information,a simple analysis method has been unable to meet user demand for advanced analytics.Therefore,it is necessary to explore advanced data analysis and data processing methods and means to effectively solve the problems in production and ensure the safe and stable operation of the power system.In this paper,the parallel computing data mining platform is constructed.The data from the provincial data measurement automation system,marketing system and the weather data system is analyzed,in order to realize the advanced application of electric power data.The tasks to be completed are:1.Developed a parallel computing data mining platform based on parallel computing platform distributed file system HDFS and parallel computing framework Map/Reduce.Using the ETL(Extract-transform-load)tool to develop a series of "jobs" related to the business.These "jobs" are used to clean,extract,transform,and "protocol cluster" rules for the data of data source.and then to generatethe text file for the intermediate server(MS),the text file is upload to the computing platform based on cloud computing;The cloud computing platform quickly retrieves the massive amount of power data.According to the characteristics of the intelligent power consumption data,data mining platform provides a parallel clustering algorithm,the classification of the parallel algorithm,parallel real-time clustering algorithm and other algorithms of data mining module,which can effectively support the research of all kinds of advanced application.2.Considering the parallel data mining platform must pay attention to the interface security,data upload,data mining platform provide webservice interface,call for an external system or existing system,at the same time,considering the differences of each system programming languages(Java,asp,PHP,etc.),need to solve the problem of cross-platform and cross-language,this paper improved the data interface,improve the efficiency of data transmission.3.ptimal complexity model and data pretreatment of line-loss-index model are established,optimized data processing algorithmes are selected,false positives for missing data,abnormal data and alarm is omission of data preprocessing,improve the validity of the application of the results,the missing data,abnormal data and alarm error report data are preprocessed to improve the correctness of the application results.4.The paper proposed the optimal algorithm combination load forecasting method based on customer personalized grouping,and built the prediction model structure.Firstly,cluster algorithm is used to group the industry customers,and the users with similar characteristics of electricity will be gathered in the same group.Then,the load modeling and prediction of the different time series prediction algorithms are used to calculate the predicted value of the industry.5.The power failure analysis model based on fuzzy neural network(FNN)is built,the K means clustering technology is used toachieve reliability index prediction evaluation,dynamic tracking and control,automatic statistical analysis function.The application results show that the parallel computing data mining platform established in this paper can achieve the expected function,obtain the correct analysis result and have the better application effect.