Research On Application Of Elasticsearch In Grid Dispatching Data Management

In the process of participating in the research and development,operation and maintenance of a power grid dispatching automation system in a city,I found the following problems:First,with the rapid growth of dispatching data and the rapid expansion of storage volume,the existing database is difficult to meet the demand of data storage.Second,With the increase of dispatching data,the data query becomes slower and slower.Third,the existing dispatching automation system does not realize log visual management.With the development of smart grid,real-time data collected by dispatching automation system,such as dispatching operation and online monitoring of power transmission and transformation equipment,and log data such as system operation and operation records will become more and more intensive,forming dispatching big data with fast sampling,large volume and various types.The relational database commonly used in the current dispatching automation system is built on the hardware background of low core,small memory and large hard disk.Before the explosive growth of scheduling big data,there are bottlenecks such as low read and write rate,poor scalability,insufficient concurrent ability and difficulty in organizing and managing log data.The dispatching automation system cannot provide the dispatching data management service of stable and reliable storage,convenient and efficient reading and log visual.Aiming at the above problems of power grid dispatching data management,this paper proposes a scheme of reliable storage,fast query and log data visualization of power grid dispatching data based on Elasticsearch distributed search engine,and applies the advantages of Elasticsearch in fast data retrieval and log visualization management to power grid dispatching data management.The main research work of this paper is as follows:First,in order to solve the storage problem of power grid dispatching data,a method to store dispatching data is proposed based on cloud computing Hadoop ecosystem and non-relational database HBase instead of the relational database currently used for power grid dispatching.The test results show that the distributed database HBase has high reliability and good performance of concurrent read and write,and can meet the data storage requirements of the dispatching automation system.Second,in order to solve the problem of slow query of power grid dispatching data,a method of establishing a secondary index through Elasticsearch inversion index on the basis of the first-level index of HBase database is proposed,and a secondary index structure of power grid dispatching monitoring data is designed and implemented to replace the query of current relational database.The test results show that the query time of Elasticsearch-based secondary index structure is much shorter than that of existing relational database,and it can meet the demand of high-speed and concurrent data query of dispatching automation system.Third,in order to solve the problem of not realizing log visualization management in power grid dispatching automation system,a method of Elasticsearch based on Elk technology stack is proposed to design and implement log visualization management in dispatching automation system,which is helpful to grasp the running state of dispatching automation system.Fourth,based on the above solutions,the software of power grid dispatching data management system is developed.Combined with the demand of power grid dispatching data management,the software uses Microsoft.NET framework,based on RESTful API to achieve back-end processing,based on WCF to provide data services,based on B/S mode for front-end interaction,design and implement power grid scheduling data management.The operation effect of power grid dispatching data management system in management of power grid dispatching data in a city shows that the system can meet the needs of stable and reliable storage of massive dispatching data,efficient concurrent reading and log visual management,and contribute to the development of intelligent and lean dispatching automation system in the future.