Deep Transfer Reinforcement Learning-integrated Active And Reactive Power Dispatch Method For Power Grids

Unified dispatch and unified decision-making are the fundamental requirements for safe and reliable operation of a strongly coupled and integrated power grid,and multi-level coordinated dispatch is another institutional advantage that China has developed over the years to ensure safe operation of power facilities.In order to solve the contradiction between comprehensive decision-making of the whole network and decentralized control of dispatching at all levels,State Grid has designed different analysis directions such as "multi-objective economic operation domain","multi-dimensional real-time evaluation and autonomous optimization technology" and"zonal grid source-load cooperative optimization control technology",among which the active and reactive power coordination control methods are "Therefore,the research of active-reactive power coordination control method is a key part to realize the intelligent dispatch and safe operation of large power grids.Usually,two main solutions are used for such problems,namely classical mathematical calculation methods and artificial intelligence heuristic algorithms.However,these artificial intelligence algorithms do not have the ability of knowledge migration,and each optimization task is performed independently,and once a brand new task is executed,it has to be re-initialized,which cannot efficiently utilize the past optimization knowledge,resulting in a long optimization search time of the algorithm,which is difficult to meet the demand of fast optimization decision of large-scale power systems.In this paper,we propose a basic idea of a class of migration-based reinforcement learning optimization algorithms for solving the integrated active-reactive power system scheduling problem.The method stores the optimal knowledge networks of historical source tasks and uses them to form the optimal knowledge networks of new optimization tasks,thus achieving the purpose of speeding up the speed of finding the optimal under new tasks.For the power system active-reactive coordination optimization problem,a hierarchical decentralized migration reinforcement learning framework is constructed in this paper.The upper layer realizes the collaboration among different intelligences through the global master intelligence,and each intelligence in the lower layer uses the migration reinforcement learning to quickly obtain the optimal solution in the local system.In general,this paper will demonstrate step-by-step the application of migrated reinforcement learning to integrated active-reactive scheduling as follows.(1)This paper first investigates how reinforcement learning single intelligences learn the knowledge of integrated scheduling decisions,and incorporates the advantages of migration learning to enable the intelligences to efficiently acquire new knowledge in an unknown new environment and improve the quality of possible solutions in a short time.Saving the optimal network of historical tasks as a guide for optimization testing of new tasks can effectively reduce the blind random search and provide the quality of initial values,thus improving the speed of finding optimal solutions,and finally the performance is verified using the classical active-reactive coordination optimization problem for power systems.(2)This paper secondly investigates how multi-intelligence optimization algorithms can be applied to multi-task nonlinear migration reinforcement learning,and proposes a framework for hierarchical optimization led by a master intelligence.The bottom layer is divided and collaborated according to different tasks,and the upper layer master intelligence comes to perform the optimal combination of actions.Coherence theory is used to achieve cooperation and collaborative learning among the intelligences,a source task library of multiple historical tasks is established,and finally the performance of the decentralized active-reactive coordinated optimization method is verified by adding a scenery storage system when designing the algorithm.