Study On Deep Reinforcement Learning Method And Its Application In Coordinated And Optimal Control Of Active Distribution Networks

With the high proportion of wind power,photovoltaic and other distributed power supply connected to the grid and the large-scale access of power electronic equipment,as well as the gradual deepening and wide application of ICT technologies such as high-speed communication and artificial intelligence in the power system,traditional distribution network is being transformed into active distribution network with multiple distributed power sources,more flexible network structure and more abundant control means.The structure of the active distribution network has changed from the previous single-point radial network to a complex network with multiple subsystems,the power flow has changed from one-way flow to two-way flow,and the degree of freedom of control has also increased greatly.Not only the output fluctuation of distributed power supply has strong uncertainty,but also the interaction between each subsystem is complex,which brings a great challenge to its coordination and optimization control.In recent years,the artificial intelligence technology represented by deep reinforcement learning has developed rapidly and has been widely used in various fields.It has obvious advantages in mass data feature extraction,complex mapping relationship learning and continuous decision making.Therefore,the active distribution network coordination control method based on deep reinforcement learning is expected to have a good application prospect.In this paper,the deep reinforcement learning method applicable to the coordination and optimization control of active distribution network and its application in the control of distribution network are taken as the research topic,so as to realize the coordinated control and efficient utilization of various controllable resources such as distributed power supply,reactive power compensation device and energy storage device,and ensure the safe and stable operation of distribution network.Since there are many researches of coordinated optimization control in active distribution network,this paper focuses on the most common and important problems of voltage optimization control,as well as loop current control in active distribution network,which also provides a reference for the research of other coordinated control problems.The main research work completed in this paper is as follows:(1)In order to solve the problem of coordinated and optimal control of various adjustable devices in active distribution network,a deep reinforcement learning framework is proposed for coordinated control of active distribution network.Firstly,this paper analyzes the models,characteristics and their respective regulating capacities of shunt capacitors,on-load voltage regulating transformers,distributed power supply and energy storage devices in active distribution networks.Secondly,this paper describes the principle of reinforcement learning and graph convolutional neural network,and integrates the perception ability of graph convolutional network to graph information and decision-making ability of reinforcement learning,and proposes a deep reinforcement learning framework for optimal control of active distribution network coordination.The framework can establish the mapping relationship between the state of the active distribution network and the actions of the adjustable devices,and can make decisions quickly according to the current state of the active distribution network,avoiding the dimension disaster caused by the scale increase and the problem of low computational efficiency caused by iterative optimization,which provides a theoretical basis for the following research.(2)In order to solve the problems of over-limit voltage and excessively high network loss in active distribution network caused by high proportion of distributed power grid connection,this paper proposes a multi-time scale active distribution network voltage optimization method based on deep reinforcement learning.Firstly,a multi-time scale voltage coordination optimization control model of active distribution network is established to solve the problems of different adjustment speeds of various devices,distributed power generation fluctuation and continuous optimization.The optimization problem is divided into two aspects:scheduling optimization in a long time scale and real-time optimization in a short time scale,and the deep reinforcement learning method is used to solve the two time-scale model building agents.This method can effectively coordinate the slow devices such as on-load voltage regulating transformer,shunt capacitor bank and the fast devices such as distributed power supply and energy storage.In addition,the load and distributed power supply do not need to be predicted in advance,so as to avoid the impact caused by the fluctuation of distributed power such as wind power and photovoltaic.Finally,the actual load and distributed power supply data of a region in Jilin,China after scaling are taken as the simulation data,and the improved IEEE33 distribution network example is taken as the simulation system.Through comparative analysis,the effectiveness and robustness of the proposed method are verified.(3)In order to ensure the safety of loop closing operation in active distribution network,a loop closing current regulation method based on deep reinforcement learning is proposed in this paper.Firstly,this paper describes the cause and calculation method of closing loop current,analyzes the influence of closing loop current on relay protection,obtains the conditions of allowing closing loop operation,and points out the necessity of closing loop current regulation.Based on this,this paper establishes the mathematical model and Markov decision model which take each adjustable device in active distribution network as control variables,and proposes the closed-loop current regulation method based on deep reinforcement learning.Finally,this paper takes a typical 10 k V closed-loop network model in a certain place as an example to carry out a simulation test,demonstrating the modeling and training process of deep reinforcement learning,and verifying the effectiveness and robustness of the proposed method.