Voltage Control For Distribution System With High Penetration Of Distributed Photovoltaics Based On Deep Reinforcement Learning

Recently,the increasing penetration rate of distributed photovoltaics has brought a profound impact on the voltage control of the distribution network.Meanwhile,the continuous development of dynamic reactive power compensation technology,the gradual maturity of energy storage technology,and the increasingly controllable distributed energy sources have brought more and more controllable resources to the voltage control of the distribution network,and also pose challenges to the existing voltage control methods.The traditional voltage control schemes require the establishment of an accurate physical model,and some approximation and simplification techniques are used to optimize the parameters.When the scale of the distribution network becomes larger,it will become more difficult to establish an accurate physical model.And with a large number of distributed photovoltaics being continuously connected,the complexity,randomness and dynamic performance of the entire network will continue to increase.Under this circumstance,continuing to use the traditional methods may result in reduced control accuracy,timeconsuming or even impossible to solve.Deep reinforcement learning methods are model free,which can provide optimal solutions within milliseconds,and have great advantages in solving complex multivariate problems.Therefore,this paper focuses on researching the voltage control methods for distribution system with high penetration of distributed photovoltaics based on deep reinforcement learning.The main contents are as follows:(1)The two-time-scale voltage control method for distribution network based on deep reinforcement learning is studied to realize centralized control of different voltage control devices.To handle voltage control devices with different response time,a two-time-scale voltage control model is established,and further modeled as a Markov decision process.The control variables at different time scales are assigned to different agents,and a deep reinforcement learning control algorithm based on deep Q network and deep deterministic policy gradient is proposed to solve the problem in order to improve computational efficiency and realize real-time voltage control.(2)The distributed voltage control method for distribution network based on multi-agent deep reinforcement learning is studied.The control variables of different types of voltage control devices are assigned to different agents,and then the optimal voltage control problem is formulated as a Markov game process.A voltage control method based on multi-agent deep reinforcement learning is proposed,which is based on multi-agent deep deterministic policy gradient algorithm,and uses Gumbel-Softmax distribution to solve discrete actions,and it can simultaneously deal with discrete and continuous action spaces.By means of centralized learning and decentralized execution,this method can adaptively obtain the optimal coordinated control strategies of various voltage controllers.(3)The voltage control method for multi-feeder distribution network considering main network voltage fluctuation based on robust deep reinforcement learning is studied.The entire multi-feeder distribution network is divided into a main agent and several sub-agents,and a multi-agent distributed voltage control model considering the voltage fluctuation of the main network is established.Based on the information uploaded by the sub-agents,the main agent models the uncertainty of the main network voltage fluctuation as a disturbance to the state,and employs a robust deep reinforcement learning method to solve the problem in order to determine the tap position of OLTC.Furtherly,based on the power flow equation of each feeder,each sub-agent uses the second-order cone relaxation technique to transform the optimal voltage control problem into a convex second-order cone programming problem to adjust the reactive power outputs of the inverters on their respective feeders.