Research On Secondary Control Strategy Of Islanded Microgrid Based On Reinforcement Learning

With the increasing importance of renewable energy in the power system,microgrid has become a popular way to integrate various small-scale renewable distributed energy and related loads.In the island mode,the primary control of the microgrid usually adopts droop control to realize the power sharing of each distributed power source.However,droop control is differential regulation,which will introduce frequency deviation in steady state.The existing secondary control mostly uses feedback linearization technology to linearize simplification the complex microgrid,which overcomes the defects of traditional centralized control,but ignores the nonlinear characteristics of the microgrid.With the rise of artificial intelligence technology,reinforcement learning,as an important method in the field of artificial intelligence,has natural advantages in solving nonlinear problems due to its excellent self-learning ability.This paper mainly focuses on the secondary frequency control of the island microgrid.In order to overcome the shortcomings of the traditional centralized control and further make up for the shortcomings of the existing secondary control methods,the reinforcement learning algorithm is introduced into the island microgrid,and the secondary control strategy of the island microgrid based on reinforcement learning is proposed.The main research contents and contributions of the paper are as follows:(1)This paper introduces the basic principle and framework of reinforcement learning and multi-agent reinforcement learning,analyzes the difference between single-agent reinforcement learning and multi-agent reinforcement learning from the perspective of Markov nature,and introduces the method of multi-agent reinforcement learning to solve the difference of Markov nature--"Markov game".According to the development order from single agent to multi-agent,the reinforcement learning algorithms involved in this paper are introduced in order,which provides theoretical support for the subsequent application of reinforcement learning in the secondary control of microgrid.(2)Aiming at the frequency deviation caused by droop control in island microgrid,a secondary frequency recovery control strategy based on Q learning is proposed.By analyzing the secondary control principle of the island microgrid,the frequency deviation is designed as the input state variable of the Q learning algorithm,and the active power compensation is the output action variable.The state-based reward function is designed for the control objective.Each distributed power source selects the optimal power compensation action through the Q table obtained from the algorithm training,and finally realizes the adaptive recovery of the frequency deviation.(3)Aiming at the problem that Q learning algorithm will affect the original power allocation of sag control in the secondary frequency recovery control,a multi-agent reinforcement learning based secondary frequency recovery control strategy for island microgrid is further proposed.The MADDPG algorithm is selected and its observation and learning conditions are adjusted to reduce the centralized training pressure of MADDPG algorithm and meet the requirements of distributed control.By analyzing the compensation actions of each distributed power source using the Q learning algorithm,the action reward function is added to constrain the compensation actions of each distributed power source,so that the action compensation value of each distributed power source is compensated according to the power distribution ratio each time.While recovering the frequency deviation,the original power distribution of droop control is not damaged.