Application Of Reinforcement Learning In Smart Grid

In order to reduce the dependence on primary energy,the power system must access new energy on a large scale to promote the clean low-carbon transformation of the grid.Different from traditional thermal power generation,renewable energy such as wind power and photovoltaic generation is random and intermittent,and grid-connected inverters composed of power electronic devices have small inertia and damping.Without intervention,large-scale renewable energy connections are bound to cause fluctuations in the frequency and voltage of the grid.First of all a large amount of renewable energy connected to the grid may cause power and frequency oscillation that threaten the stable operation of the grid.An optimal control scheme based on reinforcement learning and adaptive dynamic programming(ADP)is designed for three-phase grid connected inverter controlled by virtual synchronous generator(VSG).The dynamic compensation term is introduced into the active power loop,the VSG output is optimally adjusted through the designed adaptive optimal control strategy to suppress the power angle deviation caused by grid connection.When considering the unknown system dynamics,the reinforcement learning method of depth deterministic strategy gradient(DDPG)is used to replace the ADP algorithm.Then.aiming at the torque control problem.A method based on deep deterministic strategy gradient and the adaptive iterative learning control method is applied to the torque control of permanent magnet synchronous generator.The method can reduce the torque ripple of permanent magnet synchronous generator under stable and transient conditions.The proposed design can minimize torque ripple and improve the control performance of torque regulation during transient events.In addition,considering that the rotor position is difficult to be accurately estimated under the running state of the motor,the deep reinforcement learning strategy combined with the phase-locked loop method can effectively compensate the rotor position error.The experimental results verify that the designed controller can suppress torque ripple and compensate position error under steady and dynamic conditions.Finally.based on the model of the wind storage frequency-modulated power grid,the frequency support control link is designed.The adaptive dynamic programming method is applied to the frequency response optimal control of wind-storage grid-connected systems to suppress the frequency deviation of the power grid.and ensure the stability of the wind storage grid-connected system.Matlab/Simulink was used to build the simulation environment of the grid-connected wind storage system,and the effectiveness of the designed control strategy was verified under the condition of sudden load increase.