Deep Reinforcement Learning For Smart Generation Control Of Power Systems

Since the generation control of power systems has brought new challenges with the continuous and massive accessing of numerous distributed energies,how to efficiently and economically control the generation in power systems has become a hot topic for the engineers and researchers.The advantage reinforcement learning and deep learning algorithms are applied to solve this problem in this paper.Besides,the simulation platform of parallel systems based on cyber-physical-social systems is built for controlling the generation in power systems efficiently and economically.The major creative contributions of this paper can be summarized as follows:1.To improve the control performance of reinforcement learning for smart generation control,an artificial emotional reinforcement learning is proposed.The artificial emotion of artificial psychology is applied to this approach with three strategies,i.e.,the selection of action,the update of Q-value matrix,and the update of reward function;since three quantitative functions are designed for the artificial emotion,nine strategies of artificial emotional Q learning and nine strategies of artificial emotional Q(λ)learning are designed in this paper.2.To solve the coordination problems of conventional generation dispatch and control in power systems,the framework of the real-time economic generation dispatch and control with unified time scale is proposed;and a relaxed deep learning is proposed for this framework.Firstly,to improve the preventive ability of smart generation control for the emergency with large disturbance,this paper proposes deep reinforcement forest algorithm,which divides the systemic states into emergency case and non-emergency case with the learning of historic states and historic actions.Secondly,to improve the cognitive ability of an interconnected power system,the deep learning,which has highly cognitive ability and learning ability,is employed to the framework of reinforcement learning;thus,a deep reinforcement learning is proposed;and the proposed deep reinforcement learning is simulated in two simulations with a total of 1296 days long configured simulation time.Finally,the framework of the real-time economic generation dispatch and control with unified time scale is designed for a large scale interconnected power system;then,a relaxed deep learning is proposed for this framework;and the proposed relaxed deep learning is compared with a total of 1200 conventional non-unified time scale combined algorithms in two simulations with a total of 6.586 years long configured simulation time.3.The framework of real-time smart generation control is proposed for micro-grid with unified time scale,and a deep adaptive dynamic programming is proposed for this framework.Firstly,the framework of real-time smart generation control considering both automatic generation control and generation commands dispatch is designed.Secondly,a deep adaptive dynamic programming is proposed for this designed framework;and the deep adaptive dynamic programming contains three deep neural networks which have the ability of multiple outputs,i.e.,deep model prediction network,deep critic network,and deep action network.Finally,the proposed algorithm is applied to the simulation of micro-grid with a total of 25.155 years long of configured simulation time under six cases,which contain the normal basic case with a total of 19 AGC units,the case of plug-and-play,the case of communication failure,the case of allday long simulation,the case of varying topological graph with a total of 28 AGC units,the case of varying systemic internal parameters.4.To accelerate the learning process of the algorithms based on unified time scale,the simulation platform of parallel systems based on cyber-physical-social systems is built.To compared with the proposed relaxed deep learning,a total of 146016 type combined optimization algorithms and control algorithms are simulated in the platform with a total of 400.0493 years configured simulation time.Finally,the platform based on parallel systems is initially applied in a small-scale demonstration project.