Research On Energy Management Of Multi-energy Grid Based On Deep Reinforcement Learning

The multi-energy microgrid is an effective solution for improving the comprehensive energy utilization rate and increasing the proportion of renewable energy.The energy management is vital to the safe and efficient operation of the multi-energy microgrid.There are multiple energy flow systems coexisting,multiple renewable energy inputs coexisting,and multiple loads coexisting in the multi-energy microgrid.Besides,there are certain fluctuations and randomness in renewable energy inputs and loads.Therefore,the inputs,conversion,distribution and consumption of the multi-energy microgrid presents imbalances at diverse temporal and spatial scales,which poses a huge challenge to the energy management of the multi-energy microgrid.In this paper,an energy management method based on DRL(deep reinforcement learning)for multi-energy microgrid is proposed.The DQN(deep Q learning)algorithm is applied to learn a strategy set from environmental information such as predicted load data,predicted data of renewable energy power output and time-of-use power price.Energy management of the multi-energy microgrid is more effective through the learned strategy set.Furthermore,rapid optimization of new scenarios can be achieved through inheriting the learned strategy set.The work of this article includes:(1)Based on the research framework and equipment model of the multi-energy microgrid,the energy-bus model of the multi-energy microgrid is established according to the characteristics of the multi-energy flow.(2)On the basis of elaborating the framework of reinforcement learning,the basic theory of Q learning algorithm and DQN algorithm,the mechanism of experience replay and fixed parameters of DQN is analyzed to improve the performance.The general design method of energy management strategy based on deep reinforcement learning is summarized.The energy management for grid-connected multi-energy microgrid with the goal of economy is completed.By comparing the performance of DQN,Q-learning and genetic algorithm with different hyperparameters in the energy management,the effectiveness and superiority of deep reinforcement learning is verified.The subsequent simulation result shows that the improvement of algorithm that inherited the strategy set is significant,verifying that deep reinforcement learning possesses more feasibility and superiority than heuristic algorithm for energy management of multienergy microgrid.(3)On the basis of the foregoing,the hybrid power system of unmanned surface vehicles is regarded as the representative of the isolated multi-energy microgrid to study the application of the proposed method applied to the energy management of the independent micro-energy grid.The simulation results show that the DRL-based multi-energy microgrid energy management method has certain self-learning capabilities,which enable the vehicle effectively respond to a variety of emergency and the vehicle can adjust the energy scheduling plan in advance to ensure that the independent multi-energy microgrid system works autonomously,efficiently,safely and economically.