Reinforcement Learning Based Multi-objective Energy Management For Integrated Energy System

The structure of traditional energy system restricts the improvement of energy utilization efficiency.With the increasing shortage of fossil energy,the disadvantages of traditional energy system are becoming increasingly prominent.The need to establish an efficient and clean energy system promotes the development of a multi-energy coupled and renewable energy highly permeated integrated energy system.In order to realize the optimal allocation of the resources and maximize the system advantages,it is necessary to do research on energy managenent of integrated energy system.What’s more,in order to improve the convergence speed and precision of the traditional Q-learning algorithm,a Q-learning algorithm with variable learning rate and an adaptive Q-learning algorithm are proposed in this thesis.Contents of the thesis are as follows:1.Topology of the integrated system is studied,and devices in the system are modeled based on the coupling relationship of energy resources.The energy conversion relationship within the energy hub,represented by combined heat and power,is emphatically analyzed.The operation mode of combined heat and power is discussed,which lays a foundation for the latter research on energy management.2.Considering the operating cost and environmental cost,an evaluation system of the integrated energy system is built,and a multi-objective energy management optimization model is proposed.On the basis of the principle of Q-learning method,the optimization model is transformed into a Q-learning model,and an energy management strategy is obtained by using Q-learning algorithm.Two integrated energy system structures,with storage device and without storage device,are studied in the simulation.The effectiveness and convergence of the Q-earning algorithm are proved,what’s more,the effect of energy storage device on reducing the operating cost of the system is revealed.Additionally,by changing the weights,the coupling relationship between the two objectives in the multi-objective energy management problem is analyzed.3.As the fixed-step Q-learning algorithm can not share a high convergence speed and precision simultaneously,by analysing the effect error signal on the iterative process,a Q-learning algorithm with variable learning rate is proposed in this thesis.The effectiveness of the algorithm is verified by simulation results.In comparision with the traditional Q-learning algorithm,the proposed algorithm is proved to have advantages on accelerating convergence speed and improving accuracy.4.In order to solve the problem with continuous state space,an adaptive Q-learning algorithm with variable learning rate is proposed based on the Q-function approximation.By doing simulations,the effectiveness of the algorithm is validated,and the algorithm is proved to accelerate the convergence speed and improve the accurracy compared with the traditional Q-learning algorithm.