Hierarchical Scheduling Optimization Of Distributed Electric Heating Based On Deep Reinforcement Learning

The large-scale renewable energy network has deteriorated the regulation capacity of the "source" side of the power system,and tapping the adjustable resources on the "load" side has become an effective way to support the economic operation of the high proportion of renewable energy power system.With the implementation of electric energy substitution and clean heating incentive policies,the scale of electric heating load capacity in northern areas is gradually increasing,and these loads have time-shiftable characteristics and are high-quality regulating stock resources.To address the above issues,this paper researchs distributed electric heatings and conducts the way of tapping the regulation potential of load-side stock resources and the mechanism of promoting source-load interaction.The main research work is as follows.Analyse the power-temperature variation dynamic characteristics of distributed electric heating units.Deep learning methods and thermodynamic modelling methods are applied to characterise the load characteristics of distributed electric heating units.A large amount of distributed electric heating operation data was collected through field household electric heating operation control experiments and laboratory simulation experiments.The validity of the two methods in predicting the dynamic relationship between distributed electric heating unit power and room temperature variation is verified through the support of real measurement data.The advantages,disadvantages and differences of each of the two methods are investigated by setting up comparative experiments with different data sets of samples.The deep learning method that can characterise the load characteristics of distributed electric heating units lays the foundation for applying deep reinforcement learning algorithms for optimal control of distributed electric heating.A customer-side scheduling strategy for household electric heating is developed.An intermediary company load aggregator is introduced and a building-level control module is used to guide the customer’s electric heating load to participate in demand response through the tariff,solving the problem that the dispatch centre does not facilitate direct regulation and control of each household electric heating.An improved DDPG algorithm is proposed as the solution method.This deep reinforcement learning algorithm accumulates experience in offline training and can quickly formulate user electricity response actions according to user goals.The case study verifies the effectiveness and economy of the proposed method for scheduling household electric heating to participate in demand response,tapping into the load-side resource regulation potential and with better performance of the improved algorithm.An operational mechanism is constructed for load aggregators to consider the game with customers.The dynamic game relationship between load aggregators and electric heating customers is analysed,and a two-tier optimal dispatching model is developed,taking into account customer heating satisfaction,heating costs and load aggregators’ revenue.The existence of an equilibrium solution to the model is theoretically analysed,and the model is solved based on the user-side response behaviour obtained by a deep reinforcement learning algorithm.The simulation results verify the feasibility of the game theory model.The model can increase the integration and complementarity of "source" and "load" resources on the load side,maximize the revenue of the load aggregator and reduce the heating cost of the customers.