| Building energy consumption in China has been growing at a rate of about 10% per year over the past 20 years.In 2019,building operating energy consumption in China accounted for about 22%of country’s total energy consumption.In commercial buildings,about 40%-50% of total electricity consumption is attributed to HVAC systems.With the development of Internet of things and artificial intelligence technologies,the HVAC control algorithms based on deep reinforcement learning have received much attention.The important reason is that these control algorithms do not require explicit building thermal dynamic models and any prior knowledge of uncertain parameters.Nevertheless,existing control algorithms have low scalability as the number of commercial building zones increases.Therefore,it is very necessary to conduct further research on scalable HVAC control algorithms in multi-zone commercial buildings based on deep reinforcement learning.Firstly,this paper formulates an HVAC energy cost minimization problem in a multi-zone commercial building.Due to the existence of these factors,e.g.,uncertain system parameters,unclear building thermal dynamic model and temporal coupling constraints related to indoor temperature and carbon dioxide concentration,and spatial coupling related to air supply rates in each zone,it is difficult to solve the above optimization problem directly.To address the above challenges,we reformulate the optimization problem as a Markov game.Then,this paper proposes an energy optimization algorithm based on deep reinforcement learning to solve the Markov game.Specifically,the proposed algorithm divides the training process into several stages,and the neural networks weight parameters with high fitness in each stage will be used as the initial weight parameters of neural networks in the next stage.Meanwhile,at each stage,the multi-agent attention deep deterministic policy gradient algorithm is used for parallel training of multiple populations.After the training,the populations with high fitness are screened out by means of population hybridization and evolution.Compared with other solutions,simulation results based on real data show that the proposed algorithm can reduce the energy cost by 12.0%-47.99% while maintaining comfortable air quality and thermal comfort in the zone.Finally,the paper makes a conclusion and gives an outlook on future research directions. |
