Deep Reinforcement Learning Controllers For Active Ventilated Tiles In Raised-Floor Data Center

The data center industry has seen a boom growth since last decade.Data center consist of thousands of servers and IT equipment,the cooling system consumed huge amounts of energy in data center.In cold aisle environment,because of the unbalance between heat dissipation and cooling effect cause the hotspots,which lead the rack inlet temperature higher.This issue widely exists in various data centers.The hotspots issue in cooling system causes many problems,such as performance of IT equipment's and increased cooling energy consumption.In the multi racks area,hotspots will hide the serious security issue from data center.In previous papers,researchers focus on traditional direct solutions(such as excessive air-conditioning flooding and task allocation).Traditional direct solution cannot solve the basic problems.To solve these issues,this paper proposed Active Ventilated Tiles control algorithms which is used to solve hotspots problem.this paper used Active Ventilated Tile in data center which contain the raised floor and no containment aisle structure.This paper solve the hotspots elimination problem in the multi racks area.The multiple Active Ventilated Tile control algorithm based on deep reinforcement learning to eliminate the hotspots from multi racks,and study the joint control algorithm of multiple Active Ventilated Tiles.The control algorithm is used to combines reinforcement learning and artificial neural network to control the multiple Active Ventilated Tiles.The author deployed Active Ventilated Tiles in the cold aisle environment to implement the function to control the ventilation volume.The agent observes the rack inlet temperature,and change the ventilation volume of Active Ventilated Tile to eliminate the hotspots and cooling the temperature.The main work of this paper is as follows: 1)Markov decision process problem modeling of multi Active Ventilated Tiles joint control problem in data center;2)design and improve the multi Active Ventilated Tiles joint control algorithms,which is based on deep reinforcement learning,which are Independent Multi-Agent Deep Q-learning Network,Shared-Reward Multi-Agent Deep Q-learning Network,Branching Dueling Deep Qlearning Network,Deep Deterministic Policy Gradient;3)the establishment simulation model,equipment deployment,as well as the implementation of interactive system and algorithm;4)the verification of multi Active Ventilated Tiles effect under the control algorithm,improved the performance of control algorithm,the performance evaluation of different algorithms and the verification of temperature weight of the objective function.The experimental results show hotspot problem,the cooling effect of deep reinforcement learning control multiple Active Ventilated Tiles is better than PVTs in the joint control problem,and the convergence speed of the control algorithm perform better after the improvement of Dyna mechanism;in the performance evaluation experiment of multiple Active Ventilated Tiles control algorithms,the optimal algorithm for comprehensive performance is Deep Deterministic Policy Gradient.Finally,this paper used optimal algorithm to verify the temperature weight of the objective function.