Power-Aware Traffic Engineering With Deep Reinforcement Learning

The success of Alpha-go demonstrates the great potential of deep reinforcement learning in decision making tasks.Using machine learning to make intelligent decision-making has begun to receive attention in many engineering fields.There is a large amount of dynamic data in the network for machine learning,such as delay of links,queue length,packet loss rate,etc.These data all reflect the status of the network and can be used to guide the network administrators to make adjustments.However,applying the existing machine learning theory to network decision-making has the following difficulties:first,it is difficult to collect data in real time effectively,secondly,we can't calibrate the quality of data,while unsupervised learning can not directly trained to doing intelligent decision making,so half Supervised learning,such as reinforcement learning is worth exploring.The existing researches design mechanisms based on heuristic algorithms usually get better solutions,but often require long-term designing,testing,and experimentation.If the constraints change,you need to repeat the design.This paper attempts to use the classic framework of reinforcement learning to make this kind of decision,and improve the framework according to the environment of network control problems.We design a deep neural network as the decision-making module,and model the network congestion and power consumption as the state observed from the environment,and use the switch of the routing node in the network as the action set.By continuously making decisions to change the net-state and collect data as feedback information,the machine can learn what kind of decision is applicable to a specific environment,and finally we can build an intelligent system based on it.The main content of this paper is divided into two parts.One is how to apply the reinforcement learning to solve reality problem,especially the network control,which requires a lot of decision-making,including multiple variable factors,and Compared with the fields that have been maturely applied RL.(such as Go).The second is how to improve the reinforcement learning framework according to the corresponding conditions so that it can handle the network control problems.This paper implements a network control system and verifies the effectiveness of this system through a simulation platform based on software-defined network technology.This control system we built can produces and perform decisions in milliseconds and its performance is close to heuristic algorithm.