Application Of Deep Reinforcement Learning In Network Resource Management Problems

Research on resource management issues in computer systems and networks is ubiquitous.For example,computing task scheduling of clusters,bitrate adaptation in video streaming,relay selection in Internet telephony,virtual machine placement in cloud computing,congestion control,and so on.Most of the current methods for solving these problems are heuristic algorithms.But heuristic algorithms have some key shortcomings:when the underlying system is complex,it is usually not possible to model accurately,and whenever certain aspects of the problem changes,the model must be modified.In addition,preset models are difficult to make better real-time decisions in noise-affected inputs and in variable environments.Recently,machine learning has been successfully applied to the research of problems in some decision-making fields,especially the combination of reinforcement learning and deep learning in machine learning,such as Computer Go,playing video games,and so on.Inspired by the above,it is not impossible for the system to use deep reinforcement learning and learn resource management by itself.Given the scale and complexity of many resource management issues facing today,there are many areas where deep reinforcement learning can be used.The research focus of this thesis focuses on the following aspects:(1)Task scheduling problem of resource clusters in cloud computing resource management.This thesis expands the task model with reference to the existing task model and uses the DQN algorithm in deep reinforcement learning to solve various optimization objectives preset in the task scheduling problem.First,the task model is extended to a single-stage task and a multi-stage task,and then the task also has priority.Next,using the deep reinforcement learning algorithm to learn the strategy including minimizing the average job slowdown,minimizing the average job completion time,maximizing the task priority discrimination degree and the combination optimization objectives of the above three,the DQRM algorithm is proposed.Finally,the performance of the algorithm is analyzed from the aspects of the parameters and structure of the algorithm and the workload.After experiments,the algorithm is superior to SJF,Tetris*,Packer and other classical task scheduling algorithms in different optimization objectives in different degrees.(2)Improve the TCP connection throughput problem.This thesis improves the throughput by changing the initial TCP congestion window in several different network scenarios such as the data center network through deep reinforcement learning.First,choose four different network scenarios,such as data center network,satellite network,lossy link,and bottleneck link with small buffer.Then set the optimization objective of deep reinforcement learning to maximize the average throughput,and proposed DQCC Algorithm.Finally,the performance of the algorithm is analyzed from the parameters of the algorithm and the different configuration levels of the network scenario.The simulation test proves that the algorithm can automatically learn the optimal TCP initial congestion window size in different network environments to improve throughput.