Research On Traffic Optimization Control Method Based On Deepreinforcement Learning

With the rapid development of computer networks and increasingly diversified network services,the network traffic is growing rapidly,of which video traffic accounts for a large part.Traditional static network traffic control schemes,such as the shortest path transmission,etc.,often fail to consider the state of the network well,and at the same time cannot meet specific goals.In light of this,it is usually necessary to formally describe the goal and then propose a heuristic algorithm to approximate the solution.However,when the traffic pattern shifts,the TE heuristics may suffer performance penalty.In addition,the procedure of designing heuristic algorithms usually takes several weeks,because the process requires help of network administrators,application-related information and traffic statistics collected over a long period of time,etc.Therefore,designing a traffic optimization agent that can automate the above process,and can automatically adapt to network variation while meeting the set goals is very appealing.In recent years,artificial intelligence technology has achieved satisfactory results in some fields.Inspired by this,researchers began to use artificial intelligence technology,especially deep reinforcement learning(DRL)algorithms,to control network traffic.However,the data utilization of current DRL algorithm is low,which has become one of the main obstacles to the application of DRL algorithm for traffic control.For this reason,when applying DRL algorithm for traffic control,this article also pays attention to how to improve the data utilization of the algorithm which can speed up the learning process of the algorithm.From the aspect of stetting location of traffic control policy,this paper conducts the following two researches:(1)Dynamically Split the Traffic in Software Defined Network Based on Deep Reinforcement Learning.This type of flow control method needs to set the policy to the corresponding forwarding device,which belongs to the setting category of In-Network.Software-defined network(SDN)is a new type of network architecture,which separates the control function of the network from the forwarding device,so that the network device can be managed centrally,thus speeding up the setting of traffic control policy.In order to speed up the training process,this thesis proposes an algorithm called SDN_DRLTE,it uses three techniques to improve the original DRL algorithm,namely:TE-aware exploration,priority-based experience replay and multi-step return.Simulation results show that SDN_DRLTE performs better than the baseline methods when the traffic load is dynamically changing.(2)Service-oriented Adaptive Video Streaming Based on Deep Reinforcement Learning.This type of traffic control method needs to set the policy to the sending or receiving end of traffic,which belongs to the setting category of Out-Network.To speed up the training process of the algorithm,this thesis proposes MTABR,it first uses the meta-learning method called MAML to learn multi-service goals,so that when the new goal appears,the algorithm can use previous experience to learn the control policy rapidly.After that,this thesis analyzes the process of algorithm and finds that multiple service goals can run in parallel.Therefore,a parallel multi-objective learning algorithm is proposed.Simulation results show that,when a new goal appears,MTABR can accelerate the learning process.Meanwhile,MTABR can output a good action policy in an acceptable time as well as adapt to the changes of network condition.