Research On Congestion Control Method Based On Reinforcement Learning In SDN Data Center Network

With the rapid development of new network technologies such as cloud computing and5 G,network traffic in data centers is growing explosively.How to ensure the robust operation of data center and the high quality service experience of users has become a hot research topic in data center.In the data center,the many-to-one traffic mode is easy to lead to the typical TCP Incast problem(Multiple senders simultaneously send data to one receiver,resulting in congestion of bottleneck link)in the data center,which will directly reduce the user's favorable degree of using the product.And improper routing can easily lead to uneven distribution of traffic load in the data center,which leads to the collapse of network nodes in the data center.Therefore,more intelligent and efficient congestion control and load balancing algorithms are needed in data centers.In this paper,Deep Reinforcement Learning(DRL)is introduced based on the data center under Software Define Network(SDN).Prioritize Experience Replay DQN Congestion Control(PERDQNCC)and Advantage Actor Critical Load Balancing(A2CLB)algorithms are implemented respectively to solve the typical TCP Incast problem and uneven distribution of traffic and Load in data center.To solve the typical TCP Incast problem in data center,this paper combines the deep reinforcement learning algorithm DQN(Deep Q Network)to implement the congestion control algorithm PERDQNCC.It can sense the state of the network environment(Throughput,round-trip delay of openflow flow and packet loss rate),so as to judge whether the network is congested or not,and dynamically adjust the transmission rate of the sending host to relieve the network congestion.PERDQNCC deploys an agent(An independent entity that can think and interact with the environment)on all sending hosts and regularly collects information about the network environment.Through the reward and punishment mechanism of reinforcement learning,the most appropriate transmission rate is assigned to the sending host,so as to avoid congestion,relieve congestion and maximize the use of network resources.The experimental results show that PERDQNCC can alleviate congestion and make full use of link bandwidth when the link is idle in the TCP Incast scenario in the data center.In response to the uneven distribution of traffic load in the data center,this paper implements load balancing algorithm A2 CLB with deep reinforcement learning algorithm A2C(Advantage Actor Critic).It can dynamically select the optimal forwarding path for Open Flow flow according to the state of network environment(Throughput,round-trip delay of openflow flow,packet loss rate and available bandwidth of link).Because the traditional load-balancing algorithm ECMP maps large streams with a long life cycle to the same path,which leads to serious load imbalance and network congestion problems.This static mapping mode is not suitable for data center traffic with strong randomness.Therefore,based on the A2 C algorithm,this paper sets an agent for each pair of communication devices to monitor the network environment of each forwarding path and dynamically select the forwarding path of openflow according to the environment,so as to achieve the purpose of load balancing.The experimental results show that in the fat tree topology common in data center,A2 CLB has better load balancing effect than ECMP,more uniform traffic distribution,and can make full use of link bandwidth.Under the common traffic modes,the average packet loss rate of A2 CLB is 15.3% lower than that of ECMP,and the average round-trip delay is 49.2% lower than that of ECMP,and the average throughput was 15.4% higher than ECMP and the average flow completion time was 37.6% lower than ECMP.