Congestion Control Based On Deep Reinforcement Learning

With the explosive growth of the number of network devices and the continuous improvement of bandwidth,the network environment is becoming more and more complex.The current popular congestion control algorithm can not meet the requirements of the network environment,which brings new challenges to the design of network congestion control algorithm.Therefore,the development of new algorithms that can adapt to the current network environment has become an important topic in the field of network technology.At present,the network congestion control algorithm mainly has two shortcomings: first,the large-scale fluctuation of congestion window,and the resulting drop-packing and high roundtrip time(RTT);Second,most of the traditional congestion control algorithms are based on Presupposed window adjustment rules,which cannot be learned from the historical information of data link to improve the window adjustment strategy.These deficiencies result in poor performance tables under high bandwidth-delay product networks by traditional congestion control algorithms that do not take full advantage of bandwidth.First of all,in order to solve the problem of congestion control windows fluctuate widely and the lower average throughput.This paper presents a QLearning-based network congestion control algorithm named RL-TCP.It uses QLearning as the decision center to replace the presupposed window adjustment rules.In this way,it can adapt to complex network environments,perform well in high bandwidth-delay product(BDP)networks and have low RTT.Experiments show that the RL-TCP algorithm has a 7%-11% improvement in performance compared with the traditional congestion control algorithm.On this basis,aiming at the low efficiency of RL-TCP algorithm and the slow adding window at the beginning of data stream,which affects the average throughput.This paper puts forward a deep Q network(DQN)based network congestion control algorithm named DQN-TCP.The efficiency of the algorithm is improved by using DQN instead of the Q-Learning algorithm to make decisions.The design of utility functions and the network state selection are improved.So,the performance is promoted,and the window volatility is reduced.Experiments in the NS-3 simulation environment show that the DQN-TCP algorithm has greatly improved in efficiency,and the congestion window can reach a higher level faster and remain stable.Finally,this paper compares the performance of different network state parameters and reward functions under the DQN-TCP algorithm.Different network parameters and reward functions are selected for comparison in the same network environment,so as to show the influence of different parameters on the experimental results.