Improved TCP Congestion Control Algorithm Design Based On Multi-objective Optimization

Recently,with the rapid development of computer networking and mobile communication technology,internet services are becoming ubiquitous.Consequently,those services generate an ever increasing volume of Internet data,which brings great pressure of Internet congestion.Congestion Control algorithm,the key mechanism of Transmission Control Protocol(TCP),is an important means to solve Internet congestion.It is widely used and investigated,so as to effectively improve the multi-objective performance of data transmission and meet the user's requirement on quality of experience.Existing congestion control algorithms can be divided into rule-based congestion control algorithms and learning-based ones.The rule represents a deterministic way that the sending window size is adjusted according to a certain network feedback signal.Typical rule-based algorithm Cubic uses predefined rules to adjust the sending window size based on specific congestion signals,so it cannot adapt to dynamic network resources to realize high throughput and low transmission delay.Learning-based algorithms mainly use Reinforcement Learning(RL)to learn the appropriate congestion control algorithm to adjust the sending window based on continuous interactions between the agent and the environment,such as state,action and reward,in order to maximize the cumulative reward.However,when it optimizes multiple objectives,its multi-objective reward function is often difficult to design.To properly address the problem of multi-objective transmission performance,this thesis proposes a scheme to continuously improve the multi-objective transmission performance of TCP congestion control algorithm.First of all,the rule-based congestion control algorithm BBR proposed by Google in 2017 has been widely used in data communication networks due to its high throughput and low latency.However,when its data flow(end-to-end TCP connection)shares the link with heterogeneous data flows(using different congestion control algorithms),it aggressively fills the link,triggers the default queue management mechanism Drop Tail in the network,and causes a lot of packet loss.When packet loss occurs,the sending window of other data flows will be halved,and the BBR data stream will not respond to packet loss,hence its data flow will unfairly occupy too much bandwidth.Aiming at solving the fairness issue,this thesis proposes a fair queue management algorithm CKCD(Choose-Keep and Controlled-Delay)for queue management.The algorithm actively drops the data packets of two types of data flows to achieve fairness between heterogeneous data streams while ensuring high throughput and low latency.Those two types of data flows are: 1)data flows with excessive backlog due to no response to packet loss;2)data flows with long queuing delay.Experimental results show that compared with the default mechanism Drop Tail,CKCD improves fairness by up to 62%,while reducing average transmission delay by up to 92% and guaranteeing 96% link utilization.Secondly,RL is widely used in TCP congestion control to solve multi-objective problems,the multi-objective reward function is,however,generally difficult to design.To address this challenges,the thesis designs a constrained reinforcement learning(CRL)algorithm for multiobjective optimization in congestion control.The algorithm first formulates the multiobjective optimization problem as a constrained optimization problem,and uses Lagrangian relaxation method to transform the problem into a single-objective optimization problem,and finally employs a CRL framework for training a congestion control strategy that optimizes multi-objective performance.Extensive experimental results prove that,compared with the classic algorithm PCC,the CRL algorithm used for congestion control not only avoids the problem of difficult objective function design,but also improves fairness by 21.7%,throughput by 5.4%,and reduces transmission delay by 27.4%.The novelty of this thesis lies in the following aspects:(1)this thesis analyzes the reasons why BBR-controlled data flows unfairly occupy bottleneck bandwidth,and design a fair queue management mechanism CKCD to solve the fairness problem between heterogeneous data flows,while improving network throughput and reducing transmission delay,(2)this thesis reveals the challenge of optimizing multiple objectives for traditional RL algorithms,and proposes a learning algorithm to achieve multi-objective optimization,and(3)this thesis designs a constrained reinforcement learning framework for network congestion control to optimize multiple transmission performance.