| Multipath Transmission Control Protocol(MPTCP)allows Transmission Control Protocol(TCP)to use multiple transmission paths to maximize channel resource usage.The MPTCP scheduler evaluates the states of each path through the scheduling algorithm,and schedules data packets according to different states.However,as the network environment becomes more and more complex,and the upper-layer applications have higher and higher demands on the quality of service of network transmission,traditional scheduling algorithms have been unable to adapt to the current network environment.Therefore,the research on MPTCP scheduling algorithm is helpful to improve the transmission efficiency of MPTCP and adapt to the network environment.In this paper,a deep reinforcement learning method is used to study the multipath packet transmission scheduling method of MPTCP.The heuristic-based MPTCP packet scheduling method has achieved good transmission results in many specific network scenarios.However,static packet scheduling rules are difficult to adapt to complex network environments.To solve this problem,this paper combines deep reinforcement learning and heuristic scheduling methods,and proposes a MPTCP packet scheduling method(DQN-R)based on deep Q neural network(DQN)and minimizing roundtrip delay(Min RTT).DQN-R uses DQN to learn the optimal transmission path set in various states,and performs packet scheduling in the path set through the Min RTT heuristic rule,and trains to generate the optimal scheduling policy suitable for the asymmetric environment.Experiments are carried out on the transmission performance of MPTCP based on DQN-R in different network test environments,and the results show that the transmission performance of DQN-R is better than the comparison methods.The MPTCP packet scheduling method based on DQN models the transmission scheduling process as a discrete action space problem.The search space is small and it is easy to train.To solve this problem,this paper proposes an MPTCP packet scheduling method(RLCAS)based on the Actor-Critic framework.RLCAS divides the decision-making network into Actor network and Critic network.By training Actor and Critic network,the model can output a determined optimal scheduling action under different network states.In different network test environments,RLCAS is compared with various scheduling methods.The results show that RLCAS can achieve better transmission performance in various environments. |
