Design And Implementation Of Queue Management In Data Plane Based On Deep Reinforcement Learning

With the emergence of real-time demanding applications such as short video,live broadcast,and cloud meetings,minimizing delay has become the main target of network research.Active queue management plays an important role in improving network congestion and controlling data flow delay.However,forwarding equipment of traditional network does not support user-defined queue management algorithms as hardware limitations.However,new network architectures such as programmable networks have emerged,where users can manage data plane resources(memory,processors,queues,etc.)on network nodes by programming interfaces.Active queue management technology has been further developed accordingly.Active queue management based on programmable data planes has great significance for network congestion research.Based on the research of network programmable technology,combined with the deep reinforcement learning model,this thesis designs and implements an activate queue management mechanism based on programmable data plane,which is a supplement to the effective combination of programmable data plane queue management and deep reinforcement learning.This thesis mainly includes the following contents:(1)An active queue management scheme is proposed for the changes in the queue status of different data streams in the queue.Three queuing states are defined,and different queue management strategies are implemented in different queue queuing states.By discarding data packets to control the delay of the data flow,thereby limiting the sending rate of the sender,stabilizing the queue length and queue delay,and ensuring network transmission efficiency.(2)Use programmable data plane in-band network telemetry(INT)to collect the information of the switch,providing end-to-end packet-level network information for network state judgment and deep reinforcement learning model training.INT works directly on the data plane without the controller participates in the network information collection process,which greatly reduces the interaction between the control plane and the data plane.(3)Introduce the deep reinforcement learning model in the programmable network architecture,effectively combine active queue management with the deep reinforcement learning DDQN model.By learning the complex network environments,the queue status division threshold can be adjusted according to different network states at different times,so that the responsiveness of activate queue management follows the dynamic changes of the network environment.(4)Finally,perform functional testing and performance analysis on the queue management scheme proposed in this article.Compared with several P4-based active queue management schemes(P4-Co Del,P4-PIE and P4-RED)in terms of queue delay,throughput and other performance,the experimental results show that the queue management scheme proposed in this thesis can stabilize the queue length and ensure the minimized queue queuing delay.It has a good performance in dealing with the accumulation of data packets in the queue.