Desigh And Implementation Of A Data Forwarding Model Based On Deep Reinforcement Learning For Deterministic Network

With the development of electronic technology and communication equipment,more and more new applications are emerging,such as Virtual Reality/Augmented Reality(VR/AR),holographic communication,haptic networks,etc.These applications have ultra-low delay requirements,and many of them also have the requirement of deterministic delay(i.e.,the delay requirement is with an upper and a lower bound).This poses a great challenge to the communication networks.Although some resource management and control technologies have been proposed to meet the requirements of some low-latency applications,such as the concepts and technologies of Time-Sensitive Networking and Deterministic Networking.However,these technologies only focus on the problem of ultra-low-latency data transmission,without considering the impact of the transmissions on that of the data streams with the deterministic delay requirement.There are no solutions to guarantee the deterministic delay during data transmission either.It is of great significance to improve the overall utilization rate of network resources and the quality of service(QoS)of applications by considering the transmission of multiple data streams and optimizing the data forwarding.Hence,this thesis focuses on the Deterministic Networking data forwarding model.It investigates and analyzes the delay requirements and characteristics of future network applications,and classifies the future network applications into four categories.For the applications with deterministic delay requirement,the data packet forwarding process is regarded as a Markov Decision Process based on the idea of reinforcement learning.We design a deep reinforcement learning algorithm to model the forwarding of deterministic delay packets in the deterministic networks,taking into account the network resources and packet delay requirements.The thesis first investigates and analyzes the development of current electronics and communication technologies,and summarizes the applications brought by the new technologies and the characteristics of these applications.Then we propose a SDN-based deterministic networking architecture with intelligent control.Based on the architecture,we suggest a data forwarding model for data flows with deterministic delay.The model uses the data interaction method of SDN,and calculates the next hop for data streams by optimizing the network resources and the data transmission delay simultaneously with deep reinforcement learning algorithm,taking into account the states of networks and the delay requirement of data.Finally,we perform simulations of the transmission of multiple types of data in deterministic networks to evaluate the proposed data forwarding model,and analyze the experimental results by using different network topologies,traffic generation rates,routing table update times etc.The experimental results show that the proposed model is effective in reducing network congestion,increasing network throughput and increasing the success rate of data delivery.