Reseasrch On Deep Reinforcement Learning Based Router Resource Scheduling Mechanism

The global network is undergoing profound restructuring and transformation with the development of Software-Defined Networking(SDN),and Network Function Virtualization(NFV).However,following the improvement of the flexibility and scalability of the network,it also brings unprecedented challenges for networks control.Recently,owing to the success of Machine Learning(ML)related applications,adding intelligence to the network control plane through Artificial Intelligence & Machine Learning(AI&ML)becomes a trend and a direction of network development.However,as a large-scale geo-distributed high-dynamic system,how to place the intelligence in networking is the key to the high-efficiency operation.In this treatise,we propose a hybrid intelligent routing paradigm,where we deploy intelligence in different way for different network services.We propose a Deep Q-learning based QoS-Routing algorithm,a local reward distributed reinforcement learning routing algorithm,a centralized deep deterministic policy gradient based congestion control strategy,and a WoLF-PHC based distributed resource allocation mechanism which realize the precise matching between network resource and QoS requirement and intelligent network router allocation.The main work and contributions of this paper are as follows:Firstly,we propose a centralized QoS routing control mechanism.We employ the SDN and INT to implement a network state upload link and a decision download link to accomplish a close-loop control of a network and build a centralized intelligent agent aiming at learning the policy by interaction with a whole network.In addition,we apply DRL to effectively solve real-time large-scale network control problems.Secondly,due to the centralized paradigm brings too much overhead of both communication and computation,we propose a different reward based distributed reinforcement learning for hop-by-hop routing.In addition,we introduce the network mind to enhance the convergence of the distributed agent by collecting global knowledge.Thirdly,focusing on the increasing of the multiple interfaces of mobile devices,we propose a deep deterministic policy gradient based congestion control strategy to realize the effectively allocation of the bandwidth resource among competitive sub-flows.Fourthly,for supporting the latency-sensitive services control,the edge router as the fog node is deployed in the network edge to reduce latency between local and remote computing resources.We model the resource management and pricing problems among three players as a Stackelberg game.In addition,for searching the Nash equilibrium of this game,we apply the 'WoLF-PHC' algorithm for learning the optimal resource management strategies by interacting with the environment.