Research On WAN Traffic Scheduling Method Based On Deep Reinforcement Learning In SDN Architecture

As a remote network connecting different regional network systems,WAN(Wide Area Network)plays an important role in the whole computer network system.In recent years,with the rapid development of network applications,the traditional network traffic scheduling method cannot meet the current network needs.The emergence of SDN(Software Defined Network)architecture makes it possible for network optimization.By centralizing the control plane and evaluating from the point of the whole network view,we can control the traffic in a fine-grained way.At the same time,centralized control provides a prerequisite for the application of artificial intelligence technology in network traffic scheduling.Traffic scheduling is essentially a decision-making problem,and the recent emerging DRL(Deep Reinforcement Learning)technology is the latest breakthrough of artificial intelligence in decision-making problems.This thesis investigates some DRL based traffic scheduling schemes proposed in the past few years,and proposes a new WAN traffic scheduling scheme based on DRL in SDN architecture to avoid network congestion and optimize network performance.The main research contents of the thesis are as follows:(1)The problem of WAN traffic scheduling is analyzed.At first,the traditional WAN traffic scheduling scheme and its shortcomings are introduced.Then,a case of WAN traffic scheduling under SDN architecture is showed,and the advantages of SDN architecture in traffic scheduling are presented.At last,some requirements to be met by applying the current deep reinforcement learning technology to WAN to create intelligent traffic scheduling scheme are analyzed,including the main congestion avoidance capability,the auxiliary requirements of fault tolerance and real-time computing performance,etc.(2)A WAN traffic scheduling scheme based on deep reinforcement learning in SDN architecture is proposed.Firstly,the WAN traffic scheduling problem is modeled,and the traffic flows are merged into aggregate flows for scheduling.Then,the overall scheme of the traffic scheduling using DQN(Deep Q Network)algorithm is proposed,and the function and information flow of each module as well as the overall workflow of the scheme are designed.The real-time intelligent routing based on DQN in the scheduling scheme is used to allocate the optimal path for the new traffic according to the current state of the network.The intelligent rerouting based on DQN is used for routing reconfiguration when the network is about to congestion,and dynamically adjusting the traffic distribution.Compared with the existing dynamic adjustment algorithm based on DRL,this trigger mode reduces the calculation pressure and interaction pressure of the controller.(3)A real-time intelligent routing algorithm based on DQN is proposed.The design of DQN agent is carried out.Aiming at the problem that the existing real-time routing algorithm based on DRL is difficult to give consideration to both fault tolerance and real-time computing performance,a special data preprocessing method is proposed,which makes the agent have better fault tolerance to the network topology changes.At the same time,the eigenvector obtained by this method can effectively represent the network traffic distribution state and reduce the pressure of neural network for feature extraction,so that even a simple fully connected neural network can satisfy the need,with better real-time computing performance.The reward function is designed and the training method of DQN agent is explained in detail.(4)An intelligent rerouting algorithm based on DQN is proposed.The influence of flow order on network load balancing in dynamic routing is noticed.Then,with the ability of obtaining the real-time traffic bandwidth information in SDN architecture,an intelligent rerouting algorithm is designed to further improve the ability of congestion avoidance by reconstructing network traffic distribution with the set of the best path,which is got by numerical computation using the same DQN agent with real-time intelligent routing algorithm.(5)A simulation network scenario is built to test the proposed traffic scheduling scheme.Firstly,the simulation network is built,in which the DQN agent is trained.Then,the basic performance of the proposed traffic scheduling scheme,namely congestion avoidance,is tested,and the experiments shows that the method can effectively schedule the network traffic effectively and achieve congestion avoidance.Compared with the traditional CSPF(Constrained Shortest Path First)algorithm,the proposed framework can improve the congestion avoidance ability by 15%.Finally,the other properties of the scheme are tested and analyzed,and it is observed that the scheme can meet the requirements of fault tolerance,real-time computing in WAN traffic scheduling.