Research On Network Resource Optimization Of Data Center Based On Deep Reinforcement Learning

In recent years,with the continuous development of mobile Internet,big data,and cloud computing services,the number of network users and business traffic are also increasing rapidly,and data centers have become an important part of massive data processing and computing.While the computing power of a single node continues to increase,how the interconnection network in the data center can cope with the high dynamic data communication needs has become an urgent problem to be solved.A large number of studies have shown that there is still a lot of room for optimization in the existing data center network interconnection structure and routing forwarding strategy based on electrical switching equipment.The traditional data center routing strategy generally adopts fixed forwarding,multi-path distribution,centralized control,etc.,and often formulates corresponding forwarding strategies and routing tables according to the network topology and service traffic conditions(for example,through network telemetry and other technologies).It cannot cope with local congestion caused by burst traffic.At the same time,traditional data center network frameworks generally use fixed connection frameworks such as FatTree,Butterfly,etc.Such symmetrical and stacked interconnection frameworks also lead to poor flexibility in dealing with uneven traffic distribution.The introduction of Knowledge-Defined Networking(KDN)combined with Deep Reinforcement Learning(DRL),provides new possibilities for intelligent network resource optimization.In this paper,data center network resource optimization problem is deeply studied from two aspects:routing optimization strategy and topology reconstruction.(1)Topology:A DRL-based optical interconnection reconfigurable architecture is proposed on the KDN framework,and the mainstream DDPG(Deep Deterministic Policy Gradient)algorithm is used in the agent(Agent)in the top-level knowledge plane.In order to solve the discrete action problem that the continuous action value output by the DDPG algorithm does not satisfy the topology structure,this paper uses the KNearest Neighbor(KNN)algorithm to set up a complete set of action selection strategies,and the continuous action output by the Agent is reasonable.The nearest neighbor calculation is performed in the topology set,and K reasonable topologies are selected,and the topology actions with the highest value return are configured and learned from them.The average delay of the entire network is effectively reduced,and the optimization effect is most obvious at 50%network load(Traffic Load,TL)and 75%TL,which are increased by 27%and 30%respectively.(2)Routing:Aiming at the routing optimization problem of different TLs under complex network topology,a method for routing optimization based on the current network traffic state is proposed.When the Agent uses the DDPG algorithm to learn,there is a low correlation between the traffic distribution information and the weight information,and the learning process has the problems of insufficient training and unstable convergence value.Therefore,a set of dynamic link weight construction strategies is designed.The initial weight is constructed by using the traffic distribution information in the network,which not only ensures the priority communication of large traffic data,but also ensures that the path with the minimum number of hops is preferentially selected.Considering that the link weight does not require high refinement of the weight action when generating the routing table,the action discretization method is adopted to project the high-dimensional continuous action space onto the discrete action set,reducing unnecessary action training and space.Exploration is conducive to the selection of optimal routes and the improvement of training speed.We use the OMNeT++network simulator to verify the two optimization strategies.The data proves that the DRL Agent can formulate the routing table with the lowest average network delay according to the current user’s traffic distribution information.The traffic distribution information and initial weights are used as training inputs respectively,and the network delays obtained by the routing tables generated by different weight strategies are compared,which verifies the effectiveness of the routing optimization strategy and effectively reduces the average delay of the entire network.This paper uses the DRL method to solve the problem of network resource optimization in the data center.Based on the KDN architecture,self-analysis,self-learning,and resource optimization are realized from the aspects of routing and topology.overall resource utilization.Provides a new approach to resource optimization in large-scale data centers.