| In recent years,the Internet industry are booming,and the surge in the number of network equipment has brought great challenges to network service providers.To meet the requirements of different network services(such as firewall,code translation,and network address translation),different dedicated hardware devices must be deployed on the network.However,deploying dedicated hardware for each new network function causes Network Service Providers to incur significant Capital Expenditure and Operating Expense losses.More recently,the emergence of virtualization of network functions offers significant opportunities to address these challenges.In a network architecture that supports the virtualization of network functions,network services can be implemented as a set of ordered virtual network functions on common computing nodes,which are called service function chains.The main problem of service chain deployment is how to find the appropriate service function deployment mode under the constraints of function location and service chain requirements.This paper mainly studies the service chain deployment method based on deep reinforcement learning.Firstly,the network environment is modeled as an undirected graph containing nodes and links,and the total network delay is defined as processing delay and transmission delay.Considering the finitude of network resources,the constraint formula of network resources is given,and a method to judge network load balance is proposed based on Cauchy inequality.According to the dynamic characteristics of network,the process of resource deployment is modeled as Markov decision process,and the definition of state,action and reward in network environment is described in detail.Considering load balancing and delay optimization,the optimization goal of service function chain deployment in this paper is obtained.To solve this problem,a service chain deployment framework based on deep reinforcement learning was designed,and a service chain deployment algorithm based on depth determination gradient strategy was proposed.Centralized solutions in service chain deployment bring too much communication and computing overhead to the entire system.However,distributed solutions face serious convergence problems.In order to solve the deployment problem of multi-user service chain in network,this paper abstracts the multi-user competition behavior as a Markov game problem.In order to combine the advantages of distributed and centralized solutions,a hybrid intelligent control framework based on centralized training and distributed execution strategy was designed,and a service chain deployment algorithm based on multi-agent reinforcement learning was proposed.The simulation results and analysis show that the algorithm proposed in this paper can not only improve the utilization of network resources,but also have obvious advantages in response speed. |
