Research On Efficient Orchestration Of Network Function Virtualization Via Federated Reinforcement Learning

Network Function Virtualization(NFV)has become a key technology to drive network architecture innovation by decoupling network functions from dedicated hardware and providing services to the public in the form of Virtual Network Functions(VNFs).In NFVenabled networks,a Service Function Chain(SFC),which has an efficient deployment scheme,provides a great solution for operators in terms of resource allocation,elastic scaling,and latency optimization,becoming a popular research direction in the field of NFV.Due to the concerns of data privacy and the complex changes in requests,it is critical to study an efficient,intelligent and secure deployment strategy with network resource constraints.As a distributed intelligent learning framework,Federated Reinforcement Learning(FRL),which effectively considers the limited resources and dynamic requests,can reduce the stress of communication and deployment on the central server while protecting data privacy.Therefore,in this paper,we propose to conduct research on SFC deployment based on FRL.The main research contents and contributions are summarized as follows.To address the challenge that the traditional deployment schemes can hardly respond intelligently to dynamic network scenarios and lack scalable efforts,a scalable SFC orchestration scheme based on FRL is proposed to dynamically train the overall deployment model via the distributed machine learning framework.Each domain agent senses available resources and pre-allocated resources to learn local models iteratively while avoiding data exchange between domains.The cloud server adopts the collection model methods based on delay perception and the aggregation model methods based on weight to reduce the communication cost of the distributed system.Simulation results show that compared with other algorithms,our proposed scheme can effectively improve network resource utilization,which reduces the total network overhead.To address the challenge that the research of parallelized SFC deployment in resourceconstrained networks lacks an intelligent resolution deployment scheme to cope with parallelized structures,we have proposed a parallel VNF placement scheme via FRL.The scheme designs the applicable VNF parallelism principles to multiple scenarios through the dependencies among VNFs.The cloud server decomposes SFC requests into different domains considering the available resources of each domain and the possible parallelized SFC structure.In addition,an aggregation strategy is adopted to accelerate global model convergence based on time difference weights.The local learning is improved according to the parallelization mechanism,which improves the exploration efficiency and enhances the training stability.Simulation results show that our proposed scheme can achieve efficient SFC parallel deployment and reduce the end-to-end delay of SFC requests with reasonable utilization of network resources.