Research On Slice Resources Management And Orchestration Algorithms Based On Reinforcement Learning

The future mobile communication system will support a large number of communication services with different performance requirements.However,traditional networks difficult to meet such diversified performance demands.In this context,network slicing technology could solve such problems well.Network slicing as a key driving force of future mobile communications system,for it can create multiple customized virtual networks on a shared physical network to meet the demands of various services.Considering the limitation of physical resources and dynamic change of services requests,the resource allocation strategy which is rigid and unreasonable may not able to meet the different performance requirements and even increase the cost of the service providers.Further,with the people focus on the data security and device privacy more and more,efficient and intelligent resources management strategy is essential.Therefore,the focus of this thesis is to optimize resource management mechanism in radio access network(RAN)slicing and core network(CN)slicing scenarios,and the main innovations and research works are summarized as follows:To solve the problem of unreasonable resource allocation caused by data privacy and dynamic arrival of services requests where multiple types of slices coexist in the RAN scenario,a network slicing resource allocation strategy based on Federated Learning with Actor-Critic(FL-AC)learning is proposed.First,by jointly considering congestion control,resource allocation,and performance requirements of Device to Device(D2D)communication service,a quality of service(Qo S)of slicing maximization model based on joint allocation of power and physical resource blocks(PRBs)is established,and transformed into a Markov Decision Process(MDP).Secondly,in order to solve the problem of data security,a distributed Federated Learning(FL)framework containing multiple agents is constructed,and the agents directly use local data to make decisions.Lastly,for the local decision-making process in federated learning framework,an Actor-Critic(A-C)learning algorithm is adopted to continuously interacting with environment,so as to dynamically adjust resource allocation strategy based on traffic control.Simulation results show that the proposed algorithm could realize the dynamic allocation of resources,reduce network congestion,and effectively meet the performance requirements of different slices.Aiming at the high cost of virtual network function(VNF)orchestration caused by dynamic changes of network topology,a dynamic VNF orchestration strategy based on Double Deep Q Network(DDQN)is proposed.First,by jointly considering VNF mapping and computing resource management,virtual link mapping and bandwidth resource management,a VNF orchestration cost minimization model with the constraints of delay is established,and further transformed into an MDP model.Then,due to the transition probability of state can not be obtained and the high dimensionality of the state space and action space,a Deep Q Network(DQN)is used to solve the MDP model.Further,in order to solve the over-fitting phenomenon that may occurs in Deep Q network,decoupled action selection and value function estimation from target network,and a double Deep Q network is used to obtain an optimized dynamic virtual network function orchestration strategy.The simulation results show that the proposed algorithm could reduces the cost of VNF orchestration while satisfying the delay performance.