Research On Intelligent Handoff Algorithm In B5G End-to-End Network Slicing Scenario

With the explosive increase of new services in B5 G communication systems,network slicing has become an important solution to meet the diverse requirements of different applications.Based on software-defined networking(SDN)and network function virtualization(NFV),the network is sliced into several isolated end-to-end(E2E)logical networks,each of which is defined as a network slice(NS)running on a common physical infrastructure and providing certain services for users on-demand.In the Het Nets with E2 E network slicing functions,the user handoff problem is more complex than that in the traditional Het Nets.Since a NS may only be associated with some of the base stations(BSs)in the area,the handoff should consider the associations among users,BSs,and NSs.Besides,the transmission resource limitations of the core network(CN)also need to be considered in E2 E scenarios.Therefore,the handoff problem faces challenges of the huge state space and action space and the considerable communication overhead.This thesis focuses on the handoff problem in the B5 G end-to-end network slicing scenario,considering the resource states of the radio access network(RAN)and the core network,and studies the intelligent handoff algorithm based on deep reinforcement learning(DRL).The main research contents are as follows:1)Given the system state considered by the handoff algorithm based on the reference signal received power(RSRP)is insufficient,and the computational complexity of the centralized handoff algorithms based on the traditional optimization methods is too high,this thesis adopts a Markov decision process to model the handoff problem that considers the comprehensive system state including communication link quality,end-to-end slice resource state,and user quality of service(Qo S)requirements.Then,the cumulative reward related to the user’s profit of being served,the handoff cost,and the outage penalty is designed as the optimization goal,and a Double DQN(DDQN)-based intelligent handoff algorithm is designed to reduce the computational complexity.Numerical results confirm the convergence of the DDQN used to make handoff decisions and show that compared with typical handoff algorithms,the algorithm we proposed performs the best from the aspect of the cumulative reward.2)Given the high communication overhead and decision complexity of the centralized handoff algorithm when there are a large number of users in the network,the existing distributed handoff algorithm faces the problems of insufficient training samples and geographical restriction due to each user holding an independent Q-table for handoff decision and trains the agent locally,this thesis designs a distributed NS handoff decision method based on a decentralized Markov decision process(DECMDP)model with a unified structure.First,unlike the general distributed handoff methods,which set different feasible solution spaces for different users,we build the same state space,action space,and reward function for all users in the area according to the deployment of BSs and NSs.Then,using the advantages of the unified model,a multi-agent DDQN based distributed handoff method(MA-DDQN-DH)with a centralized training and decentralized executing framework is designed,and the penalty function is used to guide the decision-making agent to avoid infeasible solutions and the handoff intelligent sharing among users in the area is realized.To obtain theoretical performance guidance,the original distributed NS handoff problem is simplified,and a Nash equilibrium-based performance bound is given.Simulation results show that the Nash equilibrium-based performance bound is reasonable and the proposed MADDQN-DH algorithm performs well in the comparison.