Research On Dynamic Resources Management Algorithm Based On Reinforcement Learning In Network Slice

In the future,communication services will be more diverse,and different services have different communication requirements.However,traditional networks could only satisfy a single service demend,so the network slice technology emerges to adapt to diverse services.Network slice technology allows the establishment of multiple logical networks in a physical network to respectively meet the demends of various services.In order to ensure the excellent communication quality of network slices,efficient resources management mechanism is extremely important.Therefore,the target of this thesis is to optimize resource management in access network slice scenario,and the research work of this thesis is summarized as follows:Aiming at the problem that topology perception is not practical in many literatures,a multiresource joint management scheme based on Partial Observation Markov Decision Process(POMDP)is proposed.For the physical network topology,a heartbeat packet detection mechanism is designed to partially observe the current topology condition.Based on complete queue state and partial topology state,POMDP is used to determine the orchestration and mapping of computing and link resources for service function chains in slices.The mingle heuristic value iteration algorithm is used to solve the proposed POMDP model because of its low complexity,and the approximate optimal resource management strategy could be obtained.The simulation results show that the proposed scheme could optimize the slices' resource orchestration and mapping on dynamic topology,and reduce delay and minimize repair cost.As a result,the proposed scheme could realize low-cost and low-delay communication and improve resource utilization.In order to solve the high latency caused by the terminals' moving in the access network slice,a mobility-aware resource management scheme based on reinforcement learning is proposed.Based on Markov Decision Process(MDP),a MDP model is proposed.Especially,it could observe the terminals' mobility and environments' changes,and it could make decisions about the allocation of computing,link and radio resources.The Q-learning method is applied to the MDP model to solve the problem that the transition probability of mobile state and environment state can not be obtained.In order to reduce the complexity of Q-learning method,a Deep Q Network(DQN)is designed and trained to obtain the approximate optimal resource management strategy.The simulation results indicate that the proposed scheme can dynamically optimize the delay of access network slice with mobile terminals,meanwhile it reduce the data loss in the slice.So the proposed scheme could achieve low data loss and low delay,and improve the throughput of slice.