Research On Wireless Resources Management Algorithm Based On Reinforcement Learning

With the rapid development of mobile Internet,the fifth generation of mobile communication systems(5G)has become the focus of current attention.Different application scenarios have different requirements for the network.Ultra-dense deployment of base stations(BSs)is used to increase network capacity,and network slicing technology is used to meet the requirements of application with very different service qualities.Therefore,BS selection strategies and network slice resource management algorithms have become hot topics for scholars.3GPP proposed a dual connectivity(DC)solution in R14,that is,users can connect to 4G BSs and 5G BSs at the same time.In this scenario,the traditional received signal strength(RSS)algorithm is no longer suitable to the problem of base station selection,and the network slice resource management on the radio access network(RAN)side also needs to be specially designed.Therefore,in the DC scenario,this thesis focuses on user-centered and BS-centered mobile communication network resource optimization algorithm based on reinforcement learning(RL).The main research contents are as follows:(1)In order to solve the problem that the existing BS selection strategy based on RL algorithm converges slowly and cannot adapt to the dynamic changes of the network scene,a DC BS selection algorithm based on RL is proposed.The algorithm is the user equipment(UE)centered,with the goal of maximizing UE's throughput.This algorithm maps the BS selection problem to a RL problem: the UE as the agent,the wireless access technology selection strategy as the action space,and the throughput obtained by connecting to the BS at the current moment as the reward to calculate the probability of selecting each BS at the next time.Simulation results show that compared with the traditional RSS algorithm,the proposed algorithm can reduce the number of user equipment handoffs and increase the total throughput of users in the statistical time period in the DC scenario.(2)Aiming at the problem of network slice resource management in a DC scenario,a wireless slice network management system based on deep reinforcement learning(DRL)is proposed on the RAN side.In order to reasonably allocate the wireless resources in the entire system,the system takes the maximum of weighted sum of user satisfaction and resource utilization as the optimization goal.First,the problems that need to be studied in this scenario are proposed,which are the slice-level and user-level resource allocation problems.Then,a network slice resource management system is designed for this problem,which contains two modules and aims to solve the above two problems.Finally,signaling flows about system state,reward and UE information collection are designed for the proposed network slice resource management system.(3)Aiming at the proposed network slice management system,based on the deep Q-learning(DQL)algorithm,the slice-level resource management problem in the system is specifically realized.Because the traditional Q-learning(QL)algorithm cannot be applied to the continuous state space,a neural network is added and trained,that is,DQL algorithm is used to flexibly and dynamically adjust the slice resources of each system and map it to each BS.The algorithm maps the slice resource management problem into a DQL problem: the central controller as the learner,the system's slice resource allocation and user satisfaction as the state space,and the system's dynamic allocation of slice resources as the action space,the system's resource utilization and average user satisfaction as the reward value.Simulation results show that in the DC scenario,the proposed algorithm improves resource utilization while ensuring user satisfaction.