Resource Optimization For LTE-U Networks Based On Machine Learning

With the development of mobile communication and mobile equipments,people are increas-ingly demanding high-speed and low-latency communication,which has brought tremendous chal-lenges and pressure to mobile communication.In order to cope with the limited capacity of mobile communication system and the limited growth of data transmission rate caused by the shortage of licensed resource,academia and industry have proposed two types of solutions.One is to improve the utilization rate of licensed resource,and the other is to introduce LTE technology into unli-censed band,namely LTE-U technology.Due to the scarcity of licensed resource,the improvement of the first type of the schemes is always limited.Since the unlicensed band is wider and more idle,it can significantly increase the system capacity and data transmission rate of the cellular mobile communication system.Therefore,LTE-U technology has broad application prospects.However,the application of LTE-U technology still faces many challenges.The main difficul-ty is the coexistence of LTE and WiFi.Because the MAC layer of LTE-U systems adopts the same non-competitive protocol as LTE systems.If without controlled and managed,the performance of the systems which have already been working on unlicensed band will be seriously deteriorated,especially the WiFi networks which are the most widely used.Therefore,whether LTE and WiFi can coexist is the key to the successful application of the LTE-U technology.Although the coexis-tence of LTE and WiFi has been studied in depth,most of the researches are limited to the central scenarios.However,in many scenarios,central control nodes do not exist,so these schemes are not practical.In order to solve the coexistence of LTE and WiFi in distributed scenarios,we propose some machine learning based algorithms to enable small base stations to learn optimal decisions adaptively through interactions with the environment.Not only in distributed scenarios,we also study the application of machine learning algorithms in centralized scenarios to overcome some limitations of the traditional methods.Firstly,the deep Q-network learning algorithm is applied to the LTE and WiFi coexistence problem in the distributed coexistence systems.In order to ensure the fairness between WiFi and LTE systems and among the small base stations in LTE system,we propose a max-min unlicensed resource allocation problem.Since this problem is essentially a multi-player game,we reformulate it as a cooperative game,and propose a multi-agent deep Q-network learning algorithm to enable each base station to learn the optimal resource allocation decision adaptively.In order to make the algorithm converge to pure strategy Nash equilibrium faster and better,we adopt a special Q-learning rule.The simulation results show that the proposed algorithm can converge to the optimal pure strategy Nash equilibrium quickly and achieve the same performance as the optimal central algorithm.Secondly,based on the first work point,we extend the problem to a long-term unlicensed resource allocation and user association problem.Our goal is to maximize the long-term average per-user throughput of the LTE system,while ensuring the long-term fairness between different access technologies and among different small base stations.Similarly,we model the problem as a non-cooperative game.Considering the timing of the problem,we propose a deep reinforcement learning algorithm based on bi-directional long short-term memory network and strategy gradient method,which enables each small base station to learn the balanced hybrid strategy adaptively by interacting with the environment.The simulation results show that the proposed algorithm can converge to the mixed strategy Nash equilibrium.In addition,the simulation results also show that comparing with the short-term unlicensed resource allocation,the long-term unlicensed resource allocation and user association can further improve the average throughput of LTE system by learning the load variation rule of WiFi networks and making full use of the idle time resource of WiFi networks.Finally,we study an application of machine learning algorithms in central scenarios.In or-der to improve the total energy efficiency of the LTE-U system,we consider joint optimization of resource allocation,user association and base station sleeping.Our objective optimization prob-lem is a complex mixed-integer fractional programming problem.To simplify the problem,We transform the original problem into a standard mixed-integer convex programming problem by the Dinkelbach algorithm and several variable substitutions.Now we can obtain the optimal solution by the standard branch-and-bound algorithm.However,the time complexity of the branch-and-bound algorithm is exponential in the worst case which is unacceptable.Since the time complexity mostly comes from the pruning strategy,we propose a pruning strategy learning algorithm based on imitative learning to learn the optimal pruning strategy of the branch-and-bound algorithm in order to obtain a near-optimal solution while greatly reducing the time complexity of the branch-and-bound algorithm.The simulation results show that our algorithm can greatly speed up the branch-and-bound process and highly outperform the general heuristic algorithms in performance.