Reinforcement Learning And The Application To Interference Management Of Femtocell

With the gradual development of the mobile Internet and the era of big data,the traditional cellular network present already is not up to the amount of data,while the small cell to Femtocell as the representative of the base station can not only supply good service level,but also can effectively improve the network coverage rate.However,the interference caused by the introduction of the Femtocell has an impact on QoS of Macrocell users,but also has a certain impact on the construction of the Macrocell cell.So it is urgent to study the interference management of Femtocell network.In this paper,we mainly discuss the reinforcement learning and its application to the interference management of Femtocell dual layer network.Strengthen a very active research area in the field of machine learning as learning to explore,in the dynamic environment of the optimal decision problem has a great advantage in recent years,especially after 2010 became the Femtocell double layers network interference management problems become a hot research direction.Because the reward function determines the goal of the system,the choice of reward function is very important.In the prior to the application of reinforcement learning in Femtocell interference management research in the design of the reward function without considering the distance between the Femtocell cell and the Macrocell user,all Femtocell cell are in exactly the same the way of learning,its performance is limited.This paper presents a new reward function,the function will considering the distance between the Femtocell cell and the Macrocell user,and according to the new reward function is proposed to improve the return of distributed Q-learning algorithm based on interference management.Through the comparison of simulation experiments in three kinds of position relations,compared to the improved return function to improve before can in a very good guarantee and communication service quality of Macrocell users and greatly improve the capacity of the whole system.In view of the traditional reinforcement learning algorithm convergence problem,in this paper we combined with the characteristics of Femtocell base stations and introduce the concept of Docition.On this basis,this paper improves the state framework of the algorithm and proposes an interference management algorithm of Docition Q-learning.The simulation results show that the Docition Q-learning algorithm can effectively improve the convergence speed of the algorithm.