Resource Allocation Strategy For Heterogeneous Wireless Networks Based On Q-learning

With the large-scale deployment of femtocell,the femtocell heterogeneous networks will suffer serious,complex and varying interference.Moreover,users' QoS requirement is also increasing.Therefore,it is important to reduce the interference in femtocell heterogeneous networks and to guarantee delay QoS,minimum data requirements and other QoS metrics for users.Accordingly,we have done some deep research on the sub-channel allocation and power control scheme for femtocell heterogeneous networks with QoS provisioning based on Q-learning.Firstly,in his paper,we have proposed a sub-channel allocation scheme based on Q-learning.The proposed scheme ensures the delay-sensitive traffic QoS requirements and the minimum data rate requirements for different users.During sub-channel allocation,to avoid cross-layer interference between femtocells and macrocells,there are signal interaction between macro base stations and femtocell base stations.Moreover,the sub-channel allocation scheme strengthens the self-organization of femtocells through Q-learning.The simulation results demonstrated that the proposed algorithm is effective in both sparse and dense deployment scenarios with femtocell.Secondly,we have proposed a power control algorithm for two-tier femtocell networks based on Q-learning.In the proposed algorithm,we firstly made the continuous power discrete.Then we established power control framework based on Q-learning and proposed a weighted filter algorithm based Boltzmann.In the learning process,each femtocell base station as an agent reduces the cross-tier interference through a pricing function and maximizes the effective capacity to guarantee delay QoS for delay-sensitive traffics.Finally,we analyzed the proposed power control algorithm through the simulation.The simulation shows that the proposed algorithm has a rapid convergence with a little loss of effective capacity compared with non-cooperative game-based power control algorithm.Moreover,the effective pricing will well reduce the cross-tier interference.