Interference Coordination Based On Reinforcement Learning In Femtocell Networks

With the development of society and technology, people’s demand for mobile communicationstransmission rate is getting higher and higher, especially for indoor transmission. Due to the scarcityof spectrum resources, how to use the wireless resources properly and how to improve the spectralefficiency have been the focus of research. In order to improve voice and data rate of the indoor andsolve the problem of limited wireless resources, the concept of femtocell has been proposed inrecent years.In this thesis, a new dynamic spectrum allocation scheme for femtocell and macro use totallyseparate spectrum networks based on Q learning is proposed to avoid the complex optimizationproblem. The proposed algorithm allocates spectrum through Q-Learning to dynamically adjust thenumber of subchannels by different frequency reuse factors. The reward function of Q-Learningconsiders spectral efficiency of all femtocells to ensure the minimum spectral efficiency of each cellas much as possible. Simulation results show that the proposed algorithm has improved the spectralefficiency with QoS requirements of femtocell guaranteeing.In case of co-channel operation, a new dynamic distributed interference management schemeQL&game is proposed. QL&game aims at improving the performance of the femtocell system, andcontrolling the SINR of the macro user above the given threshold. The simulation results show thatthe QL&game algorithm proposed in this paper has increased femtocells and system capacity withmacro capacity guaranteed in comparison with other algorithms.Interference management based on fuzzy Q-learning taking into account both the QoSrequirements of the macro and femtocell is proposed. The algorithm combines the characteristics ofthe fuzzy inference system and reinforcement, which reduces the Learning cycle, solves thecontinuous state space problem and improves the learning accuracy. The environment state as theinput of the fuzzy inference system and the Q-learning determines the output of the FIS. Thesimulation result shows that the proposed algorithm has increased the system capacity with both theQoS requirements of the macro and femtocell guaranteed in comparison with Q-learning.