Research On Inter-Cell Interference Coordination Algorithm Based On Deep Reinforcement Learning

The rapid development and popularization of mobile communication have greatly satisfied people’s daily production and life needs.With the advent of the 5G era,various phenomenalevel applications have put forward higher requirements on the performance of mobile communication systems.However,inter-cell interference(ICI)restricts the performance of mobile communication system.In order to solve ICI,the common method is inter-cell interference coordination(ICIC)technology.In the face of extremely complex interference situations in 5G networks,traditional interference coordination schemes have been difficult to adapt.Research on more autonomous and intelligent dynamic ICIC technology is of great significance to improving the performance of communication systems.In this thesis,the deep reinforcement learning technology is applied to the dynamic ICIC of the mobile communication system,and the following research is carried out:First of all,this thesis studies the basic principles of inter-cell interference coordination technology and reinforcement learning technology.Specifically,the inter-cell interference coordination technology in frequency domain,time domain,and space domain is introduced,and the basic composition of reinforcement learning is analyzed.On this basis,deep reinforcement learning technology and multi-agent deep reinforcement learning technology are introduced.Secondly,to solve the problem that the traditional soft frequency reuse algorithm cannot dynamically adjust the frequency reuse scheme according to the user distribution and service requests of each cell,which leads to the decrease of the throughput of the edge users,a dynamic soft frequency reuse algorithm based on proximal policy optimization is proposed.This algorithm is a centralized interference coordination algorithm.By introducing a central controller,centralized joint optimization of multiple cells is realized.The central controller is modeled as an agent,and the agent can sense the user distribution and service requests information of each cell.With the optimization goal of maximizing the spectrum efficiency of edge users,the algorithm dynamically adjusts the soft frequency algorithm scheme to achieve interference coordination for each cell.In this thesis,the states,actions,rewards and neural network structure involved in the algorithm are designed,and based on the NS3-Gym open source platform,the Actor,Critic,and experience pool modules of the algorithm are implemented using Python.The simulation results show that,compared with the traditional soft frequency reuse algorithm,the proposed algorithm can effectively improve the spectrum efficiency of cell edge users.Finally,in view of the additional delay and overhead caused by the centralized interference coordination algorithm due to the collection of the status information of each cell,this thesis proposes a dynamic partial frequency reuse algorithm based on multi-agent proximal policy optimization.This algorithm models each base station in the system as an agent,and each agent dynamically adjusts the partial frequency reuse algorithm according to its own local observation state to realize distributed interference coordination.The algorithm effectively improves the cooperation performance between agents by classifying the cells and designing a time-based inter-cell communication method;the problem of illegal actions of multi-agents is solved by designing a mask operation,and realizes the complete cooperation task among multi-agent by sharing rewards.The simulation results show that compared with the traditional partial frequency reuse algorithm,the proposed algorithm effectively improves the communication performance of cell edge users.