| With the rapid development of wireless communication technology,all kinds of new Internet services are blossoming.Mobile terminal equipment and network data traffic are showing explosive growth.As one of the key technologies of 5G,the Ultra-Dense Network(UDN)can effectively improve the coverage and throughput of the network.But compact deployment of small base stations can cause serious interference and power consumption problems.It is a hot and difficult issue of current research that how to reduce the system energy consumption while ensuring the spectrum utilization rate.Therefore,this paper studies the resource allocation problem in the downlink of dense heterogeneous cellular network with spectrum efficiency and energy efficiency as the optimization goal.In this paper,distributed optimization algorithms based on Deep Reinforcement Learning(DRL)framework are proposed to solve this kind of NP-hard problems with high complexity and difficulty.The main research contents of this paper are as follows:To solve the problem of resource allocation in the downlink of dense heterogeneous cellular network with energy efficiency as the objective,a centralized resource allocation algorithm based on Deep-Q-Network(DQN)is proposed in this paper.Firstly,the mathematical optimization model aiming at energy efficiency is established according to the network system model.Secondly,a neural network is employed instead of the Q table to avoid the dimension disaster problem.The result of resource allocation algorithm is obtained according to the trained neural network model.Finally,the simulation results show that DQN algorithm can achieve higher system energy efficiency compared with Q learning and greedy algorithm.At the same time,the convergence speed and stability of DQN algorithm are significantly improved.In addition,the optimal learning rate has been found by changing the learning rate to observe the performance of the model.A centralized resource allocation algorithm based on Double Deep-Q-Network(DDQN)is proposed to solve the resource allocation problem in the downlink of dense heterogeneous cellular network,which took the weighted sum of energy efficiency and spectral efficiency as the optimization objective.Firstly,the energy efficiency and spectrum efficiency in the system are theoretically deduced.It is found that there is a trade-off relationship between the two.Secondly,the joint weighted optimization objective function of energy efficiency and spectrum efficiency is established according to the network model.And the mathematical model is established according to the resource allocation and power limitation conditions.Thirdly,the DDQN algorithm is used to solve the problem of over-estimation of Q value caused by greedy selection strategy in DQN algorithm.Finally,the simulation results on Tensor Flow show that there is a trade-off relationship between energy efficiency and spectral efficiency.At the same time,the optimal weighting factor is obtained according to the simulation results.In addition,the DDQN algorithm divides the step of Q value selection into two steps: action selection and value evaluation.It makes the obtained estimated Q value closer to the target Q value,avoiding the problem of overestimation which make the results are more accurate and higher energy efficiency is obtained. |
