A Study On Machine Learning Based Spectrum Allocation And Power Control Algorithms For D2D Communications Underlaying Cellular Networks

With the evolution of the mobile Internet,the number of user devices with access to the wireless communication network grows dramatically in recent years,causing an increasing demand for data service.Consequently,the shortage of frequency resources has become a major problem in wireless communication systems.How to use the limited spectrum resources to provide high-speed and reliable services for mobile users is an important research topic in the field of mobile communications.In recent years,with the commercialization of 4G(4th Generation)networks and the in-depth research on 5G(5th Generation)technologies,a wide range of emerging technologies have been proposed and applied to wireless communication systems,such as Massive Multiple Input and Multiple Output(Massive MIMO)antenna,Polar Code,Sparse Code Multiple Access(SCMA),Device-to-Device(D2D)communication,etc.Among them,D2 D communication was introduced in Long Term Evolution(LTE)Release 12.Through direct communication between two mobile devices,the spectrum utilization of a cellular communication system can be effectively improved,and the load of a base station can be as well.However,since D2 D users multiplex the spectrum resources of cellular users,including uplink and downlink resources,this can cause mutual interference between D2 D users and cellular users.Therefore,how to reduce the interference caused in D2 D communication has become a hot research topic in D2 D communication.Meanwhile,Big Data,Machine Learning(ML)and Artificial Intelligent(AI)technologies has developed rapidly in recent years,bringing breakthroughs to a large number of applications in the computer science field,such as image processing,speech recognition,Natural Language Processing(NLP),etc.Machine learning techniques focus on the correlation among data,which can help reduce the complexity and reach higher performance,save the computational resource through dedicated computing devices.ML has been applied to solve many wireless communication issues as well,such as resource allocation,channel coding and decoding,D2 D communication,etc.There are three main solutions to the interference caused by D2 D communication,namely resource allocation,power control and mode selection.This thesis mainly studies the spectrum allocation and power control in D2 D communication underlaying cellular networks,and proposes spectrum allocation and power allocation algorithms based on machine learning techniques.Firstly,the thesis proposes two Q learning based power control algorithms for D2 D communication,namely Centralized Q Learning based Power Control(CQLPC)algorithm and Distributed Q Learning based Power Control(DQLPC)algorithm.The two algorithmsconverge to the optimal transmission power by allowing each D2 D user iteratively to select the transmitting power and update the reward function.In CQLPC,multiple D2 D users select their own transmission power simultaneously,while in DQLPC each D2 D user selects its own transmission power independently.In the case where the same reward function is used,CQLPC and DQLPC can converge to the same optimal value.Simulation results show that CQLPC and DQLPC can achieve higher system throughput than traditional power control algorithms.Then,,the thesis proposes a Q Learning based Joint Spectrum Allocation and Power Control(QLSA-PC)algorithm,which considers both the spectrum allocation and transmission power control for D2 D user pairs.The QLSA-PC algorithm treats each D2 D user as an independent agent,and obtains the optimal solution by selecting RB and transmission power in the process of Q learning.Simulation results show that the QLSA-PC algorithm can achieve better performance than a pure spectrum allocation or power control algorithm,and can achieve higher system throughput and cellular satisfaction than traditional solutions.Finally,considering the high computational cost of Q learning,the thesis proposes two Deep Neural Network(DNN)based algorithms(DNNSA and DNNPC)for spectrum allocation and power control in D2 D communication,respectively.In DNNSA and DNNPC,the results obtained in the first two parts are used as training data.The location information on the cellular users and the D2 D users is preprocessed to form numerical features;the optimal RB and transmission power of the D2 D users are used as labels.Then a DNN is used to train on these features and the labels.Next,the trained DNN is used to predict the optimal RB and power levels of D2 D users.Simulation results show that DNNPC and DNNSA can approximate the performance of Q learning.Meanwhile,the obtained neural network model only requires training once for all the data,which significantly saves the computational time and resources.