Research On Radio Resource Management Based On Machine Learning In M2M Communication

The ultimate goal of Internet of Things(IoT)is to enable people or things to communicate with anyone or anything using any service at any moment and any place The rapid development of IoT will greatly facilitate people's lives.Machine to Machine(M2M)communication refers to the autonomous communication between devices without human intervention.It is an important part of IoT.Cellular network is an ideal carrier for M2M communication benefited from its characteristics such as broad coverage,high reliability and supporting for high-speed mobility.However,the existing resource allocation algorithm of the cellular network is mainly designed for the traditional Human to Human(H2H)communication and the M2M communication has unique features such as various service types,the power saving requirement and the large amount of users.Thus,the existing resource allocation algorithm is not fully applicable.In this paper,the reinforcement learning algorithm is introduced,and the reasonable allocation of limited resources is achieved in two scenarios.The main research contents and innovation points are summarized as follows.1.For the multi-service feature of M2M devices,the rate requirements of M2M devices are distinguished by three different Quality of Experience(QoE)functions.Improving QoE and saving power are set as the optimization target of spectrum and power allocation.In order to alleviate the burden of centralized scheduling of massive equipment by base station,a distributed learning algorithm based on reinforcement learning is proposed to solve the nonlinear mixed integer programming problem.The M2M devices are modeled as agents with reinforcement learning capabilities and could choose spectrum and power levels smartly.In this way,the complexity of solving this problem is greatly reduced.2.In order to deal with the competition of M2M devices for uplink spectrum,a multi-agent reinforcement learning algorithm based on game theory is introduced.At the same time,in order to realize the cooperation among agents without information interaction,a prediction method is introduced,which enables agents to predict the strategies of other agents by reference to their own behaviors.The simulation results show that the proposed algorithm achieves good performance in terms of QoE,power saving and computation complexity.3.In order to reduce the energy consumption of M2M devices and improve their network lifetime,the relay and energy harvesting technology are combined and applied in the M2M communication network.This paper focuses on the issues of relay selection,the power allocation of M2M devices and relays,and the selection of time partitioning coefficients.The optimization problem is divided into three sub-problems.Firstly,the relay is selected according to the channel conditions.Then,by solving the optimization problem,the transmit power of the source node is obtained.Finally,in order to maximize the long-term rate of the relay with chargeable devices,the delayed reward feature of reinforcement learning is used to determine the time partitioning coefficients and transmit power,and the value function approximation is introduced to deal with the continuous state faced by relays.Simulation results show that the proposed algorithm converges quikly and achieves good performance at outage probability and long-term average rate.