Research On Resource Optimization Techniques For Delay Sensing In Massive Machine Type Communications

The widespread application of 5G has gradually promoted the development of the Internet of Everything.With the advent of the Internet of Everything,tens of thousands of Machine Type Communication(MTC)devices will be accessed via wireless networks,resulting in heavy congestion.At the same time,considering the energy consumption of MTC devices,green energy harvesting and wireless energy transfer(WET)are new ideas for solving the energy shortage in the Internet of Things(IoT).This paper focuses on the delay optimization of uplink access in large-scale IoT systems.The specific research content includes the following aspects:1.Aiming at the problem of uplink access congestion and energy shortage of the devices in massive MTC(mMTC)systems,a distributed uplink traffic control scheme based on the Deep Distributed Recurrent Q-networks(DDRQN)algorithm is designed.There are two queues modeled at the MTC device:battery queue and data queue.The uplink traffic control problem is modeled as a Markov decision process(MDP)with the goal of minimizing the cost of delay and battery cost.Due to the Markovian nature of the problem,we use a novel centralized learning,distributed decision algorithm—DDRQN for uplink traffic control.This algorithm can obtain the similar performance with the centralized solution-Deep Deterministic Policy Gradient(DDPG)algorithm,and can effectively reduce the signaling overhead after the Q-network deployed at the device.2.In order to ensure the timeliness of information in wireless communication systems,the concept of Age of Information(AoI)is proposed to quantify the timeliness and freshness of information in wireless systems,which describes the time elapsed since the latest updated packet form the source node was successfully received at the destination node.As a novel measure of information freshness,AoI is significantly different from delay.In order to understand this new concept in depth,this article investigates the basic concepts and methods of AoI optimization,as well as several typical scenarios of AoI optimization applications.3.Focus on the user scheduling and energy allocation with AoI as the optimization target in the mMTC system with node energy constraints.The information freshness is quantified using AoI.The base station provides WET services for these energy-constrained devices.The problem of resource allocation and user scheduling of the system is modeled as an MDP problem,and each device updates two queues,including the AoI queue and the battery power queue.Through user scheduling and WET resource allocation,the sum average AoI of the devices in the system with limited node energy can be minimized,while meeting the AoI requirements of each node.We use reinforcement learning algorithm-DDPG to find the optimal resource allocation and user scheduling strategy.In summary,this article aims at delay optimization in mMTC systems,and uses two indicators of queuing delay and AoI to quantify the system delay.At the same time,considering the energy-limited characteristics of MTC nodes,the green energy harvesting and WET are used to provide energy for the nodes.Considering the Markov properties of the data queue,AoI queue,and battery energy queue,the problem is modeled as an MDP problem and the reinforcement learning algorithm is used to solve the problem.Simulation results show that the two algorithms proposed in this paper have achieved a reduction in latency,at the same time reduced the energy consumption of resource-constrained nodes,and prevented nodes from leaving the network due to power exhaustion.