Research On AoI Optimization Method For UAV-assisted IoT Networks

In the traditional Internet of Things(IoT),IoT devices generate sensory data through sensing the environment and send it back to the data center for processing.However,the battery capacity of IoT devices is limited,which greatly limits their transmission power and coverage,fundamentally affecting the timely delivery of data and reducing the freshness of data.In recent years,unmanned aerial vehicles(UAVs)has been widely used to assist the IoT for data collection due to its high flexibility and low deployment cost,so as to meet the timeliness requirements of real-time applications.However,in the actual IoT system,the generation of data at the IoT device has a certain randomness,and the available energy of UAV is limited.Therefore,how to effectively improve the freshness of information in the data center through joint dynamic optimization of data collection,data transmission and charging behavior when the timeliness of information in the IoT device cannot be accurately known has become an important problem to be solved.To solve this problem,the thesis takes age of information as an evaluation indicators of information freshness for UAV-assisted IoT system,and studies an age of information optimization method based on UAV dynamic trajectory planning by using deep reinforcement learning theory and technology.The specific research contents are as follows:(1)A single UAV dynamic trajectory planning policy is proposed,and its effectiveness in optimizing age of information and improving data timeliness is verified.For the UAV-assisted IoT system,how to design the flight trajectory of UAV to minimize the weighted average age of information of the IoT devices in the base station has become a core issue that needs to be studied under the condition of considering the random generation mode of IoT device data and the energy safety constraints of UAV.For the above scenarios,the optimization problem is described as a partially observable Markov decision process(POMDP)with non-uniform time step to solve the challenge of partially observable environmental information and non-uniform time step,where the effective action of an agent is coupled with its observed value.To solve this problem,the thesis designs a trajectory planning algorithm based on deep recurrent reinforcement learning.By comparing the proposed algorithm with the baseline algorithms,the simulation results show that the performance of the proposed algorithm is better than other baseline algorithms.(2)A completely distributed multi-UAV trajectory planning policy is proposed,and its effectiveness in improving data collection efficiency and optimizing age of information is verified.In the network scenario where multiple UAVs collaborate to collect data,how to use the trajectory planning policy of multiple UAVs to collect data is the core problem to effectively reduce the average age of information.In view of this,the dynamic trajectory planning problem of multiple UAVs is modeled to minimize the average age of information at the base station by considering the energy constraints and collision constraints of multiple UAVs in the thesis.Furthermore,the optimization problem is described as a decentralized partially observable Markov decision process(Dec-POMDP)based on the incomplete environmental state information and the independent observation information of multiple UAVs.In order to improve the cooperative efficiency of multi-UAV,a distributed trajectory planning algorithm based on multi-agent deep reinforcement learning is designed to solve the problem.The convergence and effectiveness of the proposed algorithm are verified by comparison with the baseline algorithms.