Discontinuous UAV Trajectory Planning For Data Collection In WSN Based On Reinforcement Learning

Generally speaking,wireless sensors have certain limitations in terms of power supply,communication capacity,computing and storage capacity,which affect the performance of network such as data collection efficiency and life cycle.Considering the high flexibility and mobility of unmanned aerial vehicle(UAV),it can be used as a mobile data collector to assist the network to complete the task of data acquisition.In this process,the trajectory planning of UAV is one of the important factors affecting the acquisition efficiency.Because different practical application scenarios have different requirements on the reliability and effectiveness of data collection,this paper studies the trajectory planning of single UAV assisted sensor network data collection,and optimizes the effective data rate,energy utilization rate and life cycle of the network.The main work is as follows:A discontinuous UAV trajectory planning algorithm Q-TDUD based on Q-learning is proposed for scenarios in which the data generation rate of each node is random and the states of sink nodes are inconsistent in wireless sensor networks.Firstly,according to the data rate change probability of a single node in the sensor network and the delay time of transmitting data per unit data volume and per unit distance,the delay difference model of sink node is established.Then,the UAV trajectory design is subdivided into discrete Markov process(MDP),and the Q-learning algorithm in reinforcement learning is applied to optimize the UAV trajectory.The position and motion direction of UAV are used as state set and action set in reinforcement learning.By using the reward setting in Q learning,two reward modes are set to enable the UAV to intelligently choose its hovering state or flight state,changing the traditional continuous UAV flight trajectory planning into a noncontinuous trajectory planning.The results show that the UAV can adjust its flight trajectory intelligently when the aggregation completion time of each sink node changes with the updating of the data volume of each cluster.The performance of the UAV is better than the benchmark scheme in terms of the completion time of the UAV acquisition task,the effective data rate of the collected data and the energy consumption of the UAV.Based on the previous work,a safe data acquisition trajectory planning scheme,SDCTD,is proposed for non-secure network scenarios where there may be hot nodes and captured nodes.First,hot node analysis is added to the original cluster model.The hot list is counted by the UAV and broadcast in the cluster.The hot node does not participate in the campaign of the forwarding node,and the remaining nodes campaign for the forwarding node according to their own conditions and form a small group in the cluster.Dijkstra algorithm is used to obtain the optimal multi-hop route for data collection in the cluster.In terms of security,the UAV will carry out security detection on the next cluster head before data collection.If it is illegal,a new scheme will be adopted to replace the original plan for data collection.In order to minimize the arrival time of subsequent collection points and minimize the collection delay caused by illegal nodes,UAV divides nodes in clusters into "in-route" nodes and "out-of-route" nodes.The in-route nodes are nodes appearing near the connecting line between UAV and the next destination,while the out-of-route nodes are nodes close to other uncollected clusters.The ant colony TSP algorithm was used to calculate the UAV’s best routing acquisition path and execute the task.The non-routing nodes chose the nearest access point to become members of other clusters.The simulation results show that compared with the scheme without security and hot spot analysis,the proposed scheme has significantly improved node energy consumption,data collection efficiency and network life cycle.