| Benefiting from their remarkable flexibility,mobility,and broad horizon,recently,Unmanned Aerial Vehicles(UAVs)have been widely applied into multifarious array of applications within smart cities,especially for tasks related to the intelligent Internet of Things(Io T).In complicated urban environments full of various obstacles,to complete a reliable and safe UAV navigation task,designing an efficient UAV navigation algorithm has become one of the key problems that need to be solved urgently.Cellular-connected UAVs are UAVs that communicate with ground cellular base stations.To address the autonomous navigation problem for cellular-connected UAV,the designed navigation algorithm is required to not only capable of guiding the UAV to avoid static obstacles densely distributed in high altitude or dynamic obstacles floating in the airspace,but also avoid the cellular base stations’ coverage holes..On the other hand,in order to efficiently complete the navigation task,the cellular connected UAV needs to take the shortest possible path to the specified destination.To solve this multi-objective optimization problem,this thesis proposes two autonomous navigation algorithms for cellularconnected UAV based on Deep Reinforcement Learning(DRL),aiming at the navigation problem for cellular-connected UAV in large-scale complex urban environment and high dynamic urban environment.The performance of these two methods has been greatly improved compared with baselines.The main research contents of this thesis are as follows:(1)Aiming at the characteristics of the two major urban environments,the navigation tasks for cellular-connected UAV in the two environments are established as a Markov Decision Process(MDP),respectively.Through this problem formulation process,this thesis transforms the complex and difficult autonomous navigation problem for cellular-connected UAV into a sequential decision-making problem that can be solved by DRL methods.(2)Aiming at autonomous navigation problem for cellular-connected UAV in large-scale complex urban environment,this thesis proposes a DRL algorithm named Decentralized Recurrent Proximal Policy Optimization(DecentralizedRPPO).The algorithm consists of two neural networks,SINR Prediction Network and Avoid Network,which are utilized to guide the UAV to avoid the cellular base stations’ coverage holes and static obstacles,respectively.In addition,a comprehensive actionselection method is designed.Subsequent simulation results indicate that the proposed algorithm can effectively solve the multi-objective optimization problem for cellular connected UAV navigation,and the convergence speed of the proposed algorithm is higher than that of baselines.(3)Aiming at autonomous navigation problem for cellular-connected UAV in high dynamic urban environment,this thesis proposes a DRL algorithm named Layered-RSAC.The algorithm is composed of a sub-network structure and an Integrated Network.The sub-network structure is able to motivate the UAV to evade collisions with dynamic obstacles and approach destinations,while the Integrated Network is capable of selecting a specific action from the sub-solutions generated by two sub-networks,maximizing the communication coverage probability of the flight airspace.The subsequent simulation results show that the proposed algorithm outperforms baselines when assessed by three indicators of navigation performance,security,efficiency,and communication-connectivity,and the convergence rate is also significantly improved. |
